Assessment¶

There are two tracks for the multiclass segmentation task, one track for CTA modality and one track for MRA modality. The assessment of algorithms will be the same for both CTA and MRA modalities, and with the following metrics:

Metrics for TopBrain Vessel Segmentation¶

Six evaluation metrics with equal weights for the multiclass segmentation task:

1. Class-average Dice similarity coefficient
2. Class-average centerline Dice (clDice)
3. Class-average error on number of connected components
4. Class-average Hausdorff distance 95% percentile (HD95)
5. Class-average error on number of invalid neighbors
   • Each vessel has a list of valid neighbor vessels
   • Neighborhood is defined by adjacency or "touching"
6. Average F1 score (harmonic mean of the precision and recall) for detection of the "side road" vessels
   • "Highway" vs "side road" vessels
   • "Highway" vessels are ICA, VA, BA, ACA, PCA, MCA, ECA, STA, MaxA, VoG, StS, SSS
   • "Side road" vessels are Pcom, Acom, 3rd-A2, 3rd-A3, SCA, AICA, PICA, AChA, OA, ICV, BVR, MMA
   • List of side-road vessels is documented in the constants.py in our evaluation code. See below.

Evaluation Code¶

Whatever we use for the challenge evaluation will be released and synchronized to the following repo in a transparent manner. Please refer to this repo for our assessment implementations.

Please feel free to leave an issue or let us know if you have further feedback or questions.

Further Readings¶

• Yang, Kaiyuan, et al. "Benchmarking the cow with the topcow challenge: Topology-aware anatomical segmentation of the circle of willis for cta and mra." ArXiv (2025).
• Shit, Suprosanna, et al. "clDice-a novel topology-preserving loss function for tubular structure segmentation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.