Tutorials¶

Algorithm Template Docker 🐋¶

• 👉 Please refer to our 🐋 TopCoW_Algo_Submission repo on GitHub as a template and guide on the submission process (later we will publish a version adapted for TopBrain).
• The validation phase is not used for final evaluation.
  • Please use validation phase to debug and validate your Docker submission workflow.
  • Or better still, test locally with the provided test_run.sh and use the Try-out Algorithm on your algorithm page
    • Please refer to the README of our submission template repo for more instructions
• The final test phases only allow for one submission per team for each phase
  • The test phases take much longer than the smaller validation phases to be evaluated (might take a few hours!)
  • Please plan ahead and do not cram last minute for the final test phases.

Lessons from TopCoW 📰¶

We have summarized a few lessons learned from the TopCoW winning algorithms, which could be helpful for solving the more complex TopBrain whole vessel segmentation task:

1. Benefits of mixed modality training for CTA
   • Training with both MRA and CTA modalities led to better performance for individual modality, particularly for CTA :)
2. Importance of topological optimizations
   • A wide range of topological optimizations were used by the top submissions, for example:
     • Centerline-based loss functions
       • centerline Dice (clDice) loss [Link]
       • connectivity-aware surrogate (CAS) loss [Link]
       • skeleton recall (SkelRecall) loss [Link]
       • centerline boundary Dice (cbDice) loss [Link]
     • Connectivity-based optimizations
       • e.g. originally designed for lung airway segmentation [Link]
       • e.g. with neighborhood relation in NexToU architecture [Link]

👉 More information can be found in the 📰 TopCoW summary paper's Discussion section.

FAQ¶

0. Common pitfalls in failed submissions¶

• Watch out for Time limit exceeded errors (try your docker out before submission helps)
• During inference, you can first cast/convert the input images to float before inputting them to your model
  • this prevents the following RuntimeErrors:
    • Input type (torch.cuda.ShortTensor) and weight type (torch.cuda.HalfTensor)
    • result type Float can't be cast to the desired output type Short
• Mindful of the 32GB main RAM memory limitation during the inference ⚠️

1. How do you weight the metrics and how will the final ranking be done?¶

For the MICCAI in-person event and its results annoucement, we will simply use an equal weighting of metrics mentioned in our "Assessment" page. Please visit our GitHub repo 👉 TopCoW_Eval_Metrics 📐 for our evaluation metrics for the tasks. (We will broadcast in the forum and webpages and github readme accordingly if we update any metrics.)

The ranking to be annouced for the event and awards will thus be based on the leaderboard displayed on grand-challenge. The leaderboard uses equal weights for each column (for example when you see "mean position", it is the mean of several columns' positions/ranks), and 'rank then average'.

We will summarize the results and do post-challenge analysis. Additional metrics and more advanced ranking analysis may be introduced.

2. Can I take part in just one track and one task?¶

Yes, definitely. You are welcome to submit to any track and task of your preference. There are 2 rankings in the end, one for each modality track.

3. Submission Confidentiality and Availability¶

According to grand-challenge, "Challenge organisers do not have access to the algorithms. Challenge organisers only have access to the algorithms logs and predictions for the cases in the challenges test/training archive. Challenge organisers cannot use the algorithm on any other cases. Algorithm owners can see who has access to use their algorithm in the Users tab on their algorithm page. No one is ever given access to the algorithms container image."

Having said that, please contact us with your team's information and contact details, so we can reach out to you for the MICCAI event and follow-up publications. (We organizers cannot see your profile's email due to GDPR.)