How to participate

Geometric metrics

For each patient, the participant must upload a file containing the segmentation result to the evaluation platform. Each file must be saved in NIFTI file format with the same name and image information (i.e. resolution, spacing, dimensions) as the corresponding input images.

The same information on the class labels as used for the reference volumes should be used, namely:

  • Left ventricle cavity: pixel intensity of 1
  • Healthy myocardial tissues: pixel intensity of 2
  • Infarct tissues: pixel intensity of 3
  • MVO region: pixel intensity of 4
From these files, the corresponding LV cavity, full myocardium, infarct region and MVO region, as defined in the MYOSAIQ contest page, will be automatically computed. These four regions will then be compared to the corresponding references using the following geometric metrics:
  • The Dice score
  • The Average Symmetric Surface Distance (ASSD)
  • The Hausdorff Distance (HD)

Clinical metrics

In addtion to the NIFTI files described above, participants are invited to upload a cvs file for each of the four targeted structures (i.e. LV cavity, full myocardium, infarct region and MVO region) containing the values of the estimated volumes and the cumulative distributions computed for each patient.

Each cumulative distribution must have 600 values from 0 to 599 mL. Each file must be named as LV_volumes.csv, MYO_volumes.csv, MI_volumes.csv and MVO_volumes.csv. They must have a header and contain all estimated volumes (Vol) and 600 values of the cumulative distributions in the following format, where P0 represents the probability the volume is less than or equal to 0 mL, P1 represents the probability the volume is less than or equal to 1 mL, etc:

volume file format

Participants who are not able to estimate the cumulative distributions still take part in the challenge. In this case, volumes estimated from the segmentation estimates and cumulative distributions are automatically calculated from the uploaded NIFTI files. Cumulative distributions correspond to Heaviside step functions centred on the estimated volumes.

From these files, the estimated volumes and cumulative distributions will be compared to the corresponding references using the following clinical metrics:
  • Correlation coefficient score computed from the set of volumes

  • Mean absolute error (MAE) computed from the set of volumes

  • Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of volumes

  • Continuous Ranked Probability Score (CRPS) computed from the set of cumulative distributions.