Evaluation procedure

Goals

The aims of the evaluation metrics involved in the segmentation contest were twofold:

• measure the degree of accuracy of the left ventricular endocardium and epicardium as the right ventricular endocardium. This will be done through global and local measures of similarity with the reference contours;
  
  
• measure the degree of accuracy of the derived clinical indices.
  
  
  

For this contest, we used the usual clinical and geometrical metrics.

Clinical indice metrics

The clinical metrics are the ones that are the most widely used in cardiac clinical practice. Moreover a set of metrics are computed per structure (i.e. LV cavity, RV cavity and myocardium) to allow the computation of ranking per structure. The following metrics were computed:

• Left ventricular cavity
  
  
• Correlation coefficient computed from the set of End Diastolic Volumes (EDV) measurements
  
  
• Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
  
  
• Correlation coefficient computed from the set of Ejection Fraction (EF) measurements
  
  
• Bias computed from the set of EDV measurements
  
  
• Bias computed from the set of ESV measurements
  
  
• Bias computed from the set of EF measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EDV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EF measurements
  
  

• Right ventricular cavity
  
  
• Correlation coefficient computed from the set of End Diastolic Volumes (EDV) measurements
  
  
• Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
  
  
• Correlation coefficient computed from the set of Ejection Fraction (EF) measurements
  
  
• Bias computed from the set of EDV measurements
  
  
• Bias computed from the set of ESV measurements
  
  
• Bias computed from the set of EF measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EDV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EF measurements
  
  

• Myocardium
  
  
• Correlation coefficient computed from the set of Myocardial Mass (at End Diastolic time instant) measurements
  
  
• Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
  
  
• Bias computed from the set of Myocardial Mass measurements
  
  
• Bias computed from the set of ESV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of Myocardial Mass measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
  
  

Distance error metrics

A set of geometrical metrics were computed per structure (i.e. LV cavity, RV cavity and myocardium) to allow the computation of ranking per structure. The following metrics are computed:

• Left ventricular cavity
  
  
• The average Dice value for the left ventricle cavity at ED.
  
  
• The average Dice value for the left ventricle cavity at ES.
  
  
• The average Hausdorff distance for the endocardial contour of the left ventricle at ED.
  
  
• The average Hausdorff distance for the endocardial contour of the left ventricle at ES.
  
  

• Right ventricular cavity
  
  
• The average Dice value for the LV cavity at ED.
  
  
• The average Dice value for the LV cavity at ES.
  
  
• The average Hausdorff distance for the endocardial contour of the right ventricle at ED.
  
  
• The average Hausdorff distance for the endocardial contour of the right ventricle at ES.
  
  

• Myocardium
  
  
• The average Dice value for the myocardium region at ED.
  
  
• The average Dice value for the myocardium region at ES.
  
  
• The average Hausdorff distance for the myocardial contours at ED.
  
  
• The average Hausdorff distance for the myocardial contours at ES.