Evaluation procedure

Goals

The aims of the evaluation metrics involved in the segmentation contest are twofold:

• measure the degree of accuracy of the left ventricular endocardium and epicardium as the left atrium. This will be done through global and local measures of similarity with the reference contours;
  
  
• measure the degree of accuracy of the derived clinical indices.

Clinical metrics

The clinical metrics are the ones that are the most widely used in cardiac clinical practice:

• Left ventricular cavity
  
  
• Correlation coefficient computed from the set of End Diastolic Volumes (EDV) measurements
  
  
• Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
  
  
• Correlation coefficient computed from the set of Ejection Fraction (EF) measurements
  
  
• Bias computed from the set of EDV measurements
  
  
• Bias computed from the set of ESV measurements
  
  
• Bias computed from the set of EF measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EDV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
  
  
• Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EF measurements
  
  

Geometrical metrics

A set of geometrical metrics are computed per structure, i.e. left ventricle endocardium (LV endo), left ventricle endocardium (LV epi) and left atrium (LA), to allow the computation of ranking per structure. The following metrics are computed:

• LV endo
  
  
• The average Dice value for the left ventricle cavity at ED.
  
  
• The average Dice value for the left ventricle cavity at ES.
  
  
• The average mean absolute distance for the endocardial contour of the left ventricle at ED.
  
  
• The average mean absolute distance for the endocardial contour of the left ventricle at ES.
  
  
• The average Hausdorff distance for the endocardial contour of the left ventricle at ED.
  
  
• The average Hausdorff distance for the endocardial contour of the left ventricle at ES.
  
  

• LV epi
  
  
• The average Dice value for the area composed by the myocardium and the left ventricle cavity at ED.
  
  
• The average Dice value for the area composed by the myocardium and the left ventricle cavity at ES.
  
  
• The average mean absolute distance for the epicardial contour of the left ventricle at ED.
  
  
• The average mean absolute distance for the epicardial contour of the left ventricle at ES.
  
  
• The average Hausdorff distance for the epicardial contour of the right ventricle at ED.
  
  
• The average Hausdorff distance for the epicardial contour of the right ventricle at ES.
  
  

• LA
  
  
• The average Dice value for the left atrium region at ED.
  
  
• The average Dice value for the left atrium region at ES.
  
  
• The average mean absolute distance for the left atrium contours at ES.
  
  
• The average mean absolute distance for the left atrium contours at ED.
  
  
• The average Hausdorff distance for the left atrium contours at ED.
  
  
• The average Hausdorff distance for the left atrium contours at ES.