(Sunday, 7/12/2020) [Eastern Time (GMT-4)]
Room: AAPM ePoster Library
Purpose: To investigate the beta distribution as a method to quantitatively and comprehensively represent contour agreement, specifically for evaluating the effectiveness of training interventions.
Methods: Five attending physicians and seven residents contoured twenty-six head-and-neck structures on two patients. For the residents, the contouring on the second patient was performed after educational intervention. For each structure contoured by each resident, the Simultaneous Truth and Performance Level Estimation (STAPLE) was applied to a collection of all the attending physician contours and the resident contour to generate a group consensus contour. For each resident, Jaccard coefficients (contour agreement scores) were calculated between the consensus contour versus the resident contour of each structure and patient. These 2184 Jaccard coefficients were then fitted to a beta distribution, stratified by structure, resident, and patient. Lastly, plots and heat maps of the beta distributions’ probability density functions (PDF) were generated to visualize the contour agreement.
Results: Plots of the beta distributions’ PDF could illustrate the confidence of the contour agreement. For example, for resident 1 and left level 1 lymph nodes, the training shifted the mean of the beta distribution from 0.59 to 0.76 (average agreement increased), the standard deviation from 0.24 to 0.10 (variability between resident contour compared to attending contours reduced), and kurtosis from -0.87 to 0.1 (higher peak/confidence and less outliers), indicating the training was effective in bringing the resident contour in closer agreement with the attending contours. The beta distribution could be used to evaluate which structures and residents were more susceptible to improvement with training compared to other structures and residents.
Conclusion: The beta distribution model comprehensively scored contour agreement in the context of each structure’s inherent contour agreeability. This model could be used to evaluate the effectiveness of training interventions and to provide feedback to residents about their learning progress.
Keywords
Taxonomy
IM/TH- image Segmentation: General (Most aspects)
Contact Email
