(Sunday, 7/12/2020) [Eastern Time (GMT-4)]
Room: AAPM ePoster Library
Purpose:
Knowledge-based DVH prediction systems vary in their approach to uncertainty estimation. This work establishes a framework for empirical quantification of DVH prediction accuracy, so that different knowledge-based algorithms can be fairly compared, and clinicians can properly interpret their results.
Methods:
A natural method to quantify uncertainty of different knowledge-based predictions is to calculate the root-mean-square (RMS) deviation from the actual clinical DVHs of a large validation set. In such a case, ˜68% of clinical DVHs should lie within the prediction uncertainty band (i.e., 1 s from mean); this provides an intuitive standard by which to test prediction uncertainties provided by the knowledge-based model within the treatment planning system. RMS-uncertainties were compared for both ORBIT-RT and RapidPlan™ knowledge-based prostate models, built with the same training cohort. Prediction accuracy was expressed as a census of a subset of clinical DVHs (ORBIT-RT: 73 plans, RapidPlan: 26 plans) falling within the uncertainty band. Finally, the calculated RapidPlan RMS-uncertainty was compared to the provided RapidPlan uncertainty.
Results:
Both ORBIT-RT and RapidPlan predictions exhibit comparable mean- and RMS-deviation in the 80-105% dose region. Below 80%, differences emerged: the ORBIT-RT mean-deviation from clinical (prediction bias) was consistently lower than RapidPlan, while the RapidPlan RMS-deviation (prediction uncertainty) was often lower than ORBIT-RT. As expected, the RMS uncertainty bands of both models’ predictions encompassed ˜68% of clinical DVHs. However, for the provided RapidPlan uncertainty band, the frequency of clinical DVHs within the band fell far below 68% for most OARs.
Conclusion:
The RMS-deviation method is an intuitive and testable means of quantifying DVH prediction accuracy. Provided uncertainty bands checked in this manner ensure that they comport with the expected meaning of a 1-s band. In this prostate cohort, RapidPlan RMS-uncertainty was consistently larger than the provided RapidPlan uncertainty, implying a systematic underestimate of the true modeling error.
Funding Support, Disclosures, and Conflict of Interest: K. Moore reports income for personal consulting and speaker honoraria from Varian Medical Systems. This work was supported in part by the Agency for Healthcare Research and Quality (AHRQ R01HS025440).
Keywords
Dose Volume Histograms, Treatment Planning, Quality Control
Taxonomy
Contact Email
