Click here to


Are you sure ?

Yes, do it No, cancel

Constructing a Gold Standard for ROC Analyses of a SABR Beam Detector

S Tattenberg1,2*, C Araujo1 , M Carlone1 , M Milette1 , D Hyde1 , K Parodi2 , (1) BC Cancer Agency, Kelowna, BC (2) Ludwig-Maximilians-university Muenchen, Garching b. Muenchen


(Sunday, 7/29/2018) 3:00 PM - 6:00 PM

Room: Exhibit Hall

Purpose: Receiver operating characteristic (ROC) curves are statistical tools that are largely independent of many institutionally-dependent factors. Recently, they have been applied to radiotherapy QA devices to better quantify the detectability of errors. Essential to this problem is the construction of the Gold Standard to which the investigated data sets are compared. This study examines the qualities of the ideal Gold Standard.

Methods: From a set of 29 brain stereotactic ablative radiotherapy (SABR) plans, 12 subsets of 29 plans each were created, with plans in every subset having been modified with different multileaf collimator (MLC) leaf position or gantry angle errors. All plans were delivered to Sun Nuclear’s ArcCHECK device. In Eclipse, a variety of calculations were run for every unmodified plan, using both the correct and suboptimal dosimetric leaf gap (DLG) values. The dependence of the area under the ROC curve (AUC) on the introduced error magnitudes was investigated.

Results: A Gold Standard based on a single consensus optimal DLG value was shown to be inadequate when investigating DLG and MLC leaf position errors. In these investigations, the AUC response curves for such a Gold Standard were found to exhibit unexpected, non-symmetrical behaviors. Optimizing the Gold Standard by instead using a variety of plan-specific optimal DLG values produced the expected AUC response curves. We suggest this indicates that using plan-specific values lifts the dependence of the errors that were introduced to the swept parameters for the creation of the ROC curves on the Gold Standard's DLG (model) parameters. Gantry angle error data, which is not linked to the DLG in this way, never exhibited such a behavior.

Conclusion: We have demonstrated that the desired Gold Standard for radiotherapy ROC analyses should be constructed by ensuring that the model parameters and the swept parameters are independent of one another.


Not Applicable / None Entered.


Not Applicable / None Entered.

Contact Email