Purpose: Automatic segmentation with deep learning is very useful for radiotherapy because it can reduce observer variabilities and save time. Nevertheless, physicians must spend a considerable amount of time examining the automatically generated contours slice-by-slice, which reduces the benefit of the automatic tool greatly. To circumvent this shortcoming, we developed an automatic quality-assurance method for automatic segmentation without the ground truth using convolutional neural networks (CNN).
Methods: Data were for 680 patients who underwent breast-conserving therapy. The clinical target volume (CTV) generated by experts and the automatic-segmentation model were used for analyses. We used the Dice as the index of segmentation quality, and divided the range into three levels: [0.95, 1] for â€œgoodâ€?, [0.8, 0.95) for â€œmediumâ€?, and [0, 0.8) for â€œbadâ€? quality. The quality-assurance framework was based on ResNet-101, with maps of computed tomography(CT), segmentation probability and uncertainty as inputs and the quality level as output. We selected 520 cases randomly as the training set, 80 cases as the validation set, and the remaining 80 cases as the test set. The performance of the proposed method was evaluated with quantitative metrics: balanced accuracy, sensitivity, specificity, F-score, and the area under the receiving operator characteristic curve (AUC).
Results: The proposed method achieved promising quality-prediction results for good, medium and bad quality-level prediction, respectively: balanced accuracy of 0.97, 0.94 and 0.89; sensitivity of 0.96, 0.94 and 0.80; specificity of 0.99, 0.95 and 0.98; F-score of 0.98, 0.91 and 0.81; AUC of 0.96, 0.93 and 0.88.The prediction time was about 2 s per patient.
Conclusion: Our method could predict the segmentation quality automatically without the ground truth. It could provide useful information for physicians to further verify and revise the automatic contours. Integration of our method into current automatic segmentation pipelines could improve the efficiency of radiotherapy contouring.
Funding Support, Disclosures, and Conflict of Interest: This work was supported by the National Natural Science Foundation of China (NO.11605291 and NO.11875320). The authors report no conflicts of interest with this study.