(Sunday, 7/12/2020) [Eastern Time (GMT-4)]
Room: AAPM ePoster Library
Purpose: has shown its promise for outcome prediction based on medical images. Although cross-validation (CV) is commonly used to evaluate the model performance on a single cohort, the validity of independent feature selection (FS) for each split training set has not been demonstrated. We perform the study to assess the implication of independent FS in the CV of radiomics studies.
Methods: H&N patients were collected (109 from a public database, 23 from our own institution). Their 5-year survival (72% and 78% respectively) was chosen to be the endpoint of this radiomics study. Eleven clinical features and 1130 radiomics features calculated using PyRadiomics were collected for each patient. An AUC performance matrix with 32 elements is generated, including the combinations of four feature selectors and eight machine learning classifiers for each scenario, with or without independent FS.
Results: public dataset was used in the single cohort scenario. With single global FS, the optimized model gives an average AUC of 0.864, which decreased to 0.666 with training set independent FS added to CV. The global FS best performance model AUC decreased from 0.930 to 0.675. In the two-cohort scenario, the public dataset was used for training, and the private dataset was used for testing. The global and independent FS CV of training set gives a test average AUC of 0.548 and 0.502, respectively. The global FS best-performed model AUC decreased from 0.707 to 0.672 accordingly.
Conclusion: study shows that without independent feature selection, the test result can be considerably inflated in a single-cohort radiomics study because of the hidden knowledge of the test split in the selected features. In the two-cohort study, no obvious decline was observed using the global FS in training. The single-cohort study using independent FS better predicts the model performance for cross-institution outcome prediction.
Keywords
Not Applicable / None Entered.
Taxonomy
IM- Dataset Analysis/Biomathematics: Machine learning
Contact Email
