Purpose: Many groups are developing predictive models based on radiomics features. The purpose of this work was to investigate the agreement and variations in these features as computed by several software packages using standardized quantitative imaging feature definitions and common image datasets.
Methods: Seven sites from the NCIâ€™s Quantitative Imaging Network PET-CT working group participated in this project. First, nine commonly used quantitative imaging features were selected for comparison and included features that describe morphology, intensity, shape and texture. A standard lexicon developed by the International Biomarker Standardisation Initiative (IBSI) was adopted as the feature definition reference. Each site described their software packageâ€™s implementation for each feature, based on the lexicon. Three 3D Digital Reference Objects (DROs) were developed specifically for this effort: a uniform sphere, a sphere with intensity variations, and a complex shape object with uniform intensity. To eliminate variation in radiomics features caused by segmentation differences, each DRO was accompanied by a Volume of Interest (VOI), from which the features were calculated. Each participating site reported the computed feature values for each DRO. The percent coefficient of variation (CV) was calculated across software packages for each feature on each phantom.
Results: Eight sets of results were submitted. Six of the nine features demonstrated excellent agreement across submissions with CV < 1%, including: Approximate Volume, 2D diameter, 3D diameter, Mean Intensity, Standard Deviation and Kurtosis (with Fisher adjustment). Larger variations (CV 15% and higher) were observed for Surface Area, Sphericity and GLCM Entropy Texture features, which are being investigated further.
Conclusion: By computing a subset of nine common radiomics features using a variety of software packages on DROs, we have shown that while several features agree strongly, others do not. This highlights the need for standardization in feature definitions and proof of equivalence of computational methods.
Funding Support, Disclosures, and Conflict of Interest: Supported by the National Cancer Institute's Quantitative Imaging Network (QIN) through the following grants: U01CA181156; U01CA187947; U01CA154601; U01CA143062; U01CA225427; U01CA225431; U01CA148131; U01CA179106