Click here to


Are you sure ?

Yes, do it No, cancel

Multimodal Biomarkers for Cancer Treatment Outcome Prediction by Use of Deep Learning and Canonical Correlation Analysis

M Saad1*, S He2, W Thorstad2, H Gay2, X Wu2, Y Zhao3, S Ruan4, X Wang2, H Li1,3, (1) University Of Illinois at Urbana-Champaign, Urbana, IL, USA (2) Washington University in St. Louis, St. Louis, MO, USA, (3) Carle Cancer Center, Carle Foundation Hospital,Urbana, IL, USA (4) University of Rouen, French


(Wednesday, 7/15/2020) 3:30 PM - 4:30 PM [Eastern Time (GMT-4)]

Room: Track 2

Purpose: Predicting early in treatment whether a tumor is likely to respond to treatment is a difficult yet important tasks in providing personalized cancer care. Studies have shown that multimodal biomarkers provide complementary information for treatment outcome prediction. However, current efforts to utilize multi-modal biomarkers for outcome prediction have been compromised by data heterogeneity and incomplete patient cohorts in each modal. There also lacks an efficient platform for the validation and comparison of different prediction models. A novel modularized platform is designed to effectively and seamlessly identify and combine predictive information carried by multimodal biomarkers for outcome prediction, and facility the validation and comparison of different models.perf

Methods: The platform contains three modules of deep feature extraction, multi-modal feature fusion, and classification. Informative single-modal deep features are independently extracted based on deep neural networks. A new canonical correlation analysis (CCA)-based feature fusion method is proposed to reduce the heterogeneity and redundancy of multi-modal biomarkers while increasing their homogeneity and correlations to treatment outcome, and ultimately improve the performance of the classifier.

Results: PET images and microRNA genomic features acquired from 305 oropharyngeal cancer patients were employed for a preliminary study. Three neural networks and two classifiers were implemented within the platform to 1) investigate its flexibility of integrating various feature extraction and classification methods and 2) evaluate the performance of difference methods. The results demonstrate that combining multi-modal biomarkers improves the prediction performance of all tested classifiers compared to that with uni-modal data. Given the deep-learning features as input, a neural network-based classifier yields higher performance than conventional classifiers. CNN with shallow layers yield better performance due to limited training samples.

Conclusion: The platform can effectively and seamlessly identify and combine predictive information carried by multimodal biomarkers for outcome prediction. It also facilities the validation and comparison of different models.

Funding Support, Disclosures, and Conflict of Interest: This research was supported in part by NIH awards R01CA233873 and R21CA223799.


Not Applicable / None Entered.


Not Applicable / None Entered.

Contact Email