Click here to


Are you sure ?

Yes, do it No, cancel

Tracking Imaging Phenotypes with Deep Learning for Survival and Treatment Response Prediction

Y Xu1*, A Hosny2 , T Coroller2 , R Zeleznik2 , R Mak1,2 , H Aerts1,2 , (1) Harvard Medical School, Brigham and Women's Hosp., Boston, MA, (2) Dana Farber Cancer Institute, Boston, MA


(Thursday, 8/2/2018) 7:30 AM - 9:30 AM

Room: Davidson Ballroom B

Purpose: Radiomics has demonstrated the ability to predict clinical outcomes through quantitative image analysis and subsequent tumor phenotyping. Tumors are continuously evolving dynamic biological systems, and the phenotypes we are interested in can be captured by an image series. We propose that a deep learning analysis on follow-up CT scans of locally advanced non-small cell lung cancer (NSCLC) patients treated with radiation therapy, will lead to improved prediction of survival and progression outcomes.

Methods: CT images of 179 stage III non-surgical NSCLC patients treated with radiation therapy were analyzed. A total of 581 scans were analyzed, with an average of 3.2 (range 2-4) scans per patient, of pretreatment and follow-up scans. Transfer learning through the ResNet50 convolutional neural network (CNN) was used and merged with a recurrent neural network to incorporate follow-up CT scans. Survival and pathological response predictions of metastasis, progression, and locoregional recurrence were analyzed. The AUC (rank-sums) and differences between high and low risk groups (log-rank) were assessed.

Results: A CNN model based on the pretreatment scan only demonstrated low performance for predicting two-year survival (AUC=0.53, p=0.52). Enhanced performance was observed with the addition of each follow-up into the input (AUC=0.65, 0.70, 0.73, p<0.05). Comparable results were found for one-year overall survival, and the pathological response predictions mentioned in the methods. The difference between patients through Kaplan-Meier analysis for high and low risk groups of the predictions with two or three follow-up scans were significant (p<0.05).

Conclusion: This study demonstrates that combining transfer learning, though pre-trained neural networks, and recurrent neural networks with patient scans at multiple time points has the potential to improve clinical survival and response predictions. The addition of follow-up scans may provide additional phenotypic information to the neural network and allow for prediction compared to pretreatment scans alone.


Radiation Therapy, CT, Lung


TH- response assessment : CT imaging-based

Contact Email