Click here to


Are you sure ?

Yes, do it No, cancel

Extraction of Lung Tumor On KV Projection Images for Noninvasive Real Time Tumor Motion Tracking with the Transfer Learning Deep Segmentation Net Jury Committee Algorithm

M Chao1*, J Wei2 , Y Lo1 , K Rosenzweig1 , (1) The Mount Sinai Medical Center, New York, NY, (2) City College of New York, New York, NY


(Thursday, 7/18/2019) 10:00 AM - 12:00 PM

Room: 303

Purpose: The prerequisite of non-invasive lung tumor tracking on the sequential kilovoltage (kV) X-ray images is the reliable identification of the tumor target on these low contrast and noisy images. We aimed to develop a transfer learning deep segmentation net jury committee (TL-DSN-JC) algorithm to achieve this goal.

Methods: Instead of training millions of parameters from scratch, we initialized the net with well-known successful deep nets VGG-16/-19 that have been trained over the big ImageNet with ~14 million images, and then froze all but the final two layers, which was adaptively trained using available data. When training and cross-validating the net, a randomized partitioning was again applied to train the deep segmentation net. By independently training 12 different deep nets and combining them to form a committee we determined whether a pixel in a test location belongs to the target. The training dataset included only six hundred kV images acquired along the anterior-posterior or posterior-anterior direction. The performance of the algorithm was assessed with the manual identification using three performance measures: precision, recall, and the harmonic mean of them coined as F1 and with other similar algorithms such as the singular DSN without transfer learning (sDSN), DSN jury committee without transfer learning (DSN-JC), and singular DSN with transfer learning (sDSN-TL).

Results: The average precision, recall and F1 achieved by the TL-DSN-JC algorithm were found to be (0.95, 0.95, 0.85) for typical cases compared to the manual identification where the conventional methods failed to recognize the targets. The proposed algorithm outperformed sDSN by ~50%, DSN-JC by ~20%, and sDSN-TL by ~5%, respectively, indicating the superiority over other deep learning based methods.

Conclusion: The TL-DSN-JC algorithm showed the capacity to reliably extract the lung tumor on kV radiographic images and could be utilized for non-invasive tumor motion tracking on sequential projection images.


Image-guided Therapy, Target Localization, Image Processing


IM/TH- RT X-ray Imaging: combined KV/MV imaging

Contact Email