Click here to


Are you sure ?

Yes, do it No, cancel

Multi-Organ Segmentation Through Surrogate Labels and Classification of Intermediate Network Representations

D Huff1*, A Weisman1 , T Bradshaw1 , R Jeraj1,2 , (1) University of Wisconsin-Madison, Madison, Wisconsin, (2) University of Ljubljana, Ljubljana, Slovenia


(Tuesday, 7/16/2019) 3:45 PM - 4:15 PM

Room: Exhibit Hall | Forum 5

Purpose: Automatic multi-organ CT segmentation is a challenging but important task with applications in both radiotherapy and medical image analysis. Deep learning-based approaches typically rely on many physician-contoured example images, which require enormous effort to acquire. This study developed a novel, semi-supervised method requiring far fewer labels and assessed its performance against a standard fully-supervised method.

Methods: Publicly available CT scans of lymphadenopathy patients (N=176) were cropped from vertex to mid-thigh and resampled to a 2mm grid. Liver, spleen, kidneys, lungs, and heart were manually contoured on 10 scans (5 validation, 5 test). Instead of training a network to perform segmentation directly, a 3D CNN was trained to predict the x-, y-, and z- voxel coordinates of image patches on the full dataset excepting test patients. This forces the network to learn location- and organ-specific features to correctly assign voxel coordinates. These learned features were extracted from the final fully-connected layer, and K-nearest-neighbor classifiers were trained on the 5 labelled validation scan features to predict organ membership. The method was then applied to the labeled test patient scans, and segmentation performance was assessed with Dice coefficients.

Results: Dice coefficients between manual and predicted test patient organ contours for the semi-supervised method were (mean±sd): liver:0.86±0.04, spleen:0.75±0.05, kidneys:0.59±0.11, heart:0.87±0.01, and lungs:0.86±0.01. A fully-supervised instance of the same CNN trained to perform segmentation directly on the same amount of labelled data (N=5) had significantly worse performance in liver (0.76±0.08), spleen (0.14±0.16), kidneys (0.00±0.00), and heart (0.61±0.21) (paired t-test, p<0.05). No significant performance difference was found in lungs (0.57±0.51).

Conclusion: By training with surrogate labels and classifying in feature space, we achieved impressive segmentation results superior to a fully-supervised method when using few labelled datasets. Overall, our approach leverages the power of CNN-based segmentation yet requires only a fraction of the labelled datasets.

Funding Support, Disclosures, and Conflict of Interest: This work is supported by the University of Wisconsin Carbone Cancer Center Support Grant P30 CA014520. Robert Jeraj is a co-founder of AIQ Solutions.




IM/TH- image segmentation: CT

Contact Email