MENU

Click here to

×

Are you sure ?

Yes, do it No, cancel
  1. Home
  2. Meeting Program
  3. All
  4. Ultrasound-Based Motion Tracking Using Attention-Aware Convolutional Neural Network and Convolutional Long Short-Term Memory Network

Ultrasound-Based Motion Tracking Using Attention-Aware Convolutional Neural Network and Convolutional Long Short-Term Memory Network

P Huang1*, G Yu1 , H Lu1 , D Liu1 , L Xing2 , Y Yin2 , N Kovalchuk3 , L Xing4 , D Li1 , (1) Shandong Normal University, Jinan, Shandong, (2) Shandong Cancer Hospital Affiliated to Shandong University, Jinan, Shandong, (3) Stanford University Cancer Center, Palo Alto, CA, (4) Stanford University School of Medicine, Stanford, CA

Presentations

(Sunday, 7/14/2019) 2:00 PM - 3:00 PM

Room: 303

Purpose: To develop a novel deep learning based real-time motion tracking strategy for ultrasound image-guided radiation therapy (RT).

Methods: 64 sets of 2D+time Ultrasound sequences provided in the MICCAI challenge on liver ultrasound tracking (CLUST 2015) are used in this study, the training size is 25, while the testing size is 39. For each case, the vessel center location was annotated by experts and served as ground truth (GT). The proposed approach involved an attention-aware Fully Convolutional Neural Network (FCNN) and a Convolutional Long Short-Term Memory network (CLSTM), and the architecture was end-to-end trainable. A glimpse module was built inside the attention-aware FCNN to focus on a region that contained the object of interest. FCNN enable extract discriminating spatial features of glimpse to facilitate temporal modeling for CLSTM. The saliency mask computed from CLSTM refined the features particular to the tracked landmarks. Moreover, the multi-task loss strategy including bounding box loss, localization loss, saliency loss, and adaptive loss weighting term was utilized to facilitate training convergence and avoid over/under-fitting.

Results: The average tracking error of 0.97 ± 0.52 mm and maximum tracking error of 1.94 mm were observed for 85 point-landmarks across 39 ultrasound cases compared to the GT. Furthermore, the tracking time per frame ranged from 66 and 101 frames per second, which is well below the frame acquisition time.

Conclusion: We proposed a deep learning-based approach for tracking respiration induced landmark motion using 2D ultrasound data with high accuracy and low runtime, despite of the existence of some known shortcomings of ultrasound imaging such as speckle noise. The tracking speed of the system was found to be remarkable, sufficiently fast for real-time applications in RT environment. The approach provides a valuable tool to guide RT treatment with beam gating or multi leaf collimator tracking in real time.

Funding Support, Disclosures, and Conflict of Interest: This work was funded by the National Natural Science Foundation of China (NO.61471226), Natural Science Foundation of Shandong Province (NO. JQ201516, 2018GGX101018), and the Taishan scholar project of Shandong Province (NO. tsqn20161023).

Keywords

Ultrasonics, Radiation Therapy

Taxonomy

IM- Ultrasound : Motion management

Contact Email

Additional Links