Click here to


Are you sure ?

Yes, do it No, cancel

Hierarchical Deep Reinforcement Learning for Intelligent Automatic Radiotherapy Treatment Planning

C Shen*, L Chen, Y Gonzalez, D Nguyen, S Jiang, X Jia, The University of Texas Southwestern Medical Ctr, Dallas, TX


(Sunday, 7/12/2020)   [Eastern Time (GMT-4)]

Room: AAPM ePoster Library

Purpose: a deep learning based virtual treatment planner (VTP) has been developed via deep reinforcement learning (DRL) to release human efforts in prostate cancer intensity-modulated radiotherapy (IMRT) treatment planning. Yet, the network size of VTP grows proportionally to the number of operations available in a treatment planning system (TPS), bringing computational and memory concerns, especially for more complicate cancer sites and treatment modalities. To address this issue, we developed a hierarchical DRL (HieDRL) framework that establishes a VTP with a hierarchical structure without growing with the number of available operations.

Methods: break down the problem of operating TPS into a sequence of sub-problems at organ level, parameter level, and adjustment level, respectively. Each sub-problem has a dedicated subnetwork (Organ-Net, Para-Net, or Adjust-Net) of the hierarchical VTP (HieVTP) to decide the operation at the corresponding level. Organ-Net determines which target or organ-at-risk (OAR) to be adjusted, while Para-Net further decides the specific parameter. Finally Adjust-Net resolves the detailed adjustment. The HieVTP is trained to learn the policy of operating a TPS for clinically acceptable plans via the end-to-end HieDRL. To demonstrate the feasibility of the proposed framework, we considered prostate cancer IMRT as a testbed.

Results: successfully trained a hierarchical VTP via the proposed HieDRL using 10 patient cases. It consistently improves plan quality for all 65 independent testing cases. The average plan score was improved from 4.44 to 8.62 with 9 being the highest score available in the scoring system.

Conclusion: this proof-of-principle study, we have demonstrated the feasibility of HieDRL in training HieVTP for intelligent automatic treatment planning. The network size of HieVTP does not grow with the number of available operations in a TPS. It has great potential in tackling more complicated cancer sites and treatment modalities.


Not Applicable / None Entered.


TH- External Beam- Photons: Treatment planning using machine learning/Knowledge Based Planning/automation

Contact Email