Click here to


Are you sure ?

Yes, do it No, cancel

Integrating Computer Vision and Non-Linear Optimization for Automated Deformable Registration of Three-Dimensional Medical Images

KT Huang*, Pymedix, Inc., Northfield, IL


(Thursday, 7/18/2019) 7:30 AM - 9:30 AM

Room: 225BCD

Purpose: Deformable image registration (DIR), is a challenging task with many important clinical applications. Commonly used DIR techniques chiefly rely on non-linear optimization. Various semi-dense feature-based methods drawing inspiration from mammalian systems as the basis for 2D visual processing have been implemented for automated wide baseline registration and object detection applications with great success. Extension to the 3D case could be adapted, in theory, to allow precise semi-dense global optimization.

Methods: 3D image features were detected, oriented, and described using modified biologically-inspired 2D computer vision algorithms extended to the 3D case. Features were matched using Euclidean distance thresholded by first-to-second best distance ratio, followed by topological analysis. This algorithm was hybridized with a second stage normalized correlation-based non-linear optimizer, and benchmarked with publically available 4D computed tomography (CT) datasets against published results for commercial packages. Further characterization was performed with actual clinical 3D CT images containing truncated data and very large deformations. A method for estimating registration error in the absence of ground truth was derived.

Results: The feature-based algorithm reliably and repeatably detected distinct semi-dense 3D image features with sub-voxel localization accuracy and with high matching rates. The hybrid algorithm significantly improved landmark displacement error on the benchmark cases (mean improvement 0.91 mm, range -0.13 to 4.78, p < 0.00001). Difficult clinical cases showed significant qualitative improvements.

Conclusion: The hybrid algorithm, dubbed Constrained Robust Affine Feature Transform (CRAFT), incorporates paradigms from various computer-vision techniques to combine aspects of the human visual pathway with well-characterized non-linear optimization methods to automate general deformable registration with unprecedented robustness. This hybrid technique is able to estimate registration confidence and can serve as the basis of machine perception of 3D medical images for machine learning.

Funding Support, Disclosures, and Conflict of Interest: This work was performed at the University of Cincinnati. The author is a founder of Pymedix, Inc and serves as Chief Technology Officer.


Deformation, Image Analysis, Registration


IM/TH- Image registration : Multi-modality registration

Contact Email