Share Email Print

Proceedings Paper

Endoscopic-CT: learning-based photometric reconstruction for endoscopic sinus surgery
Author(s): A. Reiter; S. Leonard; A. Sinha; M. Ishii; R. H. Taylor; G. D. Hager
Format Member Price Non-Member Price
PDF $14.40 $18.00

Paper Abstract

In this work we present a method for dense reconstruction of anatomical structures using white light endoscopic imagery based on a learning process that estimates a mapping between light reflectance and surface geometry. Our method is unique in that few unrealistic assumptions are considered (i.e., we do not assume a Lambertian reflectance model nor do we assume a point light source) and we learn a model on a per-patient basis, thus increasing the accuracy and extensibility to different endoscopic sequences. The proposed method assumes accurate video-CT registration through a combination of Structure-from-Motion (SfM) and Trimmed-ICP, and then uses the registered 3D structure and motion to generate training data with which to learn a multivariate regression of observed pixel values to known 3D surface geometry. We demonstrate with a non-linear regression technique using a neural network towards estimating depth images and surface normal maps, resulting in high-resolution spatial 3D reconstructions to an average error of 0.53mm (on the low side, when anatomy matches the CT precisely) to 1.12mm (on the high side, when the presence of liquids causes scene geometry that is not present in the CT for evaluation). Our results are exhibited on patient data and validated with associated CT scans. In total, we processed 206 total endoscopic images from patient data, where each image yields approximately 1 million reconstructed 3D points per image.

Paper Details

Date Published: 21 March 2016
PDF: 6 pages
Proc. SPIE 9784, Medical Imaging 2016: Image Processing, 978418 (21 March 2016); doi: 10.1117/12.2216296
Show Author Affiliations
A. Reiter, Johns Hopkins Univ. (United States)
S. Leonard, Johns Hopkins Univ. (United States)
A. Sinha, Johns Hopkins Univ. (United States)
M. Ishii, Johns Hopkins Medical Institutions (United States)
R. H. Taylor, Johns Hopkins Univ. (United States)
G. D. Hager, Johns Hopkins Univ. (United States)

Published in SPIE Proceedings Vol. 9784:
Medical Imaging 2016: Image Processing
Martin A. Styner; Elsa D. Angelini, Editor(s)

© SPIE. Terms of Use
Back to Top