Newport - Supporting SPIE scholarships, grants and outreach for over 10 years Get updates from SPIE Newsroom

Biomedical Optics & Medical Imaging

New approaches to bladder-surveillance endoscopy

An ultrathin scanning-fiber endoscope uses 3D digital reconstruction of endoscopic video for automated analysis.
7 April 2011, SPIE Newsroom. DOI: 10.1117/2.1201103.003578

Bladder cancer is the fifth most common cancer in the United States1 and has a 50% recurrence rate. Consequently, patients undergo frequent surveillance, where a flexible endoscope is inserted into the bladder to detect recurrent tumors. The exam can be uncomfortable, in part because of the large (5mm) devices currently employed. While use of smaller endoscopes is desirable, they suffer from reduced resolution and field of view, making examination and detection challenging. Additionally, bladder surveillance constitutes a significant percentage of urologists' time and resources, and is costly. While avenues for improved detection of bladder cancer—such as biomarkers,2 fluorescence imaging,3 and narrow-band imaging4—are areas of current investigation, conventional endoscopy remains the gold standard. Limitations of current devices have spurred development of mosaicking systems that generate panoramic views of the bladder. Constructed from multiple overlapping images, mosaics provide expanded views and greater visual context for in situ detection and assessment of mucosal changes associated with carcinoma. However, the resulting panoramas are limited to localized regions of the bladder and are unable to generate full, sweeping 360° views. Here, we report our developments toward automated surveillance that uses novel endoscopic technology and image-analysis software to reconstruct full 3D panoramas of the bladder.5

We developed an ultrathin scanning-fiber endoscope (SFE), whose small diameter (1.5mm) and superior imaging capabilities make it ideal for endoscopic surveillance (see Figure 1).6 In addition to mitigating patient discomfort, we have configured the SFE with an automated tip-bending system that allows machine-controlled surveillance endoscopy.7,8 By employing a spiral scan trajectory, we can image the entire internal surface (see Figure 2). This operationally simple system could potentially be performed by a nurse or technician, thus freeing up the urologist's time. In conjunction with these hardware advances, we developed post-processing software capable of converting endoscopic video into digital 3D models. By stitching endoscopic images onto a spherical surface, our models permit expedient review and interpretation of findings by the urologist.

Figure 1. Scanning-fiber endoscope image probe.

Figure 2. Spiral scan trajectory used to image the entire internal bladder surface.5

To test the reconstruction software, we performed endoscopy on a spherical bladder phantom (see Figure 3) using a conventional endoscope (EB-1970K, Hoya-Pentax, Tokyo, Japan) and an excised pig bladder (see Figure 4) using the SFE. Both the phantom and bladder were scanned on a rotating stage. We subsequently processed the recordings from the SFE using our reconstruction software.

Figure 3. (left) A bladder phantom, constructed from a glass light bulb, with painted vessels.5 (right) Endoscopic images acquired from a conventional endoscope.

Figure 4. (left) An excised pig bladder fixed in formalin. Before fixing, red and blue ink were injected into separate arteries feeding the hemispheres to maintain vessel contrast. (right) Endoscopic images acquired using the scanning-fiber endoscope.

Conventional planar mosaicking algorithms break down as the surface geometry of the bladder becomes increasingly nonplanar.8,9 As an alternative, our software computes image alignment by mutual reconstruction of both endoscopic motion and bladder shape.5 This method—known in computer vision as ‘structure from motion’—has many uses in reconstructing complex scenes, such as virtual tourism10 and software that explores—in 3D—collections of overlapping photographs.11We detected a set of consistent feature points from the bladder videos and matched pairs of overlapping video frames. We then performed a nonlinear least-squares-optimization method known as ‘bundle adjustment’ to yield the 3D point locations, camera positions and poses, and intrinsic camera properties needed for the reconstruction. Our software successfully generated 3D spherical mosaics comprised of several hundred images and several thousand 3D feature points with single-pixel accuracy (see Figure 5). Additionally, we recovered intrinsic endoscope parameters such as lens distortion, thereby obviating any initial calibration. To the best of our knowledge, this is the first time that these techniques have been extended to endoscopic bladder images.

Figure 5. Stitched 3D reconstructions from scanning-fiber endoscopic images of the (left) phantom5and (right) excised pig bladder.

In summary, our SFE and reconstruction software present new avenues for improved bladder-cancer surveillance. Our approach benefits from expedient urological examination, improved cancer detection, and avenues for longitudinal assessment. Currently, we are developing an automated scanning mechanism. A limitation of an automated approach is the requirement of sufficient image overlap to identify a consistent set of features. However, the reconstructed model in turn validates that the entire bladder has been scanned. Similarly, the reconstruction can be referenced during follow-up examinations to evaluate disease progression and/or response to therapy. While the 3D model promotes more immersive interaction with image data, the stitching algorithm could be improved to better blend individual frames. Our methodology may be extended to minimally invasive examination of other hollow organs (such as the stomach) and computer-aided diagnosis.

Timothy Soper, Eric Seibel
Department of Mechanical Engineering
University of Washington
Seattle, WA 

Timothy Soper received his bachelor's degree in bioengineering from the University of California, San Diego, and his PhD in bioengineering from the University of Washington. Currently, he develops image-guided procedures using endoscopy in the Human Photonics Laboratory.

Eric Seibel received BS and MS degrees in mechanical engineering from Cornell University and the University of California at Berkeley, respectively. After working in the ophthalmic-device industry, he designed and developed laser-scanning microscopes for live-tissue imaging for his PhD at the University of Washington (1996). He is currently a research associate professor, adjunct in bioengineering and electrical engineering, and director of the Human Photonics Laboratory. His multidisciplinary research program develops and translates to industry novel technologies for optical scanning for image acquisition and clinical visualization, with specific focus on early detection, diagnosis, and treatment of cancer and pre-cancer.

Michael Porter
Department of Urology
University of Washington
Seattle, WA 

Michael Porter is an assistant professor of urology and adjunct professor of epidemiology. He is also the assistant chief of urology at the Veterans Affairs Puget Sound Health System. His clinical expertise is in genitourinary oncology, with focus on bladder cancer. His research interests include health-services research, bladder-cancer screening, and outcomes of urologic oncology surgery.

1. A. Jemal, R. Siegel, J. Xu, E. Ward, Cancer statistics, 2010, CA Cancer J. Clin. 60, pp. 277-300, 2010.
2. O. P. Vrooman, J. A. Witjes, Molecular markers for detection, surveillance and prognostication of bladder cancer, Int'l J. Urol. 16, pp. 234-243, 2009. doi:10.1111/j.1442-2042.2008.02225.x
3. P. E. Spiess, H. B. Grossman, Fluorescence cystoscopy: is it ready for use in routine clinical practice?, Curr. Opin. Urol. 16, pp. 372-376, 2006. doi:10.1097/01.mou.0000240312.16324.9a
4. H. W. Herr, S. M. Donat, A comparison of white-light cystoscopy and narrow-band imaging cystoscopy to detect bladder tumour recurrences, Brit. J. Urol. Int'l 102, pp. 1111-1114, 2008. doi:10.1111/j.1464-410X.2008.07846.x
5. T. D. Soper, J. E. Chandler, M. P. Porter, E. J. Seibel, Constructing spherical panoramas of a bladder phantom from endoscopic video using bundle adjustment, Proc. SPIE 7964, In press. doi:10.1117/12.878299
6. C. M. Lee, C. J. Engelbrecht, T. D. Soper, F. Helmchen, E. J. Seibel, Scanning fiber endoscopy with highly flexible, 1 mm catheterscopes for wide-field, full-color imaging, J. Biophoton. 3, pp. 385-407, 2010. doi:10.1002/jbio.200900087
7. W. J. Yoon, P. Sangtae, P. G. Reinhall, E. J. Seibel, Development of an automated steering mechanism for bladder urothelium surveillance, J. Med. Devices 3, pp. 011004, 2009. doi:10.1115/1.3054381
8. W. J. Yoon, M. A. Brown, E. J. Seibel, Automated cystoscopic surveillance system with automated endoscopic image mosaicing, 12th Mechatron. Forum Bienn. Int'l Conf., 2010. Conf. presentation.
9. A. Behrens, T. Stehle, S. Gross, T. Aach, Local and global panoramic imaging for fluorescence bladder endoscopy, Proc. 31st Annu. Int'l Conf. IEEE Eng. Med. Biol. Soc.: Engineering the Future of Biomedicine, pp. 6990-6993, 2009.
10. N. Snavely, S. M. Seitz, R. Szeliski, Photo tourism: exploring photo collections in 3D, ACM Trans. Graph. 25, pp. 835-846, 2006. doi:10.1145/1141911.1141964
11. Microsoft Photosynth software. Accessed 5 March 2011.