New approaches to bladder-surveillance endoscopy

Bladder cancer is the fifth most common cancer in the United States¹ and has a 50% recurrence rate. Consequently, patients undergo frequent surveillance, where a flexible endoscope is inserted into the bladder to detect recurrent tumors. The exam can be uncomfortable, in part because of the large (5mm) devices currently employed. While use of smaller endoscopes is desirable, they suffer from reduced resolution and field of view, making examination and detection challenging. Additionally, bladder surveillance constitutes a significant percentage of urologists' time and resources, and is costly. While avenues for improved detection of bladder cancer—such as biomarkers,² fluorescence imaging,³ and narrow-band imaging⁴—are areas of current investigation, conventional endoscopy remains the gold standard. Limitations of current devices have spurred development of mosaicking systems that generate panoramic views of the bladder. Constructed from multiple overlapping images, mosaics provide expanded views and greater visual context for in situ detection and assessment of mucosal changes associated with carcinoma. However, the resulting panoramas are limited to localized regions of the bladder and are unable to generate full, sweeping 360^° views. Here, we report our developments toward automated surveillance that uses novel endoscopic technology and image-analysis software to reconstruct full 3D panoramas of the bladder.⁵

We developed an ultrathin scanning-fiber endoscope (SFE), whose small diameter (1.5mm) and superior imaging capabilities make it ideal for endoscopic surveillance (see Figure 1).⁶ In addition to mitigating patient discomfort, we have configured the SFE with an automated tip-bending system that allows machine-controlled surveillance endoscopy.^7,8 By employing a spiral scan trajectory, we can image the entire internal surface (see Figure 2). This operationally simple system could potentially be performed by a nurse or technician, thus freeing up the urologist's time. In conjunction with these hardware advances, we developed post-processing software capable of converting endoscopic video into digital 3D models. By stitching endoscopic images onto a spherical surface, our models permit expedient review and interpretation of findings by the urologist.

Figure 1. Scanning-fiber endoscope image probe.

Figure 2. Spiral scan trajectory used to image the entire internal bladder surface.⁵

To test the reconstruction software, we performed endoscopy on a spherical bladder phantom (see Figure 3) using a conventional endoscope (EB-1970K, Hoya-Pentax, Tokyo, Japan) and an excised pig bladder (see Figure 4) using the SFE. Both the phantom and bladder were scanned on a rotating stage. We subsequently processed the recordings from the SFE using our reconstruction software.

Figure 3. (left) A bladder phantom, constructed from a glass light bulb, with painted vessels.⁵ (right) Endoscopic images acquired from a conventional endoscope.

Figure 4. (left) An excised pig bladder fixed in formalin. Before fixing, red and blue ink were injected into separate arteries feeding the hemispheres to maintain vessel contrast. (right) Endoscopic images acquired using the scanning-fiber endoscope.

Conventional planar mosaicking algorithms break down as the surface geometry of the bladder becomes increasingly nonplanar.^8,9 As an alternative, our software computes image alignment by mutual reconstruction of both endoscopic motion and bladder shape.⁵ This method—known in computer vision as ‘structure from motion’—has many uses in reconstructing complex scenes, such as virtual tourism¹⁰ and software that explores—in 3D—collections of overlapping photographs.¹¹We detected a set of consistent feature points from the bladder videos and matched pairs of overlapping video frames. We then performed a nonlinear least-squares-optimization method known as ‘bundle adjustment’ to yield the 3D point locations, camera positions and poses, and intrinsic camera properties needed for the reconstruction. Our software successfully generated 3D spherical mosaics comprised of several hundred images and several thousand 3D feature points with single-pixel accuracy (see Figure 5). Additionally, we recovered intrinsic endoscope parameters such as lens distortion, thereby obviating any initial calibration. To the best of our knowledge, this is the first time that these techniques have been extended to endoscopic bladder images.

Figure 5. Stitched 3D reconstructions from scanning-fiber endoscopic images of the (left) phantom⁵and (right) excised pig bladder.

In summary, our SFE and reconstruction software present new avenues for improved bladder-cancer surveillance. Our approach benefits from expedient urological examination, improved cancer detection, and avenues for longitudinal assessment. Currently, we are developing an automated scanning mechanism. A limitation of an automated approach is the requirement of sufficient image overlap to identify a consistent set of features. However, the reconstructed model in turn validates that the entire bladder has been scanned. Similarly, the reconstruction can be referenced during follow-up examinations to evaluate disease progression and/or response to therapy. While the 3D model promotes more immersive interaction with image data, the stitching algorithm could be improved to better blend individual frames. Our methodology may be extended to minimally invasive examination of other hollow organs (such as the stomach) and computer-aided diagnosis.