Dynamic volumetric medical imaging (4DMI) has reduced motion artifacts, increased early diagnosis of small mobile tumors, and improved target definition for treatment planning. High speed cameras for video, X-ray, or other forms of sequential imaging allow a live tracking of external or internal movement useful for real-time image-guided radiation therapy (IGRT). However, none of 4DMI can track real-time organ motion and no camera has correlated with 4DMI to show volumetric changes. With a brief review of various IGRT techniques, we propose a fast 3D camera for live-video stereovision, an automatic surface-motion identifier to classify body or respiratory motion, a mechanical model for synchronizing the external surface movement with the internal target displacement by combination use of the real-time stereovision and pre-treatment 4DMI, and dynamic multi-leaf collimation for adaptive aiming the moving target. Our preliminary results demonstrate that the technique is feasible and efficient in IGRT of mobile targets. A clinical trial has been initiated for validation of its spatial and temporal accuracies and dosimetric impact for intensity-modulated RT (IMRT), volumetric-modulated arc therapy (VMAT), and stereotactic body radiotherapy (SBRT) of any mobile tumors. The technique can be extended for surface-guided stereotactic needle insertion in biopsy of small lung nodules.

Recent development in optical 3D surface imaging technologies provide better ways to digitalize the 3D surface and its motion in real-time. The non-invasive 3D surface imaging approach has great potential for many medical imaging applications, such as motion monitoring of radiotherapy, pre/post evaluation of plastic surgery and dermatology, to name a few. Various commercial 3D surface imaging systems have appeared on the market with different dimension, speed and accuracy. For clinical applications, the accuracy, reproducibility and robustness across the widely heterogeneous skin color, tone, texture, shape properties, and ambient lighting is very crucial. Till now, a systematic approach for evaluating the performance of different 3D surface imaging systems still yet exist. In this paper, we present a systematic performance assessment approach to 3D surface imaging system assessment for medical applications. We use this assessment approach to exam a new real-time surface imaging system we developed, dubbed "Neo3D Camera", for image-guided radiotherapy (IGRT). The assessments include accuracy, field of view, coverage, repeatability, speed and sensitivity to environment, texture and color.

Rapid optical three-dimensional (O3D) imaging systems provide accurate digitized 3D surface data in real-time, with no patient contact nor radiation. The accurate 3D surface images offer crucial information in image-guided radiation therapy (IGRT) treatments for accurate patient repositioning and respiration management. However, applications of O3D imaging techniques to image-guided radiotherapy have been clinically challenged by body deformation, pathological and anatomical variations among individual patients, extremely high dimensionality of the 3D surface data, and irregular respiration motion. In existing clinical radiation therapy (RT) procedures target displacements are caused by (1) inter-fractional anatomy changes due to weight, swell, food/water intake; (2) intra-fractional variations from anatomy changes within any treatment session due to voluntary/involuntary physiologic processes (e.g. respiration, muscle relaxation); (3) patient setup misalignment in daily reposition due to user errors; and (4) changes of marker or positioning device, etc. Presently, viable solution is lacking for in-vivo tracking of target motion and anatomy changes during the beam-on time without exposing patient with additional ionized radiation or high magnet field. Current O3D-guided radiotherapy systems relay on selected points or areas in the 3D surface to track surface motion. The configuration of the marks or areas may change with time that makes it inconsistent in quantifying and interpreting the respiration patterns. To meet the challenge of performing real-time respiration tracking using O3D imaging technology in IGRT, we propose a new approach to automatic respiration motion analysis based on linear dimensionality reduction technique based on PCA (principle component analysis). Optical 3D image sequence is decomposed with principle component analysis into a limited number of independent (orthogonal) motion patterns (a low dimension eigen-space span by eigen-vectors). New images can be accurately represented as weighted summation of those eigen-vectors, which can be easily discriminated with a trained classifier. We developed algorithms, software and integrated with an O3D imaging system to perform the respiration tracking automatically. The resulting respiration tracking system requires no human intervene during it tracking operation. Experimental results show that our approach to respiration tracking is more accurate and robust than the methods using manual selected markers, even in the presence of incomplete imaging data.

The classification of anatomical features using hyperspectral imaging has been a common goal in biomedical hyperspectral imaging. Identification and location of the common bile duct is critical in cholecystectomies, one of the most common surgical procedures. In this study, surgical images where the common bile duct is visible to the surgeon during open surgeries of patients with normal bile ducts were acquired. The effect of the spectral distribution of simulated light sources on the scene color are explored with the objective of providing the optimum spectral light distribution that can enhance contrast between the common bile duct and surrounding tissue through luminance and color differences.

The selection of well-vascularized tissue during DIEP flap harvest remains controversial. While several studies have elucidated cross-midline perfusion, further characterization of perfusion to the ipsilateral hemiabdomen is necessary for minimizing rates of fat necrosis or partial fat necrosis in bilateral DIEP flaps. Eighteen patients (29 flaps) underwent DIEP flap harvest using a prospectively designed protocol. Perforators were marked and imaged with a novel system for quantitatively measuring tissue oxygenation, the Digital Light Hyperspectral Imager. Images were then analyzed to determine if perforator selection influenced ipsilateral flap perfusion. Flaps based on a single lateral row perforator (SLRP) were found to have a higher level of hemoglobin oxygenation in Zone I (mean %HbO₂ = 76.1) compared to single medial row perforator (SMRP) flaps (%HbO₂ = 71.6). Perfusion of Zone III relative to Zone I was similar between SLRP and SMRP flaps (97.4% vs. 97.9%, respectively). These differences were not statistically significant (p>0.05). Perfusion to the lateral edge of the flap was slightly greater for SLRP flaps compared SMRP flaps (92.1% vs. 89.5%, respectively). SMRP flaps had superior perfusion travelling inferiorly compared to SLRP flaps (88.8% vs. 83.9%, respectively). Overall, it was observed that flaps were better perfused in the lateral direction than inferiorly. Significant differences in perfusion gradients directed inferiorly or laterally were observed, and perforator selection influenced perfusion in the most distal or inferior aspects of the flap. This suggests broader clinical implications for flap design that merit further investigation.

Over the course of the last several years hyperspectral imaging (HSI) has seen increased usage in biomedicine. Within the medical field in particular HSI has been recognized as having the potential to make an immediate impact by reducing the risks and complications associated with laparotomies (surgical procedures involving large incisions into the abdominal wall) and related procedures. There are several ongoing studies focused on such applications. Hyperspectral images were acquired during pancreatoduodenectomies (commonly referred to as Whipple procedures), a surgical procedure done to remove cancerous tumors involving the pancreas and gallbladder. As a result of the complexity of the local anatomy, identifying where the common bile duct (CBD) is can be difficult, resulting in comparatively high incidents of injury to the CBD and associated complications. It is here that HSI has the potential to help reduce the risk of such events from happening. Because the bile contained within the CBD exhibits a unique spectral signature, we are able to utilize HSI segmentation algorithms to help in identifying where the CBD is. In the work presented here we discuss approaches to this segmentation problem and present the results.

In this study, we investigate fluorescence optical detection and image reconstruction based on modulated light illumination using a digital micromirror device (DMD). Fluorescence detection is one of the most common methods to study various cellular dynamics labeled with fluorescent indicators. Although employed in many cell-based assays, widefield fluorescence microscopy provides poor axial sectioning capability that is insufficient to measure thick cell-based assays, e.g., those with 3D cell complex cultured in a thick extracellular matrix. Confocal fluorescence microscopy, on the other hand, provides good axial sectioning capability compared to wide-field fluorescence imaging. However, confocal microscopy is subject to temporal overhead associated with scanning to acquire fluorescence images. For image acquisition at enhanced axial sectioning with reduced processing load, we have developed a fluorescence optical detection system based on subtractive light illumination using a DMD. Compared to moving grid masks, a DMD provides fast and flexible scanning by modulating aperture patterns. Here, we report DMD-based structured light illumination for improved image reconstruction by separating in-focus and out-of-focus fluorescence components so that the system can achieve 3D fluorescence image stacks with enhanced axial sectioning capability. In this proof-of-concept study, the DMD-based structured light illumination system was evaluated by observing two-dimensionally deposited fluorescent microbeads and mixed pollen grains. Also shown was that the scanning time to obtain fluorescence images can be greatly reduced compared to confocal microscopy.

In this paper, we present a novel approach to retrieve attenuation corrected fluorescence (ACF) in the image field. This approach can be applied to improve tumor identification for both diagnosis and treatment purpose. Furthermore, this approach will facilitate the development of fluorescence image-guided surgery.

The use of the Texas Instruments DLP^® LightCrafter^TM as a compact module in lithography-based additive manufacturing technologies (AMT) is discussed in this paper. For this purpose the light engine is placed underneath a transparent vat which is coated with a PTFE-film and filled with photosensitive resin. By loading an appropriate bitmap into the light engine, the resin can be exposed selectively to obtain a photopolymerized layer. To integrate the device into the building process, a configurable I/O trigger is required since the loaded bitmap should be exposed only in a certain period (exposure time). By stacking up the individual layers with a typical layer thickness between 25 and 50μm, a three-dimensional part is built up. The current setup of the used digital LEDs in combination with a customized optical projection system ensures a spatial and temporal homogeneity of the intensity at the build platform, which is significantly better than with traditionally used engines. It could be shown that this system can fabricate threedimensional parts with a resolution < 40μm in x-y plane and 15μm in z-axis. Additionally, mechanical properties (e.g. bending strength) were measured and potential anisotropies, which might be caused by the layered manufacturing process, were assessed. Ceramic-filled polymers were also used and the necessary post-processing steps like the removal of the polymer phase after structuring and the final sintering step to obtain fully dense ceramic parts are discussed.

Scientists conceiving future space missions are interested in using DMDs as a multi-object spectrometer (MOS) slit mask. The main uncertainties in utilizing DMDs in a space-based instrument are associated with their operational longevity given the exposure to high levels of proton radiation and their ability to operate at low temperatures. Since a favored orbit is at the second Lagrangian point (L2), it is important to determine how long such Micro-Electrical Mechanical Systems (MEMS) would remain operational in the harsh L2 radiation environment, which primarily consists of solar protons and cosmic rays. To address this uncertainty, we have conducted DMD proton testing at the Lawrence Berkeley National Laboratory (LBNL) 88” Cyclotron. Three DMDs were irradiated with high-energy protons (20- 50MeV) with energies sufficient to penetrate the DMD package’s optical window and interact electrically with the device. After each irradiation step, an optical test procedure was used to validate the operability of each individual mirror on the DMD array. Each DMD was irradiated to a wide range of dosage levels and remained 100% operable up to a total dose of 30 krads. In addition, a few single event upsets were seen during each irradiation dose increment. To determine the minimal operating temperature of the DMDs, we placed a DMD in a liquid nitrogen dewar, and cooled it from room temperature to 130 K. During this test, the DMD was illuminated with a light source and monitored with a CCD camera. Additionally, the temperature was held constant at 173 K for 24 hours to test landing DMD patterns for long periods of time. There was no indication that extended periods of low temperature operation impact the DMD performance. Both of these results point to DMDs as a suitable candidate for future long duration space missions.

Rapidly programmable micro-mirror arrays, such as the DLP® digital micro-mirror device (DMD), have opened an exciting new arena in spectral imaging: rapidly reprogrammable, high spectral resolution, multi-band spectral filters that perform spectral processing directly in the optical hardware. Such a device is created by placing a DMD at the spectral plane of an imaging spectrometer, and using it as a spectral selector that passes some wavelengths down the optical train to the final image and rejects others. While simple in concept, realizing a truly practical DMD-based spectral filter has proved challenging. Versions described to date have been limited by the intertwining of image position and spectral propagation direction common to most imaging spectrometers, reducing these instruments to line-by-line scanning imagers rather than true spectral cameras that collect entire two-dimensional images at once. Here we report several optical innovations that overcome this limitation and allow us to construct full-frame programmable filters that spectrally manipulate every pixel, simultaneously and without spectral shifts, across a full 2D image. So far, our prototype, which can be programmed either as a matched-filter imager for specific target materials or as a fully hyperspectral multiplexing Hadamard transform imager, has demonstrated over 100 programmable spectral bands while maintaining good spatial image quality. We discuss how diffraction-mediated trades between spatial and spectral resolution determine achievable performance. Finally, we describe methods for dealing with the DLP’s 2D diffractive effects, and suggest a simple modification to the DLP that would eliminate their impact for this application.

Field test results are presented for a prototype long-wave adaptive imager that provides both hyperspectral imagery and contrast imagery based on the direct application of hyperspectral detection algorithms in hardware. Programmable spatial light modulators are used to provide both spectral and spatial resolution using a single element detector. Programmable spectral and spatial detection filters can be used to superimpose any possible analog spectral detection filter on the image. In this work, we demonstrate three modes of operation, including hyperspectral imagery, and one and two-dimensional imagery using a generalized matched filter for detection of a specific target gas within the scene.

Next-generation infrared astronomical instrumentation for ground-based and space telescopes could be based on MOEMS programmable slit masks for multi-object spectroscopy (MOS). This astronomical technique is used extensively to investigate the formation and evolution of galaxies. We propose to develop a 2048x1080 DMD-based MOS instrument to be mounted on the Galileo telescope and called BATMAN. A two-arm instrument has been designed for providing in parallel imaging and spectroscopic capabilities. The two arms with F/4 on the DMD are mounted on a common bench, and an upper bench supports the detectors thanks to two independent hexapods. Very good optical quality on the DMD and the detectors will be reached. ROBIN, a BATMAN demonstrator, has been designed, realized and integrated. It permits to determine the instrument integration procedure, including optics and mechanics integration, alignment procedure and optical quality. First images have been obtained and measured. A DMD pattern manager has been developed in order to generate any slit mask according to the list of objects to be observed; spectra have been generated and measured. Observation strategies will be studied and demonstrated for the scientific optimization strategy over the whole FOV. BATMAN on the sky is of prime importance for characterizing the actual performance of this new family of MOS instruments, as well as investigating the operational procedures on astronomical objects. This instrument will be placed on the Telescopio Nazionale Galileo at the beginning of next year, in 2014.

Hyperspectral imaging sensors have proven to be powerful tools for highly selective and sensitive chemical detection applications, but have some significant operational drawbacks including a detection time-lag due to the large computational overhead of the matched filter analysis of the resulting data cubes. For applications where only a single chemical is of interest or real-time detection is desired, an intelligently designed multispectral sensor can trade high resolution and continuous spectral coverage for an in-line optical matched filter, enabling snapshot chemical detection with nearly no image processing requirements. Such a system can operate with little loss of performance, greatly reduced data volume, and at a fraction of the cost. We have recently developed a high-speed, high-resolution, programmable spectral filter based on a DLP® digital micro-mirror device (DMD) that mimics a conventional band-pass filter by operating on the spectrum without disturbing the underlying image. Our DMD-based filter can independently choose or reject dozens or hundreds of spectral bands and present them simultaneously to an imaging sensor, forming a complete 2D image. With this new technology, even very complicated matched filters can be implemented directly into the optical train of the sensor, producing an image highlighting the target chemical within a spectrally cluttered scene in real-time without further processing. Examples of matched-filter images recorded with our visible-spectrum prototype will be displayed, and extensions to other spectral regions will be discussed. Finally, we will discuss strategies for implementing more sophisticated clutter-suppressing matched filters on the DMD-based system, including schemes that approximate the subtlety of post-processing algorithms by utilizing the DMD’s duty-cycle-based gray-scale capability.

A wide variety of three dimensional (3D) measurement systems that can extract shape information’s with sub millimetric accuracy is available in the industry. However, they generally are of macroscopic size and measuring on confined areas is not feasible. To miniaturize such systems, the step proposed is the integration of flexible image guides combined with compact optical probes. This miniaturization process is tested on an active stereoscopic measurement system. In the projection channel of the system, a digital micro-mirror device (DMD) generates structured binary patterns from an incoherent white light source and injects them into a first image guide. Then, a compact optical system projects the pattern on the measurement area. The same configuration principle is applied to the acquisition channel and allows the capture of the measurement area through a second image guide and finally to a digital camera. In this miniaturized system, image guides have lower resolution than in standard imaging devices. Indeed they are equivalent of 70k pixels devices to compare to the almost 800k pixels of the DMD and camera. That implies lower axial and lateral resolutions and consequently the shape reconstruction method must be carefully chosen. In this paper, several reconstruction strategies such as tuning the projected patterns frequency and also phase-shfit versus gray code based methods were compared considering the best axial resolution criteria.

The field of ghost imaging encompasses systems which can retrieve the spatial information of an object through correlated measurements of a projected light field, having spatial resolution, and the associated reflected or transmitted light intensity measured by a photodetector. By employing a digital light projector in a computational ghost imaging system with multiple spectrally filtered photodetectors we obtain high-quality multi-wavelength reconstructions of real macroscopic objects. We compare different reconstruction algorithms and reveal the use of compressive sensing techniques for achieving sub-Nyquist performance. Furthermore, we demonstrate the use of this technology in non-visible and fluorescence imaging applications.

A method to reliably extract object profiles even with surface discontinuities (that leads to 2nπphase jumps) is proposed. Proposed method uses an amplitude modulated Ronchi grating, which facilitates one to extract phase and unwrap the same with a single image. The modified grating image can be split into two images, one aids in extracting the wrapped phase using Fourier transform profilometry, and the other aids in reliable phase unwrapping. As we only need a grayscale projector that projects amplitude modulated grating, proposed method facilitates one to extract 3D profile without using full video projectors.

The paper presents super resolving configurations that are integrating two digital mirror devices (DMDs) in the aperture and/or in the intermediate image plane. The usage of the DMDs allows obtaining geometric resolution improvement, enhancing field of view and reduction of aberrations such as defocusing and blurring that is obtained due to relative movement during the integration time. The idea behind all the above mentioned applications is to use the DMDs to properly encode the space and the spatial frequency domains such that the object’s information can be separated from the above mentioned aberrations, distortions, limitations and noises.

Optical, full field testing of aspheres and especially freeform optics still remains a challenging task. Till now, various measurement setups for wavefront characterisation have been presented for functional testing. These are primarily based on microlens arrays in front of a photosensitive semiconductor in combination with an analysis logic. Compared to other sensor types for optical testing the Shack-Hartmann sensor (SHS) features a high flexibility with regard to wavefront deformations. For SHS the measurement range is limited due to the measurement principle that all measurement points are detected simultaneously by an imaging device and the signals must be separable - thus the dynamic range is defined by the number of micro-lenses and the resolution of the imaging sensor. Here, we present an approach for wavefront measurements which increases the dynamic range and the lateral resolution simultaneously. The concept is based on a selection and thereby encoding of single sub-apertures of the wavefront under test and to measure the wavefronts slope consecutively in a scanning procedure. In contrast to the LCD based approaches, here the selection of the sub-apertures and thus the scanning procedure is performed by a digital micro-mirror array (DMD). The use of a DMD allows high lateral resolution as well as a very fast scanning ability. The measurement concept and performance of this method will be demonstrated for different freeformed specimens like progressive eye glasses. Furthermore, approaches for calibration of the measurement system will be characterised comprehensively and the optical design of the detector will be discussed in detail.

We present a method to encode complex values into three or four quantized complex values for wavefront modulation using two digital micromirror devices (DMDs). This encoding offers advantages to eliminate the twin image or suppress the zero order diffraction as well to improve hologram fidelity. The optical architecture utilizes a Michelson interferometer with a DMD in Littrow configuration replacing the mirrors to combine the two holograms with the desired phase shift. System performance was examined using numerical simulations and experimental measurements to explore different encoding methods for hologram reconstruction. Both ZOD and conjugate image suppression were demonstrated for different encoding schemes.

Digital micro-mirror devices (DMD) by their high switching speed, stability, and repeatability are a promising devices for fast, reconfigurable telecommunication switches. However, their binary mirror orientation is an issue for conventional redirection of a large number of incoming ports to a similarly large number of output fibers like with analog MEMS. We are presenting here the use the DMD as a diffraction based optical switch, where Fourier diffraction patterns are used to steer the incoming beams to any output configuration. Fourier diffraction patterns are computer generated holograms that structures the incoming light into any shape in the output plane. This way, the light from any fiber can be redirected to any position in the output plane. The incoming light can also be split to any positions in the output plane. This technique has the potential to make an "any to any", true non-blocking, optical switch with high port count, solving some the problems of the present technology.

The progresses in 3D display systems and user interaction technologies will help more effective 3D visualization of 3D information. They yield a realistic representation of 3D objects and simplifies our understanding to the complexity of 3D objects and spatial relationship among them. In this paper, we describe an autostereoscopic multiview 3D display system with capability of real-time user interaction. Design principle of this autostereoscopic multiview 3D display system is presented, together with the details of its hardware/software architecture. A prototype is built and tested based upon multi-projectors and horizontal optical anisotropic display structure. Experimental results illustrate the effectiveness of this novel 3D display and user interaction system.

Based on previous prototype of the Real time 3D holographic display developed last year, we developed a new concept of auto-stereoscopic multiview display (64 views), wide angle (90°) 3D full color display. The display is based on a RGB laser light source illuminating a DMD (Discovery 4100 0,7”) at 24.000 fps, an image deflection system made with an AOD (Acoustic Optic Deflector) driven by a piezo-electric transducer generating a variable standing acoustic wave on the crystal that acts as a phase grating. The DMD projects in fast sequence 64 point of view of the image on the crystal cube. Depending on the frequency of the standing wave, the input picture sent by the DMD is deflected in different angle of view. An holographic screen at a proper distance diffuse the rays in vertical direction (60°) and horizontally select (1°) only the rays directed to the observer. A telescope optical system will enlarge the image to the right dimension. A VHDL firmware to render in real-time (16 ms) 64 views (16 bit 4:2:2) of a CAD model (obj, dxf or 3Ds) and depth-map encoded video images was developed into the resident Virtex5 FPGA of the Discovery 4100 SDK, thus eliminating the needs of image transfer and high speed links

OPTRA is developing a next-generation digital micromirror device (DMD) based two-band infrared scene projector (IRSP) with infinite bit-depth independent of frame rate and an order of magnitude improvement in contrast over the state of the art. Traditionally DMD-based IRSPs have offered larger format and superior uniformity and pixel operability relative to resistive and diode arrays, however, they have been limited in contrast and also by the inherent bitdepth / frame rate tradeoff imposed by pulse width modulation (PWM). OPTRA’s high dynamic range IRSP (HIDRA SP) has broken this dependency with a dynamic structured illumination solution. The HIDRA SP uses a source conditioning DMD to impose the structured illumination on two projector DMDs – one for each spectral band. The source conditioning DMD is operated in binary mode, and the relay optics which form the structured illumination act as a low pass spatial filter. The structured illumination is therefore spatially grayscaled and more importantly is analog with no PWM. In addition, the structured illumination concentrates energy where bright object will be projected and extinguishes energy in dark regions; the result is a significant improvement in contrast. The projector DMDs are operated with 8-bit PWM, however the total projected image is analog with no bit-depth / frame rate dependency. In this paper we describe our progress towards the development, build, and test of a prototype HIDRA SP.