Medical visualization tools use computer graphics techniques to help physicians understand imaging data. These tools commonly handle 2D medical images, but the software used to display 3D images from computerized tomography (CT) and magnetic resonance imaging (MRI) has not yet achieved widespread clinical use for processing ultrasound (US) images. This is because CT and MRI data are isotropic, while US data are anisotropic. The voxels of isotropic data, i.e. the smallest box-shaped volume elements of a 3D image, have resolutions that are the same along their x-, y- and z-axes. As shown in Figure 1, they look like a grid of little cubes with consistent spacing. In contrast, anisotropic voxels are not cubic (see Figure 2), with the result that images may appear flattened or elongated in certain views.
Figure 1. Isotropic CT images and corresponding voxel cubic structure.
The current trend in medical imaging is to integrate multiple images to provide an accurate 3D representation of patient anatomy. As a result, a key area of medical visualization is volume visualization, which involves projecting a multidimensional data set onto a 2D image plane.1 Current algorithms all have their respective strong points, some providing higher contrast, others more accurate detail. A good algorithm should be efficient at volume rendering and be able to handle a high throughput of imaging information. It should also allow the simultaneous viewing of all image patterns and provide a rapid scrolling function. Another consideration is that, while the implementation of advanced visualization algorithms is powerful enough on most commercial imaging workstations, it is not efficient on conventional PCs. (This is a special concern in China, where few hospitals are equipped with high-performance computers.) Some imaging workstations can integrate optional components to process different types of medical images. But more components increase software requirements which makes them computationally more expensive.
We have recently developed a solution that is compatible with both isotropic and anisotropic volume data, and also with all current visualization algorithms. It is provided as a plug-in that can be used on stand-alone PCs, or it can be offered as a web service.
Our software was developed using the Visualization Toolkit (VTK), an open source software system for 3D computer graphics, image processing, and visualization.2,3 The toolkit allows image information to be built into structured data sets for subsequent treatment in an object-processing pipeline. These data sets include imaging information such as spatial position information and voxel values. The pipeline mechanism ensures that all objects share the same data source, which saves significant memory space and leads to efficient processing. We developed the software as a data definition language (DDL) file. Application programs that include the interface and process the image data in Digital Imaging and Communications in Medicine (DICOM), the required format, can transfer the visualization function easily.
In a series of images, one image has rectangular planar pixels, and all images have the same number of pixels. The overall volume data is then collected on a structured grid. For medical images, the number of voxels in each dimension is constant, but the spaces between voxels are irregular (see Figure 2). Using the coordinates of each voxel, we can generate a volume data set using VTK definitions based on a compositing algorithm that can calculate the contribution of each voxel to the image.4
Figure 2. Anisotropic US images and corresponding voxel representation.
For image reconstruction, the volume can be rendered in multiplanar reconstruction (MPR) or maximum intensity projection (MIP) mode. When rendering the volume in MPR mode, the view plane is calculated according to the visual angle in real-time. The movement is reflected on other sections by shifting the cut lines to correlate with the view plane on these sections. Because of the pipeline mechanism, the movement of a plane triggers an update of all correlated objects. A volume can also be rendered simultaneously in interactive MPR and MIP modes. The view plane of an MIP object can also be represented on MPR sections, as shown in Figure 3.
Figure 3. Multiplanar reconstruction (MPR) and maximum intensity projection (MIP) views of CT images. The upper left view is a MIP volume display and is represented by red dotted lines in the upper right and bottom left MPR views.
Since VTK supports C++ and Java, it is easily portable and the DDL file can be installed on Unix-based platforms, PCs (Windows 98/ME/NT/2000/XP), and Mac OSX Jaguar.3 Our software was written in C++, using MS VC++ 6.0 and VTK 5.0.Conclusion
We developed a software package for the interactive display of DICOM images and other graphics. We used open source web technology and DICOM standards to build a PC-based architecture for medical imaging. For more widespread medical visualization applications, a smaller but more efficient architecture is still required. Our novel framework is currently compatible with CT and MRI imaging, and we are presently testing the rendering of US images. It has a high processing efficiency, and we plan to distribute it more widely, also as web-based platforms.
Jie Shuai, Jianyong Sun, Jianguo Zhang
Shanghai Institute of Technical Physics
Chinese Academy of Sciences
Jie Shuai obtained her BS degree in computer science from the University of Science and Technology of China. Currently, she is completing a PhD in the Laboratory for Medical Imaging Informatics at the Shanghai Institute of Technical Physics. Her current research is focused on multidimensional medical imaging visualization.
Jianyong Sun obtained his BS degree in electronic engineering from Shandong University and his MS and PhD degrees in 2001 and 2004, respectively, from the Institute of Technical Physics of the Chinese Academy of Sciences in Shanghai. He is currently an associate professor at the Laboratory for Medical Imaging Informatics of the Shanghai Institute of Technical Physics. His current research interests include teleradiology, image display, and visualization.
Jianguo Zhang is the director and a professor of the Laboratory for Medical Imaging Informatics at the Shanghai Institute of Technical Physics, Chinese Academy of Sciences. He was previously a visiting professor in the Department of Radiology of the University of Southern California. His current research interests include picture archiving and communication systems (PACS) technologies, medical image processing and visualization, imaging informatics, security technologies in healthcare, and electronic patient records.