Dimension Technologies Inc. has experimentally demonstrated an optical system that produces autostereoscopic images and also allows very high brightness and power efficiency to be achieved using off the shelf color LCDs. This capability is important in applications such as cockpit displays or mobile, portable, or laptop systems where brightness must be maximized but power conserved as much as possible. The effects are achieved through the creation of light line illumination, by means of which autostereoscopic images are produced, and by simultaneously concentrating the light emitted by the display toward the area the viewer's head is. By turning different illumination sources on and off, it is possible to aim both the concentration area and the 3D viewing area at the observer's head as the observer moves. A variation on the system allows two or more persons to be tracked independently. Cross talk (ghosting) can be reduced to the point that imperceptibility can be achieved.

Dimension Technologies Inc. has recently developed the first commercial implementation of an autostereoscopic head tracking display system. Utilizing proprietary and patented illumination technologies, the display system creates left and right viewing zones to the eyes of a viewer. A head tracking system is used to determine the position of the viewer, and control electronics adjust the illumination and LCD subsystems to display a correct stereo image for the current viewer position. This system requires rapid real-time processing and response by the subsystems in order to avoid visual artifacts during viewer head movement. System latency and update rates are critical factors in system performance. The research provides a clear demonstration of the near term feasibility for development of an autostereoscopic display exhibiting no visual artifacts due to viewer head position and/or movement.

We have developed an experimental stereoscopic display system for personal computers that utilizes a lenticular array and a high-resolution color liquid crystal display panel. This article discusses the principles of stereopsis used in the system, describes the system's mechanism and features, and evaluates the performance of its graphics system. We also developed a color image quantization algorithm specifically for this display device. In addition, we proposed a technique for interactive 3D manipulation of objects, which uses two trackballs to allow movement and rotation through six degrees of freedom.

A 3D high-definition television (HDTV) system suitable for attachment to a stereoscopic operating microscope allowing 3D medical documentation using a single HDTV camera and monitor is described. The system provides 3D HDTV microneurosurgical recorded images suitable for viewing on a screen or monitor, or for printing. Visual documentation using a television and video system is very important in modern medical practice, especially for the eduction of medical students, the training of residents, and the display of records in medical conferences. For the documentation of microsurgery and endoscopic surgery, the video system is essential. The printed images taken from the recording by the HDTV system of the illustrative case clearly demonstrate the high quality and definition achieved, which are comparable to that of the 35 mm movie film. As the system only requires a single camera and recorder, the cost performance and size make it very suitable for microsurgical and endoscopic documentation.

This paper describes a new robust stereo and motion analysis algorithm to meet the special demands of a multiview 3D TV system. Based on this algorithm, images can be synthesized for any virtual camera position within the array covered by the real cameras. It is possible to create any number of perspective views needed for smooth motion parallax in multiview systems. For the reconstruction of a scene, 3D line segments are estimated by triangulation of corresponding 2D line segments (disparity estimation). This estimation of 3D segments is carried out more dependably by the combination of stereo and motion information and -- to achieve further improvements -- the utilization of multiocular stereo. New measures for a more reliable estimation of the scene are proposed. To achieve subjectively error-free estimations, Prazdny's coherence principle is extended to motion and image sequences as well as sub-constellations of the multi-camera rig. This algorithm comes up with a 4D space-time model of the scene by the cooperation of multi-camera motion and stereo.

This paper deals with human factor requirements for the conception of a stereoscopic camera made up of two cameras. With this concept, geometric, optic, and electric differences between the two cameras are possible. These differences have a direct influence on the subjective quality of natural stereoscopic images. In this context, we describe the study of the admissible contrast differences between the two channels of a stereoscopic camera. The main aim of this analysis is the definition of the acceptability thresholds of the defects produced by an inadequate adjustment of the white and black levels between the two cameras. In both cases, the acceptability thresholds are determined with subjective assessments. The studies on the white and black levels are focused on the red, green, and blue levels of each of the two views. The recommendations are aimed at ensuring comfortable images to look at in a stereoscopic television service.

We rigorously present the geometric issues related to binocular imaging. We identify the minimum number and most fundamental conceptual set of parameters needed to define 3D-stereoscopic camera and display systems; the fundamental parameter that is needed to specify a 3D-stereoscopic system but not a monocular system is the pupillary distance. We analyze the constraints that are imposed on the values of the parameters by the requirement that the imagery be geometrically indistinguishable from the reality that would be perceived by the `naked' human visual apparatus. We relate our approach to those employed by several well known textbooks and graphics engines.

We discuss an approach to detect a 3D position or shape by reconstructing real images at multiple positions from a holographic fringe pattern produced on a charge-coupled device (CCD) camera. The calculation of the intensity distribution on a real image at a certain position can be performed by making the Fresnel transformation of the holographic fringe pattern. It is possible to detect the position of an object by comparing reconstructed images at multiple positions and by picking up one having peak intensity in them. In this paper, an approach to position detection using reconstructed images from a holographic fringe pattern and Fresnel transformation for image reconstruction are described. As experimental results, image reconstructions from holographic fringe patterns produced with one or two point light sources are shown. In conclusion, we discuss the accuracy of this method on z position.

Methods are described for interactively transforming flat, monoscopic images into stereoscopic 3D image pairs. These methods assume that the user is beginning with a single existing image, and that it is desirable to preserve as much as possible of the original, altering it minimally to create a stereoscopic version. Utilizing binocular vision and a stereoscopic display during the transformation process, the monocular depth cues contained in most images can be used as guides to `replace' lost stereoscopic depth.

The depth of an object in stereo is determined by the horizontal separation (i.e., disparity) of the object between the left and right images. For digitized images the disparity is in increments of pixels. Since all points in a given depth plane have the same disparity, the `cut- plane' procedure can theoretically eliminate a given depth plane by simply subtracting the stereo image pairs from each other after horizontally shifting them a specific number of pixels. Mathematical analysis and simulations with abstract objects have determined that both the length and disparity of objects with widths greater than one pixel may be modified by the `cut- plane' procedure; even when the object is not in the depth plane being eliminated. To what extent an object is modified depends on the original disparity and width of the object. The application of this procedure to chest x rays is presented with a demonstration of how certain pitfalls of the `cut-plane' procedure can be surmounted.

Crosstalk between the left and right eyes consists of the one eye's image seen faintly by the other. Image processing can reduce this. The technique is effective, but there are costs of course, and some surprises.

Translating rectangular 2D images in planes perpendicular to the line of sight can create image distortion known as keystoning or projection warp. This can occur when viewing several images on a plane from a fixed position, when cross viewing, or when using non-parallel mirrors to view stereo pairs. In the latter case, vertical disparity can become a problem. The projection warp and its inverse are derived as functions of the viewing and translation parameters. It is shown that vertical disparity can be significant, even in small image translations. The inverse transformation applied to the original image corrects the keystoning caused by the translation. Rendering of the inverse image is also discussed.

This paper is concerned with the data compression and interpolation of multi-view image. In this paper, we propose a novel disparity compensation method based on geometric relationship. We first investigate the geometric relationship between a point in the object space and its projection onto view images. Then, we propose the disparity compensation scheme that utilizes the geometric constraints between view images. This scheme is used to compress the multi-view image into the structure of the triangular patches and the texture data on the surface of the patches. This scheme not only compresses the multi-view image but also synthesizes the view images from any viewpoints in the viewing zone. We report the experiment where three sets of multi-view image were used as original images and the amount of data was reduced to 1/20 with SNR 34 dB.

This paper is about a field-sequential electro-stereoscopic virtual reality (VR) viewing device which incorporates electro-optical shutters, and lenses that aid accommodation and convergence. Each eye looks through a combination of prismatic and positive-diopter lenses, and the same shutters used in CrystalEyes^R eyewear are used for image selection. This is the first lenticular stereoscope to employ the time-multiplexing technique with superimposed images and shutters functioning as a kind of electronic septum. The result is a product with relatively low cost, high resolution, and a wide field of view.

We propose a new autostereoscopic display system employing an eye-position tracking technique for multi-viewers who do not need to wear 3-D glasses. We also describe the developed stereoscopic projectors and the lenticular screen design for this system. In this system, the 3-D screen consists of two lenticular screens and a diffusion layer. Each 3-D projector that corresponds to a viewer consists of two projectors for right and left images. The viewer's positions are detected, and the 3-D projector moves according to the viewer's movement.

Stereoscopic liquid crystal displays are described that permit the observation of a stereo pair by several persons simultaneously without the use of special glasses. One of the systems is composed of a right eye system, a left eye system, and a special infrared illuminating system. In the right eye system, a transparent type color liquid crystal plate is used with a special back light unit. The back light unit consists of a monochrome 2D display and a large format convex lens. The unit distributes the light to the viewers' right eye only. The right eye system is combined with a left eye system by a half mirror in order to function as a time-parallel stereoscopic system. Another system consists of a large format convex lens, a polarizer and liquid crystal plate, and a color 2D display. The color 2D display superposes left eye and right eye perspectives on the display as polarized images that are polarized 90 degrees to each other. The polarized images are distributed to the viewers' correct eyes because the polarizer and liquid crystal plate occludes the incorrect eye. These systems are applicable not only to stereoscopic receivers for domestic use but also medical devices for stereoscopic purposes.

Three-dimensional displays that permit simultaneous observation by plural persons have been reported in other papers, but a 3-D endoscope is only available for stereo image and is better used in the no-glasses method. Hence conventional 3-D techniques were poor for the endoscope for medicine. We produced a new 3-D device (stereoscopic liquid crystal display) which permits simultaneous observation of a stereo pair by plural persons without glasses and we improve the device for the endoscope. We combined the 3-D endoscope and the new 3-D device. The endoscope has two CCD cameras. The cameras create a stereo pair transmitted by two NTSC signals. The system is time-parallel stereoscopic.

A fully operational single-camera system for 3D medical laparoscopy has been developed. We optimized the optical design to increase the amount of light through the system, and we improved the image quality through an increase in resolution and a reduction of the flicker. We report the details of the present embodiment of the laparoscope and note the specific differences between the current version and those previously described. In order to improve image stability this model was built retaining the single camera but with the two viewpoints presented field-sequentially at 60 Hz. Shuttered glasses are used for viewing the monitor. The present embodiment retains the single-camera concept and the shuttered glasses, but now switches at 120 Hz to present a flickerless image. Light losses in the original laparoscope have been overcome through improved placement of fiber bundles, modifications to the optics to achieve higher f/number for the system, and better cameras that respond well at low light levels. Although the latest scope has twisted-nematic liquid-crystal shutters, mechanical shutters will be tried in the next generation.

This paper describes on-going research into 3-D x-ray imaging techniques conducted by the 3- D Imaging Group at The Nottingham Trent University. This work was initiated to enhance the visual interpretation of complex x-ray images, specifically in response to problems encountered in the routine screening of freight and hand luggage at airports. The interpretation of standard 2-D x-ray images by humans is difficult due to the lack of visual cues to depth in an image produced by transmitted radiation. The solution proposed is to introduce binocular parallax, a powerful physiological depth cue, into the resultant shadowgraph x-ray images. This is accomplished by implementing a stereoscopic imaging technique specifically developed by the Nottingham group for use with linear x-ray detector arrays and has culminated in the development of two experimental machines. Current research involves the investigation of techniques that allow the extraction of the 3-D data contained in the stereoscopic images such that `slice' images can be obtained. This data set may then be used with existing reconstruction software utilized by CAT scanning techniques.

Real-time x-ray stereoscopic imaging is obtained with the Reverse Geometry X-ray^R (RGX^R) system using a large raster scanning x-ray source and two small point detectors. The detectors range in size from 1 inch to 2 millimeters in diameter. The detectors can be mounted on fiber optic cables to increase accessibility. Industrial and medical applications are broad since the small detectors acquire a field of view as large as 10 inches in diameter. Polarized glasses are typically used to separate the fields on a shuttered monitor; the stereoscopic imaging displays can be used as well.

Numerous 3-D stereoscopic techniques have been explored. These previous techniques have had shortcomings precluding them from making stereoscopic imaging pervasive in mainstream applications. In the last decade, several enabling technologies have emerged and have become available and affordable. They make it possible now to realize the near-ideal stereoscopic imaging technology that can be made available to the masses making possible the inevitable transition from flat imaging to stereoscopic imaging. The ideal stereoscopic technology must meet four important criteria: (1) high stereoscopic image quality; (2) affordability; (3) compatibility with existing infrastructure, e.g., NTSC video, PC, and other devices; and (4) general purpose characteristics, e.g., the ability to produce electronic displays, hard-copy printing and capturing stereoscopic images on film and stored electronically. In section 2, an overview of prior art technologies is given highlighting their advantages and disadvantages. In section 3, the novel (mu) Pol^TM stereoscopic technology is described making the case that it meets the four criteria for realizing the inevitable transition from flat to stereoscopic imaging for mass applications.

We describe a 3-D monitor for the direct display of 3-D information. The general concept for this volumetric display is a fairly straightforward extension of a 2-D screen composed of an array of picture-elements, or `pixels.' Here, a 3-D stack of pixels (actually, volume-elements, or `voxels') that, when off, are completely transparent. When addressed, a voxel becomes optically active and emits light. In this way, a 3-D pattern may be directly built up from a set of activated voxels. In our prototypes built to date, we have made the voxels from a nugget of UV-curved optical resin doped with an organic dye. The void between voxels is filled with an index matching fluid formulated to eliminate ghost images due to Fresnel reflections off the voxel surfaces. Each voxel is addressed by an optical fiber that pipes light to the voxel. This optical energy pumps the embedded dye causing it to fluoresce at an appropriate color that depends on the choice of dye. The fibers emerge from the 3-D array of voxels and are addressed using a flat array of liquid crystals.

We have developed a flicker-free stereoscopic video system which uses commercial television components. This system has been installed on an underwater remotely operated vehicle (ROV) that is used for service and inspection tasks at a gas production platform 130 km off the North-West coast of Western Australia. We report the results of field and laboratory time trials of remote manipulation tasks and also the general experience gained in the field operation of the system. Stereoscopic video provides the operator with an intuitive sense of the depth relationships of the work site. Operators report that this reduces frustration and mental effort as well as giving them confidence in their actions. Some of the other advantages that we have observed include the increased ability to see through suspended matter (fine particles) in the water. The system is most useful in manipulative tasks but also useful for general `flying' of the ROV making navigation through the platform easier. Our results indicate that stereoscopic video will be a valuable tool in the operation of remotely operated vehicles in the underwater environment.

To determine the effectiveness of stereo imaging in aiding the detection of objects in a scene, we are conducting experiments in which subjects are shown computer-generated stereo and mono images and are asked to determine if there is an object with particular characteristics in the image. The experimental data are analyzed using receiver operating characteristic (ROC) approaches to determine which types of objects may be easier to detect using stereo viewing. In this paper, issues that rise in the design of ROC studies to determine the statistical effectiveness of stereo imagery are discussed. These include traps and pitfalls such as varying viewing conditions, image intensity differences, ghosting, flicker, the speed/accuracy tradeoff, subjects' stereo acuity, and degree of difficulty in the discrimination task. Our experimental results show that when these problems are properly addressed, stereo viewing increases the sensitivity and specificity of observer performance in detecting subtle features in simulated x- ray transmission images.

This paper reports a workstation experiment to verify the potential of stereo cueing for the declutter function in a simulated tracking task. The experimental task was designed as similar to tracking in a conventional flight director, although the format presented was a very cluttered dynamic display. The subject's task was to use a hand-controller to keep a tracking symbol on top of a target symbol. In the basic tracking task, the target symbol is presented as a red `X' and the tracking symbol as a blue `X,' which provides some declutter and makes the task more reasonable in terms of performance. The color coding was removed in one display condition to provide some additional clutter. For this condition, both the target and tracking symbols are presented as red `Xs.' Additional clutter was provided by the inclusion of moving, differently colored `X' symbols. Stereo depth was utilized by placing any clutter in a plane in front of the display monitor, placing the tracking symbol at screen depth, and placing the target symbol behind the screen. The results from eight subjects revealed that the stereo presentation effectively offset the cluttering effects.

A stereoscopic system was developed that integrates hardware and software components for image acquisition, digitization, processing, display, and measurements. The model-induced trajectories of nearly neutrally buoyant fluorescent particles, illuminated with a 15-W pulsed copper vapor laser, are tracked in a towing tank by stereoscopic time-lapse photography using two 35-mm cameras positioned at a 90-degree angle from the top and the side. A C program, HI, drives two data I/O boards hosted in a PC to set up the run parameters, control the operations of the laser and camera shutters, and acquire the stereo images. The photographic records are digitized and processed to derive the centroids of reference marks and particle images. The centroids are then fed into a Windows-based program, Track/3D, to perform image correlation, correction for image distortion, stereo conversion, stereoscopic display, and measurements. The display module incorporates a graphics library that drives a stereoscopic display adapter attached to a monitor; the stereogram must be viewed with polarizing glasses. Functions are available for image translation, rotation, zooming, and on- screen measurements. The velocity and acceleration components of the 3-D flow field induced by the model are derived from the trajectories, serving as a basis for whole-field stereoscopic quantitative flow visualization.

An application is developed for viewing scanning tunneling microscope (STM) data stereoscopically by using alternating-pair glasses and the TV-1024/32 32-bit/pixel graphics accelerator system. Arbitrary viewer position, view, and light source direction are allowed. The steps used for generation of the stereoscopic views are outlined. Due to the large size of the data set (up to 400 X 400 uniform grid) several computational optimizations are used. Symmetry of the data set is used to remove computational redundancies during triangulation and shading processes. Generation of the left eye and right eye view matrices is explained, as well as the transformation of the physical interocular distance of the viewer into the viewing coordinate system. Stereoscopic slides of gold, graphite, and other atomic structures are presented.

It is the intention of this paper to give details of the continuing research into obtaining 3-D coordinate information from an object space using non-standard video sources. Details are given on producing images with the line-scan sensor by rotating the device relative to an object space. Theoretically, this could provide picture information from a potential 360 degree panoramic view. However, initial results have demonstrated that such images are difficult for humans to interpret. Details are given in this paper on the limitations of the line-scan images produced from camera rotation for presentation to human observers.

Present tracking schemes for virtual reality position sensing have a variety of features that make their use in applications such as large classrooms or remote locations difficult. A more natural tracking method would be a lightweight, low cost, and accurate inertial tracking system. Some commercial inertial systems are discussed. As a low cost alternative, a mouse based head self-tracker has been built at North Carolina State University. Its design and operational ideas are being extended to build a less cumbersome head tracker based on the rotational axes.

Sensor Applications, Inc. has developed a variety of novel approaches in the use of low-cost, passive magnetic sensors, referenced to the earth's magnetic field, to provide a true measurement of roll, pitch, and yaw, and, as a derivative, angular acceleration and velocity. These sourceless, non-inertial sensors have a strong potential to replace existing inertial, gravity or mass-based technologies in a multitude of applications. The novel geometric arrangement of solid state magnetic sensors and the associated algorithms involved in resolving the earth's magnetic field vectors, provides a non-inertial 3-axis sensor capable of monitoring in realtime roll, pitch, and yaw. This paper also describes the current development of a solid state, adaptive, self-calibrating sensor, also referenced to the earth's magnetic field, that is immune to local field strength variations. Promising results outlined in this paper from pilot projects in virtual reality headtracking, biomechanical gait analysis, suggests that this new technology has the potential to meet the angular tracking requirements of both large volume consumer applications, and small cost-conscious research efforts.

In current virtual environment systems, the stereoscopic images presented in a head-mounted display are far from optimal. The aim is to achieve orthostereoscopy, which roughly means images should `behave as in real life.' A theoretical model of stereoscopic optics was used to implement a test and optimization system. Tests were devised to analyze the importance of many stereoscopy related parameters. The system's capability to independently set these parameters allows an optimization of the stereoscopic images, given the limitations of the display device used.

The design factors and features of high resolution head-mounted display (HMD) were studied. Based on these studies, two models of HMD were designed and made. Both models adopted monochrome cathode ray tubes (CRTs) to achieve high resolution images. These HMDs were applied for experiments of a tele-existence. For further applications, color and high resolution HMD is completed using monochrome CRTs with RGB color filters. As this HMD can be used as both a goggle type and a see-through type, many applications are expected.

Many researchers have felt that counterbalanced, stereoscopic immersive displays were an interim technology that would be supplanted as advances in LCDs and electronics made lightweight, head-mounted viewers popular. While there is still a long way to go in the development of truly practical head-mounted displays, it now seems clear that counterbalanced display will always play a significant role in the development, applications, and general dissemination of virtual environment tools. This paper hopes to explain the unexpected popularity of these devices, and to highlight features of these displays that have become apparent since the 1989 SPIE paper that described an early workable example of this genre. In addition, this paper describes the current state of this technology and the acceptance of counterbalanced displays in a wide range of applications since the original SPIE paper.

Interaction in virtual reality environments is still a challenging task. Static hand posture recognition is currently the most common and widely used method for interaction using glove input devices. In order to improve the naturalness of interaction, and thereby decrease the user-interface learning time, there is a need to be able to recognize dynamic gestures. In this paper we describe our approach to overcoming the difficulties of dynamic gesture recognition (DGR) using neural networks. Backpropagation neural networks have already proven themselves to be appropriate and efficient for posture recognition. However, the extensive amount of data involved in DGR requires a different approach. Because of features such as topology preservation and automatic-learning, Kohonen Feature Maps are particularly suitable for the reduction of the high dimensional data space that is the result of a dynamic gesture, and are thus implemented for this task.

Virtual environments as an aid for scientific discovery are becoming a topic of research for many visualization efforts. Virtual environments provide greater immersion into the world or environment of scientific data, thereby enhancing the researcher's perception of its features and forms. However, access to laboratories with the appropriate hardware for providing the immersive environments is currently limited. To make the best use of limited lab time a scientist will want to spend as much of it as possible actually looking at the data, and not debugging the virtual world source code. In an effort to get the scientist close to achieving this goal, we have developed software to emulate a suite of standard virtual environment hardware devices. Emulated hardware is designed such that it is indistinguishable to the application software. Communications with the `virtual devices' is via a serial port using the same protocol as the physical devices. Some `virtual devices' have already been written. We are now making efforts to have virtual environment programmers use them to design and test applications without using much of the increasingly precious time in the NCSA Virtual Reality Laboratory.

This paper discusses the use of hand gestures (i.e., changing finger flexion) within a virtual environment (VE). Many systems now employ static hand postures (i.e., static finger flexion), often coupled with hand translations and rotations, as a method of interacting with a VE. However, few systems are currently using dynamically changing finger flexion for interacting with VEs. In our system, the user wears an electronically instrumented glove. We have developed a simple algorithm for recognizing gestures for use in two applications: automotive design and visualization of atmospheric data. In addition to recognizing the gestures, we also calculate the rate at which the gestures are made and the rate and direction of hand movement while making the gestures. We report on our experiences with the algorithm design and implementation, and the use of the gestures in our applications. We also talk about our background work in user calibration of the glove, as well as learned and innate posture recognition (postures recognized with and without training, respectively).

As dissimilar as virtual environment interfaces are to traditional desktop interfaces, one common element will always be the need for selection. Menus are often used for this type of interaction to minimize memorization requirements. However, the direct analogy of 2-D menus to 3-D applications has not been widely accepted due to complications concerning pointing tasks in virtual space. In this paper, we report results of the use of a new technique for menu display and interaction which involves the display of menu items on a 2-D overlay onto the 3-D world. The proposed technique displays the textual menu items on a viewplane that moves relative to the user so that text will always directly face the eyes. The menu items are selected via a speech recognition system. Advantages of this technique include ease of readability and trivial interaction with the menu. The hands are always free to be used for other tasks. A number of usability issues are discussed. These findings show that this metaphor is an effective technique for not only menus in virtual spaces but for many uses of text in the 3-D domain.

There are many ways to produce the sense of `presence' or telepresence in the user of virtual reality. For example attempting to increase the realism of the visual environment is a commonly accepted strategy. In contrast, this paper explores a way for the user to feel present in an unrealistic virtual body. It investigates an unusual approach, proprioceptive illusions. Proprioceptive or body illusions are used to generate and explore the experience of virtuality and presence outside of the normal body limits. These projects are realized in art installations.

This paper describes an ongoing effort to develop one of the first fully immersive virtual environment installations that is inhabited by virtual characters and presences specially designed to respond to and interact with its users. This experience allows a visitor to become visually and aurally immersed in a 3D computer generated environment that is inhabited by many virtual animals. As a user explores the virtual space, he/she encounters several species of computer generated animals, birds, and insects that move about independently, and interactively respond to the user's presence in various ways. The hardware configuration of this system includes a head-coupled, stereoscopic color viewer, and special DSP hardware that provides realistic, 3D localized sound cues linked to characters and events in the virtual space. Also, the virtual environment and characters surrounding the user are generated by a high performance, real-time computer graphics platform. The paper describes the computer programs that model the motion of the animals, the system configuration that supports the experience, and the design issues involved in developing a virtual environment system for public installation.

We are interested in the application of computer animation to surgery. Our current project, a navigation and visualization tool for knee arthroscopy, relies on real-time computer graphics and the human interface technologies associated with virtual reality. We believe that this new combination of techniques will lead to improved surgical outcomes and decreased health care costs. To meet these expectations in the medical field, the system must be safe, usable, and cost-effective. In this paper, we outline some of the most important hardware and software specifications in the areas of video input and output, spatial tracking, stereoscopic displays, computer graphics models and libraries, mass storage and network interfaces, and operating systems. Since this is a fairly new combination of technologies and a new application, our justification for our specifications are drawn from the current generation of surgical technology and by analogy to other fields where virtual reality technology has been more extensively applied and studied.

The relationship between people and computers is currently undergoing a rapid transformation. What was once an inert, complex tool is becoming a natural extension of the user. In order to complete the transformation, the computer must become functionally transparent to the user. Ubiquitous computing holds promise toward this end but has severe limitations in terms of installation and upgrade costs. Virtual environment technology provides a viable alternative, if properly implemented, by seamlessly integrating the user's senses with the growing global information network. This paper is an overview of the InterFACE project at Future Vision Technologies, Inc. InterFACE is a new type of computing device which merges portable computing, wireless communications, synthetic environments and telepresence into a wearable package. The technology is introduced in contrast to existing systems in terms of form, flexibility and ease of use and thereby a framework for market justification is formed. The evolution of the system is also described and major application types are detailed.

A central issue in teleoperation is the provision of appropriate perceptual information for the remote human operator. We have developed an experimental teleoperation system which provides the capability of displaying graphic simulation images that accurately depict actual robot operation. Our experimentation compares the use of monocular operator-controlled- viewpoint displays to conditions in which the operation of the robot is viewed directly. The robot gripper was moved through a balanced set of trajectories relative to an object in the workspace, and subjects were required to make a forced-choice judgement as to whether or not the gripper would collide with the object. Significantly more errors were encountered in the live-monocular and fixed-simulation conditions than were found in either the live-stereo or the controlled-simulation conditions, while there were no significant performance differences between these two more effective display conditions. An analysis indicates that the angle formed between viewing direction and the line of motion of the robot gripper is a strong determiner of the number of errors that are made.

We exploit the correlations between 3D-stereoscopic left-right image pairs to achieve high compression factors for image frame storage and image stream transmission. In particular, in image stream transmission, we can find extremely high correlations between left-right frames offset in time such that perspective-induced disparity between viewpoints and motion-induced parallax from a single viewpoint are nearly identical; we coin the term `wordline correlation' for this condition. We test these ideas in two implementations, straightforward computing of blockwise cross-correlations, and multiresolution hierarchical matching using a wavelet-based compression method. We find that good 3D-stereoscopic imagery can be had for only a few percent more storage space or transmission bandwidth than is required for the corresponding flat imagery.

This paper reports on the initial developmental stages of an endoscope that can be used for medical and other applications requiring 3-D images. The conceptual basis for the design is derived from our prior efforts in the development of 3-D scopes for laparoscopy and from telepresence research with head-mounted displays for use in virtual reality. We include the initial results of the development of the viewing system and the design for the scope. Specific future research efforts and developmental phase also are described.

We describe a virtual environment architecture (VEA) that incorporates an object-oriented design approach. VEA emphasizes fine-grained multiprocessing, which takes advantage of the processing power available on today's multiprocessor workstations. Inter-object communications are supported primarily through events. VEA allows objects to process events simultaneously, subject to processor availability and data consistency constraints. Device abstraction layering is important for isolating assumptions, and for removing dependencies on certain I/O devices or modes of interaction. Finally, VEA provides facilities for loading and saving virtual environment configuration files, which allow a session to be saved and restarted.

We characterize a class of stereoscopic displays, that includes head-mounted displays that incorporate miniature cathode-ray tubes. We present a detailed viewing model that accounts for the placement, size, and orientation of the virtual display images. The model is appropriate for several types of head-mounted displays, as well as head-tracked stationary displays. The tracking algorithm accounts for the displacement between the tracking sensor and the user's eyes. We provide algorithms for calculating stereoscopic viewing and projection matrices. The algorithms are presented as parameterized homogeneous transforms. We also discuss design guidelines for avoiding accommodation/convergence conflicts, and managing the perceived field-of-view. The advantages of this viewing model and algorithm are the elimination of possible vertical parallax, and an undistorted perception of depth. Both of these factors can contribute to improved utility for the operator.

Users of virtual environments currently have few good options for controlling or interacting with their environment from within. Existing VR interaction techniques, including menus, postures, gestures, and wands, all suffer from a variety of problems, not the least of which is the unnatural, almost forced, mapping that exists between the input act and its corresponding function. This paper describes a new technique for controlling and interacting with virtual environments. The techniques superimposes an object in the real world with an augmented representation of that object in the virtual environment. Such blending of a physical object with its augmented virtual representation results in a surprising natural interaction metaphor.