Extracting 3D images from lunar orbiter Kaguya data

The Japanese lunar orbiter SELENE (Selenological and Engineering Explorer)¹ was launched from the Japan Aerospace Exploration Agency's Tanegashima Space Center on 14 September 2007. Its mission ended on 11 June 2009. The spacecraft is now better known by its nickname Kaguya, after a legendary Japanese moon princess. The scientific objectives were to determine the origin and evolution of the Moon and take measurements of the lunar environment. Fourteen scientific instruments were mounted on the orbiter to achieve these objectives, and it was also fitted with telephoto and wide-angle high-definition TV (HDTV) cameras to record still and moving pictures of the Moon and Earth.^2,3 Among these are views of the full rise of Earth and well-known craters on the lunar surface.⁴

There is substantial interest in rendering these views as realistically as possible. Accordingly, we wished to convert the 2D moving picture data captured by a single HDTV camera mounted on the orbiter into stereoscopic (3D) images.⁵ Figure 1 details the camera imaging in schematic form. Here, P_l and P_r correspond to the positions of the orbiter at times t₁ and t₁+Δt, and Δt denotes the time offset. The z_l- and z_r-axes correspond to the optical axes of the cameras at times t₁ and t₁+Δt, and P_c denotes the convergence point of the axes. If we compare the images captured by the HDTV camera at time t₁ and t₁+Δt, Earth appears to have moved vertically with respect to the lunar surface as a result of the distance traveled by the orbiter during time Δt. Consequently, rotating these two images by 90 degrees produces a pair of images with a horizontal disparity in Earth's position with respect to the Moon. Viewing the images through a device that restricts display of the left and right pictures to the corresponding eyes gives the observer a stereoscopic impression of depth.

Figure 1. Schematic diagram of capture specifics. P_land P_r correspond to the positions of the orbiter at times t₁ and t₁+Δt, and Δt denotes the time offset. The z_l- and z_r-axes correspond to the optical axes of the cameras at times t₁ and t₁+Δt, and P_c denotes the convergence point of the axes.

In general, stereoscopic images are captured using a parallel or converged (toed-in) camera configuration.⁶ Figure 1 shows the position of the left (P_l) and right (P′_r) cameras set symmetrically in this arrangement. In contrast, the position of the camera on the right as the orbiter moves around the Moon to collect the image data is P_r, i.e., the right image is displaced along the z_r-axis from point P′_r, where ideally it should be for convergence. This displacement causes spatial distortion that must be compensated to avoid viewing discomfort.

A typical approach to solving the problem is to correct for the displacement along the z_r-axis in the right image using depth data from the lunar surface that is obtainable with computer graphics software. The orbiter is also equipped with a metrological instrument for measuring the required altitude data.⁷ However, for simplicity we applied a less-complicated method to compensate for the spatial distortion. We generated depth data assuming that the lunar surface is spherical (i.e., we ignored the craters) and used this data to correct the right images taken at P_r. Figure 2 shows a flow chart that describes the process.

Figure 2. Flow chart for converting HDTV movie images into stereoscopic images corrected for spatial distortion. HD: High definition.

Figure 3 shows stereoscopic images without and with our corrections. The top right corner of each picture is enlarged. Both the left and right images were offset horizontally so that the lunar surface was not directly in front of the display. A vertical disparity can be observed in the stereoscopic image in Figure 3(a), where no correction has been made. The disparity is reduced by correcting the right image using the depth data that we generated by assuming a spherical lunar surface: see Figure 3(b).

Figure 3. Stereoscopic images (a) without and (b) with corrections for spatial distortion using data generated under the assumption that the Moon has a spherical surface.

In summary, we have described a method of converting moving pictures captured by a single HDTV camera into stereoscopic images. We rotated sequences from an orbiter movie of the Moon by 90 degrees to make Earth appear horizontally displaced in one image with respect to the other. We then used the rotated images as stereoscopic data to render pictures in 3D. However, the result was spatially distorted because the points traversed as the orbiter moved around the Moon differed from the axi-asymmetrical left and right positions of the camera in the ideal convergence configuration. Because these distortions cause discomfort for viewers, we added depth data to the right images that assumed a spherical Moon. We intend to continue our research in this direction to develop methods such as integral imaging and holography that convert 2D data into pictures that appear 3D without the aid of special glasses.