Takaaki Endo, Japan | ISBN: | Submitted: Jun 30, 1998

Image-Based Walk-Through System for Large-Scale Scenes

1. Introduction Recently, there are increasing needs to digitize existing large-scale scenes, such as Shirakawa-go (designated as World Heritage). In most cases, 3-D geometrical model rendering is adopted to generate 3-D virtual spaces. However, when building large-scale virtual spaces, this methodology cannot yield satisfactory quality of images under real-time conditions. Another promising methodology to generate 3-D virtual spaces is the methodology of image-based rendering. Lippman's "Aspen Movie Maps" was a pioneering system which allowed the user to move along a path pre-recorded on the laser disk and to select an arbitrary path at traffic junctions. However, in this system, the user was only allowed to see image sequences taken by a cameraman, and could not even look around. We have developed a novel image-based walk-through system for large-scale scenes, where the user can move and look around rather freely.

2. Data capturing system Images of existing large-scale scenes are taken by vehicle-mounted cameras. We use eight video cameras, each facing a different direction. To synthesize realistic renderings of scenes from an arbitrary viewpoint, position and orientation information are necessary. For the image capturing system, we adopted a kinematic GPS receiver as a position sensor, and adopted a gyro sensor and geomagnetic sensor as orientation sensors.

3. Image-based walk-through system First, eight images recorded at nearly the same location are integrated into a panoramic image. When the user looks around, corresponding sections of the panoramic image are displayed, and when the user moves, the appropriate panoramic image set is selected from the image database. By using RAID disk, and by using a multi-thread technique to separate the drawing procedure from hard disk access, we can draw 10 panoramic images per second (1.5 megabytes per panorama). In the prototype system, the user can move inside several hundred square meters of area (figure). In order to synthesize realistic renderings of scenes from an arbitrary viewpoint, we added a morphing technique to the prototype system. First, images are divided into regions by specifying the corresponding region of 3 original images. Then, the user can move freely inside the triangle area (whose nodes are view points of the 3 original images), by interpolating position and color among the corresponding regions of each original image. By changing the 3 original images appropriately according to user's view point, the area where the user can walk-through could be extended. Also, we can look around freely by selecting another 3 original images of appropriate direction. The merit of the panorama-based method is the simplicity to generate virtual spaces, and the merits of the morphing method are the freedom to move and the small size of the image database. By combining the two methods, we can efficiently generate image-based walk-through system. In places like a road, where it may be sufficient to move along a path, we adopted the panoramic image based method, and in places like a plaza, where the user may want to move freely, we adopted the morphing method.

4. Conclusion In this paper, we introduced a new image-based walk-through method for implementing large-scale virtual spaces. Vehicle-mounted cameras were used to capture images of existing large-scale scenes. By combining a panorama-based method and a morphing method, we can rather freely walk-through a photo-realistic large-scale virtual space.

[more information]

[other authors]
Akihiro KATAYAMA, Hideyuki TAMURA, Michitaka HIROSE, Tomohiro TANIKAWA, and Makoto SAITO

[keywords]
Large-Scale Virtual Space, Image-Based Rendering, Panoramic Image, Morphing, Vehicle-Mounted Cameras, Position Sensor, Orientation Sensor