Image-Based Rendering Technology for 3D Virtual Exhibition
Image-based Rendering Technology for 3D Virtual Exhibition
Chu-Song Chen, Yen-Shuo Huang, and Yu-Ting Chen
Institute of Information Science, Academia Sinica, Nankang, Taipei, Taiwan
Abstract
A major issue in a 3D digital archives system is to provide photo-realistic images. In this paper, we introduce technologies useful for generating photo-realistic images for 3D virtual exhibition. In particular, we emphasize on how to integrate some basic technologies, such as panoramas and object movies, to build composite views that consist of both a viewer-centered environment and object-centered image sets in a geometrically consistent way. The presented technology can be applied for digital museum or virtual mall for interactively demonstrating photo-realistic 3D objects.
I. Introduction
The interactive browsing of virtual objects or environments plays a significant role in computer graphics and virtual reality (VR) community, which has received much attention for 3D virtual exhibition in recent years. Two commonly existing techniques for constructing an image-based VR world are the panorama [4][5] and the object movie [4][10], where the former is used for viewer-center applications and the latter is for object-centered applications. Recently, a new technique called concentric mosaics [8] was proposed for view-center applications, which allows users to change their views within a small circle (while a panorama restricts the user to see an environment from a fixed viewing center). However, because this technique requires a large amount of images that are difficult to be efficiently transmitted over internet, it has not been widely adopted yet. In practice, these approaches all provide good interactivities between human and computers. They can provide photo-realistic visual effects and the rendering speeds are independent of the complexity of the scenes or objects.
In this paper, we focus on the discussions of the two popular technologies, panorama and object movie. In particular, we will introduce how a seamless integration of an object movie and a panorama can be achieved in a geometrically consistent way. We will also introduce a technique called manifold panorama [7] that is similar to common panoramas but without needs to put the camera center in a fixed point, which can be applied for distant scanning of long drawings or paintings.
This paper is organized as follows: In Section II, we introduce how to acquire and browse panoramas and object movies. In Section III, we present how to naturally integrate an object into a panorama. Section IV introduces a real-time implementation for manifold mosaic. Finally, Section V gives conclusions and discussions.
II. Acquisition and Browsing of Panoramas and Object Movies
There are two types of techniques panorama acquisition. The first is the multiple-shot approach that stitches a set of images taken from different viewing directions with roughly the same focal center [5]. The second is the single-shot approach, which uses an image device consisting of a camera and a curved mirror for omni-directional image acquisition [1]. The former technique takes much more time for building a panorama, but the image quality of the panorama is high. On the contrary, the latter technique takes less time because only a single shot is needed, but the image quality is low due to limited resolutions. In practice, the former technique is more suitable for building a 3D virtual exhibition environment currently. However, the latter approach has the potential to be very useful if its resolution can be raised in the future.
Browsing a panorama can be implemented by sliding a movable window in the panorama that is stored as a long-stripe image, or by further de-warping this window from a region on the surface of a cylinder (or sphere) to a squared region in a plane. The former browsing mode is faster but can not make a straight line remain to be straight during browsing, the latter mode is slower but can browse the panorama in a geometrically correct way.
Acquisition of an object movie also requires to take images of an object from different viewing directions, but the images taken with such an object-centered manner cannot be stitched into a single image because they are not acquired form a fixed (or roughly fixed) viewing center. In practice, acquisition of object movie can be achieved by using some device designed particularly for this purpose. For example, Figure 1 shows a device that can be used to take an object movie automatically. This object-centered image set of an object movie can be browsed interactively by using many manipulation strategies. A comprehensive introduction and evaluation of these strategies can be found in [10].
Figure 1. An image device that can acquire object movies.
III. Geometrically Consistent Composition of Panoramas and Object Movies
We have developed a technique that can composite an object movie and a panorama in a natural way [6]. This technique allows the users to set a “three-dimensional” Euclidean coordinate system in a “two-dimensional” panorama [2][6]. The users have simply to draw such a coordinate system in a panorama via his/her own perception, our system can then automatically compute the geometrical transformation between this coordinate system and the panorama coordinate system (defined in the panorama center), in the 3D space. Hence, an object movie can be put in the 3D space by referring to the 3D coordinate system specified by the users, and geometrical consistency can be achieved. In addition, this technique can generate planar shadows for an object movie by using the silhouette of each image of the object. An illustration of this technology is given in Figure 2. Details of this method can be found in [6].
IV. Manifold Mosaic and Its Real-time Implementation
Manifold mosaic was proposed by Peleg et al [7]. This technique can enlarge the field of view of a camera by stitching slices (or stripes) of the images contained in a video. It can also be treated as a “distant scanning” of a scene. We have developed an integrated strategy for creating manifold mosaic in real time from videos acquired by panning a camera. This technique exploits a robust motion-estimation strategy that is a modification of the least-medians-of-squares approach [9]. To achieve a real-time system, a fast algorithm for template matching [3] was employed, which can considerably reduce the time for finding the best matching positions. This technique can be used for distant scanning of a long painting, so that the painting can be digitized. Some illustrations of this technique are given in Figure 3.
(a) (b)
(c)
Figure 2. Illustrations of natural compositions of an object movies and a panorama.
(a)
(b)
Figure 3. Images obtained from real-time distant scanning by using the manifold-mosaic technique.
V. Conclusions and Discussions
In this paper, we introduced some image-based rendering technologies that are useful for 3D virtual exhibition for digital museums. Currently, the browsing efficiency is an issue worthy of being addressed. Since an object movie contains multiple images, the data amount is huge. When the resolution of each image is high, the browsing delay becomes large. Hence, how to compactly encode an object movie is an important problem, which is one of our on-going research works.
In the future, we plan to develop augmented reality technologies for exhibiting object movies with head-mounted displays, so that the object movies can be presented in a more immersive way. We will also employ the manifold-mosaic technology for flattening the textures on the surfaces of 3D objects, so that the textures can be completely printed and recorded.
References
[1]. S. Baker and S. K. Nayar, “A Theory of Catadioptric Image Formation,” Proceedings of International Conference on Computer Vision, ICCV’98, pp. 35-42, 1998.
[2]. C. S. Chen and W. T. Hsieh, “Composition of 3D Graphic Objects and Panoramas” Proceedings of International Conference on Artificial Reality and Tele-existence, ICAT2000, pp. 207-214, October 2000.
[3]. Y. S. Chen, Y. P. Hung, C. S. Fuh, “Fast Block Matching Algorithm Based on the Winner-Update Strategy,” IEEE Transactions on Image Processing, Vol. 10, No. 8, pp. 1212--1222, Aug. 2001.
[4]. S. E. Chen, “QuickTime VR – an image-based approach to virtual environment navigation,” Proc. Computer Graphics (SIGGRAPH 95), ACM Press, Los Angeles, Calif., 1995, pp. 29–38.
[5]. H. C. Huang, and Y. P. Hung, “Panoramic Stereo Imaging System with Automatic Disparity Warping and Seaming,” Graphical Models and Image Processing, Vol. 60, No. 3, May 1998, pp. 196-208.
[6]. Y. P. Hung, C. S. Chen, Y. P. Tsai, S. W. Lin, “Augmenting Panoramas with Object Movies by Generating Novel Views with Disparity-Based View Morphing” to appear in Journal of Visualization and Computer Animation -- Special Issue on Hallucinating the Real World from Real Images.
[7]. S. Peleg and J. Herman, “Panoramic Mosaics by Manifold Projection,” IEEE International Conference on Computer Vision and Pattern Recognition, CVPR'97, pp. 338-343, June 1997.
[8]. H. Y. Shum and L. W. He, “Rendering with Concentric Mosaics,” Proceedings of SIGGRAPH’99, pp. 299-306, 1999.
[9]. C. V. Stewart, “Robust parameter estimation in computer vision,” SIAM Review, vol. 41, pp. 513-537, 1999.
[10]. C. H. Yu, Y. P. Tsai, C. S. Chen, and Y. P. Hung, “Interactive Object Manipulation Strategies for Browsing Object Centered Image Set,” Proceedings of 2002 Computer Graphics Workshop, Tai-Nan, Taiwan, June 2002.