Seamless Heterogeneous 3D Tessellation via DWT Domain Smoothing and Mosaicking
© Khizar Hayat et al. 2010
Received: 18 September 2009
Accepted: 17 March 2010
Published: 29 April 2010
With todays geobrowsers, the tessellations are far from being smooth due to a variety of reasons: the principal being the light difference and resolution heterogeneity. Whilst the former has been extensively dealt with in the literature through classic mosaicking techniques, the latter has got little attention. We focus on this latter aspect and present two DWT domain methods to seamlessly stitch tiles of heterogeneous resolutions. The first method is local in that each of the tiles that constitute the view, is subjected to one of the three context-based smoothing functions proposed for horizontal, vertical, and radial smoothing, depending on its localization in the tessellation. These functions are applied at the DWT subband level and followed by an inverse DWT to give a smoothened tile. In the second method, though we assume the same tessellation scenario, the view field is thought to be of a sliding window which may contain parts of the tiles from the heterogeneous tessellation. The window is refined in the DWT domain through mosaicking and smoothing followed by a global inverse DWT. Rather than the traditional sense, the mosaicking employed over here targets the heterogeneous resolution. Perceptually, this second method has shown better results than the first one. The methods have been successfully applied to practical examples of both the texture and its corresponding DEM for seamless 3D terrain visualization.
The contemporary geobrowsers—like Google Earth, NASA's World Wind, or Microsoft's Virtual Earth—give little importance to individual client characteristics, and with the low-end clients, the performance is far from being satisfactory. In addition, area coverage by these browsers is slack, if not judicious, in the case of the underdeveloped and developing parts of the world as opposed to their developed counterparts. There is a need to reform all these browsers to cater for the diversity of clients through some scalable data structuring. One obvious choice is to create multiple levels of detail (LOD) by representing the shape at different levels of approximation. This is essential for the terrain tessellation adjustment as a function of the view parameters . For LOD, one can rely on the multiresolution nature of the discrete wavelet transform (DWT). It is better to employ some standard state-of-the-art DWT, like the now widely accepted JPEG2000 standard (The ISO/IEC 15444-1 standard.). In fact JPEG2000 is better than the popular pyramidal resolution representation with JPEG or other format closer to normal graphic cards that are of high storage cost: there is a need to store different resolutions. Additional advantages of the JPEG2000, in this context, include capabilities like better compression and progressive data transfer.
In essence, one can safely deduce that even for today's geobrowsers the tessellations are far from being smooth due to varying resolutions or heterogeneous luminance conditions or even both when the tiles have been acquired. Whilst, the traditional focus of the mosaicking community has been the seamless rendering of tiles that may vary in luminance, little attention has been paid to the heterogeneity due to resolution. That is why traditional mosaicking may correct the light difference phenomenon but the underlying tiles may still be hardly seamless if they vary in resolution. In this paper we focus on the resolution part, mainly, as we try to smoothen the tile interfaces to avoid the artifacts like those in Figure 1, for example. Our methods are aimed at rendering texture and DEM tessellations, from JPEG2000-coded tiles, in a scenario where only visible tiles are rendered and the focused ones, out of these, have the highest quality depending on the viewer's distance. As the viewer moves in, the tiles corresponding to the new view must be rendered and gradually refined in quality as the focus is coming near to it. This refinement is additive, as more and more subbands are added to it, at runtime. The main challenge to this strategy is that the tile must be seamlessly stitched for both the texture and the DEM and the boundaries must be diluted enough to prevent popping artifacts. In this paper, although we are more inclined to the texture, the DEM has also been treated as its integral part. We, therefore, present two different methods to look for a seamless tessellation that is composed of heterogeneous resolution tiles. In fact this work is a continuation of our earlier efforts about the synchronous unification of the DEM/texture pair through data hiding  for scalable 3D visualization. That work is our inspiration for the LOD aspect, since its scalable character can result in various tile qualities for both the DEM and its texture.
The first method is local in nature and relies on context-based smoothing of the tile boundaries. The smoothing is carried out at the subband level in the DWT domain by employing three different smoothing masks. We assume that, at worst, there can be three possible tile resolutions, namely, high, middle, and low. The lowest-resolution tiles of the panorama are not treated while the highest-resolution tile are smoothed by a radial smoothing function. The rest of the tiles are rendered at mid-level resolution. Of these the left/right neighbors of center are smoothed horizontally, while the above/below neighbors are smoothed vertically. The treated tiles can be joined, after each has been subjected to an inverse DWT, to get a smoother tessellation.
In the second method, we take the change in viewpoint focus, analogous to a window-sliding approach. The worst we expect is that the window may come up to be composed of three different tile qualities with the resultant artifacts at tile interfaces. To dilute these artifacts, we treat the tiles at the subband level, in the DWT domain, by employing operations involving suitable subband-sized composite masks, conceived with smoothing and mosaicking in perspective. The resultant composite subbands are subjected to a global inverse DWT to get the final seamless tessellation.
The rest of the paper is arranged as follows. Section 2 gives a brief literature survey on the subject. The presentation of the proposed methods is the theme of Section 3. Application of our methods to a practical worst case example is being demonstrated in Section 4. Section 5 concludes the paper.
2. State of the Art
The process of mosaicking involves the connection of two or more images to get a new wide-area image with no visible seams. In the case of aerial or satellite photographs, a wide screen-view is synthesized by the proportional combination of a set of images. The mosaic is hence commonly used to increase the view field with pasting successively overlapped images onto the same plane or other structures, like a cylinder or a sphere. In order to be efficient, a mosaicking method should be robust to light change, moving objects, image noise and to some extent rotation and zooming . An integral part of the process is image registration which pertains to the overlaying two or more images of the same scene taken at different times, from different viewpoints or by different sensors. The final information is gained from variety of sources like image fusion, change detection and multichannel image restoration. In this context Zitova and Flusser  classify the various approaches according to their nature (area-based and feature-based) and to four basic steps of image registration procedure, namely, feature detection, feature matching, mapping function design and image transformation and resampling. The thrust of the mosaicking methods, in general, is either on the interest point matching or on theoretical corner models. All these, hardly consider the aspect where a resolution image is to be stitched to a considerably lower-resolution image.
Terrain LOD algorithms can be classified , on the basis of the hierarchical structure, into four groups, namely, the triangulated irregular networks (TINs), bin-tree hierarchies, bin-tree regions and tiled blocks. The tessellation, in the last category, is carried out with the square tiles of different resolutions as has been done by Wagner . But the process requires seamless stitching at the boundaries. According to Deb et al. , Wagner's technique is not suited for the terrains with large height variations and that is why they tweak the technique by having the projection on real-time average height of terrain for a client/server terrain streaming to handle heterogeneous clients. The blending factors are calculated on a per-tile basis because of the use of a regular tile structure, thus, reducing the amount of computation. Special techniques based on mipmaps and clipmaps  can be found in the literature. Losasso and Hoppe  break the terrains into geometric clipmaps of varying metric sizes which can be used as LODs through a view-centered hierarchy resulting in a simplified spatial and temporal interlevel continuity. A thresholding scheme based on calculated ground sample distances is proposed by Tsai and Chiu  with a nested LOD system for efficient seamless visualization of large datasets. In order to remove the T-junctions they have put forward a mesh refinement algorithm. Ueng and Chuang  propose a dike structure between two adjacent blocks,for a smooth blending between two meshes of different LOD. This approach is combined by Li et al.  with heuristics to dynamically stitch the tile meshes together seamlessly. For the texture part they use high-resolution aerial photos subjected to the similar LOD mechanism as described for the meshes. Larsen and Christensen  try to avoid popping for low-end users by exploiting the low-level hardware programmability in order to maintain interactive frame-rates. They claim their work as pioneering as far as a smooth LOD implementation in commodity hardware is concerned.
Little attention has been paid to the texture images which require their own LOD structure. There have been proposals, like texture clipmaps , but texture tiling is usually preferred. The multiresolution technique, described by Döllner et al. , presents static LOD terrain models, allowing for combination of multiple large-scale textures of different size. The view-dependent texture management technique of Okamoto et al.  manages multiresolution textures in multiple caching levels—database, main memory and graphics card memory—which the authors claim to be suitable for hardware with limited resources. The approach of Buchholz and Döllner  includes dynamic texture loading for memory management and, combine several textures into atlases for avoiding too much texture switches . Their computationally complex preprocessing part helps them to significantly reduce runtime rendering overhead. Frueh et al.  described an approach to create a specialized texture atlas for building facades and supports efficient rendering for virtual flythroughs. The created atlas is static, and different texture resolutions are not considered. Based on the minimum error boundary cut idea of the image quilting algorithm as well as Efros and Freeman , Somol and Haindl  propose their minimum error path search algorithm. To avoid visible seams,for combining incompatible pieces of texture, they propose to find (possibly irregular) boundaries between the image pieces to minimize the visual error. Based on eccentric image overlapping, the source image is cropped at the prospective interface along the minimum error path and placed over the target background image. The problem is, however, if no good path exists in the error map, visible artifacts may be inevitable. Integer wavelet transform has been applied to DEM and a simple coding algorithm with high efficiency was introduced by Chen and Li . Many references can be found on the subject in the works of Danovaro et al.  and Buchholz and Döllner . A survey about 3D interactive rendering by Pajarola and Gobbetti  is worth reading.
3. The Smoothing Strategies
In this section we explain the two methods for seamlessly stitching heterogeneous resolution tiles. This heterogeneity is traced back to the concept of the level of detail (LOD). For the LOD we are relying on the multiresolution nature of DWT from the JPEG2000 codec. The reasons for choosing JPEG2000 are its scalability, its wide acceptance as a standard, its support to various wavelet forms for multiresolution, progressive transfer capability and last but not the least is its approximation paradigm for LOD. The various approximations do not differ in size but only in quality which is additive in nature. If each tile is DWTed in the JPEG2000 codec at level then a level- approximate image is the one that is constructed with percent of the total coefficients. These coefficients correspond to the available lowest subbands and for the coefficients corresponding to the rest of the subbands, zeros are stuffed. Subsequent application of -level inverse DWT yields what is known as the -approximate image. For example, level- approximate image is constructed from all the coefficients and level- approximate image is constructed from of the count of the initial coefficients.
3.2. Shape of the Smoothing Function
There can be many more functions to emulate but due to space limitation we would restrict to the three given in (1)–(3) only.
Let the high, the middle, and the low resolution of our scenario correspond to , , and approximate images, respectively. The quality gap would essentially bring with it the popping artifacts at tile boundaries. To undo that, we are proposing two different DWT domain smoothing methods in the next two sections.
3.3. Method I: Smoothing the Tiles before Stitching
The medium-resolution left/right neighbors of center require the horizontal smoothing function.
The medium-resolution above/below neighbors of center need the vertical smoothing function.
The high-resolution center is treated through the radial smoothing function.
The horizontal smoothing mask would have a central band of rows of 's. The thickness of this band depends on the maximum diameter, , of the view focus. If the subband belongs to the resolution level, then the thickness should be around . Beyond the band of 's, the rows gradually attenuate on both sides approaching at the periphery, complying anyone of the (1)–(3). An example smoothing function, for the horizontal case and loosely based on (1), is described in the appendix in the form of Algorithm 1. Along similar lines we can get a vertical smoothing mask by transposing the horizontal mask for a given subband. An example illustration of horizontal smoothing between two tiles (a high-resolution tile H and a medium-resolution tile M on right) is given in Figure 5.
Algorithm 1: Example horizontal smoothing function.
(10) end for
(11) end for
The radial smoothing mask is a bit tricky. We would have a center core of 's having a diameter of about for a the subband of the resolution level. Ideally, this core should be circular with being its radius, but due to the discrete character of the subband, the shape is at best polygonal. For the sake of simplicity of Algorithm 2 presented in the appendix, we are considering a square core of 's with dimensions . Each ring of coefficients around the core would gradually attenuate approaching at the corners (or the ring to be precise) and around in the up/down and left/right directions.
Algorithm 2: Example radial smoothing function.
(7) end for
(17) end if
(18) end for
(19) end for
After the smoothing treatment of a subset or all of the third group of subbands and inverse DWT, one can have a seamless visualization of the type given in Figure 4. In our application, we are using the same strategy for the DEM to avoid popping effect on elevation data.
3.4. Method II: Sliding Window Analogy to View Field
Note that we are affecting the quality of only a small part on the periphery of the tile during a given smoothing process. For any given subband, the smoothing function creates a scalar multiplication mask (over ) of the size of the subband. For example, the H to M horizontal smoothing mask would have a band of columns of 's on the left. The thickness of this band depends on the maximum diameter of the view focus. Beyond the band of 's the columns gradually attenuate toward right, approaching at the periphery. Along similar lines we can get a vertical smoothing mask for a given subband.
The corresponding subbands of all of the H, M, and L are nonzero implying direct concatenation according to their localization in space.
With zero subbands of L corresponding to nonzero subbands of M and H, we would follow the smoothing strategy described in Figure 8.
For nonzero subbands of H, corresponding to zero subbands of L and M, we would concatenate the former in the pretreated form with the zero subbands keeping in view their localization in space.
But we believe that if one think at the outset about the diagonal transition by starting the smoothing process from those subbands of M and H whose counterparts are zero in L, the overall smoothing may still be commendable. In addition the closest part of H to L can be programmed to be degraded more than the rest of its interfaces during smoothing.
4. Simulation Results
The results, presented above, have been interesting in the face of the fact that we presented a worst case example tessellation in terms of quality difference as well as DEM size. Whilst the two methods were at par in the case of texture stitching, method II had the edge in the case of DEM. The problem of synchronization may arise if the DEM is different enough to warrant its own LOD. One probable answer to this is the fact that since the LOD is wavelet driven, one can always synchronously unify the DEM texture pairs as explained by Hayat et al.  or even adapt the synchronization  to exclude some subbands from embedding. One can still argue that the seam elimination has been realized at the expense of blurring the better resolution tile. The counter argument to this is the fact that the resultant blur is temporary and as soon as further resolution information for a given tile arrives, one can employ complement of the mask to restore the lost information.
In our method we are using a regular grid. To increase the performance, we would like to employ an irregular grid depending on the DEM resolution. The well-known T-cracks problem will be avoided by the application of the junction technique of Pajarola and Gobbetti . In the continuation of our work we would like to extensively analyze and optimize the parameters of the functions employed for smoothing. Alongside, a better function needs to be discovered to further improve the stitching. The overhead in terms of the DWT and its inverse has to be quantified and optimization must be proposed for practical applications. The ultimate is its extension to real-time environment, with the emphasis on scalability.
This work is in part supported by the Higher Education Commission (HEC), Pakistan, as well as by VOODDO (2008–2011), a project of ANR and the region of Languedoc Roussillon, France.
- Losasso F, Hoppe H: Geometry clipmaps: terrain rendering using nested regular grids. ACM Transactions on Graphics 2004, 23(3):769-776. 10.1145/1015706.1015799View ArticleGoogle Scholar
- Hayat K, Puech W, Gesquière G: Scalable 3D terrain visualization through reversible JPEG2000-based blind data hiding. IEEE Transactions on Multimedia 2008, 10(7):1261-1276.View ArticleGoogle Scholar
- Zagrouba E, Barhoumi W, Amri S: An efficient image-mosaicing method based on multifeature matching. Machine Vision and Applications 2009, 20(3):139-162. 10.1007/s00138-007-0114-yView ArticleMATHGoogle Scholar
- Zitova B, Flusser J: Image registration methods: a survey. Image and Vision Computing 2003, 21(11):977-1000. 10.1016/S0262-8856(03)00137-9View ArticleGoogle Scholar
- Wagner D: ShaderX2: Shader Programming Tips and Tricks with DirectX 9.0, chapter Terrain Geomorphing in Vertex Shader. Wordware Publishing, Plano, Tex, USA; 2003.Google Scholar
- Deb S, Bhattacharjee S, Patidar S, Narayanan PJ: Real-time streaming and rendering of terrains. Proceedings of the 5th Indian Conference Computer Vision, Graphics and Image Processing (ICVGIP '06), 2006, Madurai, India 276-288.View ArticleGoogle Scholar
- Tanner CC, Migdal CJ, Jones MT: The clipmap: a virtual mipmap. Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '98), 1998, New York, NY, USA 151-158.View ArticleGoogle Scholar
- Tsai F, Chiu HC: Adaptive level of detail for large terrain visualization. Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences Congress (ISPRS '08), October 2008, Beijing, China XXXVII: 579-584.Google Scholar
- Ueng MF, Chuang JH: Terrain rendering with view-dependent LOD caching. Proceedings of the 10th International Conference on Artificial Reality and Tele-Existence (ICAT '00), October 2000 10: 58-65.Google Scholar
- Li BY, Liao HS, Chang CH, Chu SL: Visualization for HPC data—large terrain model. Proceedings of the 7th International Conference on High Performance Computing and Grid in Asia Pacific Region (HPCASIA '04), 2004, Washington, DC, USA 280-284.Google Scholar
- Larsen B, Christensen NJ: Real-time terrain rendering using smooth hardware optimized level of detail. Journal of Winter School of Computer Graphics 2003, 11(2):282-289.Google Scholar
- Döllner J, Baumann K, Hinrichs K: Texturing techniques for terrain visualization. Proceedings of the 11th IEEE Visualization Conference (VIS '00), 2000, Washington, DC, USA 227-234.Google Scholar
- Okamoto RM, de Mello FL, Esperança C: Texture management in view dependent application for large 3D terrain visualization. Proceedings of the Spring Simulation Multiconference (SpringSim '08), 2008, New York, NY, USA 641-647.Google Scholar
- Buchholz H, Döllner J: View-dependent rendering of multiresolution texture-atlases. Proceedings of the IEEE Visualization Conference, 2005 215-222.Google Scholar
- Wloka M: ShaderX3: Advanced Rendering with DirectX and OpenGL, chapter Improved Batching via Texture Atlases. Charles River Media, Hingham, Mass, USA; 2004.Google Scholar
- Frueh C, Sammon R, Zakhor A: Automated texture mapping of 3D city models with oblique aerial imagery. Proceedings of the 2nd International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT '04), 2004, Washington, DC, USA 396-403.Google Scholar
- Efros AA, Freeman WT: Image quilting for texture synthesis and transfer. Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '01), 2001, New York, NY, USA 341-346.View ArticleGoogle Scholar
- Somol P, Haindl M: Novel path search algorithm for image stitching and advanced texture tiling. In Proceedings of the 13th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG '05), February 2005, Plzen, Czech Republic. Edited by: Skala V. University of Western Bohemia; 155-218.Google Scholar
- Chen R, Li X: DEM compression based on integer wavelet transform. Geo-Spatial Information Science 2007, 10(2):133-136. 10.1007/s11806-007-0038-8View ArticleGoogle Scholar
- Danovaro E, Floriani L, Magillo P, Puppo E, Sobrero D: Level-of-detail for data analysis and exploration: a historical overview and some new perspectives. Computers & Graphics 2006, 30(3):334-344. 10.1016/j.cag.2006.02.006View ArticleGoogle Scholar
- Pajarola R, Gobbetti E: Survey of semi-regular multiresolution models for interactive terrain rendering. The Visual Computer 2007, 23(8):583-605. 10.1007/s00371-007-0163-2View ArticleGoogle Scholar
- Hayat K, Puech W, Gesquière G: Scalable data hiding for online textured 3D terrain visualization. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME '08) 2008, 217-220.Google Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.