In this section we explain the two methods for seamlessly stitching heterogeneous resolution tiles. This heterogeneity is traced back to the concept of the level of detail (LOD). For the LOD we are relying on the multiresolution nature of DWT from the JPEG2000 codec. The reasons for choosing JPEG2000 are its scalability, its wide acceptance as a standard, its support to various wavelet forms for multiresolution, progressive transfer capability and last but not the least is its approximation paradigm for LOD. The various approximations do not differ in size but only in quality which is additive in nature. If each tile is DWTed in the JPEG2000 codec at level
then a level-
approximate image is the one that is constructed with
percent of the total coefficients. These coefficients correspond to the available lowest
subbands and for the coefficients corresponding to the rest of the
subbands, zeros are stuffed. Subsequent application of
-level inverse DWT yields what is known as the
-approximate image. For example, level-
approximate image is constructed from all the coefficients and level-
approximate image is constructed from
of the count of the initial coefficients.
3.1. Scenario
We are assuming a simplistic scenario of a panorama of
tiles given in Figure 2. This use case should represent almost all the possibilities we will have in our application. An individual tile is large enough to circumscribe the field of view if the observer is close enough to require highest visual resolution. If the observer is far off, his view is limited to at least
tessellation with all the tiles at their lowest visual resolutions but as soon as he draws closer, further details may be added to the tile of focus and, to some extent, its
-neighbors to improve their visual resolution. This leads to variations in the tile qualities of the view tessellation. The tessellation is thus heterogeneous in terms of resolution and a given tile can have one of the three qualities, namely, high, middle and low. The high-quality resolution (H) corresponds to the center which is the viewpoint focus for a given distance to the observer in a flyby process. We are rendering the
-neighbors of the center with the middle-resolution (M) tiles and the rest, that is, the corners, with low resolution (L). The terms high, low and middle are not static and their extent depends on the distance of observer from the view as well as the time passed after the viewer changed his position. It also depends on the capacities of the media employed, as we may have M and L or just L for poorer conditions. There may be a tessellation having concentric square rings with each successive ring having a different quality [7] but we believe that at any instant one would hardly be dealing with a rendering involving more than three resolutions.
3.2. Shape of the Smoothing Function
Figure 3 depicts the shape of an example smoothing function in the simplest of forms. Note that we are affecting the quality of only a small part on the periphery, of the higher resolution tiles in the view, during a given smoothing process. For any given subband, the smoothing function creates a scalar multiplication mask (over
) of the size of the subband. This depends on the position of the tile, to which the subband belongs, in the view. for example, for the subbands of a tile having low resolution neighbors on left/right, we may consider the central column of coefficients as the origin in the figure and
may then be the offset from this central column on right or left with
being half the width of the subband. In a similar fashion, for a tile with up/down neighbors in the view, the origin may be the center row with row offset represented by
. Finally for a tile in the
-neighbor environment, the center of its subband may be the origin and
may be the offset of the polygonal ring of coefficients. The function
has two parameters,
and
. the parameter
decides where to start the smoothing process in the subband whereas
denotes the index where it should fall to zero. To have a full smoothing in a subband we must have
and
, but to dilute the seam, it is preferable to have
and to not deteriorate too much the subband it is preferable to have
. To not have a strong smoothing between
and
and avoiding the full deterioration of the subband, one can locate
at one quarter of the subband, that is,
, where, as already stated,
is roughly half the subband dimension (width, length or radius).
In Figure 3 the given function is of the form,
Of special importance is the shape of the fall from
to
which may be streamlined to emulate a more sophisticated function, for example, tail of a Gaussian distribution. One such function could be:
Note that beyond
we shall be dealing with
consecutive integers as
; therefore
. One can borrow from sinusoidal functions too and a smoothing function that we tested is given below:
There can be many more functions to emulate but due to space limitation we would restrict to the three given in (1)–(3) only.
Let the high, the middle, and the low resolution of our scenario correspond to
,
, and
approximate images, respectively. The quality gap would essentially bring with it the popping artifacts at tile boundaries. To undo that, we are proposing two different DWT domain smoothing methods in the next two sections.
3.3. Method I: Smoothing the Tiles before Stitching
In this section we present a wavelet-domain context-dependent smoothing strategy illustrated in Figure 4. At the corners (labeled L) are the
-approximate images obtained from lowest
subbands with the rest of
subbands being replaced by
s. For the rest of the tiles one can partition their subbands into the following three sets.
-
(i)
The lowest
subbands will remain untouched since they represent the lowest rendered quality.
-
(ii)
The highest
subbands of center tile (H) and
subbands of its
-neighbors (labeled M) are stuffed with
s.
-
(iii)
Last are the remaining
and
subbands of central tile and its
-neighbors, respectively.
It is the third set (or its subset thereof) which will be subjected to the DWT domain treatment with three different smoothing functions on each subband individually. These smoothing functions are the following.
-
(1)
The medium-resolution left/right neighbors of center require the horizontal smoothing function.
-
(2)
The medium-resolution above/below neighbors of center need the vertical smoothing function.
-
(3)
The high-resolution center is treated through the radial smoothing function.
The horizontal smoothing mask would have a central band of rows of
's. The thickness of this band depends on the maximum diameter,
, of the view focus. If the subband belongs to the
resolution level, then the thickness should be around
. Beyond the band of
's, the rows gradually attenuate on both sides approaching
at the periphery, complying anyone of the (1)–(3). An example smoothing function, for the horizontal case and loosely based on (1), is described in the appendix in the form of Algorithm 1. Along similar lines we can get a vertical smoothing mask by transposing the horizontal mask for a given subband. An example illustration of horizontal smoothing between two tiles (a high-resolution tile H and a medium-resolution tile M on right) is given in Figure 5.
Algorithm 1: Example horizontal smoothing function.
Input: dimensions (
) of the DWT domain subband and smoothing parameters (
)
Output: Horizontal
smoothing mask 
(
) begin
(
) set 
(
) set the coefficients of the first and last
rows of
to 
(
) set the coefficients of the
to
rows of
to 
(5) for
do
(6) 
(7) for
do
(8) 
(9) 
(10) end for
(11) end for
(12) end
The radial smoothing mask is a bit tricky. We would have a center core of
's having a diameter of about
for a the subband of the
resolution level. Ideally, this core should be circular with
being its radius, but due to the discrete character of the subband, the shape is at best polygonal. For the sake of simplicity of Algorithm 2 presented in the appendix, we are considering a square core of
's with dimensions
. Each ring of coefficients around the core would gradually attenuate approaching
at the corners (or the
ring to be precise) and around
in the up/down and left/right directions.
Algorithm 2: Example radial smoothing function.
Input: dimensions (
) of the DWT domain subband and smoothing parameters (
)
Output: Radial smoothing square mask
of dimensions (
)
(
) begin
(
) set 
(
) declare an array
with all elements initialized to 
(
) for
do
(5) 
(6) 
(7) end for
(8) declare a
array 
(9) for
do
(10) for
do
(11) if
then
(12) 
(13) 
(14) else
(15) 
(16) 
(17) end if
(18) end for
(19) end for
(20) end
After the smoothing treatment of a subset or all of the third group of subbands and inverse DWT, one can have a seamless visualization of the type given in Figure 4. In our application, we are using the same strategy for the DEM to avoid popping effect on elevation data.
3.4. Method II: Sliding Window Analogy to View Field
For method II we introduce a sliding window analogy, as shown in Figure 6, where the red square/window represent our initial point of focus which had been rendered with high-quality resolution, H. Let the focus changes and the window slides to the position represented by the blue square. The new window may have, at the worst, three different resolutions. This situation is illustrated by the
panorama in the right part of Figure 6. Rendering three different resolutions, side by side, will obviously give rise to nonsmooth transitions at the interface of heterogeneous tiles.
For a 1D viewpoint displacement from H to M or M to L, whether horizontal or vertical, a smooth transition can be realized by following the scheme given in Figure 7. This figure illustrates the DWT domain, H to M seamless smoothing, in the horizontal direction. For H we have all the subbands available but for the M part we have just level-2 subbands available, and six highest-frequency subbands are missing which will be replaced with zero coefficients. This gives rise to two types of combinations. First, the direct concatenation of nonzero subbands with their counterparts from H. Second, the concatenation of each of the zero subbands with the pretreated corresponding subband of H. The pretreatment involves scalar multiplication with a subband-sized mask generated by one of the functions,
, outlined Section 3.2, with
being the column index of the subband coefficient. Note that for a vertical transition, column index will have to be replaced with the row index in the function
. The resultant concatenated H/M subbands, upon inverse DWT, results the required seamlessly smoothed image.
Note that we are affecting the quality of only a small part on the periphery of the tile during a given smoothing process. For any given subband, the smoothing function creates a scalar multiplication mask (over
) of the size of the subband. For example, the H to M horizontal smoothing mask would have a band of columns of
's on the left. The thickness of this band depends on the maximum diameter of the view focus. Beyond the band of
's the columns gradually attenuate toward right, approaching
at the periphery. Along similar lines we can get a vertical smoothing mask for a given subband.
In two dimensions, the diagonal transition from H to L is also to be taken into account. One can think of the 2D version as an operation involving either two horizontal 1D smoothings followed by a vertical 1D smoothing, as shown in Figure 8, or two vertical 1D smoothings followed by a horizontal smoothing. Keeping this in view, these operations can be easily carried out in parallel to avoid time delay, thanks to the involvement of subbands and the independence of blocks as envisioned in the JPEG2000 standard. Taking the argument further, since we have three different resolution tiles, for concatenation we may face three subband combinations as follows.
-
(1)
The corresponding subbands of all of the H, M, and L are nonzero implying direct concatenation according to their localization in space.
-
(2)
With zero subbands of L corresponding to nonzero subbands of M and H, we would follow the smoothing strategy described in Figure 8.
-
(3)
For nonzero subbands of H, corresponding to zero subbands of L and M, we would concatenate the former in the pretreated form with the zero subbands keeping in view their localization in space.
But we believe that if one think at the outset about the diagonal transition by starting the smoothing process from those subbands of M and H whose counterparts are zero in L, the overall smoothing may still be commendable. In addition the closest part of H to L can be programmed to be degraded more than the rest of its interfaces during smoothing.