Rate distortion performance analysis of nested sampling and coprime sampling

Abstract

In this paper, rate distortion performance of nested sampling and coprime sampling is studied. It is shown that with the increasing of distortion, the data rate decreases. With these two sparse sampling algorithms, the data rate is proved to be much less than that without sparse sampling. With the increasing of sampling spacings, the data rate decreases at certain distortion, which is because with more sparse sampling, less number of bits is required to represent the information. We also prove that with the same sampling pairs, the rate of nested sampling is less than that of coprime sampling at the same distortion. The reason is that nested sampling collects a little less number of samples than coprime sampling with the same length of data, which is a little sparser than coprime sampling.

1 Introduction

The twenty-first century is awash with data. Data are flooding in at rates never seen before, doubling almost every 18 months [1], as result of new information gathered from Radar, Web communities, newly deployed smart assets, and customer data from public, proprietary, purchased sources, and so forth. For example, oil companies, telecommunication companies, and other data-centric industries have had huge data for long time. Data is being collected and transmitted at unprecedented scale [2, 3] in a wide range of application areas nowadays. The phrase ‘Big Data’ as defined by US National Science Foundation in its recent solicitation, refers to large, diverse, complex, distributed data sets generated from instruments, sensors, Internet transactions, email, video, click streams, and all other digital sources available today and in the future.

Unstructured data is data that does not follow a specified format, which is really most of the Big Data. Radar or sonar data is one typical example, which includes meteorological, vehicular, and oceanographic seismic information, such as in [4], Big Data from O’Reilly radar was described.

And many efforts have been made to develop suitable compression techniques for Big Data. However, traditional compression methods [5, 6] are all based on Nyquist rate, which will have poor efficiency in terms of both sampling rate and computational complexity. Unlike traditional compression techniques, some sparse sampling algorithms have been proposed to overcome Nyquist sampling requirement, like compressive sensing, nested sampling, and coprime sampling.

Nested sampling [7] is an non-uniform sampling, using two different samplers in each period. Although the signal is sampled sparsely and non-uniformly, the autocorrelation of signal could be estimated at all lags. Therefore, although the samples can be arbitrarily sparse, it keeps the signal’s statistical information [8]. While coprime sampling uses two uniform samplers, with sample spacings P and Q coprime integers. The authors in [8] have already proved that these two sets of samples of the signal could fully estimate all lags of autocorrelation of the original signal. As both nested sampling and coprime sampling could keep the statistical property of the original signal, these two sampling algorithms could be applied to Big Data to highly reduce the transmission or storage cost of Big Data.

Information rate distortion function is a measure of distortion between the original source and its representation. In this paper, we will provide theoretical rate distortion performance, because of these two sparse sampling algorithms, either nested sampling (NS) or coprime sampling (CS). We will show that with these two sparse sampling algorithms, the data rate is much less than that without sparse sampling for a given distortion. With the increasing of sampling spacings, the data rate decreases at certain distortion, which is because with more sparse sampling, less number of bits is required to represent the information. We will also prove that with the same sampling pairs, the rate of nested sampling is less than that of coprime sampling at the same distortion.

The rest of this paper is organized as follows. In Section 2, we give a brief introduction of nested sampling and coprime sampling separately. Theoretical derivation of rate distortion performance of nested sampling and coprime sampling is detailed in Sections 3.1 and 3.2. Also, the theoretical analysis and comparison of these two sparse sampling is given in Section 3.3. In Section 4, numerical results are provided to verify the theoretical rate distortion results derived in Section 3. Conclusions are given in Section 5.

2 Preliminaries

2.1 Nested sampling

The nested array was first introduced in [7] as an effective approach to array processing with enhanced degrees of freedom [9, 10]. In time domain, the signal’s autocorrelation could also be obtained from nested sampling structure [11]. And although the samples from this nested sparse sampling are sparsely and non-uniformly located, the samples of the autocorrelation can be computed at any specified rate. Some applications which depend on the difference co-array, or autocorrelation, like Direction-of-arrival (DOA) estimation and beamforming could be done based on nested sampling.

In the simplest form, there are two levels of sampling density in nested sampling [11–14], with the level 1 samples at the N₁ locations and the level 2 samples at the N₂ locations.

$$1\le l\le N_1,\text{for level 1}$$

$$(N_1+1)m,1\le m\le N_2,\text{for level 2}$$

An example of periodic sparse sampling using nested sampling structure is shown in Figure 1, with N₁=3 and N₂=5. The cross differences are given by

$$k=(N_1+1)m-l,1\le m\le N_2,1\le l\le N_1.$$

(1)

Figure 1

Nested sampling with N ₁ = 3, N ₂ = 5.

The cross differences (2) are in the following range with the maximum value (N₁+1)N₂−1 [9, 11], except the integers and the corresponding negated versions shown in (3).

$$\begin{array}{l}-\left[\right(N_1+1)N_2-1]\le k\le \left[\right(N_1+1)N_2-1]\end{array}$$

(2)

$$\begin{array}{l}(N_1+1),2(N_1+1),\cdots ,(N2-1)(N_1+1)\end{array}$$

(3)

For example, consider the example in Figure 1, where N₁=3, N₂=5, 1≤m≤5 and 1≤l≤3, the cross differences k=(N₁+1)m−l will achieve these values

$$1,2,3,\left(\right),5,6,7,\left(\right),9,10,11,\left(\right),13,14,15,\left(\right),17,18,19.$$

The difference 0 is also missing, for the reason that m and l are nonzero. Meanwhile, we notice that all of the missing differences could be covered by the self differences among the second array,

$$(N_1+1)(m_1-m_2),1\le m_1,m_2\le N_2.$$

(4)

Combining the cross differences and the self-differences is the difference-co-array, which is a filled difference co-array as shown in (2). This indicates that using nested array structure, although sparse samples are obtained, the degrees of freedom is enhanced,

$$2\left[\right(N_1+1)N_2-1]+1=2(N_1+1)N_2-1.$$

(5)

Based on the principle above, a sparse non-uniform sampling using nested sampling structure could be performed as in Figure 1. There are two levels of nesting, with N₁ level-1 samples and N₂ level-2 samples in each period, with period (N₁+1)N₂. It is obvious that nested sampling is non-uniform and the samples obtained are very sparse.

We could notice that, in (N₁+1)N₂T seconds, there are totally N₁+N₂ samples. Therefore, the average sampling rate is

$$f_{s,\text{nested}}=\frac{N_1+N_2}{(N_1+1)N_{2}T}\approx \frac{1}{N_{1}T}+\frac{1}{N_{2}T}<\frac{1}{T}$$

(6)

Here, T=1/f _n , f _n ≥2 f_max is the Nyquist sampling frequency. As the Nyquist sampling rate is 1/T, the average sampling rate of nested sampling is smaller than the conventional Nyquist sampling rate.

2.2 Co-prime sampling

Different from nested sampling, coprime sampling involves two sets of uniformly spaced samplers [8, 12–14] as shown in Figure 2, each of them are sparsely sampled.

Figure 2

Coprime sampling.

The coprime sampling uniformly sample the source signal using two sub-Nyquist samplers, with sample spacing PT and QT respectively, where P and Q are coprime integers with P<Q. 1/T Hz is the Nyquist rate for a bandlimited process, i.e., 1/T=2 f_max, f_max being the highest frequency.

$$x\left(n\right)=x_c\left(\mathit{\text{nT}}\right).$$

(7)

Consider the product

$$x\left({\mathit{\text{Pn}}}_1\right)x^{\ast }\left({\mathit{\text{Qn}}}_2\right),$$

(8)

where x(Pn₁) and x(Qn₂) comes from the first and the second sampler. Let the difference as

$$k={\mathit{\text{Pn}}}_1-{\mathit{\text{Qn}}}_2.$$

(9)

It has been shown that k can achieve any integer value in the range 0≤k≤P Q−1 in [11], if n₁ and n₂ are in the ranges 0≤n₁≤2Q−1 and 0≤n₂≤P−1.

For coprime sampling, the two samplers totally collect P+Q samples in PQT seconds, so the average sampling rate is

$$f_{s,\text{coprime}}=\frac{P+Q}{\mathit{\text{PQT}}}=\frac{1}{\mathit{\text{PT}}}+\frac{1}{\mathit{\text{QT}}}<\frac{1}{T}.$$

(10)

Same as in nested sampling, T=1/f _n , f _n ≥2 f_max is the Nyquist sampling frequency. It is obvious the average sampling rate of coprime sampling is also much smaller than the conventional Nyquist sampling rate. However, the signal’s second-order statistics, like the autocorrelation, is kept, which allows us to sample a signal sparsely and estimate some aspects of the signal (spectra, DOA, and so on) at a significantly higher resolution.

3 Rate distortion performance

Information rate distortion function is a measure of distortion between the original source and its representation. Our purpose is to construct a distortion function which can measure the distortion because of these two sparse sampling algorithms, either nested sampling (NS) or coprime sampling (CS). Sparse sampling can cause possible distortion because less number of samples are used. A wide variety of distortion functions, such as Euclidean distance, Hamming distance, Mahalanobis distance, and Itakura-Saito distance have been used. In this paper, squared error distortion is used. The original samples are denoted as x _i ,i=1,⋯,L, where L is the total number of samples. Assume that all original information from L samples is X^L= [ x₁,x₂,⋯,x _L ], the selected information after sparse sampling can be represented as [15]

$${\widehat{X}}^{L^{\prime }}=S\left(X^L\right),$$

(11)

where S(·) denotes sparse sampling, either nested sampling or coprime sampling. ${\widehat{X}}^{L^{\prime }}=[{\widehat{x}}_1,{\widehat{x}}_2,\cdots ,{\widehat{x}}_{L^{\prime }}]$ and L^′<L. The distortion associated with the sparse sampling between all original samples and the selected samples is

$$D=\mathit{\text{Ed}}(X^L,{\widehat{X}}^{L^{\prime }}),$$

(12)

where d(·) is the distortion function.

The expectation in (12) is with respect to the probability distribution on X^L. The rate distortion function R(D) is the minimum of data rates R such that (R,D) is in the rate distortion region for a given distortion. From [16, 17], we know that information rate distortion function is defined as

$$R\left(D\right)=\underset{\mathit{\text{Ed}}(X^L,{\widehat{X}}^{L^{\prime }})\le D}{min}I(X^L;{\widehat{X}}^{L^{\prime }}),$$

(13)

where $I(X^L;{\widehat{X}}^{L^{\prime }})$ is the mutual information between X^L and ${\widehat{X}}^{L^{\prime }}$.

$$\begin{array}{ll}I(X^L;{\widehat{X}}^{L^{\prime }})& =H\left(X^L\right)-H\left(X^L\right|{\widehat{X}}^{L^{\prime }})\\ =H\left(X^L\right)-H(X^L-{\widehat{X}}^{L^{\prime }}|{\widehat{X}}^{L^{\prime }})\\ \stackrel{\left(a\right)}{\ge }H\left(X^L\right)-H(X^L-{\widehat{X}}^{L^{\prime }}),\end{array}$$

(14)

where inequality (a) follows from the fact that condition reduces the entropy.

From formula (13), we know that

$$\mathit{\text{Ed}}(X^L,{\widehat{X}}^{L^{\prime }})\le D$$

(15)

For squared error distortion,

$$\begin{array}{ll}\mathit{\text{Ed}}(X^L,{\widehat{X}}^{L^{\prime }})& =E{(X^L-{\widehat{X}}^{L^{\prime }})}^2\\ =\sum E{(x_i-{\widehat{x}}_j)}^2\\ \stackrel{\left(b\right)}{=}\sum _k{D}_k\le D,\end{array}$$

(16)

where i=1,⋯,L and j=1,⋯,L^′, and (b) follows from the definition that $E{(x_i-{\widehat{x}}_j)}^2=D_k$.

Since Gaussian assumption is a classical modeling assumption heavily used in areas such as signal processing and communication system [18], from [16], the rate distortion function for a single Gaussian source N(0,σ²) with squared error distortion is

$$R\left(D\right)=\left\{\begin{array}{cc}\frac{1}{2}log\frac{{\sigma }^2}{D}& 0\le D\le {\sigma }^2,\\ 0& D>{\sigma }^2\end{array}\right.$$

(17)

For L-independent zero mean Gaussian sources x₁,⋯,x _L with variance ${\sigma }_1^2,{\sigma }_2^2,\cdots ,{\sigma }_L^2$, the rate distortion performance with squared error distortion is given by [16, 17, 19, 20]

$$R\left(D\right)=\sum _{i=1}^L\frac{1}{2}log\frac{\underset{i}{\overset{2}{\sigma }}}{D_i},$$

(18)

where

$$D_i=\left\{\begin{array}{c}\lambda \text{if}\lambda <{\sigma }_i^2,\\ {\sigma }_i^2\text{if}\lambda \ge {\sigma }_i^2\end{array}\right.$$

(19)

where λ is chosen so that $\sum _i^L{D}_i=D$, and $D_i=E{(x_i-{\widehat{x}}_i)}^2$. This gives rise to a kind of reverse waterfilling. We choose a constant λ and only describe those random variables with variance greater than λ, and no bits are used to describe random variables with variance less than λ.

3.1 For nested sampling

Theorem 1

(Rate distortion for nested sampling of Gaussian source) Let $x_i\sim N(0,{\sigma }_i^2)$, i=1,2,⋯,L, be independent Gaussian random variables, and under squared error distortion. The rate distortion between the original Gaussian source and after nested sampling of these Gaussian random variables is given by

$$R_{\text{NS}}\left(D\right)=\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}-\sum _{k=1}^{K_{\text{NS}}}\frac{1}{2}log2\mathrm{\pi e}{D}_k,$$

(20)

where K_NS is given in (24) and

$$D_k=\left\{\begin{array}{l}\lambda \mathit{\text{if}}\lambda <{\sigma }_k^2,\\ \frac{K_{\text{NS}}}{L}{\sigma }_k^2\mathit{\text{if}}\lambda \ge {\sigma }_k^2.\end{array}\right.$$

(21)

where λ is chosen so that $\sum _{k=1}^{K_{\text{NS}}}{D}_k=D$.

Proof 1

For nested sampling (NS), all L original information is X^L= [ x₁,x₂,⋯,x _L ].

And less number of samples L^′ will be selected based on nested sampling as described,

$$\begin{array}{ll}{\widehat{X}}_{\text{NS}}^{L^{\prime }}& =[{\widehat{x}}_1,{\widehat{x}}_2,\cdots ,{\widehat{x}}_{L^{\prime }}]\\ =[x_1,\cdots ,x_{N_1},x_{(N_1+1)},\cdots ,x_{N_2(N_1+1)},\cdots ]\end{array}$$

(22)

Therefore, (16) becomes

$$\mathit{\text{Ed}}(X^L,{\widehat{X}}_{\text{NS}}^{L^{\prime }})=\sum _{k=1}^{K_{\text{NS}}}{D}_k,$$

(23)

where the length of K_NS could be determined based on the following formula, here we assume Y=L(mod (N₁+1)N₂)

$$K_{\text{NS}}=\left\{\begin{array}{c}\lfloor \frac{L}{(N_1+1)N_2}\rfloor N_1(N_2-1),\mathit{\text{if}}Y\le (N_1+1)\\ \lfloor \frac{L}{(N_1+1)N_2}\rfloor N_1(N_2-1)+Z,\mathit{\text{if}}Y>(N_1+1),\end{array}\right.$$

(24)

where

$$Z=\lfloor (\frac{Y}{N_1+1}-1)\rfloor \ast N_1+U$$

(25)

in which U=(Y−(N₁+1))(mod (N₁+1)).

If all samples are assumed to be independent Gaussian $N(0,{\sigma }_i^2)$, hence, the corresponding rate distortion function for nested sampling will be

$$\begin{array}{ll}{R}_{\text{NS}}\left(D\right)& =\underset{\mathit{\text{Ed}}(X^L,{\widehat{X}}_{\text{NS}}^{L^{\prime }})\le D}{min}I(X^L;{\widehat{X}}_{\text{NS}}^{L^{\prime }})\\ \ge \underset{\mathit{\text{Ed}}(X^L,{\widehat{X}}_{\text{NS}}^{L^{\prime }})\le D}{min}H\left(X^L\right)-H(X^L-{\widehat{X}}_{\text{NS}}^{L^{\prime }})\\ \stackrel{\left(c\right)}{\ge }\underset{{\sum }_{k=1}^{K_{\text{NS}}}{D}_k=D}{min}H\left(X^L\right)-H\left(N\right(0,E{(X^L-{\widehat{X}}_{\text{NS}}^{L^{\prime }})}^2)\\ =\underset{{\sum }_{k=1}^{K_{\text{NS}}}{D}_k=D}{min}\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}-\sum _{k=1}^{K_{\text{NS}}}\frac{1}{2}log2\mathrm{\pi e}{D}_k,\end{array}$$

(26)

where inequality (c) follows from the fact that the normal distribution maximizes the entropy for a given second moment, and ${\sum }_{k=1}^{K_{\text{NS}}}{D}_k=D$.

To find the minimum value, we could use Lagrange multipliers

$$J\left(D\right)=\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}-\sum _{k=1}^{K_{\text{NS}}}\frac{1}{2}log2\mathrm{\pi e}{D}_k+\sum _{k=1}^{K_{\text{NS}}}{D}_k$$

(27)

and differentiating with respect to D _k and setting equal to 0, we have

$$\frac{\mathrm{\partial J}}{\partial D_k}=-\frac{1}{D_{k}2ln2}+\lambda =0$$

(28)

or

$$D_k=\frac{1}{\lambda 2ln2}={\lambda }^{\prime }$$

(29)

which results in an equal distortion for each random variable, if the constant λ^′ is less than ${\sigma }_i^2$ for all i. As the increase of the total allowable distortion D, the constant λ^′ increases until it exceeds ${\sigma }_i^2$ for some i. Kuhn-Tucker conditions could be used to find the minimum in (26) if we increase the total distortion D. In this case, the Kuhn-Tucker conditions yield

$$\frac{\mathrm{\partial J}}{\partial D_k}=-\frac{1}{D_{k}2ln2}+\mathrm{\lambda .}$$

(30)

Therefore,

$$D_k=\left\{\begin{array}{c}\lambda \mathit{\text{if}}\lambda <{\sigma }_k^2\\ \frac{K_{\text{NS}}}{L}{\sigma }_k^2\mathit{\text{if}}\lambda \ge {\sigma }_k^2,\end{array}\right.$$

(31)

where λ is chosen so that ${\sum }_{k=1}^{K_{\text{NS}}}{D}_k=D$.

3.2 For coprime sampling

Theorem 2.

(Rate distortion for coprime sampling of Gaussian source) Let $x_i\sim N(0,{\sigma }_i^2)$, i=1,2,⋯,L, be independent Gaussian random variables, and under squared error distortion. The rate distortion between the original Gaussian source and after coprime sampling of these Gaussian random variables is given by

$$R_{\text{CS}}\left(D\right)=\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}-\sum _{k=1}^{K_{\text{CS}}}\frac{1}{2}log2\mathrm{\pi e}{D}_k,$$

(32)

where K_CS is given in (36) and

$$D_k=\left\{\begin{array}{c}\lambda \mathit{\text{if}}\lambda <{\sigma }_k^2\\ \frac{K_{\text{CS}}}{L}{\sigma }_k^2\mathit{\text{if}}\lambda \ge {\sigma }_k^2,\end{array}\right.$$

(33)

where λ is chosen so that $\sum _{k=1}^{K_{\text{CS}}}{D}_k=D$.

Proof 2.

For coprime sampling (CS), we still assume the original information with length L, i.e., X^L=[ x₁,x₂,⋯,x _L ].

And based on coprime sampling, less number of samples L^′′ will be selected,

$$\begin{array}{ll}{\widehat{X}}_{\text{CS}}^{L^{\mathrm{\prime \prime }}}& =[{\widehat{x}}_1,{\widehat{x}}_2,\cdots ,{\widehat{x}}_{L^{\mathrm{\prime \prime }}}]\\ =[x_P,x_Q,x_{2P},x_{2Q},\cdots ].\end{array}$$

(34)

Similarly, (16) becomes

$$\mathit{\text{Ed}}(X^L,{\widehat{X}}_{\text{CS}}^{L^{\mathrm{\prime \prime }}})=\sum _{k=1}^{K_{\text{CS}}}{D}_k,$$

(35)

where the length of K_CS could be determined based on the following formula

$$K_{\text{CS}}=L-\lfloor \frac{L}{P}\rfloor -\lfloor \frac{L}{Q}\rfloor +\lfloor \frac{L}{\mathit{\text{PQ}}}\rfloor .$$

(36)

Therefore, the corresponding rate distortion function for coprime sampling of independent Gaussian source $N(0,{\sigma }_i^2)$ is

$$\begin{array}{ll}{R}_{\text{CS}}\left(D\right)& =\underset{\mathit{\text{Ed}}(X^L,{\widehat{X}}_{\text{CS}}^{L^{\mathrm{\prime \prime }}})\le D}{min}I(X^L;{\widehat{X}}_{\text{CS}}^{L^{\mathrm{\prime \prime }}})\\ \ge \underset{\mathit{\text{Ed}}(X^L,{\widehat{X}}_{\text{CS}}^{L^{\mathrm{\prime \prime }}})\le D}{min}H\left(X^L\right)-H\left(N\right(0,E{(X^L-{\widehat{X}}_{\text{CS}}^{L^{\mathrm{\prime \prime }}})}^2)\\ =\underset{\sum D_k=D}{min}\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}-\sum _{k=1}^{K_{\text{CS}}}\frac{1}{2}log2\mathrm{\pi e}{D}_k.\end{array}$$

(37)

The minimum value could be obtained using the similar procedure as described in nested sampling.

3.3 Theoretical analysis

Without sparse sampling, the rate distortion function would be

$$\begin{array}{ll}{R}_{\text{WSS}}\left(D\right)& =\underset{\mathit{\text{Ed}}(X^L,X^L)\le D}{min}I(X^L;X^L)\\ \ge H\left(X^L\right)-H\left(N\right(0,E{(X^L-X^L)}^2)\\ =\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}-0\\ =\sum _{i=1}^L\frac{1}{2}log2\mathrm{\pi e}\underset{i}{\overset{2}{\sigma }}\end{array}$$

(38)

which is much greater than that with sparse sampling.

From the above derivation of rate distortion function of nested sampling and coprime sampling, we could notice that if the sampling spacings are assumed to be the same, i.e., N₁=P and N₂=Q for these two sparse sampling methods, then the minimum value of $K_{{\text{NS}}_{\text{min}}}$ could be achieved when Y=L(mod (N₁+1)N₂)=0, therefore

$$K_{{\text{NS}}_{\text{min}}}=\frac{N_1(N_2-1)L}{(N_1+1)N_2}=\frac{P(Q-1)L}{(P+1)Q}=\frac{(P^{2}Q-P^2)L}{\mathit{\text{PQ}}(P+1)}.$$

(39)

While for coprime sampling, the minimum value of $K_{{\text{CS}}_{\text{min}}}$ could be achieved when L (mod P)=0, L(mod Q)=0, and L(mod P Q)=0, therefore

$$K_{{\text{CS}}_{\text{min}}}=\frac{(P-1)(Q-1)L}{\mathit{\text{PQ}}}=\frac{(P^{2}Q-P^2-Q+1)L}{\mathit{\text{PQ}}(P+1)}.$$

(40)

As we know that for these two sparse sampling algorithms, the sampling interval is for sure greater than Nyquist sampling spacing, which indicates that Q>1, therefore,

$$K_{{\text{NS}}_{\text{min}}}>K_{{\text{CS}}_{\text{min}}}$$

(41)

which indicates that in most cases, K_NS>K_CS. Table 1 shows some example of K_NS and K_CS with respect to sampling intervals when N₁=P, N₂=Q, and L=1,000. It is clear that with the increase of sampling spacings, samples are selected more sparsely by both nested sampling and coprime sampling, which results in a increase of K_NS and K_CS. In addition, we could notice that K_NS>K_CS as proved.

Table 1 K _NS and K _CS with respect to sampling intervals when N ₁ = P , N ₂ = Q , and L =1,000

With our assumption that all samples are independent Gaussian $N(0,{\sigma }_i^2)$, we could conclude that

$$R_{\text{NS}}\left(D\right)<R_{\text{CS}}\left(D\right)<R_{\text{WS}}\left(D\right)$$

(42)

which indicates that both nested sampling and coprime sampling use less number of bits to describe the information compared that without sparse sampling (WS).

As we know from the introduction part, in (N₁+1)N₂T seconds, there are totally N₁+N₂ samples for nested sampling, while coprime sampling totally collect P + Q samples in PQT seconds. If the sampling intervals are the same, i.e., N₁=P and N₂=Q, it is obvious that nested sampling is a little sparser than coprime sampling method. R_NS(D)<R_CS(D) is because nested sampling collects a little less number of samples than coprime sampling with the same length L of data. The rate R(D) at a given distortion for both sparse sampling algorithms is less than that without sparse sampling. The reason is that with sparse sampling, less number of bits is used to describe the original information.

4 Numerical results

The total length of the information is set to be L=1,000. Each sample is assumed to follow a Gaussian distribution N(0,1) with zero mean and unit variance. We also assume D _k =λ<σ²=1, which is equal distortion for each random variable.

Figure 3 shows the rate distortion performance of nested sampling with different sampling spacings. It is clear that with the increasing of distortion, the rate decreases. When the sampling intervals N₁ and N₂ becomes larger, i.e., less samples are acquired, the rate becomes smaller. For example, when D=0.3, N₁=3,N₂=5, the data rate R(D)≈1,350, while with the increase of sampling pairs to N₁=3,N₂=11, then R(D)≈1220, which is much smaller. This is because with more sparse sampling, less number of bits is required to represent the information.

Figure 3

Rate distortion performance-nested sampling.

The rate distortion performance of coprime sampling with different sampling spacings is shown in Figure 4. Similarly as nested sampling, with the increasing of distortion, the rate R(D) decreases. When the sampling intervals P and Q becomes larger, the rate becomes smaller.

Figure 4

Rate distortion performance-coprime sampling.

Figure 5 compares the rate distortion performance between nested sampling and coprime sampling, where D is the distortion between the original source and its sparse-sampled representation, and R(D) is the corresponding rate at a particular distortion D. With the same sampling spacings chosen, N₁=P, and N₂=Q, at the same distortion, the rate of nested sampling is less than that of coprime sampling. For example, when N₁=P=3, and N₂=Q=17, when D=0.3, the rate for nested sampling is R_NS(D)≈1,200, while the rate for coprime sampling is R_CS(D)≈1,300. This verifies the result that R_NS(D)<R_CS(D), because nested sampling collects a little less number of samples than coprime sampling with the same length L of data, which is a little sparser than coprime sampling.

Figure 5

Comparison of nested sampling and coprime sampling.

5 Conclusions

Information rate distortion function is a measure of distortion between the original source and its representation. Our purpose in this paper is to construct a distortion function which can measure the distortion because of these two sparse sampling algorithms. Information theoretical rate distortion performance for these two sparse sampling methods, nested sampling and coprime sampling, is studied in this paper. It is showed that with these two sparse sampling algorithms, the data rate is proved to be much less than that without sparse sampling at a given distortion. With the increasing of sampling spacings, i.e., data are more sparsely acquired, the data rate decreases at certain distortion. This is because with sparser sampling, less number of bits is required to represent the information. We also show that with the same sampling pairs, the rate of nested sampling is less than that of coprime sampling at the same distortion.