Application of compressed sensing for image compression based on optimized Toeplitz sensing matrices

In compressed sensing, the Toeplitz sensing matrices are generated by randomly drawn entries and further optimizes them with suitable optimization methods. However, during an optimization process, state-of-the-art optimization methods tend to lose control over the structure of measurement matrices. In this paper, we proposed the novel approach for optimization of Toeplitz sensing matrices based on evolutionary algorithms such as Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO) for compression of an image signal. Furthermore, we investigated the performance of Basis Pursuit (BP) and Orthogonal Matching Pursuit (OMP) algorithms for the reconstruction of the images. The proposed optimized Toeplitz sensing matrices based on evolutionary algorithms such as GA, SA, and PSO exhibit a significant reduction in the mutual coherence (μ) and thus improved the recovery performance of 2D images compared to state-of-the-art non-optimized Toeplitz sensing matrices. The result reveals that the optimized Toeplitz sensing matrices with Basis Pursuit (BP) achieved more accurate results with a robust and uniform reconstruction guarantee compared to the OMP algorithm. However, BP shows the slow reconstruction performance of the image signal. On the other hand, an optimized Toeplitz sensing matrix with OMP shows a fast reconstruction guarantee, but at the cost of a reduction in the PSNR. Furthermore, the proposed approach retains the structure of Toeplitz sensing matrices and improves the image recovery performance of compressed sensing. Finally, the experimental results validate the effectiveness of the proposed method based on evolutionary algorithms for image compression.


Introduction
The conventional Nyquist sampling depends on the highest amount of rate of alteration of a signal. In this sampling scheme, samples of the signal are captured at double the highest frequency in the signal. However, this method restricts the efficient compression of a signal. Since this scheme places an enormous burden on an encoder side reconstructed using a compressed sensing technique which, reduces the overall operational cost of a system.
In a nutshell, the Nyquist sampling put an enormous burden on the encoder side due to a massive number of samples of an acquired signal, particularly for audio/speech, ECG, image, and video signals. This fact inspired the study of compressed sensing as a potential solution for sampling, compression, and reconstruction of a signal. Intuitively, sparsity represents a large amount of energy concentration in a few numbers of coefficients. Several real-world signals such as speech and image are sparse or compressible in some transform domain. For example, images are compressible in basis algorithms such as JPEG and JPEG2000. The compressed sensing uses this sparsity property to compress and recover the signal effectively.
Traditionally, the random sensing matrices widely employed for signal compression in CS. The random Gaussian sensing matrices are entirely unstructured. Therefore, these matrices resulted in an enhanced computational complexity and increased memory storage requirement. Hence, the practical implementation of random Gaussian sensing matrices is costly. Further, sensing technologies need structured measurement matrices to accomplish different applications. Thus, the Toeplitz measurement matrix is one of the structured class matrix and widely used in a different field of applications, such as MRI [14], Synthetic Aperture Radar (SAR) [15], and channel estimations [16]. The Toeplitz matrices possess some exceptional features, such as these matrices generated with a smaller number of entries. Moreover, different techniques are available to speed up the matrix multiplication, which further may result in fast signal reconstruction. So far, the work carried out under the statement that the Toeplitz matrices generated using randomly drawn entries. Recently, Dirksen et al. [17] proposed partial Gaussian circulant matrices for 1-bit compressed sensing. Furthermore, Jie et al. [18] proposed compressed sensing matrices using vector spaces for signal processing.
In the literature, so far, different methods are proposed for the optimization of random Gaussian measurement matrices [19,20]. However, these methods randomly draw entries to generate random Toeplitz matrices and further optimize them with proper optimization methods. Nevertheless, during an optimization process, these methods lose control over the structure of measurement matrices. Also, researchers Abolghasemi, Jarchi, and Sanei [20] proposed the gradient-decent-based method to optimize mutual coherence. In this method, the modified cost function is followed by the gradient-descent minimization method to optimize the sensing matrices iteratively. This method shows the robustness in handling complex values. The researcher Duarte-Carvajalino and Sapiro [21] proposed the non-iterative way to calculate t-averaged mutual coherence. This method intended to make a Gram matrix closer to the identity matrix. However, this method ignores the negative eigenvalues and thus presents the problem of complex values which, causes the algorithm to fail.
It had proved that if sensing matrices satisfy restricted isometry property (RIP) [22], then there has been a high probability of superior quality signal reconstruction. On the contrary, the RIP is impractical to evaluate. Therefore, another way to satisfy the RIP and guarantee the exact reconstruction of a signal is to compute the mutual coherence (μ) between the sensing matrix (Φ) and the sparsifying matrix (Ψ). The mutual coherence (μ) of a dictionaryD M × N = Φ M × N × Ψ N × N is defined as the biggest absolute and normalized inner product among different columns of D [19] and given by the equation (1).
where N is the length of the input signal. From linear algebra, 1 ≤ μðΦ; ΨÞ ≤ ffiffiffiffi N p . Thus, the minimization of mutual coherence may be one of the effective ways to boost the recovery performance of compressed sensing matrices [23].
This paper proposes the optimization of Toeplitz sensing matrices based on evolutionary algorithms such as Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO) algorithm for image compression. The minimization of mutual coherence may be one of the effective ways to boost the recovery performance of compressed sensing matrices. Thus, this paper proposed the minimization of the mutual coherence (μ) between the sensing matrix (Φ) and the sparsifying matrix (Ψ) using evolutionary algorithms. The proposed optimization approach provides the best random Toeplitz vector, which consequently minimizes the mutual coherence (μ) between the sensing matrix (Φ) and the sparsifying matrix (Ψ). Further, the Toeplitz measurement matrix generated using the best random Toeplitz vector and finally applied for image compression. Furthermore, this proposed approach retains the structure of the Toeplitz matrix and improves the image recovery performance of compressed sensing.
Since the novel approach of the proposed optimization method is based on an evolutionary algorithm, and hence, it is entirely different from the state-of-the-art optimization methods. Until now, all the state-of-the-art optimization methods use non-evolutionary approaches for optimization of sensing matrices and thus tend to lose the structure of sensing matrices. Therefore, it is not practicable to compare proposed evolutionary approaches directly with non-evolutionary approaches of state-of-the-art methods. Thus, rather than, we have compared the performance of the proposed optimized Toeplitz sensing matrices based on evolutionary algorithms with non-optimized Toeplitz sensing matrices.
The main contributions of the proposed work are as follows: 1. We proposed a novel approach for the optimization of Toeplitz sensing matrices based on Evolutionary algorithms. 2. We proposed the first approach for the optimization of Toeplitz sensing matrices based on the Genetic Algorithm. 3. We proposed the second approach for the optimization of Toeplitz sensing matrices based on the Simulated Annealing (SA) Algorithm. 4. We proposed the third approach for the optimization of Toeplitz sensing matrices based on the Particle Swarm Optimization (PSO) Algorithm. 5. We investigated the signal reconstruction performance using Basis Pursuit (BP) and Orthogonal Matching Pursuit (OMP) algorithm for GA, SA, and PSO-based optimization approaches.
6. Finally, GA, SA, PSO-based optimization approaches exhibit a significant reduction in the mutual coherence (μ) and thus improved the recovery performance of 2D images compared to non-optimized Toeplitz sensing matrices.
The organization of the paper is as follows: Section 1 presents the formulation of an optimization problem. Section 2 elaborates on the proposed optimization method based on evolutionary algorithms. Section 4 presents results and discussion. Finally, Section 5 presents the conclusions.

Method
Let the problem needs to optimize in terms of the minimization problem. Therefore, objective/fitness function is required to minimize. This problem consists of two matrices, namely the measurement/sensing matrix (Φ M × N , where M<N) and the sparsifying transform matrix (Ψ N × N ). The Toeplitz matrix is used as a sensing matrix to compress the given signal, whereas the Discrete Cosine Transform (DCT) is used as a sparsifying transform matrix. The problem description is as follows: The dictionary matrix (D) given as: Now, the dictionary matrix D must be optimized, such that the mutual coherence (inner product) of matrix D is as small as possible as given by the equation (1).
Thus, the objective function is to optimize (minimize) the mutual coherence (μ) of the dictionary matrix (D) using an evolutionary algorithm such as Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization.
The minimization of the mutual coherence (μ) of the dictionary matrix (D) will satisfy the RIP condition and further results in improving the recovery performance of a sparse signal.
The statement of the optimization problem is as follows: The objective function is defined by equation (2) as follows. Here, N is the length of the input signal.
2.1 Proposed optimization of Toeplitz sensing matrices using Genetic Algorithm (GA) This section presents the optimization of Toeplitz sensing matrices based on the Genetic Algorithm (GA). The initial population was generated by using different random Toeplitz vectors. Then the fitness function f(x) is evaluated for all the random Toeplitz vectors. The new population made using three steps: selection, crossover, and mutation. The Roulette wheel selection technique was used to select the best parents to create new offspring. In this technique, the best parents have chosen to depend on the fitness of the population. Higher fitness indicates a greater chance to get selected. Then the bit sequence of the two parents is swapped to create the new offspring. The diversity in the new population is achieved with a mutation. Next, the old population was replaced by the new population [24]. Then the algorithm is tested for the convergence criterion such as the maximum number of iterations. Finally, when an algorithm is converged, it Parkale and Nalbalwar EURASIP Journal on Advances in Signal Processing (2021) 2021:70 will return the best Toeplitz vector, which results in minimum mutual coherence (μ). Thus, this best solution vector is used to generate the Toeplitz matrix. The algorithmic steps for optimization of Toeplitz sensing matrices using Genetic Algorithm (GA) [25] are given as follows: (i) Generate an initial population of n chromosomes using random Toeplitz vectors: Here, each Toeplitz vector is considered as one of the candidate solutions for the given problem. Each Toeplitz vector corresponding to the Toeplitz matrix is generated. (ii) Then evaluate the fitness function f(x) of each chromosome in the population: (iii)Further, evaluate the mutual coherence (i.e., fitness function) (μ) between the generated Toeplitz matrix (Φ) and the sparsifying matrix (Ψ). Thus, we had evaluated the fitness function for the generated Toeplitz matrix. Similarly, evaluate the fitness function for all the generated Toeplitz vectors. When an algorithm is converged, it returns the best candidate solution, i.e., it returns the Toeplitz vector, which resulted in minimum mutual coherence (μ). Otherwise, go to step (2) and repeat the procedure still end criterion meets.
The simulation shows excellent results with the following specifications: Algorithm: Binary GA Algorithm Population Size = 100 Selection technique: Roulette wheel selection technique Selection rate of parents for generating offspring = 0.5 or 50% of the initial population Number of encoding bits = 16 Mutation method: Single point crossover Mutation rate = 0. 15 The maximum iterations (stopping criteria) = 100 Figure 1 shows the convergence characteristics of GA. It is observed from Fig. 1 that the value of the fitness function reduces with an increase in the number of iterations.
Moreover, the cost of the fitness function remains constant from the 10th iteration to the 100th iteration. Thus, GA shows good and stable convergence characteristics up to the 100th iteration.

Proposed optimization of Toeplitz sensing matrices using Simulated Annealing (SA) algorithm
This section presents the optimization of Toeplitz sensing matrices using Simulated Annealing (SA). Primarily, the temperature parameter (T 0 ) is set to some high value, and then it is gradually reduced using the temperature reduction factor (α). The initial solution vector (s) and the new solution vector (s 0 ) are randomly generated using a random generator and then evaluate the fitness function for both the solution vectors.
If a new solution has better fitness than the current solution, it is selected as the next solution. On the contrary, if the new solution has a worse fitness compared to the current solution, the algorithm still considers it as the next solution. The acceptance or rejection of the new solution vector as the future solution vector depends on the Metropolis-Hasting criterion [26]. The Metropolis-Hasting principle given as follows: where δ f = f(s)-f (s 0 ) is the fitness difference between the new solution vector and the old solution vector, T = temperature parameter, P(r) = probability of acceptance or rejection of the new solution vector as a next solution vector and r∈ (0, 1), and k = 1 (Boltzmann's constant). A random number (r) is generated such that r∈ (0, 1). If a random number (r) < exp [(-δf)/(T)], then a new solution is selected as the next solution (s 0 = s); otherwise, a new solution is discarded. Then the temperature parameter is reduced gradually to narrow search the optimum solution. The algorithm repeated until it meets the stopping criterion, such as minimum temperature value or a number of iterations. Finally, when an algorithm is converged, it will return, the best Toeplitz vector which, results in a minimum mutual coherence (μ). Thus, this best solution vector is used to generate the Toeplitz matrix. The algorithmic steps for the optimization of Toeplitz sensing matrices using Simulated Annealing (SA) [27] are given as follows: The simulations are conducted with the following specifications: The maximum numbers of iterations = 500 Initial Temperature=1 Temperature reduction factor=0.8 Stopping Temperature value= 1e-8 Figure 2 shows the convergence characteristics of the SA. It is observed from Fig. 2 that the value of the fitness function reduces with an increase in the number of iterations. Moreover, the value of the fitness function remains relatively constant from the 200th iteration to the 500th iteration. Thus, SA exhibited good and stable convergence characteristics up to the 500th iteration.

Optimization (PSO) algorithm
This section presents the optimization of Toeplitz sensing matrices using Particle Swarm Optimization (PSO). Here, each particle in the population is equivalent to the candidate solution in the solution space. This solution space is generated using a random number generator. Then the fitness function is evaluated for each solution in the population. If the current particle fitness is higher than the previous local best particle fitness, then we have to update the local best particle position (PBest). Similarly, if the current particle fitness is higher than the previous global best particle fitness, then we have to update the global best particle position (GBest). Then update the position and velocity of each particle using equation (position update equation) and equation (velocity update equation). Finally, when an algorithm is converged, it will return the best Toeplitz vector, which results in minimum mutual coherence (μ). Thus, this best solution vector is used to generate the Toeplitz matrix.
The algorithmic steps for the optimization of Toeplitz sensing matrices using Particle Swarm Optimization (PSO) [28] are given as follows.
(i) Randomly generate the initial population. (ii) Randomly initialize the positions and velocities of particles in the population. (iii)Evaluate the fitness function for each particle in the population. (iv) If the current particle fitness is higher than the previous local best particle fitness, then update the local best particle position (PBest). (v) Similarly, if the current particle fitness is higher than the previous global best particle fitness, then update the global best particle position (GBest). (vi) Then update the position and velocity of each particle.
The velocity update equation is given by the equation number (3), and (4): where v i (k) is the initial velocity, and v i (k+1) is the updated velocity of the ith particle, w: inertia weight, c1, c2: Positive constant c1 and c2 are personal (cognitive) and social The position update equation (5) given as: where x i (k) is the initial position of the particle, and x i (k+ 1) is the updated position of the ith particle.
(vii)Go to step (iii), and repeat until stopping condition meets.
The simulations conducted with the following specifications: Population size = 100 Inertia weight (maximum) = 0.9 Inertia weight (minimum) = 0.4 Personal/cognitive factor (c1) = 2 Social learning factor (c2) = 2 The maximum numbers of iterations =100 Figure 3 shows the convergence characteristics of the PSO. It is observed from Fig. 3 that the value of the fitness function reduces with an increase in the number of iterations. Moreover, the value of the fitness function remains constant from the 50th iteration to the 100th iteration. Thus, PSO showed good and stable convergence characteristics up to the 100th iteration.

Conclusion
In this paper, a novel approach to optimize Toeplitz sensing matrices using evolutionary algorithms such as Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO) algorithms for compressed sensing is discussed. Furthermore, the performance of Basis Pursuit (BP) and Orthogonal Matching Pursuit (OMP) algorithms are investigated for the reconstruction of the 2D image signal.
The minimization of mutual coherence (μ) between the sensing matrix (Φ) and the sparsifying matrix (Ψ) is one of the effective ways to boost the recovery performance of compressed sensing matrices.
The following significant conclusions are drawn based on the investigations: The proposed optimized Toeplitz sensing matrices based on evolutionary algorithms exhibit a significant reduction in the mutual coherence (μ). Thus the proposed method clearly outperforms the non-optimized Toeplitz sensing matrices for image compression application.
The proposed optimized Toeplitz sensing matrices based on Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO) algorithms with Basis Pursuit (BP) as a reconstruction algorithm achieve more accurate results (i.e., reduction in MSE) with higher PSNR compared to the non-optimized Toeplitz sensing matrices with Basis Pursuit (BP). Furthermore, these matrices achieve significantly faster image reconstruction as well as faster construction of sensing matrices.
Further, it is noted that the proposed optimized Toeplitz sensing matrices based on Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO) algorithms with Orthogonal Matching Pursuit (OMP) as a reconstruction algorithm exhibits the lower values of PSNR and higher values of MSE compared to the non-optimized Toeplitz sensing matrices with BP. Generally, this behavior present due to the greedy approach of the OMP algorithm. Moreover, this is due to the fact that the reconstruction guarantees of OMP are weak and show non-uniform behavior.
On the contrary, the optimized Toeplitz sensing matrices with OMP achieved significantly faster image reconstruction as well as rapid sensing matrix construction performance as compared to the non-optimized Toeplitz sensing matrices.
Thus, in general, the optimized Toeplitz sensing matrices with Basis Pursuit (BP) achieve more accurate results with a robust and uniform reconstruction guarantee. However, they showed a slow signal reconstruction performance. On the other hand, the optimized Toeplitz sensing matrices with OMP show a fast reconstruction guarantee, but at the cost of the reduction in the PSNR.
Finally, the result shows the successful implementation of the proposed optimized Toeplitz sensing matrices using evolutionary algorithms such as Genetic Algorithm (GA), Simulated Annealing (SA), and Particle Swarm Optimization (PSO) algorithms for compressed sensing. Furthermore, Toeplitz sensing matrices are easy for hardware realization because of its lower computational complexity compared to random Gaussian sensing matrices.