 Research
 Open Access
 Published:
Formal analysis of 2D image processing filters using higherorder logic theorem proving
EURASIP Journal on Advances in Signal Processing volume 2022, Article number: 53 (2022)
Abstract
Twodimensional (2D) image processing systems are concerned with the processing of the images represented as 2D arrays and are widely used in medicine, transportation and many other autonomous systems. The dynamics of these systems are generally modeled using 2D difference equations, which are mathematically analyzed using the 2D ztransform. It mainly involves a transformation of the difference equationsbased models of these systems to their corresponding algebraic equations, mapping the 2D arrays (2D discretetime signals) over the (\(z_1\),\(z_2\))domain. Finally, these (\(z_1\),\(z_2\))domain representations are used to analyze various properties of these systems, such as transfer function and stability. Conventional techniques, such as paperandpencil proof methods, and computerbased simulation techniques for analyzing these filters cannot assert the accuracy of the analysis due to their inherent limitations like human error proneness, limited computational resources and approximations of the mathematical expressions and results. In this paper, as a complimentary technique, we propose to use formal methods, higherorder logic (HOL) theorem proving, for formally analyzing the image processing filters. These methods can overcome the limitations of the conventional techniques and thus ascertain the accuracy of the analysis. In particular, we formalize the 2D ztransform based on the multivariate theories of calculus using the HOL Light theorem prover. Moreover, we formally analyze a generic (\(L_1,L_2\))order 2D infinite impulse response image processing filter. We illustrate the practical effectiveness of our proposed approach by formally analyzing a secondorder image processing filter.
Introduction
Twodimensional (2D) image processing systems [1, 2] typically involve image filtering, editing, enhancement, compression and restoration of the images represented as 2D arrays (2D discretetime signals). Image processing filters [2] are the fundamental components of the 2D image processing systems that are widely used for image filtering. These filters are categorized as highpass, bandpass and lowpass filters based on the passage of the allowable range of frequencies. For example, a highpass filter permits a range of frequencies greater than a certain threshold. Moreover, these filters are widely used in autonomous vehicles [3, 4] and medicine [5]. For example, they are used to perform various image processing tasks for controlling the autonomous vehicles, such as noise reduction, color normalization, histogram equalization and edge detection, to enhance the quality of the images captured using various devices such as closedcircuit television (CCTV) and webcams [6]. Similarly, they are widely used in medicine for performing various image pre and postprocessing tasks, such as image quality enhancement, noise removal and image smoothing [5].
The dynamics of these image processing systems are generally modeled using 2D difference equations. Next, the 2D ztransform is used to mathematically analyze these systems. It mainly involves a transformation of the difference equationsbased models of these systems to their corresponding algebraic equations, using the definition and various classical properties of the 2D ztransform, while mapping 2D arrays over the (\(z_1\),\(z_2\))domain. Finally, these (\(z_1\),\(z_2\))domain representations are used to analyze various properties of these image processing systems like transfer function and stability [2].
Conventionally, the image processing filters have been analyzed using paperandpencil proof techniques and computerbased symbolic and numerical methods. However, in the former case, the analysis is errorprone due to the highly involved human manipulation, particularly for analyzing the larger and complex image processing systems, and thus we cannot ascertain an absolute accuracy of the analysis in this approach. Similarly, the later approaches suffer from some of their inherent limitations. For example, the symbolic methods involve a large number of unverified symbolic procedures residing in the root of the associated tools [7]. Similarly, the numerical techniques include a finite number of iterations due to the limited power of the computing machines. Moreover, they are based on the mathematical results that are approximated due to the finite precision arithmetic of computers. Therefore, these conventional approaches cannot be trusted when analyzing the image processing filters utilized in various safetycritical areas, such as autonomous driving and medicine.
Formal methods [8] are system analysis techniques that are based on developing a mathematical model of the system using logic and verifying its various properties using deductive reasoning. Higherorder logic (HOL) theorem proving [9, 10] is a widely utilized formal method for analyzing many safetycritical systems. In this paper, we propose a HOL theorem provingbased framework for analyzing the image processing filters. In particular, we formalize the 2D ztransform based on the multivariate theories of calculus using the HOL Light theorem prover [11]. The main motivation of selecting HOL Light for the proposed formalization is the presence of the fundamental libraries of multivariate calculus,^{Footnote 1} vectors^{Footnote 2} and matrices,^{Footnote 3} which are required to formally analyze the 2D image processing systems.
Contributions of the paper
The novel contributions of the paper are:

Formalization of 2D ztransform and its region of convergence (ROC).

Formal verification of various classical properties of 2D ztransform, such as linearity, shifting in timedomain, scaling in (\(z_1\), \(z_2\))domain and complex conjugation.

Formal analysis of a generic (\(L_1, L_2\))order 2D IIR image processing filter.

Formal analysis of a secondorder image processing filter
Preliminaries
This section provides an introduction to the HOL Light theorem prover and the formalization of some fundamental concepts from the multivariate calculus libraries of HOL Light that facilitate the understanding of the rest of the paper.
HOL Light Theorem prover
HOL Light[12] is a proof assistant for developing proofs of the mathematical concepts written as theorems in higherorder logic. HOL Light is implemented in the strongly typed functional programming language ML [13]. A theorem is a statement that is formalized as an axiom or can be implied from already verified theorems using inference rules. Soundness is assured in a theorem proving environment as every new theorem is verified using the primitive inference rules or any other previously verified theorems. HOL Light provides an extensive support of theories, such as Boolean algebra, arithmetic, real numbers, vectors and matrices, which are extensively used in our formalization. Indeed, one of the motivations for selecting the HOL Light theorem prover for the proposed framework is the availability of extensive libraries of vectors and matrices.
Multivariable calculus theories in HOL Light
This section presents an introduction to some fundamental concepts formalized in HOL Light, such as summability, infinite summation and vector summation, and some HOL Light notations that help understanding the rest of the paper.
An Ndimensional vector in HOL Light is formalized as a \(\mathbb {R}^N\) column matrix capturing individual elements as real numbers. All vector operations are then considered as matrix manipulations. Most of the theorems in multivariable calculus theories of HOL Light are proved for functions with an arbitrary data type of \(\mathbb {R}^M \rightarrow \mathbb {R}^N\). Similarly, complex numbers (\(\mathbb {C}\)) can be described as \(\mathbb {R}^2\) instead of defining a new data type. The HOL Light symbol &: \(\mathbb {N} \rightarrow \mathbb {R}\) represents an injection of natural numbers to real numbers. Similarly, the symbol Cx: \(\mathbb {R} \rightarrow \mathbb {C}\) typecasts real numbers to complex numbers. The symbols Re: \(\mathbb {C} \rightarrow \mathbb {R}\) and Im: \(\mathbb {C} \rightarrow \mathbb {R}\) represent the real and imaginary components of a complex number, respectively. The HOL Light symbol \(\mathtt {\%}\) captures the scalar multiplication of a vector or matrix. Similarly, a matrix–vector multiplication is modeled as \(\mathtt {**}\) in HOL Light.
The generalized summation over an arbitrary function fn: A \(\rightarrow \mathbb {R}^N\) is formalized in HOL Light as follows:
Definition 1
Generalized Summation of Vector \(\vdash _{\textit{def}}\) \(\forall\) st fn. vecsum st fn = (lambda k. summ st (\(\lambda\) x. fn x$k))
where vecsum accepts a set st: A \(\rightarrow\) bool over which the summation occurs and a function fn of data type A \(\rightarrow \mathbb {R}^N\) and returns a generalized vector summation over the set st. Here, the HOL Light function summ provides a finite summation for a fn over real numbers. For example, a mathematical expression \(\sum \limits _{k = 0}^{n}{f(k)}\) is described in HOL Light as vecsum (0..n) fn.
Definition 2
Summs \(\vdash _{\textit{def}}\) \(\forall\) st fn lt. (fn summs lt) st \(\Leftrightarrow\) ((\(\lambda\) n. vecsum (st INTER (0..n)) fn) \(\rightarrow\) lt) sequentially
The HOL Light function summs accepts a set of natural numbers st: N \(\rightarrow\) bool, a function fn: \(\mathbb {N}\) \(\rightarrow\) \(\mathbb {R}^N\) and a limit value lt: \(\mathbb {R}^N\) and returns the traditional mathematical expression \(\sum \limits _{k = 0}^{\infty }{f(k)} = L\). Here, INTER captures the intersection of two sets. Similarly, sequentially represents a net providing a sequential growth of a function f, i.e., \(f(k), f(k + 1), f(k + 2), \ldots ,\) etc. This is mainly used in modeling the concept of an infinite summation.
We provide the formalization of the summability of a function fn: \(\mathbb {N}\) \(\rightarrow\) \(\mathbb {R}^N\) over st: N \(\rightarrow\) bool, which ensures that there exist some limit value L: \(\mathbb {R}^N\), such that \(\sum \limits _{k = 0}^{\infty }{f(k)} = L\) in HOL Light as:
Definition 3
Summability of a Function \(\vdash _{\textit{def}}\) \(\forall\) fn st. summable fn st \(\Leftrightarrow\) (\(\exists\) lt. (fn summs lt) st)
The limit of a function fn: A \(\rightarrow \mathbb {R}^N\) is formalized as:
Definition 4
Limit of a Function \(\vdash _{\textit{def}}\) \(\forall\) net f. limt net fn = (\(\in\) lt. (fn \(\rightarrow\) lt) net)
where the function limt takes a net with components of data type A and a function fn and returns a limit value lt: \(\mathbb {R}^N\) to which fn converges at the given net. It is formalized using the Hilbert choice operator \(\in\). Similarly, the concept tends to (\(\rightarrow\)) is formalized in HOL Light as:
Definition 5
Tends to \(\vdash _{\textit{def}}\) \(\forall\) fn lt net. (fn \(\rightarrow\) lt) net \(\Leftrightarrow\) \(\forall\)e. &0 < e \(\Rightarrow\) eventually (\(\lambda\) x. dist (fn x, lt) < e) net
Now, we provide a formalization of an infinite summation, which is used in the formal definition of the 2D ztransform presented in Sect. 5.1.
Definition 6
Infinite Summation of a Function \(\vdash _{\textit{def}}\) \(\forall\) fn st. inftsumm st fn = (\(\in\) lt. (fn summs lt) st)
where the HOL Light function inftsumm accepts st: num \(\rightarrow\) bool specifying the starting point and a function fn of data type \(\mathbb {N} \rightarrow \mathbb {R}^N\), and returns a limit value lt: \(\mathbb {R}^N\) to which the infinite summation of fn converges from the given st.
Next, we formally verify an equivalence of the infinite summation (Definition 6) to its alternate form in terms of sequential limit as the following HOL Light theorem:
Theorem 1
Relationship Between Infinite Summation and the Sequential Limit \(\vdash _{\textit{thm}}\) \(\forall\) st fn. inftsumm st fn = limt sequentially (\(\lambda\) k. vecsum (st INTER (0..k)) fn)
Methods
Figure 1 depicts our proposed method for analyzing the image processing filters using HOL theorem proving. The user provides the 2D difference equations that model the dynamics of the image processing system, which needs to be analyzed. This 2D difference equation is modeled in higherorder logic using the multivariate calculus theories of HOL Light. In the next step, we formalize the 2D ztransform that is required for mathematically analyzing the image processing systems. It mainly transforms the difference equationsbased models of these systems to their corresponding algebraic equations, using the definition and various classical properties, such as linearity, shifting and scaling, of the 2D ztransform, while mapping 2D arrays over the (\(z_1\),\(z_2\))domain. Finally, these (\(z_1\),\(z_2\))domain representations are used to analyze various properties of these systems, such as transfer function and the solution of the corresponding difference equations.
Results
Formalization of the 2D ztransform
The 2D ztransform of a 2D discretetime function (2D array) \(f (n_1, n_2)\) is mathematically expressed as follows [2]:
where f is a function of type \(\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}\), and \(z_1\) and \(z_2\) are complex variables. The limits from 0 to \(\infty\) make Eq. (1) as a mathematical representation of a unilateral 2D ztransform. We have opted for this representation based on the same motivation that was considered for onedimensional ztransform [14] and the Laplace transform [15].
We formalize the 2D ztransform [Eq. (1)] in HOL Light as follows:
Definition 7
2D zTransform \(\vdash _{\textit{def}}\) \(\forall\) f z1 z2. z_transform_2d f z1 z2 = inftsumm (from 0) (\(\lambda\) n1. inftsumm (from 0) (\(\lambda\) n2. f n1 n2 / (z1 pow n1 \(*\) z2 pow n2)))
where z_transform_2d accepts a function of type \(\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}\) and two complex variables z1: \(\mathbb {C}\) and z2: \(\mathbb {C}\) and returns a complex number, which represents the 2D ztransform of f: \(\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}\) according to Eq. (1).
An essential issue with the applicability of the 2D ztransform of \(f (n_1, n_2)\) is the existence of \(F (z_1, z_2)\) that occurs due to the presence of the infinite summations in Eq. (1). Thus, we need to identify conditions for the existence of the 2D ztransform. A set of all those values of \(z_1\) and \(z_2\) for which the infinite summations are converging and \(F (z_1, z_2)\) is finite (or summable) is known as the ROC. It is mathematically expressed as follows:
We formalize the ROC of the 2D ztransform as follows:
Definition 8
Region of Convergence (ROC) \(\vdash _{\textit{def}}\) \(\forall\) f n1. ROC_2d f n1 = {(z1, z2)  \(\lnot\)(z1 = Cx( &0)) \(\wedge\) \(\lnot\)(z2 = Cx( &0)) \(\wedge\) z_tr_summable f z1 z2 n1 \(\wedge\) z_tr_td_summable f z1 z2}
where ROC_2d accepts a function f: \(\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}\) and n1 capturing the starting point of the outer summation of the 2D ztransform [Eq. (1)] and returns a set of nonzero values of variables z1 and z2 for which the 2D ztransform of f exists. It is necessary to specify the associated ROC_2d to compute the 2D ztransform. Moreover, the functions z_tr_summable and z_tr_td_summable capture the summability of the function f for the inner and the outer (double) summations, respectively, and are formalized in HOL Light as follows:
Definition 9
Summability of Function for Inner Summation \(\vdash _{\textit{def}}\) \(\forall\) f z n1. z_tr_summable f z1 z2 n1 = (\(\forall\) n1. summable (from 0) (\(\lambda\) n2. f n1 n2 / (z1 pow n1 \(*\) z2 pow n2)))
Definition 10
Summability of Function for Outer (Double) Summation \(\vdash _{\textit{def}}\) \(\forall\) f z1 z2. z_tr_td_summable f z1 z2 = summable (from 0) (\(\lambda\) n1. inftsumm (from 0) (\(\lambda\) n2. f n1 n2 / (z1 pow n1 \(*\) z2 pow n2)))
Moreover, we verify two fundamental properties of ROC, such as the linearity of the ROC and scaling of the ROC, which are quite helpful for formally verifying the classical properties of the 2D ztransform in Sect. 5.2.
Theorem 2
Linearity of ROC \(\vdash _{\textit{thm}}\) \(\forall\) z1 z2 a b f g n1. [A1]: (z1, z2) IN ROC_2d f n1 \(\wedge\) [A2]: (z1, z2) IN ROC_2d g n1 \(\Rightarrow\) (z1, z2) IN ROC_2d (\(\lambda\) n1 n2. a \(*\) f n1 n2) n1 INTER ROC_2d (\(\lambda\) n1 n2. b \(*\) g n1 n2) n1
Theorem 3
Scaling of ROC
\(\vdash _{\textit{thm}}\) \(\forall\) z1 z2 a f n1. [A]: (z1, z2) IN ROC_2d f n1 \(\Rightarrow\) (z1, z2) IN ROC_2d (\(\lambda\) n1 n2. f n1 n2 / a) n1
Theorem 2 ensures that if (z1, z2) is inside ROC_2d f n1 and ROC_2d g n1 for functions f and g then it is also inside the intersection of both ROCs for the scaled version of these functions. Similarly, Theorem 3 provides the scaling property with respect to the division by a complex number a.
Formal verification of the classical properties of the 2D ztransform
We use Definitions 7 and 8 and Theorems 2 and 3 for verifying some of the classical properties of the 2D ztransform in HOL Light. This verification plays a vital role in reducing the effort required for analyzing image processing systems, as described later in Sects. 5.3 and 5.4.
Linearity of the 2D zTransform The linearity of the 2D ztransform is mainly used in decomposing complex (larger) systems to subsystems or combining smaller systems to larger ones having different scaling inputs. It can be mathematically expressed as follows:
If \(\mathcal {Z}[f(n1, n2)] = F (z_1, z_2)\) and \(\mathcal {Z}[g(n_1, n_2)] = G(z_1, z_2),\) then the following holds:
The 2D ztransform of a linear combination of 2D sequences (or arrays) is equal to the linear combination of the 2D ztransform of the individual arrays. We verify linearity property in HOL Light as:
Theorem 4
Linearity of the 2D zTransform
\(\vdash _{\textit{def}}\) \(\forall\) f g z1 z2 a b n1. [A1]: (z1, z2) IN ROC_2d f n1 \(\wedge\) [A2]: (z1, z2) IN ROC_2d g n1 \(\Rightarrow\) z_transform_2d (\(\lambda\) n1 n2. a \(*\) f n1 n2 ± b \(*\) g n1 n2) z1 z2 = a \(*\) z_transform_2d f z1 z2 ± b \(*\) z_transform_2d g z1 z2
where a: \(\mathbb {C}\) and b: \(\mathbb {C}\) are arbitrary complex constants. Assumptions A1 and A2 capture the regions of the convergence of functions f and g, respectively. The proof of the above theorem is mainly based on Theorem 2 and the linearity of the infinite summation along with some complex arithmetic reasoning.
Shifting Property of the 2D zTransform The shifting property of the 2D ztransform is mostly used for analyzing the 2D linear shiftinvariant (LSI) systems. In particular, it is used to solve the difference equations capturing the dynamics of these systems. The shifting property expresses the transform of the shifted signal \(f(n_1  m_1, n_2  m_2)\) in terms of its 2D ztransform \(F(z_1, z_2)\).
If \(\mathcal {Z}[f(n_1, n_2)] = F(z_1, z_2)\) and assuming \(f (n_1, n_2) = 0\), \(f (n_1, n_2) = 0\) and \(f (n_1, n_2) = 0\), \(\forall n_1 = 1, 2, \ldots , m_1\) and \(\forall n_2 = 1, 2, \ldots , m_2\), i.e., \(f(n_1, n_2)\) is nonzero in the first quadrant only, then the shifting of a 2D array is mathematically expressed as follows:
We formally verify the above property in HOL Light as:
Theorem 5
Shifting in Time Domain
\(\vdash _{\textit{thm}}\) \(\forall\) f z1 z2 m1 m2 n1. [A1]: (z1, z2) IN ROC_2d f n1 \(\wedge\) [A2]: in_fst_quad_2d f \(\Rightarrow\) z_transform_2d (\(\lambda\) n1 n2. f (n1  m1 n2  m2)) z1 z2 = z_transform_2d f z1 z2 / (z1 pow m1 \(*\) z2 pow m2)
where the function in_fst_quad_2d ensures that the function f is nonzero in the first quadrant only and is formalized in a relational form, i.e., f (n1  m1, n2  m2), \(\forall\) m1 m2. m1 < n1, m2 < n2. The verification of Theorem 5 is mainly based on the properties of complex numbers along with two properties regarding the negative offset of series and infinite summation. More details about the proof process of this theorem can be found in our proof script.^{Footnote 4}
Scaling in (\(z_1, z_2\))domain Property of the 2D zTransform The scaling property of the 2D ztransform results in shrinking or expansion of the (\(z_1, z_2\))domain, i.e., 4D complex (\(z_1, z_2\))plane. If \(\mathcal {Z}[f (n_1, n_2)] = F (z_1, z_2)\), then two different types of scaling are defined as:
If \(h_1\) and \(h_2\) are positive real numbers, then the scaling is interpreted as expansion of the 4D complex (\(z_1, z_2\))plane. On the other hand, multiplication by \({w_1}^{n_1}\) and \({w_2}^{n_2}\) [Eq. (6)] shrinks the (\(z_1, z_2\))domain.
We verify the above theorems in HOL Light as:
Theorem 6
Scaling in (\(z_1, z_2\)) Domain (Positive/Expansion)
\(\vdash _{\textit{thm}}\) \(\forall\) f z1 z2 n1 h1 h2. [A1]: (inv h1 \(*\) z1, inv h2 \(*\) z2) IN ROC_2d f n1 \(\wedge\) [A2]: (z1, z2) IN ROC_2d f n1 \(\Rightarrow\) z_transform_2d (\(\lambda\) n1 n2. h1 pow n1 \(*\) h2 pow n2 \(*\) f n1 n2) z1 z2 = z_transform_2d f (inv h1 \(*\) z1, inv h2 \(*\) z2)
Theorem 7
Scaling in (\(z_1, z_2\))Domain (Negative/Shrinking)
\(\vdash _{\textit{thm}}\) \(\forall\) f z1 z2 n1 w1 w2. [A1]: (w1 \(*\) z1, w2 \(*\) z2) IN ROC_2d f n1 \(\wedge\) [A2]: (z1, z2) IN ROC_2d f n1 \(\Rightarrow\) z_transform_2d (\(\lambda\) n1 n2. w1 pow (n1) \(*\) w2 pow (n2) \(*\) f n1 n2) z1 z2 = z_transform_2d f (w1 \(*\) z1) (w2 \(*\) z2)
Complex Conjugation Property of the 2D zTransform The complex conjugation property facilitates an easy manipulation of the 2D ztransform of conjugated arrays. It is mathematically expressed as follows:
where \(f^{*}(n_1, n_2)\) represents the complex conjugate of an array \(f (n_1, n_2)\). The corresponding formalization of the complex conjugation property in HOL Light is given as follows:
Theorem 8
Complex Conjugation
\(\vdash _{\textit{thm}}\) \(\forall\) f z1 z2 n1. [A]: (cnj z1, cnj z2) IN ROC_2d f n1 \(\Rightarrow\) z_transform_2d (\(\lambda\) n1 n2. cnj (f n1 n2)) z1 z2 = cnj (z_transform_2d f (cnj z1, cnj z2))
Formal verification of a (\(L_1, L_2\))order 2D infinite impulse response (IIR) image processing filter
2D digital filters [1] are integral components of the image processing systems. Their main responsibility includes the decomposition of an image to multiple frequency bands, restricting a 2D array/signal to a certain frequency band and providing the input–output relationship of these systems. For example, a lowpass filter allows a range of frequencies less than a certain threshold [2]. The analysis of an image processing filter mainly involves developing its mathematical model using a 2D difference equation. The next step is to apply 2D ztransform on both sides of the difference equation. Finally, the definition and the classical properties of the 2D ztransform are used to perform transfer functionbased analysis of the given filter.
The impulse response of a discretetime system captures its behavior for the scenario when diracdelta function is acting as an input array [2]. 2D image processing infinite impulse response (IIR) filters have a nonzero impulse response function over an infinite length of time. For these filters, the present output depends on the present input and all previously computed input and output values.
Mathematically, the 2D image processing filters are described using the following difference equation [16]:
where \(\alpha (l_1, l_2)\) and \(\beta (k_1, k_2)\) are input and output coefficients, respectively. The output array \(y (n_1, n_2)\) is a linear combination of the previous \(K_1  1\) and \(K_2  1\) output samples, the present input \(x (n_1, n_2))\), and \(L_1  1\) and \(L_2  1\) previous input samples. Moreover, for the shiftinvariant filter, \(\alpha (l_1, l_2)\) and \(\beta (k_1, k_2)\) are the complex constants (\(\mathbb {C}\)). Therefore, Eq. (8) is known as a linear constant coefficient difference equation (LCCDE). The 2D ztransform of a \({(L_1,L_2)}^{th}\) difference represented in the form of \(f (n_1, n_2)\) is given as:
The corresponding transfer function of the 2D IIR filter is mathematically expressed as [16]:
To formally verify the transfer function of the 2D filter [Eq. (10)], we formalize the \({(L_1,L_2)}^{th}\) difference as follows:
Definition 11
Formalization of the \({(L_1,L_2)}^{th}\) Difference \(\vdash _{\textit{def}}\) \(\forall\) f c L1 L2 n1 n2. l1l2th_difference f c L1 L2 n1 n2 = vecsum (0..L1) (\(\lambda\) l1. vecsum (0..L2) (\(\lambda\) l2. c l1 l2 \(*\) f (n1  l1) (n2  l2)))
The function l1l2th_difference accepts a function f: \(\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}\), coefficients of the difference equation c l1 l2, the order (L1, L2) of the 2D difference equation and the variables n1 and n2 and returns the \((L_1, L_2)^{th}\) difference. It uses the function vsum s f twice to capture the double summation.
Next, we formalize a general LCCDE [Eq. (8)] as follows:
Definition 12
Formalization of the LCCDE \(\vdash _{\textit{def}}\) \(\forall\) y x M1 M2 N1 N2 n1 n2 a b. LCCDE x y a b M1 M2 N1 N2 n1 n2 \(\Leftrightarrow\) y (n1, n2) = l1l2th_difference y a M1 M2 n1 n2  l1l2th_difference x b N1 N2 n1 n2
Next, we verify the 2D ztransform of the \((L_1, L_2)^{th}\) difference [Eq. (9)] as:
Theorem 9
The 2D zTransform of the \((L_1, L_2)^{th}\) Difference
\(\vdash _{\textit{thm}}\) \(\forall\) f c L1 L2 z1 z2 n1. [A1]: (z1, z2) IN ROC_2d f n1 \(\wedge\) [A2]: in_fst_quad_2d f \(\Rightarrow\) z_transform_2d (\(\lambda\) n1 n2. l1l2th_difference f c L1 L2 n1 n2) z1 z2 = z_transform_2d f z1 z2 \(*\) vecsum (0..L1) (\(\lambda\) l2. vecsum (0..L2) (\(\lambda\) l1. z1 cpow  Cx ( &l1) \(*\) z2 cpow  Cx ( &l2) \(*\) c l1 l2))
where Assumption A1 ensures that \((z_1, z_2)\) are in the region of convergence of the function f. Assumption A2 implies that the function f is in the first quadrant. Finally, the conclusion provides the 2D ztransform of the \((L_1, L_2)^{th}\) difference. The verification of the above theorem is mainly based on induction on N1 and N2 and Theorems 2 and 4 along with the following lemma about the summability of \((L_1, L_2)^{th}\) difference equation.
Lemma 1
Summability of the \((L_1, L_2)^{th}\) Difference
\(\vdash _{\textit{thm}}\) \(\forall\) f c L1 L2 z1 z2 n1. [A1]: (z1, z2) IN ROC_2d f n1 \(\wedge\) [A2]: in_fst_quad_2d f \(\Rightarrow\) (z1, z2) IN ROC_2d (\(\lambda\) n1 n2. l1l2th_difference f c L1 L2 n1 n2) n1
To verify the transfer function of the 2D filter [Eq. (10)], we have to ensure that the 2D input and output arrays exist in the first quadrant only. Moreover, the denominator of Eq. (10) should be nonzero. We formalize both these requirements in HOL Light as follows:
Definition 13
First Quadrant Input and Output 2D Arrays for LCCDE \(\vdash _{\textit{def}}\) in_fst_quad_2d_lccde x y \(\Leftrightarrow\) in_fst_quad_2d x \(\wedge\) in_fst_quad_2d y
Definition 14
ROC LCCDE \(\vdash _{\textit{def}}\) \(\forall\) x y K1 K2 lst n1 ROC_2d_LCCDE x y K1 K2 lst n1 = (ROC_2d x n1) INTER (ROC_2d y n1) DIFF {(z1, z2)  vecsum (0..K1) (\(\lambda\) k2. vecsum (0..K2) (\(\lambda\)k1. z1 cpow  Cx ( &k1) \(*\) z2 cpow  Cx ( &k2) \(*\) EL k1 lst)) = Cx ( &0)} DIFF {(z1, z2)  z_transform_2d x z1 z2 = Cx ( &0)}
where the function in_fst_quad_2d_lccde (Definition 13) accepts the input and output 2D arrays x and y and asserts the first quadrant condition for both arrays. Similarly, ROC_2d_LCCDE (Definition 14) provides the ROC of the input and output 2D arrays. It uses the HOL Light function DIFF to exclude all values of the denominator, where the transfer function of the 2D IIR filter becomes undefined.
Now, we provide the formal verification of the transfer function of a 2D IIR filter in HOL Light as follows:
Theorem 10
Transfer Function of a 2D IIR Filter
\(\vdash _{\textit{thm}}\) \(\forall\) x y a b L1 L2 K1 K2 z1 z2 n1.[A1]: (z1, z2) IN ROC_2d_LCCDE x y K1 K2 blst n1 \(\wedge\)[A2]: in_fst_quad_2d_lccde x y \(\wedge\)[A3]: (\(\forall\) n1 n2. LCCDE x y a b L1 L2 K1 K2 n1 n2)\(\Rightarrow\) z_transform_2d y z1 z2 / z_transform_2d x z1 z2 = vecsum (0..K1) (\(\lambda\) k2. vecsum (0..K2) (\(\lambda\) k1. z1 cpow  Cx ( &k1) \(*\) z2 cpow  Cx ( &k2) \(*\) a k1 k2)) / vecsum (0..L1) (\(\lambda\) l2. vecsum (0..L2) (\(\lambda\) l1. z1 cpow  Cx ( &l1) \(*\) z2 cpow  Cx ( &l2) \(*\) b l1 l2))
Assumption A1 provides the ROC for LCCDE. Assumption A2 ensures that the input and output 2D arrays are in the first quadrant. Assumption A3 captures the timedomain model of the 2D IIR filter, i.e., the LCCDE (Eq. (8)). Finally, the conclusion presents the transfer function of the 2D IIR filter. The proof process of the above theorem is based on the linearity and shifting properties of the 2D ztransform (Theorems 4 and 5) and summability of the \((L_1, L_2)^{th}\) difference (Lemma 1) along with some complex arithmetic reasoning. Theorem 10 provides the transfer function of a generic 2D IIR image processing filter and is quite useful in the verification of the secondorder 2D medical image processing filter described in Sect. 5.4.
Formal verification of a secondorder 2D image processing filter
To illustrate the practical utilization and effectiveness of the proposed formalization of the 2D ztransform, we apply it to formally analyze a secondorder image processing filter that is widely used for performing various tasks, such as noise removal [1], image smoothing [2] and quality enhancement [5].
A secondorder image processing filter is graphically represented by the flow graph shown in Fig. 2. A flow graph is a collection of branches (directed connections) and nodes (input and output 2D arrays), where nodes are connected using branches. The constants \(c_{01}\), \(c_{10}\), \(c_{11}\), \(c_{02}\), \(c_{12}\), \(c_{20}\), \(c_{21}\) and \(c_{22}\) in Fig. 2 represent the gains of each branches, whereas \({z_1}^{1}\) and \({z_2}^{1}\) present the shift right (horizontal delay) and shift up (vertical delay) operations, respectively. We can mathematically describe this filter using the following linear difference equation.
Alternatively, Eq. (11) can be represented as:
The transfer function corresponding to the difference equationbased model [Eq. (11)] is given as:
Alternatively, the above equation can be represented as:
To verify the transfer function expressed in Eq. (13), we need to formalize the difference equationbased model of the filter [Eq. (11)], which is given in HOL Light as:
Definition 15
Difference EquationBased Model of the SecondOrder Filter \(\vdash _{\textit{def}}\) \(\forall\) y x n1 n2 a b. second_order_filter x y a b n1 n2 \(\Leftrightarrow\) y (n1, n2) = l1l2th_difference y a 2 2 n1 n2  l1l2th_difference x b 0 0 n1 n2
where a and b are the coefficients of input and output 2D arrays. The function second_order_filter accepts input and output 2D arrays, their coefficients a and b and returns the linear difference equation describing the secondorder image processing filter.
Now, we formally verify the transfer function [Eq. (13)] in HOL Light as follows:
Theorem 11
Transfer Function of a SecondOrder Filter
\(\vdash _{\textit{thm}}\) \(\forall\) x y a b z1 z2 n1 c11 c12 c21 c22. [A1]: (z1, z2) IN ROC_2d_LCCDE x y 2 2 b n1 \(\wedge\) [A2]: in_fst_quad_2d_lccde x y \(\wedge\) [A3]: cond_2d_diff_eq_coeff a b b01 b10 b11 b02 b12 b20 b21 b22 \(\wedge\) [A4]: \(\lnot\) (z1 = Cx ( &0)) \(\wedge\) [A5]: \(\lnot\) (z2 = Cx ( &0)) \(\wedge\) [A6]: (\(\forall\) n1 n2. second_order_filter x y a b n1 n2)\(\Rightarrow\) z_transform_2d y z1 z2 / z_transform_2d x z1 z2 = Cx ( &1) / (Cx ( &1)  b01 \(*\) z2 cpow  Cx ( &1)  b10 \(*\) z1 cpow  Cx ( &1)  b11 \(*\) z1 cpow  Cx ( &1) \(*\) z2 cpow  Cx ( &1)  b02 \(*\) z2 cpow  Cx ( &2)  b12 \(*\) z1 cpow  Cx ( &1) \(*\) z2 cpow  Cx ( &2)  b20 \(*\) z1 cpow  Cx ( &2)  b21 \(*\) z1 cpow  Cx ( &2) \(*\) z2 cpow  Cx ( &1)  b22 \(*\) z1 cpow  Cx ( &2) \(*\) z2 cpow  Cx ( &2))
Assumption A1 provides the ROC for the differential equationbased model of the secondorder filter. Assumption A2 ensures that the input and output 2D arrays x and y are in the first quadrant. Assumption A3 asserts that the input and output coefficients are constant. Assumptions A4 and A5 ensure that the complex variables z1 and z2 are nonzero. Assumption A6 captures the timedomain model of the secondorder filter, i.e., Eq. (11). Finally, the conclusion presents the transfer function of the secondorder filter. The verification of the above theorem is mainly based on Theorem 10 along with some complex arithmetic reasoning. Theorem 11 is the formal verification result of the secondorder image processing filter based on our formalization of the 2D ztransform described in Sects. 5.1 and 5.2.
Now, a specialized case of a secondorder image processing filter is graphically represented by the flow graph shown in Fig. 3. This filter can be mathematically represented, by setting the values of the gains of each branch as \(c_{01} = c_{11} = c_{20} = c_{21} = c_{22} = 0\), \(c_{10} = \frac{1}{4}\), \(c_{02} =  \frac{1}{4}\) and \(c_{12} = \frac{1}{4}\) in Eq. (11), as follows.
The transfer function corresponding to the difference equationbased model [Eq. (15)] is described as:
We formally verify the transfer function [Eq. (16)] as:
Theorem 12
Transfer Function of a Specialized SecondOrder Filter
\(\vdash _{\textit{thm}}\) \(\forall\) x y c d z1 z2 n1. [A1]: (z1, z2) IN ROC_2d_LCCDE_spec x y 2 2 b n1 \(\wedge\) [A2]: in_fst_quad_2d_lccde_spec x y \(\wedge\) [A3]: \(\lnot\) (z1 = Cx ( &0)) \(\wedge\) [A4]: \(\lnot\) (z2 = Cx ( &0)) \(\wedge\) [A5]: (\(\forall\) n1 n2. second_order_filter_spec x y c d n1 n2)\(\Rightarrow\) z_transform_2d y z1 z2 / z_transform_2d x z1 z2 = Cx ( &1) / (Cx ( &1)  (1 / 4) \(*\) z1 cpow  Cx ( &1) + (1 / 4) \(*\) z2 cpow  Cx ( &2)  (1 / 4) \(*\) z1 cpow  Cx ( &1) \(*\) z2 cpow  Cx ( &2))
Assumption A1 captures the ROC for the differential equationbased model of the specialized secondorder filter. Assumption A2 asserts the first quadrant conditions on the input and output 2D arrays x and y. Assumptions A3 and A4 ensure that the complex variables z1 and z2 are nonzero. Assumption A5 presents the timedomain model of the specialized secondorder filter, i.e., Eq. (15). Finally, the conclusion captures the transfer function of the specialized secondorder filter. The verification of the above theorem is done almost automatically using Theorem 11, which illustrates the effectiveness of our proposed approach.
Next, we implement the transfer function of the specialized secondorder filter, verified as Theorem 12, in Python. In particular, we implemented the poles [denominator of Eq. (16)] of the transfer function, i.e., the characteristic equation \(1  \frac{1}{4} {z_1}^{1} + \frac{1}{4} {z_2}^{2}  \frac{1}{4} {z_1}^{1} {z_2}^{2} = 0\) on the complex plane \(z_2\) for \(z_1 = e^{i {\omega }_1}, {\omega }_1 \in [0, \pi ]\). Figure 4 provides the root map capturing the poles of the transfer function, and their placement with respect to unit circle in the complex plane can be used for analyzing the 2D stability of the corresponding system. In the case of the specialized secondorder filter (Fig. 4), the presence of poles inside the unit circle provides a sufficient condition for the stability of the corresponding system. However, in case of poles outside the unit circle, the corresponding system will be unstable. Similarly, the onedimensional (1D) stability can be analyzed by implementing the characteristic equation for all \(z_1\) with \(z_2 = 1\) and observing the placement of the poles in the complex \(z_1\) plane.
Discussions
The distinguishing feature of our proposed framework, as compared to the traditional analysis techniques, is that all verified theorems are of generic nature, i.e., all of the functions and variables involved in these theorems are universally quantified and thus can be specialized based on the requirement of the analysis of the image processing filter of any order. For example, Theorem 10 provides the verification of the transfer function of a generic \((L_1, L_2)\)order 2D IIR image processing filter and it can be directly used for analyzing an image processing filter of any order, such as secondorder filter (Theorems 11 and 12). We only need to specialize the gains (\(\alpha (l_1, l_2)\), \(\beta (k_1, k_2)\) in Eqs. (8), (9) and (10) of an image processing filter based on a particular scenario, whereas, in the case of computerbased simulations, we need to model each filter based on its corresponding order, individually that can add a lot of complexity for the case of higherorder filters. Thus, the generic nature of the formalized theorems in our proposed approach makes it better than the transitional analysis methods. Another advantage of our proposed approach is the inherent soundness of the theorem proving technique. It ensures that all the required assumptions are explicitly present along with the theorem, which are often ignored in the conventional simulationsbased analysis, and their absence may affect the accuracy of the corresponding analysis. For example, for a given system (secondorder image processing filter), if we do not incorporate the constraints captured as Assumptions A3, A4 and A5 of Theorem 11 and Assumptions A3 and A4 of Theorem 12, it may lead to some undesired results; for example, it may result into a transfer function that can make a stable system as an unstable system. One of the main limitations of the proposed approach is the significant user involvement in the proposed formalization of ztransform, due to the undecidable nature of the higherorder logic. However, we have developed simplifiers, such as ROC_SIMP_TAC, DIFF_EQ_SIMP_TAC and TRANS_FUN_TAC, that significantly reduce the user guidance in the reasoning process. More details of the proof process can be viewed in our proof script.^{Footnote 5}
Conclusions
2D image processing systems include processing of the images, such as image filtering, enhancement, compression and restoration. These systems are typically analyzed using the 2D ztransform. This paper proposed a HOL theorem provingbased framework for formally analyzing 2D image processing filters. In particular, we formalized the 2D ztransform and formally verified its various classical properties, such as linearity, shifting in time, scaling in (\(z_1, z_2\))domain and complex conjugation. Moreover, we formally analyzed a generic 2D IIR image processing filter. Finally, to demonstrate the practical utilization and effectiveness of the proposed framework, we presented the formal analysis of a secondorder image processing filter.
In future, we aim to formalize the 2D inverse ztransform [16] that will enable us to find the timedomain solutions of the timedomain models of the image processing systems. Another future direction is to formalize the 2D convolution [2] that can greatly simplify the reasoning about systemsofsystems [16].
Availability of data and materials
All data generated or analyzed during this study are included in this paper.
Abbreviations
 2D :

Twodimensional
 CCTV :

Closedcircuit television
 IIR :

Infinite impulse response
 LCCDE :

Linear constant coefficient difference equation
References
J.S. Lim, TwoDimensional Signal and Image Processing (Prentice Hall, Englewood Cliffs, 1990)
J.W. Woods, Multidimensional Signal, Image, and Video Processing and Coding (Elsevier, Amsterdam, 2006)
R. Hussain, S. Zeadally, Autonomous cars: research results, issues, and future challenges. IEEE Commun. Surv. Tutor. 21(2), 1275–1313 (2018)
H. Blasinski, J. Farrell, T. Lian, Z. Liu, B. Wandell, Optimizing image acquisition systems for autonomous driving. Electron. Imaging 2018(5), 161–1 (2018)
C. Behrenbruch, S. Petroudi, S. Bond, J. Declerck, F. Leong, J. Brady, Image filtering techniques for medical image postprocessing: an overview. Br. J. Radiol. 77(suppl–2), 126–132 (2004)
G. Hemalatha, C. Sumathi, Preprocessing techniques of facial image with median and Gabor filters, in: Information Communication and Embedded Systems (IEEE, 2016), pp. 1–6
A.J. Durán, M. Pérez, J.L. Varona, The Misfortunes of a Mathematicians’ Trio using Computer Algebra Systems: Can We Trust? CoRR. arXiv:1312.3270 (2013)
O. Hasan, S. Tahar, Formal Verification Methods. Encyclopedia of Information Science and Technology (IGI Global Pub, Hershey, 2015), pp. 7162–7170
J. Harrison, Handbook of Practical Logic and Automated Reasoning (Cambridge University Press, Cambridge, 2009)
M.J. Gordon, HOL: a proof generating system for higherorder logic, in VLSI Specification, Verification and Synthesis. SECS, vol. 35 (Springer, Berlin, 1988), pp. 73–128
J. Harrison, HOL light: a tutorial introduction, in Formal Methods in ComputerAided Design. LNCS, vol. 1166 (Springer, 1996), pp. 265–269
J. Harrison, HOL light: a tutorial introduction, in Proceedings of the First International Conference on Formal Methods in ComputerAided Design (FMCAD’96). Lecture Notes in Computer Science, vol. 1166, ed. by M. Srivas, A. Camilleri (Springer, Berlin, 1996), pp. 265–269
L. Paulson, ML for the Working Programmer (Cambridge University Press, Cambridge, 1996)
U. Siddique, M.Y. Mahmoud, S. Tahar, On the formalization of ztransform in HOL, in Interactive Theorem Proving (Springer, 2014), pp. 483–498
S.H. Taqdees, O. Hasan, Formalization of laplace transform using the multivariable calculus theory of HOL light, in Logic for Programming Artificial Intelligence and Reasoning(Springer, 2013), pp. 744–758
D.E. Dudgeon, Multidimensional Digital Signal Processing (Prentice Hall, Engewood Cliffs, 1983)
Acknowledgements
This work was supported and funded by Kuwait University, Research Project No. (EO 07/19).
Funding
This work was funded by Kuwait University, Research Project No. (EO 07/19).
Author information
Authors and Affiliations
Contributions
All authors shared, read and approved the manuscript. The results of this paper are mainly compiled by the first author.
Author's information
Adnan Rashid received his Ph.D. degree in Information Technology from School of Electrical Engineering and Computer Science (SEECS), National University of Science and Technology (NUST), Islamabad, Pakistan, in 2019. Prior to this, he received his M.Sc. and M.Phil. degrees in Electronics from the Department of Electronics, QuaidiAzam University (QAU), Islamabad, Pakistan, in 2008 and 2012, respectively. He worked as a postdoctoral fellow at the System Analysis and Verification (SAVe) laboratory of NUST for 1 year till March 2020. He has also worked as a Visiting Researcher at Hardware Verification Group (HVG), Concordia University, Canada, in 2018. Currently, he is an Assistant Professor at SEECS. NUST, Islamabad, Pakistan. He has a strong interest in formal methods, with their applications in control systems, analog circuits, biological systems, robotic systems, cell injection systems, communication systems and transportation systems. He has served as a chair of the Doctoral program at Conference on Intelligent Computer Mathematics, Edinburgh, UK, in 2017.
Sa’ed Abed received his B.Sc. and M.Sc. in Computer Engineering from Jordan University of Science and Technology, Jordan, in 1994 and 1996, respectively. In 2008, he received his Ph.D. in Computer Engineering from Concordia University, Canada. He has previously worked at Hashemite University, Jordan, as an Assistant Professor from 2008 to 2014. Currently, he is an Associate Professor in Computer Engineering Department at Kuwait University. His research interests include formal methods, SAT solvers and VLSI design. Dr. Abed also served as a reviewer for various international conferences and journals. He published over 110 papers in reputable journals and conferences.
Osman Hasan received his B.Eng. (Hons) degree from the University of Engineering and Technology, Peshawar, Pakistan, in 1997, and the MEng and Ph.D. degrees from Concordia University, Montreal, Quebec, Canada, in 2001 and 2008, respectively. Before his Ph.D., he worked as an ASIC Design Engineer from 2001 to 2004 at LSI Logic. He worked as a postdoctoral fellow at the Hardware Verification Group (HVG) of Concordia University for 1 year until August 2009. Currently, he is a Professor at the School of Electrical Engineering and Computer Science of National University of Science and Technology (NUST), Islamabad, Pakistan. He is the founder and director of System Analysis and Verification (SAVe) Lab at NUST, which mainly focuses on the design and formal verification of energy, embedded and ehealthrelated systems. He has received several awards and distinctions, including the Pakistan’s Higher Education Commission’s Best University Teacher (2010) and Best Young Researcher Award (2011) and the President’s gold medal for the best teacher of the University from NUST in 2015. Dr. Hasan is a senior member of IEEE, member of the ACM, Association for Automated Reasoning (AAR) and the Pakistan Engineering Council.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
All procedures performed in this paper were in accordance with the ethical standards of research community.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Rashid, A., Abed, S. & Hasan, O. Formal analysis of 2D image processing filters using higherorder logic theorem proving. EURASIP J. Adv. Signal Process. 2022, 53 (2022). https://doi.org/10.1186/s13634022008823
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s13634022008823
Keywords
 Formal analysis
 2D image processing systems
 2D ztransform
 Theorem proving
 HOL Light
 Higherorder logic