Formal analysis of 2D image processing filters using higher-order logic theorem proving

Abstract

Two-dimensional (2D) image processing systems are concerned with the processing of the images represented as 2D arrays and are widely used in medicine, transportation and many other autonomous systems. The dynamics of these systems are generally modeled using 2D difference equations, which are mathematically analyzed using the 2D z-transform. It mainly involves a transformation of the difference equations-based models of these systems to their corresponding algebraic equations, mapping the 2D arrays (2D discrete-time signals) over the ($z_1$,$z_2$)-domain. Finally, these ($z_1$,$z_2$)-domain representations are used to analyze various properties of these systems, such as transfer function and stability. Conventional techniques, such as paper-and-pencil proof methods, and computer-based simulation techniques for analyzing these filters cannot assert the accuracy of the analysis due to their inherent limitations like human error proneness, limited computational resources and approximations of the mathematical expressions and results. In this paper, as a complimentary technique, we propose to use formal methods, higher-order logic (HOL) theorem proving, for formally analyzing the image processing filters. These methods can overcome the limitations of the conventional techniques and thus ascertain the accuracy of the analysis. In particular, we formalize the 2D z-transform based on the multivariate theories of calculus using the HOL Light theorem prover. Moreover, we formally analyze a generic ($L_1,L_2$)-order 2D infinite impulse response image processing filter. We illustrate the practical effectiveness of our proposed approach by formally analyzing a second-order image processing filter.

1 Introduction

Two-dimensional (2D) image processing systems [1, 2] typically involve image filtering, editing, enhancement, compression and restoration of the images represented as 2D arrays (2D discrete-time signals). Image processing filters [2] are the fundamental components of the 2D image processing systems that are widely used for image filtering. These filters are categorized as high-pass, band-pass and low-pass filters based on the passage of the allowable range of frequencies. For example, a high-pass filter permits a range of frequencies greater than a certain threshold. Moreover, these filters are widely used in autonomous vehicles [3, 4] and medicine [5]. For example, they are used to perform various image processing tasks for controlling the autonomous vehicles, such as noise reduction, color normalization, histogram equalization and edge detection, to enhance the quality of the images captured using various devices such as closed-circuit television (CCTV) and webcams [6]. Similarly, they are widely used in medicine for performing various image pre- and post-processing tasks, such as image quality enhancement, noise removal and image smoothing [5].

The dynamics of these image processing systems are generally modeled using 2D difference equations. Next, the 2D z-transform is used to mathematically analyze these systems. It mainly involves a transformation of the difference equations-based models of these systems to their corresponding algebraic equations, using the definition and various classical properties of the 2D z-transform, while mapping 2D arrays over the ($z_1$,$z_2$)-domain. Finally, these ($z_1$,$z_2$)-domain representations are used to analyze various properties of these image processing systems like transfer function and stability [2].

Conventionally, the image processing filters have been analyzed using paper-and-pencil proof techniques and computer-based symbolic and numerical methods. However, in the former case, the analysis is error-prone due to the highly involved human manipulation, particularly for analyzing the larger and complex image processing systems, and thus we cannot ascertain an absolute accuracy of the analysis in this approach. Similarly, the later approaches suffer from some of their inherent limitations. For example, the symbolic methods involve a large number of unverified symbolic procedures residing in the root of the associated tools [7]. Similarly, the numerical techniques include a finite number of iterations due to the limited power of the computing machines. Moreover, they are based on the mathematical results that are approximated due to the finite precision arithmetic of computers. Therefore, these conventional approaches cannot be trusted when analyzing the image processing filters utilized in various safety-critical areas, such as autonomous driving and medicine.

Formal methods [8] are system analysis techniques that are based on developing a mathematical model of the system using logic and verifying its various properties using deductive reasoning. Higher-order logic (HOL) theorem proving [9, 10] is a widely utilized formal method for analyzing many safety-critical systems. In this paper, we propose a HOL theorem proving-based framework for analyzing the image processing filters. In particular, we formalize the 2D z-transform based on the multivariate theories of calculus using the HOL Light theorem prover [11]. The main motivation of selecting  HOL Light for the proposed formalization is the presence of the fundamental libraries of multivariate calculus,¹ vectors² and matrices,³ which are required to formally analyze the 2D image processing systems.

2 Contributions of the paper

The novel contributions of the paper are:

• Formalization of 2D z-transform and its region of convergence (ROC).

• Formal verification of various classical properties of 2D z-transform, such as linearity, shifting in time-domain, scaling in ($z_1$, $z_2$)-domain and complex conjugation.

• Formal analysis of a generic ($L_1, L_2$)-order 2D IIR image processing filter.

• Formal analysis of a second-order image processing filter

3 Preliminaries

This section provides an introduction to the HOL Light theorem prover and the formalization of some fundamental concepts from the multivariate calculus libraries of HOL Light that facilitate the understanding of the rest of the paper.

3.1 HOL Light Theorem prover

HOL Light[12] is a proof assistant for developing proofs of the mathematical concepts written as theorems in higher-order logic. HOL Light is implemented in the strongly typed functional programming language ML [13]. A theorem is a statement that is formalized as an axiom or can be implied from already verified theorems using inference rules. Soundness is assured in a theorem proving environment as every new theorem is verified using the primitive inference rules or any other previously verified theorems. HOL Light provides an extensive support of theories, such as Boolean algebra, arithmetic, real numbers, vectors and matrices, which are extensively used in our formalization. Indeed, one of the motivations for selecting the HOL Light theorem prover for the proposed framework is the availability of extensive libraries of vectors and matrices.

3.2 Multivariable calculus theories in HOL Light

This section presents an introduction to some fundamental concepts formalized in HOL Light, such as summability, infinite summation and vector summation, and some HOL Light notations that help understanding the rest of the paper.

An N-dimensional vector in HOL Light is formalized as a $\mathbb {R}^N$ column matrix capturing individual elements as real numbers. All vector operations are then considered as matrix manipulations. Most of the theorems in multivariable calculus theories of HOL Light are proved for functions with an arbitrary data type of $\mathbb {R}^M \rightarrow \mathbb {R}^N$. Similarly, complex numbers ($\mathbb {C}$) can be described as $\mathbb {R}^2$ instead of defining a new data type. The HOL Light symbol &: $\mathbb {N} \rightarrow \mathbb {R}$ represents an injection of natural numbers to real numbers. Similarly, the symbol Cx: $\mathbb {R} \rightarrow \mathbb {C}$ typecasts real numbers to complex numbers. The symbols Re: $\mathbb {C} \rightarrow \mathbb {R}$ and Im: $\mathbb {C} \rightarrow \mathbb {R}$ represent the real and imaginary components of a complex number, respectively. The HOL Light symbol $\mathtt {\%}$ captures the scalar multiplication of a vector or matrix. Similarly, a matrix–vector multiplication is modeled as $\mathtt {**}$ in HOL Light.

The generalized summation over an arbitrary function fn: A $\rightarrow \mathbb {R}^N$ is formalized in HOL Light as follows:

Definition 1

Generalized Summation of Vector $\vdash _{\textit{def}}$ $\forall$ st fn. vecsum st fn = (lambda k. summ st ($\lambda$ x. fn x$k))

where vecsum accepts a set st: A $\rightarrow$ bool over which the summation occurs and a function fn of data type A $\rightarrow \mathbb {R}^N$ and returns a generalized vector summation over the set st. Here, the HOL Light function summ provides a finite summation for a fn over real numbers. For example, a mathematical expression $\sum \limits _{k = 0}^{n}{f(k)}$ is described in HOL Light as vecsum (0..n) fn.

Definition 2

Summs $\vdash _{\textit{def}}$ $\forall$ st fn lt. (fn summs lt) st $\Leftrightarrow$                    (($\lambda$ n. vecsum (st INTER (0..n)) fn) $\rightarrow$ lt) sequentially

The HOL Light function summs accepts a set of natural numbers st: N $\rightarrow$ bool, a function fn: $\mathbb {N}$ $\rightarrow$ $\mathbb {R}^N$ and a limit value lt: $\mathbb {R}^N$ and returns the traditional mathematical expression $\sum \limits _{k = 0}^{\infty }{f(k)} = L$. Here, INTER captures the intersection of two sets. Similarly, sequentially represents a net providing a sequential growth of a function f, i.e., $f(k), f(k + 1), f(k + 2), \ldots ,$ etc. This is mainly used in modeling the concept of an infinite summation.

We provide the formalization of the summability of a function fn: $\mathbb {N}$ $\rightarrow$ $\mathbb {R}^N$ over st: N $\rightarrow$ bool, which ensures that there exist some limit value L: $\mathbb {R}^N$, such that $\sum \limits _{k = 0}^{\infty }{f(k)} = L$ in HOL Light as:

Definition 3

Summability of a Function $\vdash _{\textit{def}}$ $\forall$ fn st. summable fn st $\Leftrightarrow$ ($\exists$ lt. (fn summs lt) st)

The limit of a function fn: A $\rightarrow \mathbb {R}^N$ is formalized as:

Definition 4

Limit of a Function $\vdash _{\textit{def}}$ $\forall$ net f. limt net fn = ($\in$ lt. (fn $\rightarrow$ lt) net)

where the function limt takes a net with components of data type A and a function fn and returns a limit value lt: $\mathbb {R}^N$ to which fn converges at the given net. It is formalized using the Hilbert choice operator $\in$. Similarly, the concept tends to ($\rightarrow$) is formalized in HOL Light as:

Definition 5

Tends to $\vdash _{\textit{def}}$ $\forall$ fn lt net. (fn $\rightarrow$ lt) net $\Leftrightarrow$                    $\forall$e. &0 < e $\Rightarrow$ eventually ($\lambda$ x. dist (fn x, lt) < e) net

Now, we provide a formalization of an infinite summation, which is used in the formal definition of the 2D z-transform presented in Sect. 5.1.

Definition 6

Infinite Summation of a Function $\vdash _{\textit{def}}$ $\forall$ fn st. inftsumm st fn = ($\in$ lt. (fn summs lt) st)

where the HOL Light function inftsumm accepts st: num $\rightarrow$ bool specifying the starting point and a function fn of data type $\mathbb {N} \rightarrow \mathbb {R}^N$, and returns a limit value lt: $\mathbb {R}^N$ to which the infinite summation of fn converges from the given st.

Next, we formally verify an equivalence of the infinite summation (Definition 6) to its alternate form in terms of sequential limit as the following HOL Light theorem:

Theorem 1

Relationship Between Infinite Summation and the Sequential Limit $\vdash _{\textit{thm}}$ $\forall$ st fn. inftsumm st fn = limt sequentially ($\lambda$ k. vecsum (st INTER (0..k)) fn)

4 Methods

Figure 1 depicts our proposed method for analyzing the image processing filters using HOL theorem proving. The user provides the 2D difference equations that model the dynamics of the image processing system, which needs to be analyzed. This 2D difference equation is modeled in higher-order logic using the multivariate calculus theories of HOL Light. In the next step, we formalize the 2D z-transform that is required for mathematically analyzing the image processing systems. It mainly transforms the difference equations-based models of these systems to their corresponding algebraic equations, using the definition and various classical properties, such as linearity, shifting and scaling, of the 2D z-transform, while mapping 2D arrays over the ($z_1$,$z_2$)-domain. Finally, these ($z_1$,$z_2$)-domain representations are used to analyze various properties of these systems, such as transfer function and the solution of the corresponding difference equations.

Fig. 1

Proposed framework

5 Results

5.1 Formalization of the 2D z-transform

The 2D z-transform of a 2D discrete-time function (2D array) $f (n_1, n_2)$ is mathematically expressed as follows [2]:

$$\begin{aligned} F (z_1, z_2) = \sum _{n_1 = 0}^{\infty } \sum _{n_2 = 0}^{\infty } f (n_1, n_2) {z_1}^{-{n_1}} {z_2}^{-{n_2}} \end{aligned}$$

(1)

where f is a function of type $\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}$, and $z_1$ and $z_2$ are complex variables. The limits from 0 to $\infty$ make Eq. (1) as a mathematical representation of a unilateral 2D z-transform. We have opted for this representation based on the same motivation that was considered for one-dimensional z-transform [14] and the Laplace transform [15].

We formalize the 2D z-transform [Eq. (1)] in HOL Light as follows:

Definition 7

2D z-Transform $\vdash _{\textit{def}}$ $\forall$ f z1 z2. z_transform_2d f z1 z2 = inftsumm (from 0)              ($\lambda$ n1. inftsumm (from 0) ($\lambda$ n2. f n1 n2 / (z1 pow n1 $*$ z2 pow n2)))

where z_transform_2d accepts a function of type $\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}$ and two complex variables z1: $\mathbb {C}$ and z2: $\mathbb {C}$ and returns a complex number, which represents the 2D z-transform of f: $\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}$ according to Eq. (1).

An essential issue with the applicability of the 2D z-transform of $f (n_1, n_2)$ is the existence of $F (z_1, z_2)$ that occurs due to the presence of the infinite summations in Eq. (1). Thus, we need to identify conditions for the existence of the 2D z-transform. A set of all those values of $z_1$ and $z_2$ for which the infinite summations are converging and $F (z_1, z_2)$ is finite (or summable) is known as the ROC. It is mathematically expressed as follows:

$$\begin{aligned} ROC = {z_1, z_2 \in \mathbb {C}: \exists k. \sum _{n_1 = 0}^{\infty } \sum _{n_2 = 0}^{\infty } f (n_1, n_2) {z_1}^{-{n_1}} {z_2}^{-{n_2}} = k} \end{aligned}$$

(2)

We formalize the ROC of the 2D z-transform as follows:

Definition 8

Region of Convergence (ROC) $\vdash _{\textit{def}}$ $\forall$ f n1. ROC_2d f n1 = {(z1, z2) | $\lnot$(z1 = Cx( &0)) $\wedge$ $\lnot$(z2 = Cx( &0)) $\wedge$ z_tr_summable f z1 z2 n1 $\wedge$ z_tr_td_summable f z1 z2}

where ROC_2d accepts a function f: $\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}$ and n1 capturing the starting point of the outer summation of the 2D z-transform [Eq. (1)] and returns a set of nonzero values of variables z1 and z2 for which the 2D z-transform of f exists. It is necessary to specify the associated ROC_2d to compute the 2D z-transform. Moreover, the functions z_tr_summable and z_tr_td_summable capture the summability of the function f for the inner and the outer (double) summations, respectively, and are formalized in HOL Light as follows:

Definition 9

Summability of Function for Inner Summation $\vdash _{\textit{def}}$ $\forall$ f z n1. z_tr_summable f z1 z2 n1 =         ($\forall$ n1. summable (from 0) ($\lambda$ n2. f n1 n2 / (z1 pow n1 $*$ z2 pow n2)))

Definition 10

Summability of Function for Outer (Double) Summation $\vdash _{\textit{def}}$ $\forall$ f z1 z2. z_tr_td_summable f z1 z2 = summable (from 0) ($\lambda$ n1. inftsumm (from 0) ($\lambda$ n2. f n1 n2 / (z1 pow n1 $*$ z2 pow n2)))

Moreover, we verify two fundamental properties of ROC, such as the linearity of the ROC and scaling of the ROC, which are quite helpful for formally verifying the classical properties of the 2D z-transform in Sect. 5.2.

Theorem 2

Linearity of ROC $\vdash _{\textit{thm}}$ $\forall$ z1 z2 a b f g n1. [A1]: (z1, z2) IN ROC_2d f n1 $\wedge$ [A2]: (z1, z2) IN ROC_2d g n1 $\Rightarrow$ (z1, z2) IN ROC_2d ($\lambda$ n1 n2. a $*$ f n1 n2) n1 INTER ROC_2d ($\lambda$ n1 n2. b $*$ g n1 n2) n1

Theorem 3

Scaling of ROC

$\vdash _{\textit{thm}}$ $\forall$ z1 z2 a f n1. [A]: (z1, z2) IN ROC_2d f n1 $\Rightarrow$ (z1, z2) IN ROC_2d ($\lambda$ n1 n2. f n1 n2 / a) n1

Theorem 2 ensures that if (z1, z2) is inside ROC_2d f n1 and ROC_2d g n1 for functions f and g then it is also inside the intersection of both ROCs for the scaled version of these functions. Similarly, Theorem 3 provides the scaling property with respect to the division by a complex number a.

5.2 Formal verification of the classical properties of the 2D z-transform

We use Definitions 7 and 8 and Theorems 2 and 3 for verifying some of the classical properties of the 2D z-transform in HOL Light. This verification plays a vital role in reducing the effort required for analyzing image processing systems, as described later in Sects. 5.3 and 5.4.

Linearity of the 2D z-Transform The linearity of the 2D z-transform is mainly used in decomposing complex (larger) systems to subsystems or combining smaller systems to larger ones having different scaling inputs. It can be mathematically expressed as follows:

If $\mathcal {Z}[f(n1, n2)] = F (z_1, z_2)$ and $\mathcal {Z}[g(n_1, n_2)] = G(z_1, z_2),$ then the following holds:

$$\begin{aligned} \mathcal {Z}[\alpha *f (n_1, n_2) \pm \beta *g (n_1, n_2)] = \alpha *F (z_1, z_2) \pm \beta *G (z_1, z_2) \end{aligned}$$

(3)

The 2D z-transform of a linear combination of 2D sequences (or arrays) is equal to the linear combination of the 2D z-transform of the individual arrays. We verify linearity property in HOL Light as:

Theorem 4

Linearity of the 2D z-Transform

$\vdash _{\textit{def}}$ $\forall$ f g z1 z2 a b n1.      [A1]: (z1, z2) IN ROC_2d f n1 $\wedge$ [A2]: (z1, z2) IN ROC_2d g n1             $\Rightarrow$ z_transform_2d ($\lambda$ n1 n2. a $*$ f n1 n2 ± b $*$ g n1 n2) z1 z2 =                    a $*$ z_transform_2d f z1 z2 ± b $*$ z_transform_2d g z1 z2

where a: $\mathbb {C}$ and b: $\mathbb {C}$ are arbitrary complex constants. Assumptions A1 and A2 capture the regions of the convergence of functions f and g, respectively. The proof of the above theorem is mainly based on Theorem 2 and the linearity of the infinite summation along with some complex arithmetic reasoning.

Shifting Property of the 2D z-Transform The shifting property of the 2D z-transform is mostly used for analyzing the 2D linear shift-invariant (LSI) systems. In particular, it is used to solve the difference equations capturing the dynamics of these systems. The shifting property expresses the transform of the shifted signal $f(n_1 - m_1, n_2 - m_2)$ in terms of its 2D z-transform $F(z_1, z_2)$.

If $\mathcal {Z}[f(n_1, n_2)] = F(z_1, z_2)$ and assuming $f (-n_1, n_2) = 0$, $f (n_1, -n_2) = 0$ and $f (-n_1, -n_2) = 0$, $\forall n_1 = 1, 2, \ldots , m_1$ and $\forall n_2 = 1, 2, \ldots , m_2$, i.e., $f(n_1, n_2)$ is nonzero in the first quadrant only, then the shifting of a 2D array is mathematically expressed as follows:

$$\begin{aligned} \mathcal {Z}[f(n_1 - m_1, n_2 - m_2)] = z_1^{- m_1} *z_2 ^{- m_2} *F (z_1, z_2) \end{aligned}$$

(4)

We formally verify the above property in HOL Light as:

Theorem 5

Shifting in Time Domain

$\vdash _{\textit{thm}}$ $\forall$ f z1 z2 m1 m2 n1. [A1]: (z1, z2) IN ROC_2d f n1 $\wedge$ [A2]: in_fst_quad_2d f $\Rightarrow$ z_transform_2d ($\lambda$ n1 n2. f (n1 - m1 n2 - m2)) z1 z2 = z_transform_2d f z1 z2 / (z1 pow m1 $*$ z2 pow m2)

where the function in_fst_quad_2d ensures that the function f is nonzero in the first quadrant only and is formalized in a relational form, i.e., f (n1 - m1, n2 - m2), $\forall$ m1 m2. m1 < n1, m2 < n2. The verification of Theorem 5 is mainly based on the properties of complex numbers along with two properties regarding the negative offset of series and infinite summation. More details about the proof process of this theorem can be found in our proof script.⁴

Scaling in ($z_1, z_2$)-domain Property of the 2D z-Transform The scaling property of the 2D z-transform results in shrinking or expansion of the ($z_1, z_2$)-domain, i.e., 4D complex ($z_1, z_2$)-plane. If $\mathcal {Z}[f (n_1, n_2)] = F (z_1, z_2)$, then two different types of scaling are defined as:

$$\begin{aligned}&\mathcal {Z}[{h_1}^{n_1} {h_2}^{n_2} f(n_1, n_2)] = F \ \left( \dfrac{z_1}{h_1}\right) \ \left( \dfrac{z_2}{h_2}\right) \end{aligned}$$

(5)

$$\begin{aligned}&\mathcal {Z}[{w_1}^{-n_1} {w_2}^{-n_2} f(n_1, n_2)] = F \ (w_1 z_1) \ (w_2 z_2) \end{aligned}$$

(6)

If $h_1$ and $h_2$ are positive real numbers, then the scaling is interpreted as expansion of the 4D complex ($z_1, z_2$)-plane. On the other hand, multiplication by ${w_1}^{-n_1}$ and ${w_2}^{-n_2}$ [Eq. (6)] shrinks the ($z_1, z_2$)-domain.

We verify the above theorems in HOL Light as:

Theorem 6

Scaling in ($z_1, z_2$) -Domain (Positive/Expansion)

$\vdash _{\textit{thm}}$ $\forall$ f z1 z2 n1 h1 h2.      [A1]: (inv h1 $*$ z1, inv h2 $*$ z2) IN ROC_2d f n1 $\wedge$      [A2]: (z1, z2) IN ROC_2d f n1       $\Rightarrow$ z_transform_2d ($\lambda$ n1 n2. h1 pow n1 $*$ h2 pow n2 $*$ f n1 n2) z1 z2 =         z_transform_2d f (inv h1 $*$ z1, inv h2 $*$ z2)

Theorem 7

Scaling in ($z_1, z_2$)-Domain (Negative/Shrinking)

$\vdash _{\textit{thm}}$ $\forall$ f z1 z2 n1 w1 w2.      [A1]: (w1 $*$ z1, w2 $*$ z2) IN ROC_2d f n1 $\wedge$      [A2]: (z1, z2) IN ROC_2d f n1   $\Rightarrow$ z_transform_2d ($\lambda$ n1 n2. w1 pow (-n1) $*$ w2 pow (-n2) $*$ f n1 n2) z1 z2 =         z_transform_2d f (w1 $*$ z1) (w2 $*$ z2)

Complex Conjugation Property of the 2D z-Transform The complex conjugation property facilitates an easy manipulation of the 2D z-transform of conjugated arrays. It is mathematically expressed as follows:

$$\begin{aligned} \mathcal {Z}[f^{*}(n_1, n_2)] = F^{*}({z_1}^{*}, {z_2}^{*}) \end{aligned}$$

(7)

where $f^{*}(n_1, n_2)$ represents the complex conjugate of an array $f (n_1, n_2)$. The corresponding formalization of the complex conjugation property in HOL Light is given as follows:

Theorem 8

Complex Conjugation

$\vdash _{\textit{thm}}$ $\forall$ f z1 z2 n1. [A]: (cnj z1, cnj z2) IN ROC_2d f n1 $\Rightarrow$ z_transform_2d ($\lambda$ n1 n2. cnj (f n1 n2)) z1 z2 = cnj (z_transform_2d f (cnj z1, cnj z2))

5.3 Formal verification of a ($L_1, L_2$)-order 2D infinite impulse response (IIR) image processing filter

2D digital filters [1] are integral components of the image processing systems. Their main responsibility includes the decomposition of an image to multiple frequency bands, restricting a 2D array/signal to a certain frequency band and providing the input–output relationship of these systems. For example, a low-pass filter allows a range of frequencies less than a certain threshold [2]. The analysis of an image processing filter mainly involves developing its mathematical model using a 2D difference equation. The next step is to apply 2D z-transform on both sides of the difference equation. Finally, the definition and the classical properties of the 2D z-transform are used to perform transfer function-based analysis of the given filter.

The impulse response of a discrete-time system captures its behavior for the scenario when dirac-delta function is acting as an input array [2]. 2D image processing infinite impulse response (IIR) filters have a nonzero impulse response function over an infinite length of time. For these filters, the present output depends on the present input and all previously computed input and output values.

Mathematically, the 2D image processing filters are described using the following difference equation [16]:

$$\begin{aligned} \begin{aligned} y (n_1, n_2)&= \sum _{l_1 = 0}^{L_1 - 1} \sum _{l_2 = 0}^{L_2 - 1} \alpha (l_1, l_2) x (n_1 - l_1, n_2 - l_2) \\&\quad - \sum _{k_1 = 1}^{K_1 - 1} \sum _{k_2 = 1}^{K_2 - 1} \beta (k_1, k_2) y (n_1 - k_1, n_2 - k_2) \end{aligned} \end{aligned}$$

(8)

where $\alpha (l_1, l_2)$ and $\beta (k_1, k_2)$ are input and output coefficients, respectively. The output array $y (n_1, n_2)$ is a linear combination of the previous $K_1 - 1$ and $K_2 - 1$ output samples, the present input $x (n_1, n_2))$, and $L_1 - 1$ and $L_2 - 1$ previous input samples. Moreover, for the shift-invariant filter, $\alpha (l_1, l_2)$ and $\beta (k_1, k_2)$ are the complex constants ($\mathbb {C}$). Therefore, Eq. (8) is known as a linear constant coefficient difference equation (LCCDE). The 2D z-transform of a ${(L_1,L_2)}^{th}$ difference represented in the form of $f (n_1, n_2)$ is given as:

$$\begin{aligned} \begin{aligned} \mathcal {Z} \left[ \sum _{l_1 = 0}^{L_1 - 1} \sum _{l_2 = 0}^{L_2 - 1} \alpha (l_1, l_2) f (n_1 - l_1, n_2 - l_2)\right] = F ({z_1}, {z_2}) \sum _{l_1 = 0}^{L_1 - 1} \sum _{l_2 = 0}^{L_2 - 1} \alpha (l_1, l_2) {z_1}^{-l_1} {z_2}^{-l_2} \end{aligned} \end{aligned}$$

(9)

The corresponding transfer function of the 2D IIR filter is mathematically expressed as [16]:

$$\begin{aligned} H (z_1, z_2) = \dfrac{Y (z_1, z_2)}{X (z_1, z_2)} = \dfrac{\sum \limits _{l_1 = 0}^{L_1 - 1} \sum \limits _{l_2 = 0}^{L_2 - 1} \alpha (l_1, l_2) {z_1}^{-{l_1}} {z_2}^{-{l_2}}}{\sum \limits _{k_1 = 0}^{K_1 - 1} \sum \limits _{k_2 = 0}^{K_2 - 1} \beta (k_1, k_2) {z_1}^{-{k_1}} {z_2}^{-{k_2}}} \end{aligned}$$

(10)

To formally verify the transfer function of the 2D filter [Eq. (10)], we formalize the ${(L_1,L_2)}^{th}$ difference as follows:

Definition 11

Formalization of the ${(L_1,L_2)}^{th}$ Difference $\vdash _{\textit{def}}$ $\forall$ f c L1 L2 n1 n2. l1l2th_difference f c L1 L2 n1 n2 = vecsum (0..L1) ($\lambda$ l1. vecsum (0..L2) ($\lambda$ l2. c l1 l2 $*$ f (n1 - l1) (n2 - l2)))

The function l1l2th_difference accepts a function f: $\mathbb {N} \rightarrow \mathbb {N} \rightarrow \mathbb {C}$, coefficients of the difference equation c l1 l2, the order (L1, L2) of the 2D difference equation and the variables n1 and n2 and returns the $(L_1, L_2)^{th}$ difference. It uses the function vsum s f twice to capture the double summation.

Next, we formalize a general LCCDE [Eq. (8)] as follows:

Definition 12

Formalization of the LCCDE $\vdash _{\textit{def}}$ $\forall$ y x M1 M2 N1 N2 n1 n2 a b. LCCDE x y a b M1 M2 N1 N2 n1 n2 $\Leftrightarrow$   y (n1, n2) = l1l2th_difference y a M1 M2 n1 n2 - l1l2th_difference x b N1 N2 n1 n2

Next, we verify the 2D z-transform of the $(L_1, L_2)^{th}$ difference [Eq. (9)] as:

Theorem 9

The 2D z-Transform of the $(L_1, L_2)^{th}$ Difference

$\vdash _{\textit{thm}}$ $\forall$ f c L1 L2 z1 z2 n1. [A1]: (z1, z2) IN ROC_2d f n1 $\wedge$ [A2]: in_fst_quad_2d f $\Rightarrow$ z_transform_2d ($\lambda$ n1 n2. l1l2th_difference f c L1 L2 n1 n2) z1 z2 = z_transform_2d f z1 z2 $*$ vecsum (0..L1) ($\lambda$ l2. vecsum (0..L2) ($\lambda$ l1. z1 cpow - Cx ( &l1) $*$ z2 cpow - Cx ( &l2) $*$ c l1 l2))

where Assumption A1 ensures that $(z_1, z_2)$ are in the region of convergence of the function f. Assumption A2 implies that the function f is in the first quadrant. Finally, the conclusion provides the 2D z-transform of the $(L_1, L_2)^{th}$ difference. The verification of the above theorem is mainly based on induction on N1 and N2 and Theorems 2 and 4 along with the following lemma about the summability of $(L_1, L_2)^{th}$ difference equation.

Lemma 1

Summability of the $(L_1, L_2)^{th}$ Difference

$\vdash _{\textit{thm}}$ $\forall$ f c L1 L2 z1 z2 n1.      [A1]: (z1, z2) IN ROC_2d f n1 $\wedge$      [A2]: in_fst_quad_2d f   $\Rightarrow$ (z1, z2) IN ROC_2d ($\lambda$ n1 n2. l1l2th_difference f c L1 L2 n1 n2) n1

To verify the transfer function of the 2D filter [Eq. (10)], we have to ensure that the 2D input and output arrays exist in the first quadrant only. Moreover, the denominator of Eq. (10) should be nonzero. We formalize both these requirements in HOL Light as follows:

Definition 13

First Quadrant Input and Output 2D Arrays for LCCDE $\vdash _{\textit{def}}$ in_fst_quad_2d_lccde x y $\Leftrightarrow$ in_fst_quad_2d x $\wedge$ in_fst_quad_2d y

Definition 14

ROC LCCDE $\vdash _{\textit{def}}$ $\forall$ x y K1 K2 lst n1 ROC_2d_LCCDE x y K1 K2 lst n1 =    (ROC_2d x n1) INTER (ROC_2d y n1) DIFF      {(z1, z2) | vecsum (0..K1) ($\lambda$ k2. vecsum (0..K2)        ($\lambda$k1. z1 cpow - Cx ( &k1) $*$ z2 cpow - Cx ( &k2) $*$ EL k1 lst)) = Cx ( &0)} DIFF         {(z1, z2) | z_transform_2d x z1 z2 = Cx ( &0)}

where the function in_fst_quad_2d_lccde (Definition 13) accepts the input and output 2D arrays x and y and asserts the first quadrant condition for both arrays. Similarly, ROC_2d_LCCDE (Definition 14) provides the ROC of the input and output 2D arrays. It uses the HOL Light function DIFF to exclude all values of the denominator, where the transfer function of the 2D IIR filter becomes undefined.

Now, we provide the formal verification of the transfer function of a 2D IIR filter in HOL Light as follows:

Theorem 10

Transfer Function of a 2D IIR Filter

$\vdash _{\textit{thm}}$ $\forall$ x y a b L1 L2 K1 K2 z1 z2 n1.[A1]: (z1, z2) IN ROC_2d_LCCDE x y K1 K2 blst n1 $\wedge$[A2]: in_fst_quad_2d_lccde x y $\wedge$[A3]: ($\forall$ n1 n2. LCCDE x y a b L1 L2 K1 K2 n1 n2)$\Rightarrow$ z_transform_2d y z1 z2 / z_transform_2d x z1 z2 = vecsum (0..K1) ($\lambda$ k2. vecsum (0..K2) ($\lambda$ k1. z1 cpow - Cx ( &k1) $*$ z2 cpow - Cx ( &k2) $*$ a k1 k2)) / vecsum (0..L1) ($\lambda$ l2. vecsum (0..L2)  ($\lambda$ l1. z1 cpow - Cx ( &l1) $*$ z2 cpow - Cx ( &l2) $*$ b l1 l2))

Assumption A1 provides the ROC for LCCDE. Assumption A2 ensures that the input and output 2D arrays are in the first quadrant. Assumption A3 captures the time-domain model of the 2D IIR filter, i.e., the LCCDE (Eq. (8)). Finally, the conclusion presents the transfer function of the 2D IIR filter. The proof process of the above theorem is based on the linearity and shifting properties of the 2D z-transform (Theorems 4 and 5) and summability of the $(L_1, L_2)^{th}$ difference (Lemma 1) along with some complex arithmetic reasoning. Theorem 10 provides the transfer function of a generic 2D IIR image processing filter and is quite useful in the verification of the second-order 2D medical image processing filter described in Sect. 5.4.

5.4 Formal verification of a second-order 2D image processing filter

To illustrate the practical utilization and effectiveness of the proposed formalization of the 2D z-transform, we apply it to formally analyze a second-order image processing filter that is widely used for performing various tasks, such as noise removal [1], image smoothing [2] and quality enhancement [5].

A second-order image processing filter is graphically represented by the flow graph shown in Fig. 2. A flow graph is a collection of branches (directed connections) and nodes (input and output 2D arrays), where nodes are connected using branches. The constants $c_{01}$, $c_{10}$, $c_{11}$, $c_{02}$, $c_{12}$, $c_{20}$, $c_{21}$ and $c_{22}$ in Fig. 2 represent the gains of each branches, whereas ${z_1}^{-1}$ and ${z_2}^{-1}$ present the shift right (horizontal delay) and shift up (vertical delay) operations, respectively. We can mathematically describe this filter using the following linear difference equation.

Fig. 2

Flow graph of a second-order image processing filter

$$\begin{aligned} \begin{aligned} y (n_1, n_2) = x (n_1, n_2) + \sum \limits _{k_1 = 0}^{2}&\sum \limits _{k_2 = 0}^{2} c_{k_1 k_2} y (n_1 - k_1, n_2 - k_2), \\&(k_1, k_2) \ne 0 \end{aligned} \end{aligned}$$

(11)

Alternatively, Eq. (11) can be represented as:

$$\begin{aligned} \begin{aligned} y (n_1, n_2) &=x (n_1, n_2) + c_{01} y (n_1, n_2 - 1) + c_{10} y (n_1 - 1, n_2) + \\ c_{11}&y (n_1 - 1, n_2 - 1) + c_{02} y (n_1, n_2 - 2) + \\ c_{12}&y (n_1 - 1, n_2 - 2) + c_{20} y (n_1 - 2, n_2) + \\&c_{21} y (n_1 - 2, n_2 - 1) + c_{22} y (n_1 - 2, n_2 - 2) \end{aligned} \end{aligned}$$

(12)

The transfer function corresponding to the difference equation-based model [Eq. (11)] is given as:

$$\begin{aligned} \begin{aligned} H (z_1, z_2) = \dfrac{Y (z_1, z_2)}{X (z_1, z_2)} &=\dfrac{1}{1 - \sum \limits _{n_1 = 1}^{2} \sum \limits _{n_2 = 1}^{2} c_{n_1 n_2} {z_1}^{-{n_1}} {z_2}^{-{n_2}}}, (k_1, k_2) \ne 0 \end{aligned} \end{aligned}$$

(13)

Alternatively, the above equation can be represented as:

$$\begin{aligned} \begin{array}{ll} H (z_1, z_2) = \dfrac{1}{\begin{array}{ll} \quad {1 - c_{01} {z_2}^{-1} - c_{10} {z_1}^{-1} - c_{11} {z_1}^{-1} {z_2}^{-1} } \\ \quad {- c_{02} {z_2}^{-2} - c_{12} {z_1}^{-1} {z_2}^{-2} - c_{20} {z_1}^{-2}}\\ \quad - c_{21} {z_1}^{-2} {z_2}^{-1} - c_{22} {z_1}^{-2} {z_2}^{-2} \end{array}}\\ \end{array} \end{aligned}$$

(14)

To verify the transfer function expressed in Eq. (13), we need to formalize the difference equation-based model of the filter [Eq. (11)], which is given in HOL Light as:

Definition 15

Difference Equation-Based Model of the Second-Order Filter $\vdash _{\textit{def}}$ $\forall$ y x n1 n2 a b. second_order_filter x y a b n1 n2 $\Leftrightarrow$      y (n1, n2) = l1l2th_difference y a 2 2 n1 n2 - l1l2th_difference x b 0 0 n1 n2

where a and b are the coefficients of input and output 2D arrays. The function second_order_filter accepts input and output 2D arrays, their coefficients a and b and returns the linear difference equation describing the second-order image processing filter.

Now, we formally verify the transfer function [Eq. (13)] in HOL Light as follows:

Theorem 11

Transfer Function of a Second-Order Filter

$\vdash _{\textit{thm}}$ $\forall$ x y a b z1 z2 n1 c11 c12 c21 c22. [A1]: (z1, z2) IN ROC_2d_LCCDE x y 2 2 b n1 $\wedge$ [A2]: in_fst_quad_2d_lccde x y $\wedge$ [A3]: cond_2d_diff_eq_coeff a b b01 b10 b11 b02 b12 b20 b21 b22 $\wedge$ [A4]: $\lnot$ (z1 = Cx ( &0)) $\wedge$    [A5]: $\lnot$ (z2 = Cx ( &0)) $\wedge$ [A6]: ($\forall$ n1 n2. second_order_filter x y a b n1 n2)$\Rightarrow$ z_transform_2d y z1 z2 / z_transform_2d x z1 z2 = Cx ( &1) / (Cx ( &1) - b01 $*$ z2 cpow - Cx ( &1) - b10 $*$ z1 cpow - Cx ( &1) - b11 $*$ z1 cpow - Cx ( &1) $*$ z2 cpow - Cx ( &1) - b02 $*$ z2 cpow - Cx ( &2) - b12 $*$ z1 cpow - Cx ( &1) $*$ z2 cpow - Cx ( &2) - b20 $*$ z1 cpow - Cx ( &2) - b21 $*$ z1 cpow - Cx ( &2) $*$ z2 cpow - Cx ( &1) - b22 $*$ z1 cpow - Cx ( &2) $*$ z2 cpow - Cx ( &2))

Assumption A1 provides the ROC for the differential equation-based model of the second-order filter. Assumption A2 ensures that the input and output 2D arrays x and y are in the first quadrant. Assumption A3 asserts that the input and output coefficients are constant. Assumptions A4 and A5 ensure that the complex variables z1 and z2 are nonzero. Assumption A6 captures the time-domain model of the second-order filter, i.e., Eq. (11). Finally, the conclusion presents the transfer function of the second-order filter. The verification of the above theorem is mainly based on Theorem 10 along with some complex arithmetic reasoning. Theorem 11 is the formal verification result of the second-order image processing filter based on our formalization of the 2D z-transform described in Sects. 5.1 and 5.2.

Now, a specialized case of a second-order image processing filter is graphically represented by the flow graph shown in Fig. 3. This filter can be mathematically represented, by setting the values of the gains of each branch as $c_{01} = c_{11} = c_{20} = c_{21} = c_{22} = 0$, $c_{10} = \frac{1}{4}$, $c_{02} = - \frac{1}{4}$ and $c_{12} = \frac{1}{4}$ in Eq. (11), as follows.

Fig. 3

Flow graph of a specialized second-order image processing filter

$$\begin{aligned} \begin{aligned} y (n_1, n_2) = \&x (n_1, n_2) + \frac{1}{4} y (n_1 - 1, n_2) - \frac{1}{4} y (n_1, n_2 - 2) + \frac{1}{4} y (n_1 - 1, n_2 - 2) \\ \end{aligned} \end{aligned}$$

(15)

The transfer function corresponding to the difference equation-based model [Eq. (15)] is described as:

$$\begin{aligned} \begin{aligned} H (z_1, z_2) = \dfrac{Y (z_1, z_2)}{X (z_1, z_2)} &=\dfrac{1}{1 - \frac{1}{4} {z_1}^{-1} + \frac{1}{4} {z_2}^{-2} - \frac{1}{4} {z_1}^{-1} {z_2}^{-2}} \end{aligned} \end{aligned}$$

(16)

We formally verify the transfer function [Eq. (16)] as:

Theorem 12

Transfer Function of a Specialized Second-Order Filter

$\vdash _{\textit{thm}}$ $\forall$ x y c d z1 z2 n1. [A1]: (z1, z2) IN ROC_2d_LCCDE_spec x y 2 2 b n1 $\wedge$ [A2]: in_fst_quad_2d_lccde_spec x y $\wedge$ [A3]: $\lnot$ (z1 = Cx ( &0)) $\wedge$ [A4]: $\lnot$ (z2 = Cx ( &0)) $\wedge$ [A5]: ($\forall$ n1 n2. second_order_filter_spec x y c d n1 n2)$\Rightarrow$ z_transform_2d y z1 z2 / z_transform_2d x z1 z2 = Cx ( &1) / (Cx ( &1) - (1 / 4) $*$ z1 cpow - Cx ( &1) + (1 / 4) $*$ z2 cpow - Cx ( &2) - (1 / 4) $*$ z1 cpow - Cx ( &1) $*$ z2 cpow - Cx ( &2))

Assumption A1 captures the ROC for the differential equation-based model of the specialized second-order filter. Assumption A2 asserts the first quadrant conditions on the input and output 2D arrays x and y. Assumptions A3 and A4 ensure that the complex variables z1 and z2 are nonzero. Assumption A5 presents the time-domain model of the specialized second-order filter, i.e., Eq. (15). Finally, the conclusion captures the transfer function of the specialized second-order filter. The verification of the above theorem is done almost automatically using Theorem 11, which illustrates the effectiveness of our proposed approach.

Fig. 4

Root map for the specialized second-order filter

Next, we implement the transfer function of the specialized second-order filter, verified as Theorem 12, in Python. In particular, we implemented the poles [denominator of Eq. (16)] of the transfer function, i.e., the characteristic equation $1 - \frac{1}{4} {z_1}^{-1} + \frac{1}{4} {z_2}^{-2} - \frac{1}{4} {z_1}^{-1} {z_2}^{-2} = 0$ on the complex plane $z_2$ for $z_1 = e^{i {\omega }_1}, {\omega }_1 \in [0, \pi ]$. Figure 4 provides the root map capturing the poles of the transfer function, and their placement with respect to unit circle in the complex plane can be used for analyzing the 2D stability of the corresponding system. In the case of the specialized second-order filter (Fig. 4), the presence of poles inside the unit circle provides a sufficient condition for the stability of the corresponding system. However, in case of poles outside the unit circle, the corresponding system will be unstable. Similarly, the one-dimensional (1D) stability can be analyzed by implementing the characteristic equation for all $z_1$ with $z_2 = 1$ and observing the placement of the poles in the complex $z_1$ plane.

6 Discussions

The distinguishing feature of our proposed framework, as compared to the traditional analysis techniques, is that all verified theorems are of generic nature, i.e., all of the functions and variables involved in these theorems are universally quantified and thus can be specialized based on the requirement of the analysis of the image processing filter of any order. For example, Theorem 10 provides the verification of the transfer function of a generic $(L_1, L_2)$-order 2D IIR image processing filter and it can be directly used for analyzing an image processing filter of any order, such as second-order filter (Theorems 11 and 12). We only need to specialize the gains ($\alpha (l_1, l_2)$, $\beta (k_1, k_2)$ in Eqs. (8), (9) and (10) of an image processing filter based on a particular scenario, whereas, in the case of computer-based simulations, we need to model each filter based on its corresponding order, individually that can add a lot of complexity for the case of higher-order filters. Thus, the generic nature of the formalized theorems in our proposed approach makes it better than the transitional analysis methods. Another advantage of our proposed approach is the inherent soundness of the theorem proving technique. It ensures that all the required assumptions are explicitly present along with the theorem, which are often ignored in the conventional simulations-based analysis, and their absence may affect the accuracy of the corresponding analysis. For example, for a given system (second-order image processing filter), if we do not incorporate the constraints captured as Assumptions A3, A4 and A5 of Theorem 11 and Assumptions A3 and A4 of Theorem 12, it may lead to some undesired results; for example, it may result into a transfer function that can make a stable system as an unstable system. One of the main limitations of the proposed approach is the significant user involvement in the proposed formalization of z-transform, due to the undecidable nature of the higher-order logic. However, we have developed simplifiers, such as ROC_SIMP_TAC, DIFF_EQ_SIMP_TAC and TRANS_FUN_TAC, that significantly reduce the user guidance in the reasoning process. More details of the proof process can be viewed in our proof script.⁵

7 Conclusions

2D image processing systems include processing of the images, such as image filtering, enhancement, compression and restoration. These systems are typically analyzed using the 2D z-transform. This paper proposed a HOL theorem proving-based framework for formally analyzing 2D image processing filters. In particular, we formalized the 2D z-transform and formally verified its various classical properties, such as linearity, shifting in time, scaling in ($z_1, z_2$)-domain and complex conjugation. Moreover, we formally analyzed a generic 2D IIR image processing filter. Finally, to demonstrate the practical utilization and effectiveness of the proposed framework, we presented the formal analysis of a second-order image processing filter.

In future, we aim to formalize the 2D inverse z-transform [16] that will enable us to find the time-domain solutions of the time-domain models of the image processing systems. Another future direction is to formalize the 2D convolution [2] that can greatly simplify the reasoning about systems-of-systems [16].