An innovative document image binarization approach driven by the non-local p-Laplacian

Abstract

Text image binarization is fairly a tedious task and a significant problem in document image analysis. This process, as a necessary pretreatment for noisy images with stains, non-uniform background, or degraded text characters, can successfully improve the quality of the image and facilitate the subsequent image processing steps. A theoretically well-motivated non-local method for document image binarization is addressed in this paper. This approach enhances degraded images by estimating and then removing the undesirable background. Extensive experiments conducted on degraded document images evince the greater effectiveness of the proposed non-local algorithm.

1 Introduction

Image binarization is a very active research area in document image processing. It basically consists of separating the handwritten and printed text from the non-uniform background. This task becomes extremely difficult when dealing with ancient historical documents, which may suffer from various degradations, such as non-uniform illumination, ink intensity variation, large dark stains, or ink bleed-through. Accordingly, correcting these deterioration effects on document images can significantly enhance the performance of subsequent treatments.

There have long been numerous studies of document image binarization techniques. They are widely categorized as adaptive methods and global methods [1, 2]. Global approaches aspire to find one single thresholding parameter to extract the text from the background based on their gray-level distribution. The popular nonparametric Otsu algorithm [3] is very simple and fast, but it cannot successfully handle images with low contrast, uneven illumination, heavy noises, and bleed-through. To overwhelm those limitations, local threshold methods attempt to compute a mutable threshold for each sub-image. These approaches yield better performance on complex text images by reducing the influence of degradations adaptively. Bernsen [4] first proposed a local method based on image contrast. In this technique, the mean value of the maximum and minimum intensities within a local window centered at the considered pixel is used to compute the threshold value. Niblack’s method [5] uses the mean M and the standard deviation S to deduce the threshold $T=M+k.S$, where k is a user-defined parameter. This approach is pursued by Sauvola [6] and [7], who proposed modified versions of Niblack’s algorithm to tackle the background noise problem. Singh et al. [8] proposed an improved algorithm based on four steps: local contrast analysis, contrast stretching, thresholding, and noise removal. Di Lu et al. [9] proposed a local thresholding method that focuses on the differences in image grayscale contrast in different areas. In general, local methods give accurate results while remaining computationally expensive because they need to specify several thresholds for the same image.

For decades, partial differential equations (PDEs) have been attractive tools in image processing ranging from denoising smoothing and image inpainting to shape extraction and remote sensing, both from a theoretical and experimental point of view. For example, Wang and He [10] proposed an evolution equation-based binarization method for document images produced by cameras where the evolution is controlled by a global force and a local force, both of which have an opposite sign inside and outside the object of interest in the original image. Jacobs and Momoniat’s method [11] is based on the dynamic process of diffusion, coupled with a nonlinear Fitzhugh–Nagumo-type source term that exhibits binarizing properties: Then, the authors extend this model by selecting the thresholding parameter in the source term based on local information making the process locally adaptive [12]. Recently, Yagoubi et al. [13] proposed a historical document compression scheme combined with a novel automatic enhancement scheme, both based on PDE-analysis. In the same principle, Guo et al. [14] proposed a novel edge-preserving equation by incorporating an adaptive source term expressing binarization properties into a nonlinear diffusion model, wherein the obtained results assure remarkable performance compared to six benchmark binarization algorithms and four PDE-based binarization methods. The work in [15] is among the first ones to apply level set framework in degraded document image binarization, which uses the image edges that can usually be detected around the text stroke boundary.

With the purpose of achieving better binarization results, the use of non-local operators may provide a solid foundation. One of the motivations for the exploitation of this kind of operator may date back to the work of [16], introducing a very general framework to treat image and signal processing by adopting the so-called non-local PDE. Non-local models involve integral equations and fractional derivatives allowing non-local interactions, that is to say, the interaction may occur even when the closures of two domains have an empty intersection. Such models are effective in modeling material singularities and are widely considered in a variety of applications, including image processing [17,18,19], phase transition [20, 21], machine learning [22], and obstacle problem [23]. On the other hand, in [24], the authors offer a rigorous mathematical analysis of non-local models by describing the analogy between the two classical and non-local frameworks.

This paper presents a new non-local document image binarization algorithm based on the non-local p-Laplacian operator. First and foremost, our main concern is separating the text from the background. One of the main features of the proposed image decomposition is that it is induced by human perception and can function very well under changing lighting conditions since it is inspired by many Retinex models [25, 26]. To recover the degraded text by estimating the smooth component of the degraded document image, we suggest considering a non-local p-Laplacian-type diffusion equation. Considering the non-local p-Laplacian operator is motivated by the need to represent anomalous diffusion to estimate complex background; moreover, the exponent “p” allows the control of the degree of smoothing to treat any degradation. Maintaining full advantage of the characteristics of non-local operators, the proposed approach succeeds in preserving text textures, small details, and complex structures.

The rest of this paper is arranged as follows. The second section focuses on a description of the proposed non-local approach. In Section 3, we present a very simple algorithm to solve the proposed non-local model and analyze the obtained results. A summary is presented in Section 4.

2 Proposed approach

2.1 Text image decomposition

To extract textual information contained in an image, it is necessary to eliminate the unwanted background. Here, taking inspiration from many Retinex models that consider that the observed image is the product of the varying illumination and the reflectance, we derive a two-component decomposition model for document images.

In a text document that does not include images, graphs, or tables, the text emanates from the foreground, while the remaining parts belong to the background. Our fundamental assumption is that the background noise, bleed-through, and non-uniform illumination affect the text multiplicatively. We propose decomposing a given document image “I” into two main parts: the foreground text “T” and the background component “B” as follows:

$$\begin{aligned} I(x,y)\;=\;T(x,y)B(x,y){.} \end{aligned}$$

(1)

We logarithmically transform (1) to obtain

$$\begin{aligned} i(x,y)\;=\;t(x,y)+b(x,y){.} \end{aligned}$$

(2)

This model allows the deduction of the text by subtraction if one can estimate the background which gathers all various degradations of the considered image.

2.2 Background estimation

The classical PDE-based methods and the frequency domain filters restore images using local information, which means that they fail to preserve the fine structure, details, and texture. To overcomes this drawback, the non-local (NL) means filter [27] tries to take advantage of the high degree of redundancy of any natural image. The NL-means algorithm estimates the value of x as an average of the values of all the pixels whose Gaussian neighborhood looks like the neighborhood of x:

$$\begin{aligned} NL(u)(x)=\frac{1}{C(x)}\int _{\Omega }e^{-\frac{(G_a*\mid u(x+.)-u(y+.)\mid ^2)(0)}{h^2}}u(y){\text {d}}y{,} \end{aligned}$$

where u is the original image defined in a bounded domain $\Omega \subset {\mathbb {R}}^2$, and denoted by u(x) for a pixel $x = (x_1, x_2) \in {\mathbb {R}}^2$, $G_a$ is a Gaussian kernel of standard deviation a, h acts as a filtering parameter, and C(x) is the normalizing factor.

By replacing local operators in variational models with the non-local ones, the authors of [16] generalized the non-local means filter into a variational formulation and proposed a non-local functional with a total variation (TV) regularization:

$$\begin{aligned} \displaystyle { \underset{u}{\min }\{\int _{\Omega }\sqrt{\int _{\Omega }(u(y)-u(x))^2w(x,y)dydx}+\lambda \parallel u-f\parallel _{L^2}^2\}}{,} \end{aligned}$$

where $\lambda >0$ controls the trade-off between data fidelity and regularization, w is the weighting function, and f is the noisy input image or signal. The corresponding non-local TV flow is given by:

$$\begin{aligned} \frac{\partial u}{\partial t}(x)=\int _{\Omega }(u(y)-u(x))w(x,y)\left( \frac{1}{\mid \nabla _w u\mid (x)}+\frac{1}{\mid \nabla _w u\mid (y)}\right) {\text {d}}y{,} \end{aligned}$$

where

$$\begin{aligned} \mid \nabla _w u\mid (q):=\sqrt{\int _{\Omega }(u(z)-u(q))^2w(q,z){\text {d}}z}. \end{aligned}$$

This non-local TV model can apply to denoising, sharpening, deblurring, and inpainting problems.

Motivated by the huge interest in the current literature, exploiting non-local methods, we introduce the following non-local p-Laplacian equation with homogeneous Neumann boundary conditions to estimate the document background:

$$\begin{aligned} \left\{ \begin{array}{llllll} \textstyle {u_t(x,t)}&{}=&{}\textstyle {\int _{\Omega }J(x-y)|u(y,t)-u(x,t)|^{p-2}(u(y,t)-u(x,t)){\text {d}}y}&{}\text{ in }&{}\textstyle {\Omega ,\;t>0}\\ \textstyle {u(x,0)}&=&\textstyle {u_0}&{}\text{ in }&{}\textstyle {\Omega }{,} \end{array} \right. \end{aligned}$$

(3)

where $\Omega \subset {\mathbb {R}}^2$ is a bounded domain, $\textstyle {J:{\mathbb {R}}^2\rightarrow {\mathbb {R}}}$ is a nonnegative continuous radial function with compact support and $\textstyle {J(0)>0}$ and $\textstyle {1< p < +\infty }$.

In summary, the main contribution of introducing the non-local p-Laplacian operator is to use more diffusion properties, controlled by the parameter p, to dispatch the information from a clean neighborhood to a degraded one. This can generalize the classical non-local TV and other non-local filtering.

The differences between our proposed method and the PDE-based methods for binarization of degraded document images lie in two aspects. Firstly, unlike traditional methods that use differential operators, we present the first attempt to incorporate non-local operators for image binarization. Secondly, the existing methods are based on evolution PDEs with a source term exhibiting binarization properties. The presence of a source term exhibiting binarization features advances remarkably the desired task, but in this work, we prefer to shed light on the robustness of non-local operators.

The basic concept of the proposed method is to recover the complex background of a document image so as to obtain a binarized image offering readable and clear text. The proposed approach is strongly inspired by the intuition that, for document images, the background is redundant and smooth, while the foreground contains text and sharp edges.

The non-local p-Laplacian equation (3) does not rely on the gradient to extract the diffusion direction so that the diffusion of the density at a point x and time t depends on values of u in a neighborhood of x. This property avoids an exaggerated smoothing effect and motivates the ability of the model to preserve the textures and details of the text.

Solutions to (3) will be understood in the following sense:

Definition 1

A solution to (3) u in [0, T] is a function

$$\begin{aligned} u\in W^{1,1}(0,T;L^1(\Omega )) \end{aligned}$$

that satisfies

$$\begin{aligned} u(x, 0) = u_0(x) \;\text{ a.e. } x \in \Omega {,} \end{aligned}$$

(4)

and

$$\begin{aligned} u_t(x,t)\;=\;\int _{\Omega }J(x-y)|u(y,t)-u(x,t)|^{p-2}(u(y,t)-u(x,t)){\text {d}}y\;\;\;a.e. \;in\; \Omega \times (0,T){,} \end{aligned}$$

(5)

where $L^1(\Omega )$ is the Lebesgue space of measurable functions and

$$\begin{aligned} W^{1,1}(0,T;L^1(\Omega ))=\{u\in L^1(0,T;L^1(\Omega ))\mid u^{'}\in L^1(0,T;L^1(\Omega ))\}{.} \end{aligned}$$

In [28], the authors discussed many non-local evolution models with different boundary conditions and proved convergence, under rescaling, to local problems. We state their main result (Theorem 6.2, page 124) which guarantees the existence of a global and unique solution to the proposed non-local binarization problem:

Theorem 1

Suppose $p>1$ and let $u_0\in L^p(\Omega )$. Then, for any $T>0$, there exists a unique solution to (3).

The mathematical properties and the convergence of solutions to the non-local p-Laplacian equation were established in [28] where the authors studied a non-local analog of the p-Laplacian evolution equation for $1< p < \infty$ with Dirichlet or Neumann boundary conditions. One of the main tools for the proof (in [28, pp. 125–127]) is nonlinear semigroup theory.

3 Experiments and discussion

The principal steps of the proposed discrete algorithm to compute approximated solutions to the problem (3) are as follows:

Let h and $\triangle t$ be space and time steps. Let $u_{i}^n=u(i_1,i_2, n\triangle t )$ with $n\ge 0$.

Equation (3) can be implemented via a simple explicit finite difference scheme

$$\begin{aligned} \frac{u_i^{n+1}-u_i^{n}}{\triangle t}=\underset{j\in {\mathcal {N}}_i}{\sum }J(i-j)|u_j^n-u_i^n|^{p-2}(u_j^n-u_i^n)\;\;\;\;{.} \end{aligned}$$

(6)

In all numerical experiments, we choose the following nonnegative real-valued kernel function:

$$\begin{aligned} J(i-j)\;:=\; {\left\{ \begin{array}{ll} \exp \left( \frac{{-\parallel i-j\parallel ^2}}{h^2}\right) \quad \quad {\text{if}}\,\parallel i-j\parallel <d,\\ 0, \quad \;\;\quad \quad \quad \quad \quad {\text{if}}\;\,\parallel i-j\parallel \ge d. \end{array}\right. } \end{aligned}$$

(7)

where $\parallel . \parallel$ denotes the Euclidean distance and ${\mathcal {N}}_i = \{j : \parallel i-j\parallel < d\}$ denotes the neighbors set of the pixel $i=(i_1,i_2)$.

Algorithm for solving (3)
Input: The acquired image u, iteration number $N=10$ and convergence parameter $\epsilon$.
Step 1: Initialize $B^{0}=\log (u+1)$, choose $dt>0$, $h>0$ and $p>1$ and set $n=0.$
Step 2: Normalize $B^{0}$ into [0, 1].
Step 3: $B^{n+1}_i=B^{n}_i+dt\triangle _{NL}^p (B^{n}_i)\;\;\;\;$for $n= 1, . . . ,N$
if $\parallel B^{n+1}-B^{n}\parallel \le \epsilon$ or $n> N$, go to step 4.
Step 4: Compute $U=\exp (B^{0}-B)$.
Output: The binarized image $U_B$.

The proposed model does not take into account any preprocessing step to enhance the quality of the degraded image. The color-to-grayscale conversion of the input image in Step 2 is considered using the simplest and most widely used approach: It is directly obtained using a mean of the RGB channels. The binary output image $U_B$ is obtained using the following projection:

$$\begin{aligned} U_{B_i}=\left\{ \begin{array}{llllll} 1\quad \quad if\; U_{i}>0{,}\\ 0\quad \quad if \;U_{i}\le 0{.}\\ \end{array} \right. \end{aligned}$$

3.1 Image binarization outcomes

Now, we return to summarize performed tests and obtained results to demonstrate the efficiency of our binarization approach.

Throughout the experimental section, we set $dt = 0.3$, $h = 80$, a patch size of $15 \times 15,$ while the exponent parameter $p\in ]1, 11]$ is adjusted for each image.

Indeed, we have been able to obtain sufficiently satisfactory results for $60 \le h \le 110$ and note that if the value of h is too small, the weight function becomes less important and consequently the background is not well estimated; if h is sufficiently high, the estimated text becomes darker and wider than desired. Concerning the choice of the parameter p, we have tried to use a bilevel optimization framework to automatically determine this parameter, but we encountered a problem of the regularity on p and also big flows in the convergence results. As an alternative way, we tried to use some learning techniques based on CNN configuration, which will be the object of a future work.

We experimentally evaluate our proposed model using the DIBCO testing dataset. DIBCO 2009, DIBCO 2011, and DIBCO 2013 contain both handwritten and machine-printed images, while DIBCO 2010, DIBCO 2012, DIBCO 2014, and DIBCO 2016 include only handwritten images. This dataset consists of 86 document images containing diverse representative degradations that commonly appear such as non-uniform illumination, stains, bleed-through, noise, and smudges. In Figs. 1, 2, 3, 4, 5, 6 and 7, we select some examples of degraded images and our proposed binarized solutions. The binarization performance on machine-printed images and handwritten images is comparable even in the presence of pale character and bleed-through background.

Fig. 1

Images binarization results from DIBCO 2009: a Original degraded images, b recovered text by the proposed non-local model

Fig. 2

Images binarization results from DIBCO 2010: a Original degraded images, b recovered text by the proposed non-local model

Fig. 3

Images binarization results from DIBCO 2011: a Original degraded images, b recovered text by the proposed non-local model

Fig. 4

Images binarization results from DIBCO 2012: a Original degraded images, b recovered text by the proposed non-local model

Fig. 5

Images binarization results from DIBCO 2013: a Original degraded images, b recovered text by the proposed non-local model

Fig. 6

Images binarization results from DIBCO 2014: a Original degraded images, b recovered text by the proposed non-local model

Fig. 7

Images binarization results from DIBCO 2016: a Original degraded images, b recovered text by the proposed non-local model

It is worth mentioning that traditional binarization methods fail to handle images with large ink stains. In Fig. 8, we test three representative degraded images from DIBCO 2013 (“PR04.bmp,” “PR05.bmp,” and “PR08.bmp”). A visual analysis of obtained results shows that the proposed method can detect and remove all smudges.

Fig. 8

a Original images with stains, b binarization results of the proposed model

The results produced in this subsection prove the efficiency of our PDE for binarizing ancient and severely degraded document images.

3.2 Comparison with State of the Art

We compare the binarization visual quality of the proposed method with eleven widely used state-of-the-art image binarization approaches. For evaluation, the considered comparison criteria include a set of measures that are suitable for evaluation in the context of document analysis. These evaluation measurements are:

(1) F-Measure (FM):

$$\begin{aligned} {\text {FM}}=\frac{2\times {\text {Recall}} \times {\text {Precision}}}{{\mathrm {Recall}} + {\text {Precision}}}{,} \end{aligned}$$

where ${\text{Recall}} = \frac{{\text {TP}}}{{\mathrm {TP}}+{\text {FN}}}$, ${\text {Precision}} = \frac{{\text {TP}}}{{\mathrm {TP}}+{\text {FP}}}$ , and TP, FP, FN are true positive, false positive, and false negative values, respectively.

(2) Pseudo-F-Measure (Fps):

$$\begin{aligned} {\text {Fps}}=\frac{2\times {\text {psRecall}} \times {\text {psPrecision}}}{{\mathrm {psRecall}} + {\text {psPrecision}}}{,} \end{aligned}$$

where psRecall and psPrecision are the pseudo-recall and the pseudo-precision [29].

(3) Peak Signal-to-Noise Ratio (PSNR):

$$\begin{aligned} {{\text {PSNR (dB)}} = 10 \log \frac{255^2}{\frac{1}{MN}{\displaystyle \sum _{x_1=1}^{M}\displaystyle \sum _{x_2=1}^{N}\left( I_{BN}(x_1,x_2)-I_{GT}(x_1,x_2)\right) ^2}}}{,} \end{aligned}$$

where $I_{BN}$ is the binary image output and $I_{GT}$ is the hand-annotated ground truth binary result.

(4) Distance Reciprocal Distortion (DRD):

$$\begin{aligned} {\text {DRD}}=\frac{\displaystyle \sum _{k=1}^{N}{\text {DRD}}_k}{{\mathrm {NUBN}}}{,} \end{aligned}$$

where ${\text {DRD}}_k$ is the reciprocal distortion distance of the kth modified pixel and NUBN is defined as the number of non-uniform blocks of $8 \times 8$ sizes in the ground truth image [30]. Good results minimize the DRD and maximize the first three metrics.

Figure 9 illustrates how Otsu, Niblack, and Bernsen’s methods cannot deal with the dark background, while the other methods ignore some faint characters. Figure 10 illustrates the efficacy of our method on an image with contrast variation.

Fig. 9

Binarization results of a sample image by different methods. a Original image, b Otsu [3], c Niblack [5], d Bernsen [4], e Sauvola [6], f Singh [8], g Lu [9], and h proposed method

Fig. 10

Binarization results of a sample image by different methods. a Original image, b Otsu [3], c Niblack [5], d Bernsen [4], e Sauvola [6], f Singh [8], g Lu [9], and h proposed method

In Fig. 11, we select a handwritten image with bleed-through from the DIBCO 2013. The first four results in Fig. 11 show some problems with the extreme bleed-through due to the edge confusion between background and foreground text, while our binarization result and result of [14] can better identify the text and background areas.

Fig. 11

Binarization results of a sample image by different methods. a Original image , b ground truth , c Jacobs [11], d Jacobs [12], e Rivest-Hénault [15], f Wang [10], g Guo [14], and h proposed method

The last example is a handwritten document image with uneven illumination, taken from the DIBCO 2012. Once more, our method is robust to this type of degradation and continues to excel in the binarizing process.

According to Figs. 9, 10, 11 and 12, it can be certainly seen that the global threshold method cannot effectively handle the documents with degradations such as bleed-through, uneven background, and low contrast. Local threshold methods can remedy the limitations of the global method while remaining computationally expensive because they need to specify several thresholds for the same image. On the other hand, we touch the flexible adaptation of the proposed non-local method to some types of degradations related to image contrast variation, non-uniform illumination, or ink bleed-through. Moreover, the simple explicit finite difference scheme for solving our non-local evolution equation numerically allows for a very fast binarization process compared with many different traditional PDE-based methods; we obtain the desired binarization after only one or two iterations.

Fig. 12

Binarization results of a sample image by different methods. a Original image , b ground truth , c Jacobs [11], d Jacobs [12], e Rivest-Hénault [15], f Wang [10], g Guo [14], and h proposed method

Tables 1, 2 and 3 depict the evaluated performance of the selected methods on the DIBCO testing dataset where the bold font marks the best values. All considered comparison criteria reflect the outperformance of the proposed method.

Table 1 Comparison of F-measure for seven methods

Table 2 Comparison of PSNR for seven methods

Table 3 Comparison of the performance of the proposed and the other algorithms against DIBCO dataset

4 Conclusions

The proposed binarization method is a nonlinear p-Laplacian diffusion process that can elegantly model the non-local nature of document images degradations. Fundamentally, background variations are estimated from an image input to acquire a binary outcome. Extensive experiments claimed that our method is highly successful.

As future prospects, we want to study how to adjust automatically the exponent “p.” For instance, it would be interesting to adaptively select this parameter instead of a global choice in order to allow a careful study of smudges, fading characters, or any non-uniform degradation.