Skip to main content

Fast and Accurate Ground Truth Generation for Skew-Tolerance Evaluation of Page Segmentation Algorithms


Many image segmentation algorithms are known, but often there is an inherent obstacle in the unbiased evaluation of segmentation quality: the absence or lack of a common objective representation for segmentation results. Such a representation, known as the ground truth, is a description of what one should obtain as the result of ideal segmentation, independently of the segmentation algorithm used. The creation of ground truth is a laborious process and therefore any degree of automation is always welcome. Document image analysis is one of the areas where ground truths are employed. In this paper, we describe an automated tool called GROTTO intended to generate ground truths for skewed document images, which can be used for the performance evaluation of page segmentation algorithms. Some of these algorithms are claimed to be insensitive to skew (tilt of text lines). However, this fact is usually supported only by a visual comparison of what one obtains and what one should obtain since ground truths are mostly available for upright images, that is, those without skew. As a result, the evaluation is both subjective; that is, prone to errors, and tedious. Our tool allows users to quickly and easily produce many sufficiently accurate ground truths that can be employed in practice and therefore it facilitates automatic performance evaluation. The main idea is to utilize the ground truths available for upright images and the concept of the representative square [9] in order to produce the ground truths for skewed images. The usefulness of our tool is demonstrated through a number of experiments with real-document images of complex layout.


  1. 1.

    Antonacopoulos A: Page segmentation using the description of the background. Computer Vision and Image Understanding 1998, 70(3):350–369. 10.1006/cviu.1998.0691

    Article  Google Scholar 

  2. 2.

    Antonacopoulos A, Brough A: Methodology for flexible and efficient analysis of the performance of page segmentation algorithms. Proc. 5th International Conference on Document Analysis and Recognition (ICDAR '99), September 1999, Bangalore, India 451–454.

    Google Scholar 

  3. 3.

    Garris MD: Evaluating spatial correspondence of zones in document recognition systems. Proc. International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 304–307.

    Article  Google Scholar 

  4. 4.

    Garris MD, Janet SA, Klein WW: Federal register document image database. In Document Recognition and Retrieval VI, January 1999, San Jose, Calif, USA, Proceedings of SPIE Edited by: Lopresti DP, Zhou J. 3651: 97–108.

    Google Scholar 

  5. 5.

    Hobby JD: Matching document images with ground truth. International Journal on Document Analysis and Recognition 1998, 1(1):52–61.

    Google Scholar 

  6. 6.

    Hull JJ: Performance evaluation for document analysis. International Journal of Imaging Systems and Technology 1996, 7(4):357–362. 10.1002/(SICI)1098-1098(199624)7:4<357::AID-IMA10>3.0.CO;2-T

    Article  Google Scholar 

  7. 7.

    Kanai J, Rice SV, Nartker TA, Nagy G: Automated evaluation of OCR zoning. IEEE Trans. Pattern Anal. Machine Intell. 1995, 17(1):86–90. 10.1109/34.368146

    Article  Google Scholar 

  8. 8.

    Kanungo T, Haralick RM: An automatic closed-loop methodology for generating character groundtruth for scanned documents. IEEE Trans. Pattern Anal. Machine Intell. 1999, 21(2):179–183. 10.1109/34.748827

    Article  Google Scholar 

  9. 9.

    Okun O, Pietikäinen M: Automatic ground-truth generation for skew-tolerance evaluation of document layout analysis methods. Proc. 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 376–379.

    Article  Google Scholar 

  10. 10.

    Phillips IT, Ha J, Haralick RM, Dori D: The implementation methodology for a CD-ROM English document database. Proc. 2nd International Conference on Document Analysis and Recognition (ICDAR '93), October 1993, Tsukuba Science City, Japan 484–487.

    Google Scholar 

  11. 11.

    Randriamasy S, Vincent L: Benchmarking page segmentation algorithms. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), June 1994, Seattle, Wash, USA 411–416.

    Google Scholar 

  12. 12.

    Yanikoglu BA, Vincent L: Pink Panther: a complete environment for ground-truthing and benchmarking document page segmentation. Pattern Recognition 1998, 31(9):1191–1204. 10.1016/S0031-3203(97)00137-4

    Article  Google Scholar 

Download references

Author information



Corresponding author

Correspondence to Oleg Okun.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Okun, O., Pietikäinen, M. Fast and Accurate Ground Truth Generation for Skew-Tolerance Evaluation of Page Segmentation Algorithms. EURASIP J. Adv. Signal Process. 2006, 012093 (2006).

Download citation


  • Ground Truth
  • Segmentation Algorithm
  • Objective Representation
  • Document Image
  • Text Line