Skip to content


  • Research Article
  • Open Access

Fast and Accurate Ground Truth Generation for Skew-Tolerance Evaluation of Page Segmentation Algorithms

EURASIP Journal on Advances in Signal Processing20062006:012093

  • Received: 15 February 2005
  • Accepted: 12 July 2005
  • Published:


Many image segmentation algorithms are known, but often there is an inherent obstacle in the unbiased evaluation of segmentation quality: the absence or lack of a common objective representation for segmentation results. Such a representation, known as the ground truth, is a description of what one should obtain as the result of ideal segmentation, independently of the segmentation algorithm used. The creation of ground truth is a laborious process and therefore any degree of automation is always welcome. Document image analysis is one of the areas where ground truths are employed. In this paper, we describe an automated tool called GROTTO intended to generate ground truths for skewed document images, which can be used for the performance evaluation of page segmentation algorithms. Some of these algorithms are claimed to be insensitive to skew (tilt of text lines). However, this fact is usually supported only by a visual comparison of what one obtains and what one should obtain since ground truths are mostly available for upright images, that is, those without skew. As a result, the evaluation is both subjective; that is, prone to errors, and tedious. Our tool allows users to quickly and easily produce many sufficiently accurate ground truths that can be employed in practice and therefore it facilitates automatic performance evaluation. The main idea is to utilize the ground truths available for upright images and the concept of the representative square [9] in order to produce the ground truths for skewed images. The usefulness of our tool is demonstrated through a number of experiments with real-document images of complex layout.


  • Ground Truth
  • Segmentation Algorithm
  • Objective Representation
  • Document Image
  • Text Line

Authors’ Affiliations

Infotech Oulu and Department of Electrical and Information Engineering, Machine Vision Group, University of Oulu, P.O.Box 4500, FI-90014, Finland


  1. Antonacopoulos A: Page segmentation using the description of the background. Computer Vision and Image Understanding 1998, 70(3):350–369. 10.1006/cviu.1998.0691View ArticleGoogle Scholar
  2. Antonacopoulos A, Brough A: Methodology for flexible and efficient analysis of the performance of page segmentation algorithms. Proc. 5th International Conference on Document Analysis and Recognition (ICDAR '99), September 1999, Bangalore, India 451–454.Google Scholar
  3. Garris MD: Evaluating spatial correspondence of zones in document recognition systems. Proc. International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 304–307.View ArticleGoogle Scholar
  4. Garris MD, Janet SA, Klein WW: Federal register document image database. In Document Recognition and Retrieval VI, January 1999, San Jose, Calif, USA, Proceedings of SPIE Edited by: Lopresti DP, Zhou J. 3651: 97–108.View ArticleGoogle Scholar
  5. Hobby JD: Matching document images with ground truth. International Journal on Document Analysis and Recognition 1998, 1(1):52–61.Google Scholar
  6. Hull JJ: Performance evaluation for document analysis. International Journal of Imaging Systems and Technology 1996, 7(4):357–362. 10.1002/(SICI)1098-1098(199624)7:4<357::AID-IMA10>3.0.CO;2-TView ArticleGoogle Scholar
  7. Kanai J, Rice SV, Nartker TA, Nagy G: Automated evaluation of OCR zoning. IEEE Trans. Pattern Anal. Machine Intell. 1995, 17(1):86–90. 10.1109/34.368146View ArticleGoogle Scholar
  8. Kanungo T, Haralick RM: An automatic closed-loop methodology for generating character groundtruth for scanned documents. IEEE Trans. Pattern Anal. Machine Intell. 1999, 21(2):179–183. 10.1109/34.748827View ArticleGoogle Scholar
  9. Okun O, Pietikäinen M: Automatic ground-truth generation for skew-tolerance evaluation of document layout analysis methods. Proc. 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 376–379.View ArticleGoogle Scholar
  10. Phillips IT, Ha J, Haralick RM, Dori D: The implementation methodology for a CD-ROM English document database. Proc. 2nd International Conference on Document Analysis and Recognition (ICDAR '93), October 1993, Tsukuba Science City, Japan 484–487.View ArticleGoogle Scholar
  11. Randriamasy S, Vincent L: Benchmarking page segmentation algorithms. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), June 1994, Seattle, Wash, USA 411–416.Google Scholar
  12. Yanikoglu BA, Vincent L: Pink Panther: a complete environment for ground-truthing and benchmarking document page segmentation. Pattern Recognition 1998, 31(9):1191–1204. 10.1016/S0031-3203(97)00137-4View ArticleGoogle Scholar


© Okun and Pietikäinen 2006