- Research Article
- Open Access
Fast and Accurate Ground Truth Generation for Skew-Tolerance Evaluation of Page Segmentation Algorithms
EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 012093 (2006)
Many image segmentation algorithms are known, but often there is an inherent obstacle in the unbiased evaluation of segmentation quality: the absence or lack of a common objective representation for segmentation results. Such a representation, known as the ground truth, is a description of what one should obtain as the result of ideal segmentation, independently of the segmentation algorithm used. The creation of ground truth is a laborious process and therefore any degree of automation is always welcome. Document image analysis is one of the areas where ground truths are employed. In this paper, we describe an automated tool called GROTTO intended to generate ground truths for skewed document images, which can be used for the performance evaluation of page segmentation algorithms. Some of these algorithms are claimed to be insensitive to skew (tilt of text lines). However, this fact is usually supported only by a visual comparison of what one obtains and what one should obtain since ground truths are mostly available for upright images, that is, those without skew. As a result, the evaluation is both subjective; that is, prone to errors, and tedious. Our tool allows users to quickly and easily produce many sufficiently accurate ground truths that can be employed in practice and therefore it facilitates automatic performance evaluation. The main idea is to utilize the ground truths available for upright images and the concept of the representative square  in order to produce the ground truths for skewed images. The usefulness of our tool is demonstrated through a number of experiments with real-document images of complex layout.
Antonacopoulos A: Page segmentation using the description of the background. Computer Vision and Image Understanding 1998, 70(3):350–369. 10.1006/cviu.1998.0691
Antonacopoulos A, Brough A: Methodology for flexible and efficient analysis of the performance of page segmentation algorithms. Proc. 5th International Conference on Document Analysis and Recognition (ICDAR '99), September 1999, Bangalore, India 451–454.
Garris MD: Evaluating spatial correspondence of zones in document recognition systems. Proc. International Conference on Image Processing (ICIP '95), October 1995, Washington, DC, USA 3: 304–307.
Garris MD, Janet SA, Klein WW: Federal register document image database. In Document Recognition and Retrieval VI, January 1999, San Jose, Calif, USA, Proceedings of SPIE Edited by: Lopresti DP, Zhou J. 3651: 97–108.
Hobby JD: Matching document images with ground truth. International Journal on Document Analysis and Recognition 1998, 1(1):52–61.
Hull JJ: Performance evaluation for document analysis. International Journal of Imaging Systems and Technology 1996, 7(4):357–362. 10.1002/(SICI)1098-1098(199624)7:4<357::AID-IMA10>3.0.CO;2-T
Kanai J, Rice SV, Nartker TA, Nagy G: Automated evaluation of OCR zoning. IEEE Trans. Pattern Anal. Machine Intell. 1995, 17(1):86–90. 10.1109/34.368146
Kanungo T, Haralick RM: An automatic closed-loop methodology for generating character groundtruth for scanned documents. IEEE Trans. Pattern Anal. Machine Intell. 1999, 21(2):179–183. 10.1109/34.748827
Okun O, Pietikäinen M: Automatic ground-truth generation for skew-tolerance evaluation of document layout analysis methods. Proc. 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 4: 376–379.
Phillips IT, Ha J, Haralick RM, Dori D: The implementation methodology for a CD-ROM English document database. Proc. 2nd International Conference on Document Analysis and Recognition (ICDAR '93), October 1993, Tsukuba Science City, Japan 484–487.
Randriamasy S, Vincent L: Benchmarking page segmentation algorithms. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '94), June 1994, Seattle, Wash, USA 411–416.
Yanikoglu BA, Vincent L: Pink Panther: a complete environment for ground-truthing and benchmarking document page segmentation. Pattern Recognition 1998, 31(9):1191–1204. 10.1016/S0031-3203(97)00137-4
About this article
Cite this article
Okun, O., Pietikäinen, M. Fast and Accurate Ground Truth Generation for Skew-Tolerance Evaluation of Page Segmentation Algorithms. EURASIP J. Adv. Signal Process. 2006, 012093 (2006). https://doi.org/10.1155/ASP/2006/12093
- Ground Truth
- Segmentation Algorithm
- Objective Representation
- Document Image
- Text Line