Skip to main content

Advertisement

Design of Experiments for Performance Evaluation and Parameter Tuning of a Road Image Processing Chain

Abstract

Tuning a complete image processing chain (IPC) is not a straightforward task. The first problem to overcome is the evaluation of the whole process. Until now researchers have focused on the evaluation of single algorithms based on a small number of test images and ad hoc tuning independent of input data. In this paper, we explain how the design of experiments applied on a large image database enables statistical modeling for IPC significant parameter identification. The second problem is then considered: how can we find the relevant tuning and continuously adapt image processing to input data? After the tuning of the IPC on a typical subset of the image database using numerical optimization, we develop an adaptive IPC based on a neural network working on input image descriptors. By testing this approach on an IPC dedicated-to-road obstacle detection, we demonstrate that this experimental methodology and software architecture can ensure continuous efficiency. The reason is simple: the IPC is globally optimized, from a large number of real images and with adaptive processing of input data.

References

  1. 1.

    Haralick RM: Performance characterization protocol in computer vision. Proceedings of the ARPA Image Understanding Workshop, November 1994, Monterey, Calif, USA I: 667–673.

  2. 2.

    Courtney P, Thacker N, Clark A: Algorithmic modeling for performance evaluation. Proceedings of the ECCV Workshop on Performance Characteristics of Vision Algorithms, April 1996, Cambridge, UK 13.

  3. 3.

    Forstner W: 10 pros and cons against performance characterization of vision algorithms. Proceedings of the ECCV Workshop on Performance Characteristics of Vision Algorithms, April 1996, Cambridge, UK

  4. 4.

    Bowyer KW, Phillips PJ: Empirical Evaluation Techniques in Computer Vision. Wiley-IEEE Computer Society Press, Los Alamitos, Calif, USA; 1998.

  5. 5.

    Meer P, Matei B, Cho K: Input guided performance evaluation. Theoretical Foundations of Computer Vision (TFCV '98), March 1998, Dagstuhl, Germany 115–124.

  6. 6.

    Phillips IT, Chhabra AK: Empirical performance evaluation of graphics recognition systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 1999, 21(9):849–870. 10.1109/34.790427

  7. 7.

    Blanc-Talon J, Ropert V: Evaluation des chaînes de traitement d'images. Revue Scientifique et Technique de la Défense 2000, (46):29–38.

  8. 8.

    Philipp-Foliguet S: Evaluation de la segmentation. ETIS, Cergy-Pontoise, France; 2001.

  9. 9.

    Sebe N, Tian Q, Loupias E, Lew MS, Huang TS: Evaluation of salient point techniques. Proceedings of International Conference on Image and Video Retrieval (CIVR '02), July 2002, London, UK 2383: 367–377.

  10. 10.

    Rosin PL, Ioannidis E: Evaluation of global image thresholding for change detection. Pattern Recognition Letters 2003, 24(14):2345–2356. 10.1016/S0167-8655(03)00060-6

  11. 11.

    Yitzhaky Y, Peli E: A method for objective edge detection evaluation and detector parameter selection. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003, 25(8):1027–1033. 10.1109/TPAMI.2003.1217608

  12. 12.

    Ropert V: Proposition d'une architecture de contrôle pour un système de vision, Thèse de l'Université René Descartes (Paris 6). , Paris, France; Décembre 2001.

  13. 13.

    Levner I, Bulitko V: Machine learning for adaptive image interpretation. Proceedings of the 16th Innovative Applications of Artificial Intelligence Conference (IAAI '04), July 2004, San Jose, Calif, USA 870–876.

  14. 14.

    Schimmerling P, Sisson J-C, Zaïdi A: Pratique des Plans d'Expériences. Lavoisier Tec & Doc, Paris, France; 1998.

  15. 15.

    Treuillet S: Analyse de l'influence des paramètres d'une chaîne de traitements d'images par un plan d'expériences. 19e colloque GRETSI sur le traitement du signal et des images (GRETSI '03), September 2003, Paris, France

  16. 16.

    Treuillet S, Driouchi D, Ribereau P:Ajustement des paramètres d'une chaîne de traitement d'images par un plan d'expériences fractionnaire. Traitement du Signal 2004, 21(2):141–155.

  17. 17.

    Wright MH: The nelder-mead simplex method: recent theory and practice. Proceedings of the 16th International Symposium on Mathematical Programming (ISMP '97), August 1997, Lausanne, Switzerland

  18. 18.

    Domingues A, Lucas Y, Baudrier D, Marché P: Détection et suivi d'objets en temps réel par un système embarqué multi capteurs. Proceedings of the 18th Symposium GRETSI on Signal and Image Processing (GRETSI '01), Septembre 2001, Toulouse, France

  19. 19.

    Domingues A: Système embarqué multicapteurs pour la détection d'obstacles routiers—Développement du prototype et réglage automatique de la chaîne de traitement d'images, Thèse de l'Université d'Orléans. , Orléans, France; Juillet 2004.

  20. 20.

    Lucas Y, Domingues A, Boubal M, Marché P: Système de vision embarqué pour la détection d'obstacles routiers. Techniques de l'Ingénieur—Recherche & Innovation 2005, 9. IN-24

  21. 21.

    Lamaty P: Opérateurs de niveau intermédiaire pour le traitement temps réel des images, Thèse de Doctorat. Thèse de l'Université de Cergy-Pontoise, Cergy-Pontoise, France; 2000.

  22. 22.

    Lucas Y, Domingues A, Driouchi D, Marché P: Modeling, evaluation and control of a road image processing chain. Proceedings of the 14th Scandinavian Conference on Image Analysis (SCIA '05), June 2005, Joensuu, Finland 3540: 1076–1085.

  23. 23.

    Fries A, Hunter WG:Minimum aberration designs. Technometrics 1980, 22(4):601–608. 10.2307/1268198

  24. 24.

    Rechtschaffner RL: Saturated fractions of 2n and 3n factorial designs. Technometrics 1967, 9: 569–575. 10.2307/1266195

  25. 25.

    Cheng H-D, Sun Y: A hierarchical approach to color image segmentation using homogeneity. IEEE Transactions on Image Processing 2000, 9(12):2071–2082. 10.1109/83.887975

Download references

Author information

Correspondence to Yves Lucas.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Lucas, Y., Domingues, A., Driouchi, D. et al. Design of Experiments for Performance Evaluation and Parameter Tuning of a Road Image Processing Chain. EURASIP J. Adv. Signal Process. 2006, 048012 (2006). https://doi.org/10.1155/ASP/2006/48012

Download citation

Keywords

  • Input Image
  • Parameter Tuning
  • Image Database
  • Software Architecture
  • Numerical Optimization