Tuning Range Image Segmentation by Genetic Algorithm

Several range image segmentation algorithms have been proposed, each one to be tuned by a number of parameters in order to provide accurate results on a given class of images. Segmentation parameters are generally a ﬀ ected by the type of surfaces (e.g., planar versus curved) and the nature of the acquisition system (e.g., laser range ﬁnders or structured light scanners). It is impossible to answer the question, which is the best set of parameters given a range image within a class and a range segmentation algorithm? Systems proposing such a parameter optimization are often based either on careful selection or on solution space-partitioning methods. Their main drawback is that they have to limit their search to a subset of the solution space to provide an answer in acceptable time. In order to provide a di ﬀ erent automated method to search a larger solution space, and possibly to answer more e ﬀ ectively the above question, we propose a tuning system based on genetic algorithms. A complete set of tests was performed over a range of di ﬀ erent images and with di ﬀ erent segmentation algorithms. Our system provided a particularly high degree of e ﬀ ectiveness in terms of segmentation quality and search time.


INTRODUCTION
Image segmentation problems can be approached with several solution methods.The range image segmentation subfield has been addressed in different ways.But, since an algorithm should work correctly for a large number of images in a class, such a program is normally characterized by a high number of tuning parameters in order to obtain a correct, or at least satisfactory, segmentation.
Usually the correct set of parameters is given by the developers of the segmentation algorithm, and it is expected to give satisfactory segmentations for the images in the class used to tune the parameters.But it is possible that, given changing input image class, the results are not satisfactory.To avoid exhaustive test tuning, an expert system to tune parameters should be proposed.In this way, it should be pos-sible to easily direct the chosen segmentation algorithm to work correctly with a chosen class of images.
Several expert systems have been proposed by other teams.We can quote [1] that performs the tuning of a color image segmentation algorithm by a genetic algorithm (GA).The same technique can be applied to range segmentation algorithms.Up till now, only techniques that partition the parameter space and work on a successive approximation have been used (such as in [2,3,4,5]).Such techniques obtain results similar to those provided by the algorithm teams' tuning.
In this paper, we propose a tuning system based on GAs.To prove the validity of this method, we will show results obtained using well-tuned segmentation algorithms of range images (in particular the ones proposed at the University of Bern and University of South Florida).Genetic solutions are evaluated according to a fitness function that accounts for different types of errors such as under/oversegmentation or miss-segmentation.
The paper is organized as follows.In Section 2, we summarize the related works.In Section 3, we describe in detail our approach.In Section 4, we show the experimental results, while in Section 5, we present our conclusions.

Range image segmentation
Range images are colored according to the distance from the sensor that scans the image.In fact, each pixel in a range image indicates the value of the distance from the sensor to the foreground object point.Image segmentation is the refinement of an image into patches corresponding to the represented regions.So the range image segmentation algorithm aims at partitioning and labeling range images into surface patches that correspond to surfaces of 3D objects.
Surface segmentation is still a challenging problem.Currently, many different approaches have been proposed.The known algorithms devoted to range segmentation may be subdivided into at least three broad categories [6]: (1) those based on a region-growing strategy, (2) those based on clustering method, (3) those based on edge detection and completion followed by surface filling.
Many algorithms addressing range segmentation have been proposed.In [6], there is a complete analysis of four segmentation algorithms-from the University of South Florida (USF), the University of Bern (UB), the Washington State University (WSU), and the University of Edinburgh (UE).The authors show that a careful parameter tuning has to be performed according to the chosen segmentation algorithm and image set.Such algorithms are based on the above methods, and show different performances and results in terms of segmentation quality and segmentation time.
Jiang and Bunke [7] describe an evolution of the segmentation algorithm built at the University of Bern and in [5], the same segmentation algorithm is used for other tests.Recently, a different segmentation algorithm was presented, based on the scan-line grouping technique [8], but using a region-growing strategy and showing good segmentation results and a quasi-real-time computation capability.Zhang et al. [9] presented two algorithms, both edge based, segmenting noisy range images.By these algorithms, the authors investigated the use of the intensity edge maps (IEMs) in noisy range image segmentation, and the results compared against the corresponding ones are obtained without using IEMs.Such algorithms use watershed and scan-line grouping techniques.Chang and Park [10] proposed a segmentation of range images based on the fusion of range and intensity images, and the estimation of parameters for surface patches representation is performed by a least-trimmed squares (LTS) method.Baccar et al. [11] describe a method to extract, via classification, edges from noisy range images.Several algorithms (particularly color segmentation algorithms) are described or summarized in [12].
Parameters tuning is still a main task, and a possible solution is proposed.A different method to tune set parameters is given by Min et al. in [2,3,4].The main drawback seems to be that a limited subset of the complete solution space is allowed to be explored, but exposes the method to the possibility of missing the global optimum or a good enough local optimum.But such a method is fast and efficient enough to represent a fine-tuning step: given a set of rough local suboptima, the algorithm proposed in [2] could quickly explore a limited space around these suboptima to reach, if they exist, local optima.
In [6], for the first time, an objective performance comparison of range segmentation algorithms has been proposed.Further results on such comparison have been proposed in [3,4,13,14].Another comparison has been presented in [15], where another range segmentation algorithm is proposed.This is based on a robust clustering method (used also for other tasks).But the need for tuning algorithm parameters is still present.

Genetic algorithms and their application to image segmentation
GA is a well-known spread technique for exploring in parallel a solution space by encoding the concept of evolution in the algorithmic search: from a population of individuals representing possible problem solutions, evolution is carried out by means of selection and reproduction of new solutions.Basic principles of GAs are now well known.Quoted references are the books of Goldberg [16] and Michalewicz [17]; a survey is presented in [18], while a detailed explanation of a basic GA for solving NP-hard optimization problem, presented by Bhanu et al., can be found in [1].Many GA-driven segmentation algorithms have been proposed in the literature; in particular, an interesting solution was presented by Yu et al. [19], an algorithm that can segment and reconstruct range images via a method called RESC (RESidual Consensus).Chun and Yang [20] presented an intensity image segmentation by a GA split-and-merge exploiting strategies; and Andrey and Tarroux [21] proposed an algorithm which can segment intensity images by including production rules in the chromosome, that is, a data string representing all the possible features present in a population member.Methods for segmenting textured images are described by Yoshimura and Oe [22] and Tseng and Lai [23].The first one adopts a "small region"-representing chromosome, while the second one uses GAs to improve the iterated conditional modes (ICM) algorithm [24].Cagnoni et al. [25] presented a GA based on a small set of manually traced contours of the structure of interest (anatomical structures in three-dimensional medical images).The method combines the good trade-off between simplicity and versatility offered by polynomial filters with the regularization properties that characterize elastic-contour models.Andrey [26] proposed another interesting work, in which the image to be segmented is considered as an artificial environment.In it, regions with different characteristics are presented as a set of ecological niches.A GA is then used to evolve a population distributed all over this environment.The GA-driven evolution leads distinct species to spread over different niches.Consequently, the distribution of the various species at the end of the run unravels the location of the homogeneous regions on the original image.The method has been called selectionist relaxation because the segmentation emerges as a by-product of a relaxation process [27] mainly driven by selection.
As previously stated, the algorithm presented in [1] tunes a color-image segmentation algorithm, namely, phoenix [28], by a chromosome formed by the program parameters, and not formed by image characteristics as in [19,20,21].
A complete survey on GA used in image processing is that one compiled by Alander [29].

GASE: GENETIC ALGORITHM SEGMENTATION ENVIRONMENT
Using the same rationale as in [1], we adopted a GA for tuning the set of parameters of a range segmentation algorithm.Different approaches to the tuning of parameters could be represented by evolutionary programming (EP) and evolution strategy (ES).
The first one places emphasis on the behavioral linkage between parents and their offsprings (the solutions).Each solution is replicated into a new population and is mutated according to a distribution of mutation types.Each offspring solution is assessed by computing its fitness.Similarly, the second one tries random changes in the parameters defining the solution, following the example of natural mutations.
Like both ES and EP, GA is a useful method of optimization when other techniques, such as gradient descent or direct analytical discovery, are not possible.Combinatoric and real-valued function optimization in which the optimization surface or fitness landscape is rugged, possessing many locally optimal solutions, are well suited for GA.
We chose GA because it is a well-tested method in image segmentation and a good starting point to explore the evolutionary framework.
Because of the universal model, we have the possibility of changing the segmentation algorithm with few consequent changes in the GA code.These changes mainly involve the chromosome composition and the generation definition.The fitness evaluation has been modeled for the problem of range segmentation and can be kept constant as the reproduction model.This is one of the features of our proposal that we called GASE or genetic algorithm segmentation environment (introduced as GASP in [30]).
The main goal of GASE is to suggest a signature for a class of images, that is, the best fitted set of parameters performing the optimal segmentation.In this way, when our system finds a good segmentation for an image or for a particular surface, we can say that the same parameters will work correctly for the same class of images or for the same class of surfaces (i.e., all the surfaces presenting a big curvature radius).

The GASE architecture
In Figure 1, we show the architecture of our system.Following the block diagram, we see that an input image I i is first segmented by a program s (range segmentation algorithm) with a parameter set Π s j , producing a new image having labeled surface patches M s i j .All such segmented images are stored in a database that we call "phenotype repository."Briefly, we may write The quality of the segmentation process may be assessed by means of the so-called fitness evaluation (in block "geneticbased learning") computing a score F s i j by comparing the segmented image M s i j with the ground truth segmented image G i .We assume that our fitness function evaluates a cost, therefore positively valued (or zero valued if the segmented image coincides exactly with the ground truth one).Thus This process is fulfilled for all available images with different parameter sets.The sets that produce the best results (called Π w ) are stored in the so-called "final genotype repository" (if fitness function is under a given threshold).Once the score is assigned, a tuple P i j containing the genotype, the score value, the phenotype identifier, and the generation (Π s j , F s i j , i j, k) is written in a database called "evaluation repository."The genetic computation selects two individuals to be coupled among the living ones (mating individuals selection); these genotypes are processed by the "crossover" block that outputs one or more offsprings that could be mutated.The generated individuals will be the new genotypes Π s j in the next generation step.At the end of a generation, a "to-be-deleted individuals selection" is performed.The decision on which individuals are to be erased from the evaluation repository is made by fixing a killing probability p k depending on the fitness and the age of the individuals (their k value).If an individual has a score greater than p k , the solution it represents will be no longer considered.In this way, we have a limited number of evaluated points in the solution space.

GASE features
When building a GA, some features have to be specifically designed.Among others, we mention the fitness function, the chromosome, described in Sections 3.3 and 3.4, and the crossover.
The fitness function is a heuristic function that indicates to the GA whether an individual fits or not the environment.The chromosome is the data structure that contains the characters of the individuals.The crossover is the method that indicates how parents' characteristics are inherited by children.For this work, we used modified versions of multiple point crossover [31] and uniform crossover [32], as described in [30].
Fitness evaluation

Training set & prototype repository
Final genotype repository (best parameters) Figure 1: GA architecture for range image segmentation.

Fitness function
The most critical step in the genetic evolution process is the definition of a reliable fitness function which ensures monotonousness with respect to the improvement provided by changing the segmentation parameters.The fitness function could be used for comparing both different algorithms and different parameter sets within the same algorithm.In [6] the problem of comparing range segmentation algorithms has been thoroughly analyzed, nevertheless the authors' evaluations take into account a number of separate performance figures and no global merit value is provided.More precisely, the authors consider five figures that are functions of a precision percentage: (1) correct segmentation, (2) oversegmentation, (3) undersegmentation, (4) miss-segmentation, (5) noise segmentation.
Conversely, we are in the need of a single value which will then guide our feedback loop within the optimization process, and therefore, we define a unique performance value specifically accounting for all points.In [33] and in [34] a function assigning a scalar to a segmentation is used.Particularly in [34], that function is the probability error between the ground truth and the machine-segmented image.But such a way of assessing fitness is judged not suitable [6].This means that a more robust way to have a scalar could be to order a vector of properties.Of course the ordering of vectors is not straightforward without using particular techniques; one of them could be to adopt a weighted sum of the components.
We define the fitness function as a weighted sum of a number of components: where w 1 , w 2 , w 3 , and w 4 are tuned to weigh differently the single components.
The fitness takes into account two levels of errors (and therefore is a cost to be minimized); the former is a measure at pixel level computed with a pixel-by-pixel comparison, the latter is a measure at surface level considering the number of computed surfaces.At the pixel level, C is the cost associated with erroneously segmented pixels and U accounts for unsegmented pixels.At the surface levels, we add two factors (handicaps), one due to undersegmentation (H u ) and one due to oversegmentation (H o ).
Let G be the ground truth image, having N G regions called R Gi composed by P Gi pixels, i = 1, . . ., N G , and let MS be the machine-segmented image, having N M regions called R Mj composed by P Mj pixels, j = 1, . . ., N M .We define the overlap map O so that where #(•) indicates the number of (•).The number of pixels with the same coordinates in the two regions is the value O i j .
The expression (4) could be written as O i j = R Gi ∩ R Mj .It is straightforward that if there is no overlap between the two regions, O i j = 0; while in case of complete overlap, O i j = P Gi = P Mj .
Starting from O i j , we search the index x j for all R Mj : x j = argmax NG i=1 (O i j ) to compute the cost C: In other words, C should be a kind of distance between the real and the ideal segmentation at pixel level.
The term U accounts for the unlabeled pixels, that is, those pixels that at the end of the process do not belong to any region (this holds only for the USF segmentation algorithm since the UB segmentation algorithm allocates all unlabeled pixels to the background region): Then we can create another (boolean) matching map with entries m i j so that The handicap H u is accounting for the number of undersegmented regions (those which appear in the resulting image as a whole whilst separated in the ground truth image): In fact, in each row i of the matching map, only one entry is set to 1, while more entries in a column can be set to 1 if undersegmentation occurs and a segmented region covers more ground truth regions.
Finally, H o is a handicap accounting for the number of oversegmented regions (those which appear in ground truth image as a whole whilst split in the resulting image): The handicaps H o and H u are both multiplied by a constant k just to enlarge the variability range.
Some results about the effectiveness of the adopted fitness function have been presented in [35].

Coding the chromosomes
One of the main tasks in GASE was to code the chromosome, that is, to code the parameter set for a given segmentation algorithm.
To simplify the generation of new solution by a correct chromosome manipulation, we should use a binary coding, but since some genes (i.e., parameters) could assume real values, this coding is not sufficient.So we decided to adopt an extended logical binary coding in order to represent real values with a fixed-point code (with a defined number of decimals).Thus we define the symbol set as {0, 1, dot} to allow a representation (of fixed but arbitrary precision) of the decimal of the number.The choice of a fixed precision could seem wrong, but we can consider that, beyond a certain precision, segmentation algorithm performances are not affected.We could have used a floating-point representation of the chromosome, as suggested in [36], but in the case we studied, a fixed-point representation seems to be sufficient.The binary strings are formed by the juxtaposition of BCD-coded genes, memory consuming but giving accuracy to and from decimal conversion.The choice of extending the symbols set including dot was a help for visual inspection of the created population databases (listed in Figure 1).
Our chromosome contains all the parameters (their meanings are listed in Tables 1 and 2) of the chosen segmentation algorithm.In this way, the solution spaces considered are n-dimensional with n = 5 for USF and n = 10 for UB.

EXPERIMENTAL RESULTS
Experiments carried out on GASE are used as a benchmark of the Michigan State University/Washington State University synthetic image database (that we will refer to as MSU/WSU database, http://sampl.eng.ohio-state.edu/∼sampl/data/3DDB/RID/index.htm) and as a subset of the University of Bern real database (referred to as ABW).The tests performed are very time consuming since each segmentation process is iterated for a single experiment many times (i.e., for each individual of the solution population and for each generation).
Since we tested our GA with both a fixed and random number of children crossover, according to [30], we have to use an alternative definition of generation.The term generation in GAs is often used as a synonym of the iteration step and is related to the process of creating a new solution.In our case, a generation step is given by the results obtained in a fixed time slice.In this manner, we can establish a time slice in function of the reference workstation; for instance, with a standard PC (AMD Duron 700 MHz) running Linux OS, we could define the time slice as one minute of computation.In order to compare the efficacy and efficiency of results, we will define a convergence trend maximum time to get the optimal solution in a given Max G generations.

Tuning the UB algorithm
The first experiment was the tuning of the UB segmentation algorithm [7].This algorithm initially tries to detect the edges (jump and crease [37]) of the segmenting image by computing the "scan lines."After finding the candidates for area borders, it accomplishes an edge-filling process.This segmentation algorithm is capable of segmenting curved surfaces and the available version [38] can segment images of the GRF2-K2T database (named after the brand and model of the structured light scanner used).We used a version, slightly modified at the University of Modena, which is able to segment also synthetic images of the MSU/WSU database.A set of 35 images was chosen and a tuning task as in [6] was executed.
While the tuning done should provide very good results, it is our opinion that a training set should not be too large.We then chose a subset of 6 images as our training set.This set was input to GASE, and the resulting parameters set were used to segment the test set (formed by the remaining 29 images) and to find the most suitable set.
We fixed our generation in 1 minute and the maximum number of generations in 30, that is to say, about 30 minutes of computation for every image of the training set.It took a total of about 3 hours to obtain 6 possible solutions and to select the most suitable for the test set.During this time our algorithm performed about 10000 segmentations on the images.An exhaustive search should explore all the enormous space of solution (the space has 10 dimensions, and one parameter potentially ranges from 0 to ∞) and all the instances of the test set.In our case, the exhaustive search was substi-tuted by the GA-based search.Nevertheless, it is critical to test an individual on all images and measure the fitness as a function of the goodness over the whole training set.
As an acceptable approximation, to save computational time, we evaluated the fitness of every individual, applied on a single image at a time.We assumed that, thanks to the genetic evolution, when the individual genotype becomes common in the population, it will be tested on different images.At the end, the best scored individuals are tested on all images of the training set and the one that outperforms the others in average is selected as the best.
In Table 3, we show the parameters used for this test.With "original opt.val." we refer to the parameters tuned by the algorithm author, while with "GASE opt.val." we refer to those tuned by GASE.In Table 4, we show the average scores obtained in this test.Although the improvement could seem poor, it is not because of the presence of images with very different characteristics, which were not considered in the training set.As a matter of fact, the fitness improvement is in most of the cases of one or more units (see Figures 2 and  3 where original and GASE opt.val.are compared).The best improvement was of 11.26 points, while in one case only the GASE optimization generated the worst result with respect to the manual selection.

Tuning the USF algorithm
The second experiment was performed on the USF segmentation algorithm [6].Based on a region growing strategy, it computes the normal vector for each pixel within a parametric-sized window.After that first computation, it selects seed points on the basis of a reliability measure.From these seed points, it accomplishes the region growing, aggregating surfaces until at least one of four parametric criteria is met.This segmentation algorithm has been tuned using a set of parameters proposed by its authors.As we can see in [6], the given results are very impressive, so we knew how difficult it will be to improve them.Nevertheless, we performed the following experiment: given the original training set (10 images of the ABW database), we chose an image as our training set and the other 9 as the test set.Then we compared the results on this subset to the corresponding former results on the same subset, using the comparison tool presented in [6].The comparison tool considers five types of region classification: correct detection, oversegmentation, undersegmentation, miss-segmentation, and noise segmentation.When all region classifications have been determined, a metric describing the accuracy of the recovered geometry is computed; any pair of regions R 1 and R 2 in the ground truth image, representing adjacent faces of the same object, have their angle A n recorded in the truth data.If R 1 and R 2 are classified as correct detections, the angle A m between the surface normals of their corresponding regions in the machinesegmented image is computed.Then |A n − A m | is computed for every correct detection classification.The number of angle comparisons, the average error, and the standard deviation are reported, giving an indirect estimation of the accuracy of the recovered geometry of the correctly segmented portion of image.The set as tuned by GASE is in Table 5, and we refer to as "GASE opt.val."The same table also includes the parameters as tuned in [6] which are referred as "original opt.val."The results are not better than those presented in [6], but in a limited amount of time (we fixed the search in 15 generations), we reached a good result considering that the solution space was larger than that considered in [6].Moreover, no information is given about the time spent to select the solution space, while an average time can be easily determined to explore the whole solution space to select the original opt.val.
In Table 6, we present the results determined by the two sets with a precision tolerance of 80% (see [6]).In Figure 4, we show the plots corresponding to the experiment.The comparison tool provides five error measures, in addition to a measure of correctness.All these measures are related with a tolerance percentage.Plots of Figures 4a, 4b, 4c, 4d, and  4e show the results on the training set of the original opt.val.(curve labeled as HE) versus GASE opt.val.(with label GA).The comparison is very interesting, especially considering that the heuristic selection was performed on a small solution space and tuned on all 10 images, while the GASE one, although optimized by GAs, was tuned on a single image only.
In particular, Figure 4a indicates that both parameter sets achieve the same number of correct instances over the training set, while Figures 4b and 4c demonstrate that, for problems of over-and undersegmentation, GASE and orig-inal opt.val.have an opposite behavior since GASE produces less undersegmentation errors but higher oversegmentation.Finally, the last two plots show that there is no noticeable difference in noise segmentation and misssegmentation.

DISCUSSION AND CONCLUSIONS
The segmentation of range images is a challenging problem both for the selection of the more appropriate algorithm (region growing, edge filling, clustering, etc.) and for the obtained accuracy.A variety of systems to perform this task have been presented in the literature (we recall [6,15]), and all of them need an accurate parameters tuning, according to the image characteristics.
A tool to compare results was proposed in [6], and it has been used to address the parameters tuning (as in [2,3,4]), using only one of the given measures.The tuning methods are based either on careful selection or on solution spacepartitioning search which limits the dimensions of the solution space.
We proposed an automated search method, based on genetic algorithms, that allows us to search a large solution space while requiring a manageable amount of computation time (according to the chosen segmentation algorithm).To address the search, we used a fitness function that combines different measures given by the comparison tool (although using a different source code).We thus implemented a system called GASE to test different segmentation algorithms, namely, UB and USF.
We saw that for the UB, we obtained excellent results, improving segmentation quality and the speed of segmentation.For the USF, we obtained reasonable results, similar to the one proposed by the authors, but without having any knowledge about the nature of the parameters.In fact, GAs start from random values of the parameter set and are able to reach a similar solution in relatively few generations.Finally, embedded in GASE and as a stand-alone tool, an algorithm to robustly award a scalar value to a segmentation was proposed.
We believe that this work provides the basis to design a wizard (or expert system) helping human operators in segmenting images.Our final aim is to build an interactive system that, after an unsupervised training time, will help human operators in the task of obtaining good segmentations.The expert system will provide the framework for the operator to decide the parameters to segment a single or a subset of surfaces in a complex scene (as done in [39]).
Average missed regions on 10 ABW images.

Figure 4 :
Figure 4: Results, as measured by the comparison tool, obtained by the original opt.val.(labeled "HE") and GASE opt.val.(labeled "GA") on 10 images of the ABW database.

Table 1 :
USF parameters: meaning and variability range.

Table 2 :
UB parameters: meaning and variability range.
*The range is limited according to the observed lack of meaning of greater values when segmenting MSU/WSU images, so the shown limits are less than possible.* * Fixed by UB task force; a range from 2 to 4 is allowed.

Table 3 :
Parameters sets for modified UB as tuned by the algorithm authors and by GASE.

Table 4 :
Average fitness values as allowed by original opt.val.and by GASE opt.val.

Table 5 :
Parameters sets for USF as tuned by the algorithm authors and by GASE.

Table 6 :
Average results of USF segmentation algorithm with original opt.val.and GASE opt.val. on 10 ABW images at 80% of compare tolerance (we recall that tool measures segmentation algorithm performances with respect to a certain precision tolerance, ranging from 51 to 95%).