# Craniofacial reconstruction based on a hierarchical dense deformable model

- Yongli Hu
^{1}, - Fuqing Duan
^{2}, - Mingquan Zhou
^{2}, - Yanfeng Sun
^{1}Email author and - Baocai Yin
^{1}

**2012**:217

https://doi.org/10.1186/1687-6180-2012-217

© Hu et al.; licensee Springer. 2012

**Received: **31 October 2011

**Accepted: **29 August 2012

**Published: **9 October 2012

## Abstract

Craniofacial reconstruction from skull has deeply been investigated by computer scientists in the past two decades because it is important for identification. The dominant methods construct facial surface from the soft tissue thickness measured at a set of skull landmarks. The quantity and position of the landmarks are very vital for craniofacial reconstruction, but there is no standard. In addition, it is difficult to accurately locate the landmarks on dense mesh without manual assistance. In this article, we propose an automatic craniofacial reconstruction method based on a hierarchical dense deformable model. To construct the model, we collect more than 100 head samples by computerized tomography scanner. The samples are represented as dense triangle mesh to model face and skull shape. As the deformable model demands all samples in uniform form, a non-rigid registration algorithm is presented to align the samples in point-to-point correspondence. Based on the aligned samples, a global deformable model is constructed, and three local models are constructed from the segmented patches of the eye, nose, and mouth. For a given skull, the global and local deformable models are matched with it, and the reconstructed facial surface is obtained by fusing the global and local reconstruction results. To validate our method, a face deformable model is constructed and the reconstruction results are evaluated in its coefficient domain. The experimental results indicate that the proposed method has good performance for craniofacial reconstruction.

## Keywords

## Introduction

Craniofacial reconstruction is an efficient method to get a visual outlook of an individual in the case of only skull and bone remaining. The traditional plastic methods[1–3] depend on the time-consuming manual work of artists. The reconstruction result is generally determined by the experience of practitioners. To reduce reconstruction time and eliminate subjective biases, different computer-aid craniofacial reconstruction methods have been proposed[4–17]. The state-of-the-art of the computer-aid craniofacial reconstruction have comprehensively been reviewed in the surveys[18–21]. The soft tissue thickness measured on skull is the foundation for craniofacial reconstruction. To get complete tissue thickness, the head samples are usually measured by different equipments such as computerized tomography (CT), magnetic resonance imaging and ultrasound scanner. Most computer-aid craniofacial reconstruction methods fit a selected facial template to the target skull according to the average soft tissue thickness at the skull landmarks[4–8]. Others deform a reference skull to match the remaining skull according to the skull feature such as anthropologic points[9], lines[10], and other features[11]. Applying an extrapolation of the skull deformation to the face template, the reconstructed face will be achieved.

The selection of the template or reference is vital for accurate craniofacial reconstruction. In general, a generic or a specific craniofacial template with similar shape attributes is chosen. But it is difficult to get suitable reference for every dry skull because of the diversity of skull and face modality. In addition, as the complex deformation between the reference and the target skull, the warping methods should intensively be studied to get accurate reconstruction result. So many deformation methods are proposed to model the non-rigid shape deformation of skull and face, such as radial basis functions (RBF)[22, 23], or more exactly, a thin plate spline (TPS)-based deformation[12, 24, 25] for its smoothness. Instead of using fixed template, the recently proposed statistical craniofacial reconstruction methods[12–17] construct a type of deformable model from a set of 3D heads by the principle component analysis (PCA) technique. The statistic deformable model can be regarded as a dynamic template for the given skull. The template deformation is a model fitting procedure driven by the difference between the input skull and the template, in which the model parameters are adjusted by optimization method. The reconstruction result of the deformable model depends on the diversity of samples in the 3D heads database. If there are sufficient samples, good reconstruction results will be achieved. So the statistic method is regarded as the dominant method with great potential application in practice.

Essentially, the craniofacial reconstruction is to figure out the face of unknown skull by the knowledge of skull and face dependency, which is concretely represented as the distribution of the tissue thickness on skull. Most current methods utilize the soft tissue thickness of a set of skull landmarks for craniofacial reconstruction, but it is considered not an ideal approach to model the relationship between face and skull. One reason is that the statistical soft tissue thickness at a set of sparse landmarks is far less than enough to reflect the whole distribution of tissue depth. The other reason is that the quantity and position of the landmarks are indefinite. Different landmark sets have been proposed for craniofacial reconstruction[26–31], though there are definite anatomical points in biometrics[32, 33]. Moreover, it is difficult to detect the landmarks accurately on the complex surface of skull without manual interactive work. In order to reflect the complete tissue thickness distribution and eliminate the disadvantages of the sparse representations, the methods which measure tissue depth at all points have been proposed. In these methods, the face and skull are generally represented in dense form. For examples, Tu et al.[34] constructed a face space for craniofacial reconstruction from the dense skull and face surfaces extracted from head CT images. Vandermeulen et al.[35] also used dense representations (implicit surfaces) for both skin and skull in craniofacial reconstruction. Pei et al.[22] presented a dense tissue depth image representation for craniofacial reconstruction, namely tissue-map. The dense tissue depth methods utilize more information of the relationship between skull and face, it generally has better craniofacial reconstruction results. To represent the dense tissue depth exactly, the dense point registration of skull or face is usually demanded. Although many registration methods[12, 24, 25, 36] have been proposed to construct correspondence between surfaces and point sets, it is still a challenging problem for further investigation because of the complex skull mesh with gross errors or outliers.

To the complex skull and face surfaces, the modality variety is composed of global shape and local detail. However, most current craniofacial reconstruction systems generally take the whole face or skull for shape analysis, while the local feature of skull and face is not emphasized. The recent research reveals that the local shape model is better than the global model to represent local shape variety[37, 38]. Inspired by this point, we propose a hierarchical craniofacial reconstruction model which integrates the global model with several local models to improve craniofacial reconstruction result. To construct the model, the face and skull samples are represented as dense mesh and aligned in point-to-point form by a proposed automatic dense registration algorithm, which contributes to a fully automatic craniofacial reconstruction method. In addition, to get valid evaluation for the craniofacial reconstruction results, we transform the reconstructed face into the coefficient domain of a face deformable model and the distance in the coefficient space is used as the similarity measurement. Comparing with the current measurement methods, such as the mean correspondence point distance or the Euclidean distance matrix[12], the proposed measurement is more suitable for face recognition.

## Data acquisition and preprocessing

In order to construct the craniofacial reconstruction model, we have constructed a head database from CT images. The CT images were obtained by a clinical multi-slice CT scanner (Siemens SOMATOM Sensation 16) in the affiliated hospital of Shaanxi University of Chinese Medicine located in western China. More than 100 patients planned for preoperative osteotomy surgery gave informed consent to scan the whole head for scientific research. The images of each subject are stored in DICOM 3.0 with 512×512 resolution. To get complete head data, 250 to 320 slices are captured for different persons. Most of the patients belong to the Han ethnic group in northern China. In this article, 110 samples are used for craniofacial reconstruction experiments. There are 48 female and 62 male subjects in the collection. The age distribution ranges from 20 to 60.

*L*, the section will be deleted if its length is smaller than

*L*pixels. In our experiment,

*L*is set to be 10. Because the skull is non-convex, the rough contour may disrupt in some regions where it should be connected. We adopt an 8-neighborhood boundary tracing approach to connect each point of the rough contour if they are disrupted, and obtain the final contour, as shown in Figure2d. It is easier for skin to find the outer contour, as the skin contour is simple and generally close for all CT images (Figure2e). So we only need to find a point in the above second step to get the outer skin contour, as shown in Figure2f. After retrieving the outer contours for skull and skin from all CT images, the skull and face surfaces can be represented as triangle meshes by the marching cube algorithm[39]. Usually, the raw skin surface consists of about 220,000 points with 450,000 triangles, while the skull surface contains about 150,000 points with 320,000 triangles, as shown in Figure2g–j. It is dense enough to describe the rich details of skull and face shapes.

To eliminate the inconsistence of position, pose, and scale caused by data acquisition, all samples are transformed into a uniform coordinate system. The uniform coordinate system is determined by four skull feature points, the left and right porion, the left (or right) orbitale and the glabella, denoted by *L*_{
p
}, *R*_{
p
}, *L*_{
o
}, *G*. From three points, *L*_{
p
}, *R*_{
p
}, and *L*_{
o
}, the Frankfurt plane[40] is determined. The coordinate origin (denoted by *O*) is produced from the intersection of the line *L*_{
p
}*R*_{
p
}and the plane which contains point *G* and orthogonally intersects with *L*_{
p
}*R*_{
p
}. We take the line *O* *R*_{
p
}as *x*-axis. The line contains point *O* and has the same direction as the normal of the Frankfurt plane is set as *z*-axis. *y*-axis is obtained by the cross product of *z*- and *x*-axis. The scale of the samples is standardized by setting the distance between *L*_{
p
} and *R*_{
p
} to unit, i.e., every vertex (*x* *y* *z*) of the skull and face is replaced by$(\frac{x}{\left|{L}_{p}{R}_{p}\right|},\frac{y}{\left|{L}_{p}{R}_{p}\right|},\frac{z}{\left|{L}_{p}{R}_{p}\right|})$. The uniform coordinate system of skull and face is shown in Figure2k,l.

## Dense registration for skull and face

### TPS-based non-rigid registration

*S*

_{ref}= {

*P*

_{ rp }|

*P*

_{ rp }= (

*x*

_{ rp },

*y*

_{ rp },

*z*

_{ rp }),

*p*= 1,…,

*N*

_{1}}, and the target, the

*i*th sample, is denoted by

*S*

_{ i }= {

*P*

_{ iq }|

*P*

_{ iq }= (

*x*

_{ iq },

*y*

_{ iq },

*z*

_{ iq }),

*q*= 1,…,

*N*

_{2}}, where

*N*

_{1}and

*N*

_{2}are the points number of

*S*

_{ref}and

*S*

_{ i }such that

*N*

_{1}≤

*N*

_{2}. Then the TPS transformation can be regarded as a map from

*S*

_{ref}to

*S*

_{ i }, denoted by

*f*(.). The correspondent random controlling point sets of

*S*

_{ref}and

*S*

_{ i }are denoted by${M}_{r}=\left\{{L}_{\mathit{\text{rj}}}\right|{L}_{\mathit{\text{rj}}}=({x}_{\mathit{\text{rj}}}^{\ast},{y}_{\mathit{\text{rj}}}^{\ast},{z}_{\mathit{\text{rj}}}^{\ast}),j=1,\dots ,M\}$,${M}_{i}=\left\{{L}_{\mathit{\text{ij}}}\right|{L}_{\mathit{\text{ij}}}=({x}_{\mathit{\text{ij}}}^{\ast},{y}_{\mathit{\text{ij}}}^{\ast},{z}_{\mathit{\text{ij}}}^{\ast}),j=1,\dots ,M\}$, where

*M*is the count of the controlling points. From the definition of TPS,

*f*(.) will satisfy the following interpolation conditions:

*X*=(

*x*,

*y*,

*z*)

^{ T },${F}_{X}={(\frac{\partial f}{\partial x},\frac{\partial f}{\partial y},\frac{\partial f}{\partial z})}^{T}$ and

*I*

_{3}= (1,1,1)

^{ T }. It is proved that TPS can be decomposed into affine and non-affine components[45]. This fact is generally represented as the following formula:

where *P*∈*S*_{ref} with the homogeneous coordinate (1,*x*,*y*,*z*). *d* is a 4×4 affine transformation matrix. *K* is the TPS kernel, a 1×*M* vector in the form of *K* = (*K*_{1}(*P*),…,*K*_{
M
}(*P*)), where *K*_{
j
}(*P*) = ∥*P*−*L*_{
rj
}∥,*j* = 1,…,*M*. *w* is a *M*×4 warping coefficient matrix representing the non-affine deformation.

*d*and

*w*must be determined. There are two solutions to this problem, namely, the interpolating and non-interpolating methods. In the interpolating case, formula (1) is satisfied. Putting formula (3) into (1) and confining

*w*to non-affine transformation, i.e.,${M}_{1}^{\phantom{\rule{2.77695pt}{0ex}}\phantom{\rule{2.77695pt}{0ex}}\prime T}w=0$, it leads to a direct solution for

*d*and

*w*formed by the following matrix relation:

*M*×4 matrixes corresponding to the controlling points sets

*M*

_{ r }and

*M*

_{ i }in homogeneous coordinate form.

*K*

^{ ′ }is a

*M*×

*M*symmetry matrix representing the spatial relation of

*M*

_{ r }with the element

*k*

_{ uv }= ∥

*L*

_{ ru }−

*L*

_{ rv }∥,

*u*= 1,…,

*M*,

*v*= 1,…,

*M*. In the non-interpolating case, formula (1) is not strictly satisfied. The following energy function can be minimized to find the optimized answer.

where *λ* is the weight to control the blending component, and given a fixed *λ* there is a unique minimum for the energy function. It is conducted that the non-interpolating solution has a parallel form replacing *K*^{
′
} in formula (4) by *K*^{
′
} + *λI*. As the correspondent controlling points on the reference and target acquired by ICP closest points searching is not exactly correct, the condition in formula (1) is not satisfied. So the non-interpolating method is adopted in this article.

*f*(.), the reference

*S*

_{ref}can be deformed by the formula (3). The deformed reference is denoted by${S}_{i}^{\prime}$. Then the correspondence between${S}_{i}^{\prime}$ and

*S*

_{ i }can be obtained using ICP closest point searching. But the closest point searching procedure of ICP is a time consuming procedure with computation in

*O*(

*N*

_{1}×

*N*

_{2}). To get high efficiency, we adopt a

*K*-dimensional binary search tree (KD-tree)[48] to model the target, which is proved having a complexity with

*O*(

*N*

_{1}×log

*N*

_{2}) for the pairwise closest point searching. Considering that the closest point matching based on the initial TPS deformation is more inaccurate and the beginning alignment refers to the global correspondence while the later to the local area, a deterministic annealing strategy is applied in the stepwise TPS-based registration procedure. At the beginning of the registration, the points move a little to its deformed points, and the step size increases gradually when TPS deformation result becomes desirable. At the same time, the number of the random points increase from a small initial number for enhancing the holistic deformation at the beginning, and the blending weight of TPS in (5) decreases to relax the global constrains. The following gives the implementation of the proposed non-rigid TPS-based registration method.

- 1.
Create KD-tree for the

*i*th sample*S*_{ i }, denote it by*T*_{ i }; - 2.
Apply ICP alignment between

*S*_{ref},*S*_{ i }, then transform*S*_{ref}by the rigid transformation of ICP, the transformed sample is denoted by ${S}_{i}^{\prime}$; - 3.
Produce random controlling point set ${M}_{i}^{\ast}$ with cardinality of

*M*for ${S}_{i}^{\prime}$; - 4.
For each point in ${M}_{i}^{\ast}$, search its correspondent point on

*S*_{ i }by querying on*T*_{ i }, the correspondent point set is denoted by*M*_{ i }; - 5.
Determine the TPS transformation

*f*from ${M}_{i}^{\ast}$,*M*_{ i }with blending weight*λ*; - 6.
Apply the TPS transformation

*f*on ${S}_{i}^{\prime}$, the deformed ${S}_{i}^{\prime}$ is denoted by ${S}_{i}^{\mathrm{\prime \prime}}$; - 7.
Update ${S}_{i}^{\prime}$ by adding a movement to each point ${P}^{\prime}\in {S}_{i}^{\prime}$:

*P*^{ ′ }=*P*^{ ′ }+*δ*(*f*(*P*^{ ′ })−*P*^{ ′ }), where $f\left({P}^{\prime}\right)\in {S}_{i}^{\mathrm{\prime \prime}}$ and*δ*is the step size; - 8.
For each ${P}^{\prime}\in {S}_{i}^{\prime}$, search its correspondent point

*P*^{ ′′ }∈*S*_{ i }by querying on*T*_{ i }; - 9.
Update the parameters:

*M*=*M*+ △*M*,*δ*=*δ*+ △*δ*,*λ*=*λ*−△*λ*, where △*M*, △*δ*and △*λ*are the pre-assigned increments; - 10.
If the iterations

*l*<*l*_{0}and $\frac{1}{{N}_{1}}\sum _{{P}^{\prime}\in {S}_{i}^{\prime}}\parallel {P}^{\prime}-{P}^{\mathrm{\prime \prime}}\parallel >{\epsilon}_{0}$, where*l*_{0}is the given maximum loops and*ε*_{0}is the given threshold, goto 3; - 11.
The final correspondence of

*S*_{ref}and*S*_{ i }is achieved from the equivalent correspondence of ${S}^{\prime}$ and*S*_{ i }, denote it by ${S}_{i}^{0}$.

In our experiments, *M* ranges from$\frac{1}{500}$ to$\frac{1}{80}$ of the point number of the reference, *l*_{0} = 30, *ε*_{0} = 10^{−6}, the initial *δ* = 0 with$\u25b3\delta =\frac{1}{{l}_{0}}$, and the initial *λ* = 0.01 with △*λ* = *λ*∗0.05.

### Group registration by linear combination

It is important to select a closest reference for all samples to get good alignment, but the fixed reference may greatly differ with some samples as there is much variety in skull and face modality. Considering there are enough samples in our database, we try to improve the above registration by a group registration method based on a linear combination model. Instead of using a fixed reference, we utilize the combination of the above aligned samples to generate dynamic reference for every samples. As the dynamic reference is closer to the given sample, aligning the dynamic reference to the target sample will give better result. Based on the new correspondence result, we can construct new dynamic reference by linear combination, which will get more accurate aligning result. By this iterative procedure, the registration precise will be improved gradually. In the following, the iterative registration procedure by linear combination is described in detail.

*S*

_{ref}as a

*N*

_{1}×1 vector in form of${({x}_{r1},{y}_{r1},{z}_{r1},\dots ,{x}_{\mathit{\text{rp}}},{y}_{\mathit{\text{rp}}},{z}_{\mathit{\text{rp}}},\dots ,{x}_{r{N}_{1}},{y}_{r{N}_{1}},{z}_{r{N}_{1}})}^{T}$, then from the point-to-point correspondence, the

*N*aligned samples$\left\{{S}_{i}^{0}\right|i=1,\dots ,N\}$ in the first-step can be formatted as vectors with the same form as

*S*

_{ref}, i.e.,${S}_{i}^{0}={({x}_{i1},{y}_{i1},{z}_{i1},\dots ,{x}_{\mathit{\text{ip}}},{y}_{\mathit{\text{ip}}},{z}_{\mathit{\text{ip}}},\dots ,{x}_{i{N}_{1}},{y}_{i{N}_{1}},{z}_{i{N}_{1}})}^{T}$, where the point$({x}_{\mathit{\text{ip}}},{y}_{\mathit{\text{ip}}},{z}_{\mathit{\text{ip}}})\in {S}_{i}^{0}$ is the correspondent point of (

*x*

_{ rp },

*y*

_{ rp },

*z*

_{ rp }). By this representation, we can get a new object by the following linear combination:

**a**= (

*a*

_{1},…,

*a*

_{ N }) is the linear combination coefficient vector. For each original sample

*S*

_{ i }, a dynamic reference${S}_{i}^{\ast}$ can be determined by the following minimizing formula:

- 1.
Align

*S*_{ref}to each sample*S*_{ i }by TPS-based method and get the primary aligning result ${S}_{i}^{0}$; - 2.
Produce the dynamic reference ${S}_{i}^{\ast}$ for each

*S*_{ i }by linear combination; - 3.
Align the dynamic reference ${S}_{i}^{\ast}$ to

*S*_{ i }by TPS-based method and get the aligning result ${S}_{i}^{1}$; - 4.
If the iterations are less than the given maximum loops and the global difference

*Eg*is great than the given threshold, update ${S}_{i}^{0}$ by ${S}_{i}^{0}={S}_{i}^{1}$ and goto 2; - 5.
Get the final aligning result from $\left\{{S}_{i}^{1}\right\}$.

## The construction of the hierarchical deformable model

*i*th head sample as a high dimension vector composed of skull and face vectors in the following form:

where${S}_{i}={({x}_{i1}^{S},{y}_{i1}^{S},{z}_{i1}^{S},\dots ,{x}_{\mathit{\text{im}}}^{S},{y}_{\mathit{\text{im}}}^{S},{z}_{\mathit{\text{im}}}^{S})}^{T}$ and${F}_{i}={({x}_{i1}^{F},{y}_{i1}^{F},{z}_{i1}^{F},\dots ,{x}_{\mathit{\text{in}}}^{F},{y}_{\mathit{\text{in}}}^{F},{z}_{\mathit{\text{in}}}^{F})}^{T}$ are the vectors of the *i* th skull and face with dimensions of 3*m* and 3*n*, respectively.

*H*

_{ i }|

*i*= 1,…,

*N*} will produce new skull and face. Given an unknown skull, the closest combination skull can be achieved by the model matching procedure. Extrapolating the combination of the skull vectors to the face vectors in the model will get a reconstructed face for the given skull. The detail of the model matching will be given in the next section. As the prototypic skull and face samples have high dimension data with large redundance, PCA is applied to construct the following deformable model:

*h*

_{ i }|

*i*=1,…,

*N*

^{ ′ }} are the former

*N*

^{ ′ }components corresponding to the eigenvalues {

*σ*

_{ i }|

*i*=1,…,

*N*

^{ ′ }} of the covariance matrix of the subtracting vectors$\{{H}_{i}-\overline{H}|i=1,\dots ,N\}$ in descending order.

*N*

^{ ′ }is determined by 98% of the cumulative eigenvalues of the variance. The combination coefficient$\alpha =({\alpha}_{1},\dots ,{\alpha}_{{N}^{\prime}})$ is the parameter for the deformable model. To generate a plausible face, the probability of α is constrained by the following formula:

The model in (9) is the global model referring to the modality of whole skull and face. To characterize the local shape variety, we construct several local deformable models with respect to the main organs of face, the eye, nose, and mouth. The first step for constructing the local models is to segment the organs. It is difficult to get an ideal automatic segment for different skull and face. As our samples have been aligned, getting the segments of the reference, the segments of other samples can be obtained from the correspondence of points. So we segment the local patches of the reference by hand. The segmented local patches of the reference are shown in Figure4e–h. Based on the segmented data, the local models can be constructed by the similar method of the global model. The hierarchical deformable model is constructed by integrating the local models with the global model.

## Craniofacial reconstruction

*S*

_{ md }(α) and the given skull, after assigning the initial values for the combination coefficients, we align

*S*

_{ md }(α) to the given skull by the TPS-based registration. From the obtained correspondence, we format the given skull as a vector

*S*

_{ gv }in the same form as

*S*

_{ md }(α). So the difference between the given skull

*S*

_{ gv }and the model combination skull

*S*

_{ md }(α) can be represented as the square module of the subtracting vector as following form:

*S*

_{ md }(α) will change as the combination coefficients updating in the optimization procedure. So the registration between

*S*

_{ md }(α) and

*S*

_{ gv }is implemented every 20 loops to ensure the error in 11 computed in correct correspondence with the updated

*S*

_{ gv }. To solve the optimization, we adopt a gradient descent algorithm to resolve the optimization problem. The core of the method is to find the gradient descent direction of

*E*(α) about α, which is equal to the negative derivative of

*E*(α). From 11, the partial derivative of

*E*(α) can be formed as follows:

*E*(α) depends on$\frac{{S}_{\mathit{\text{md}}}^{\partial}\left(\alpha \right)}{\partial \alpha}$, which can be deduced from formula 9 as the form$\frac{{S}_{\mathit{\text{md}}}^{\partial}\left(\alpha \right)}{\partial \alpha}=\frac{\partial (\overline{S}+\mathbf{s}{\alpha}^{T})}{\partial \alpha}=\frac{\partial \left(\mathbf{s}{\alpha}^{T}\right)}{\partial \alpha}$, where$\overline{S}$ is the skull part of$\overline{H}$,$\mathbf{s}=({s}_{1}\dots {s}_{i}\dots {s}_{{N}^{\prime}})$ and

*s*

_{ i }is the skull part of

*h*

_{ i }. For a

*α*

_{ i },$\frac{\partial \left(\mathbf{s}{\alpha}^{T}\right)}{\partial {\alpha}_{i}}=\frac{\partial \left(\sum _{i=1}^{{N}^{\prime}}{\alpha}_{i}{s}_{i}\right)}{\partial {\alpha}_{i}}={s}_{i}$. Putting it into 12 we get the following partial derivative computation for

*α*

_{ i }:

As the vectors *s*_{
i
},$\overline{S}$, and *S*_{
gv
} on the above formulas have the dimension same as the dimension of the reference skull vector, it is time-consuming to implement the optimization of the gradient descent algorithm with the high dimension in 3*m*. To reduce computation, we extract a random sub-vectors of these vectors to replace them on the above equations. That is a subset indices$\{{i}_{0},\dots ,{i}_{{m}^{\prime}}\}$ are selected randomly from the continuous indices {1,…,*m*} in each gradient descent iteration. As *m*^{
′
}≪*m*, the computation is greatly reduced. As the indices of the vector element is correspondent to the points on the skull, this method is equivalent to select a subset of random points, representing the model and given skull for similarity error computation. So the similar random point selection approach used in the TPS-based non-rigid registration can be applied here to get the random sub-vector. Considering that the model deformation scale is smaller than the deformation between the reference and the target in data registration, and the local deformation dominates in the model matching, we use more quantity of random points in the model matching procedure. In our experiment, a subset with$\frac{1}{20}$ indices of the model skull vector is selected to implement the model matching computation. The maximal iteration is set to 500 for the global model and 1000 for the local models. To avoid the influences of noise and make use of the contribution of every points, this random subset is updated at each iteration. However, this will lead instability for the error in 11 at the beginning tens of iterations, but it behaves steadily in the later iterations and converges to a minimal value. We have tested different sizes of random subsets in the model matching experiment, smaller size than the assigned number generally cannot get satisfied precision and even not convergent. While more points added in the subset, the improvement for the model matching is insignificant. By this model matching procedure, the best matched model skull will be obtained. Then the reconstructed face can be calculated by the combination of the face part of *h*_{
i
} in 9 with the same coefficients as the model skull.

*F*and the sub-mesh

*F*

_{sub}, the local meshes are set onto the global mesh in proper position using translation and rotation transformation

*R*

^{∗},

*T*

^{∗}, where

*R*

^{∗},

*T*

^{∗}are determined by minimizing the average distance of the correspondent points of the sub-mesh and the global mesh, i.e.,$({R}^{\ast},{T}^{\ast})=\text{arg}\underset{R,T}{\text{min}}\sum _{{P}_{0}\in {F}_{\text{sub}}}\parallel R{P}_{0}+T-{P}_{1}\parallel $, where

*P*

_{1}∈

*F*is the correspondent points of

*P*

_{0}. The first step fusion result is shown in Figure6b. Second, the inconsistence at the boundary is removed by a mesh stitching algorithm, in which both the points on the sub-mesh and the global mesh near the boundary are deformed to an interspaced position by interpolation method. The detail of the mesh interpolation is shown in Figure6d. For the sub-mesh boundary

*B*

_{0}and a point

*P*

_{0}∈

*B*

_{0}, we can get the correspondent contour (denoted by

*B*

_{1}) on the global mesh and the correspondent point

*P*

_{1}∈

*B*

_{1}of

*P*

_{0}by the above segmentation of the reference. Then the interpolating point

*P*

_{2}is calculated by${P}_{2}=\frac{({P}_{0}+{P}_{1})}{2}$. Given a scale

*l*

_{0}, the boundary

*B*

_{0}will shrink into interior with

*l*

_{0}step and get a contour${B}_{0}^{\prime}$ which is indicated by a point${Q}_{0}\in {B}_{0}^{\prime}$ in Figure6d, while the counter

*B*

_{1}shrink oppositely on the global mesh and get a contour${B}_{1}^{\prime}$ which is indicated by a point${Q}_{1}\in {B}_{1}^{\prime}$ in Figure6d. The stitching method is to find a pair of interpolation functions

*f*

_{0},

*f*

_{1}have the following conditions:

There are many interpolation methods can be used to meet the above conditions, such as RBF function. For convenience, we adopt the above TPS to solve the interpolation. Having determined the interpolation functions, the final fusion result (Figure6c) can be achieved by applying *f*_{0} to the points between the contours${B}_{0},{B}_{0}^{\prime}$ on the sub-mesh and *f*_{1} to the points between the contours${B}_{1},{B}_{1}^{\prime}$ on the global mesh.

## Experimental results and discussion

where$\overline{F}$ is the average face, {*f*_{
i
}|*i*=1,…,*k*} are the former *k* components. When two faces *F*_{1}, *F*_{2} are compared, their model coefficients β_{1}, β_{2} are computed by the model-matching procedure. Then the difference between *F*_{1} and *F*_{2} is measured by the distance between β_{1}, β_{2} in the coefficients space.

## Conclusion

We proposed a hierarchical dense deformable model for automatic craniofacial reconstruction. The feature of proposed model is that the skull and face are represented as dense mesh without landmarks. The advantage of this representation is that the dense meshes contain more meta-data for exploring the intrinsic relation between skull and face. In addition, the presented non-rigid dense meshes registration and the model matching procedure can be implemented automatically, which contributes to the fully automatic craniofacial reconstruction method. The craniofacial reconstruction experiments show that the hierarchical model has better reconstruction results than the single global model. The craniofacial reconstruction evaluation problem is also explored in this article. We present an evaluation method based on a deformable facial model. By comparing with the average distance of correspondent points method in face recognition experiment, the evaluation method may be the potential method for identification in the application of craniofacial reconstruction. In the future work, we plan to capture more head scans to increase the plenty of the samples, which is important for the model deformable capacity. Based on the abundant samples, the personal properties, such as gender, age, and BMI, are considered integrating with the hierarchical dense deformable model. The reconstruction result will be improved if these properties information are properly utilized. In addition, it is worthy of exploring ideal evaluation methods for the results of craniofacial reconstruction.

## Declarations

### Acknowledgements

This study was partly supported by the 973 Program of China (No. 2011CB302703) and the National Natural Science Foundation of China (Nos. 60825203, 61171169, 61133003, 60973057, 60736008, 61272363).

## Authors’ Affiliations

## References

- Gerasimov M:
*The Face Finder*. 1971.Google Scholar - Snow C, Gatliff B, Williams KM: Reconstruction of the facial features from skull: an evaluation of its usefulness in forensic anthropology.
*Am. J. Phys. Anthropol*1970, 33: 221-228. 10.1002/ajpa.1330330207View ArticleGoogle Scholar - Lebedinskaya G, Balueva T, Veselovskaya E: Development of methodological principles for reconstruction of the face on the basis of skull material. In
*Forensic Analysis of the Skull*. Wiley-Liss Inc.; 1993.Google Scholar - Vanezis P: Application of 3-D computer graphics for facial reconstruction and comparison with sculpting techniques.
*Forensic Sci. Int*1989, 42: 69-84. 10.1016/0379-0738(89)90200-4View ArticleGoogle Scholar - Vanezis P, Vanezis M, McCombe G, Niblet T: Facial reconstruction using 3-D computer graphics.
*Forensic Sci. Int*2000, 108: 81-95. 10.1016/S0379-0738(99)00026-2View ArticleGoogle Scholar - Evenhouse R, Rasmussen M, Sadler L: Computer-aided forensic facial reconstruction.
*J. Biocommun*1992, 19: 22-28.Google Scholar - Shahrom AW, Vanezis P, Chapman RC, Gonzales A, Blenkinsop C, Rossi ML: Techniques in facial identification: computer-aided facial reconstruction using laser scanner and video superimposition.
*Int. J. Legal Med*1996, 108: 194-200. 10.1007/BF01369791View ArticleGoogle Scholar - Tyrell AJ, Evison MP, Chamberlain AT, Green MA: Forensic three-dimensional facial reconstruction: historical review and contemporary developments.
*J. Forensic Sci*1997, 42: 653-661.Google Scholar - Jones MW: Facial Reconstruction Using Volumetric Data.
*Proceedings of the Sixth International Vision Modelling and Visualisation Conference*2001, 135-150.Google Scholar - Quatrehomme G, Cotin S, Subsol G, Delingette H, Garidel Y, Grevin G, Fidrich M: A fully three-dimensional method for facial reconstruction based on deformable models.
*J. Forensic Sci*1997, 42: 649-652.Google Scholar - Nelson LA, Michael SD: The application of volume deformation to three-dimensional facial reconstruction: a comparison with previous techniques.
*Forensic Sci. Int*1998, 94: 167-181. 10.1016/S0379-0738(98)00066-8View ArticleGoogle Scholar - Claes P, Vandermeulen D, De Greef S, Willems G, Suetens P: Craniofacial reconstruction using a combined statistical model of face shape and soft tissue-depths: methodology and validation.
*Forensic Sci. Int*2006, 159(1):147-158.View ArticleGoogle Scholar - Claes P, Vandermeulen D, De Greef S, Willems G, Suetens P: Statistically deformable face models for cranio-facial reconstruction.
*CIT*2006, 14(1):21-30. 10.2498/cit.2006.01.03View ArticleGoogle Scholar - Claes P, Vandermeulen D, De Greef S, Willems G, Clement JG, Suetens P: Bayesian estimation of optimal craniofacial reconstructions.
*Forensic Sci. Int*2010, 201: 146-152. 10.1016/j.forsciint.2010.03.009View ArticleGoogle Scholar - Berar M, Tilotta FM, Glaunes JA, Rozenholc Y: Craniofacial reconstruction as a prediction problem using a Latent Root Regression model.
*Forensic Sci. Int*2011, 210(1-3):228-236. 10.1016/j.forsciint.2011.03.010View ArticleGoogle Scholar - Berar M, Desvignes M, Bailly G, Payan Y: 3d semi-landmark-based statistical face reconstruction.
*CIT*2006, 14(1):31-43. 10.2498/cit.2006.01.04View ArticleGoogle Scholar - Paysan P, Luthi M, Albrecht T, Lerch A, Amberg B, Santini F, Vetter T: Face reconstruction from skull shapes and physical attributes.
*Proceedings of 31st DAGM Symposium for Pattern Recognition (DAGM’09)*2009, 232-241.Google Scholar - Claes P, Vandermeulen D, De Greef S, Willems G, Clement JG, Suetens P: Computerized craniofacial reconstruction: conceptual framework and review.
*Forensic Sci. Int*2010, 201: 138-145. 10.1016/j.forsciint.2010.03.008View ArticleGoogle Scholar - Stavrianos Ch, Stavrianou I, Zouloumis L, Mastagas D: An introduction to facial reconstruction.
*Balkan J. Stomatol*2007, 11(2):76-83.Google Scholar - De Greef S, Claes P, Mollemans W, Vandermeulen D, Suetens P, Willems G: Computer-assisted facial reconstruction: recent developments and trends.
*Rev. Belge Med. Dent*2005, 60(3):237-249.Google Scholar - Wilkinson C: Computerized forensic facial reconstruction: a review of current systems.
*Forensic Sci. Med. Pathol*2005, 1(3):173-177. 10.1385/FSMP:1:3:173View ArticleGoogle Scholar - Pei Y, Zha H, Yuan Z: Tissue map based craniofacial reconstruction and facial deformation using rbf network.
*The Third International Conference on Image and Graphics (ICIG’04)*2004, 398-401.Google Scholar - Tu P, Hartley R, Lorensen W, Allyassin M, Gupta R, Heier L: Face reconstruction using flesh deformation modes. In
*Computer-graphic Facial Reconstruction*. Academic Press; 2005:145-162.Google Scholar - Deng Q, Zhou M, Shui W, Wu Z, Ji Y, Bai R: A novel skull registration based on global and local deformations for craniofacial reconstruction.
*Forensic Sci. Int*2011, 208: 95-102. 10.1016/j.forsciint.2010.11.011View ArticleGoogle Scholar - Turner WD, Brown REB, Kelliher TP, Tu PH, Taister MA, Miller KWP: A novel method of automated skull registration for forensic facial approximation.
*Forensic Sci. Int*2005, 154: 149-158. 10.1016/j.forsciint.2004.10.003View ArticleGoogle Scholar - De Greef S, Claes P, Vandermeulen D, Mollemans W, Suetens P, Willems G: Large-scale in-vivo caucasian facial soft tissue thickness database for craniofacial reconstruction.
*Forensic Sci. Int*2006, 159(1):126-146.View ArticleGoogle Scholar - Tilotta F, Richard F, Glaunes JA, Berar M, Gey S, Verdeille S, Rozenholc Y, Gaudy JF: Construction and analysis of a head CT-scan database for craniofacial reconstruction.
*Forensic Sci. Int*2009, 191: 112.e1-112.e12. 10.1016/j.forsciint.2009.06.017View ArticleGoogle Scholar - Martin R, Saller K:
*Lehrbuch der Anthropologie in Systematischer Darstellung*. 1956.Google Scholar - Menin C: La population gallo-romaine de la necropole de Maule (France, Yvelines): etude anthropologique. PhD thesis, University Paris 6, Pierre et Marie Curie, France. 1977.Google Scholar
- Welcker RH:
*Schillers Schdel und todenmaske, nebst mittheilungen ber Schadel und todenmaske Kants*. 1883.Google Scholar - His W: Anatomische forschungen ueber johann sebastian bachos gebeine und antliz nebst bemerkungen ueber dessen bilder. abhandlungen dermathematisch physikalischen klasse der konigl.
*Sachsischen Gesellschaft derWissenchaften*1895, 22: 379-420.Google Scholar - Kollmann J, Buchly W: Die persistenz der rassen und die reconstruction derphysiognomie prahistorischer schadel.
*Archiv fur Anthropologie*1898, 25: 329-359.Google Scholar - Rhine JS, Campbell HR: Thickness of facial tissues in American Blacks.
*J. Forensic Sci*1980, 24(4):847-858.Google Scholar - Tu P, Book R, Liu X, Krahnstoever N, Adrian C, Williams P: Automatic face recognition from skeletal remains.
*IEEE Conference on Computer Vision and Pattern Recognition (CVPR’07)*2007, 1-7.Google Scholar - Vandermeulen D, Claes P, Loeckx D, De Greef S, Willems G, Suetens P: Computerized craniofacial reconstruction using CT-derived implicit surface representations.
*Forensic Sci. Int*2006, 159: S164-S174.View ArticleGoogle Scholar - Berar M, Desvignes M, Bailly G, Payan Y:
*3D meshes registration: application to statistical skull model*. 2004.Google Scholar - Pei Y, Zha H, Yuan Z: The craniofacial reconstruction from the local structural diversity of skulls.
*Comput. Graphic Forum*2008, 27(7):1711-1718. 10.1111/j.1467-8659.2008.01315.xView ArticleGoogle Scholar - Tilotta F, Glaunes J, Richard F, Rozenholc Y: A local technique based on vectorized surfaces for craniofacial reconstruction.
*Forensic Sci. Int*2010, 200: 50-59. 10.1016/j.forsciint.2010.03.029View ArticleGoogle Scholar - Lorensen WE, Cline HE: Marching cubes: a high resolution 3D surface construction algorithm.
*Comput. Graph*1987, 21(4):163-169. 10.1145/37402.37422View ArticleGoogle Scholar - Frankfurt plane http://en.wikipedia.org/wiki/Frankfurt_planeGoogle Scholar
- Chui H, Rangarajan A: A new algorithm for non-rigid point matching.
*IEEE Conference on Computer Vision and Pattern Recognition (CVPR’00)*2000, 44-51.Google Scholar - Chui H, Rangarajan A: A new point matching algorithm for non-rigid registration.
*Comput. Vis Image Understand*2003, 89: 114-141. 10.1016/S1077-3142(03)00009-2View ArticleGoogle Scholar - Hutton TJ, Buxton BF, Hammond P: Automated registration of 3D faces using dense surface models.
*The 2003 British Machine Vision Conference (BMVC’03)*2003, 439-448.Google Scholar - Myronenko A, Song XB: Point set registration: coherent point drift.
*IEEE Trans. Pattern Anal. Mach. Intell*2010, 32(12):2262-2275.View ArticleGoogle Scholar - Bookstein FL: Principal Warps: thin-plate Splines and the Decomposition of Deformations.
*IEEE Trans. PAMI*1989, 11(6):567-585. 10.1109/34.24792View ArticleGoogle Scholar - Besl P, McKay N: A method for registration of 3-d shapes.
*IEEE Trans. PAMI*1992, 14(2):239-256. 10.1109/34.121791View ArticleGoogle Scholar - Moenning C, Dodgson NA: Fast marching farthest point sampling for implicit surfaces and point clouds. Technical Report 565. Computer Laboratory, University of Cambridge, UK, 2003Google Scholar
- Greenspan MA, Yurick M: Approximate K-D tree search for efficient ICP.
*The 4th International Conference on 3D Digital Imaging and Modeling (3DIM’03)*2003, 442-448.Google Scholar - GB 10000-88: Human dimensions of Chinese adult, National Standard of the People’s Republic of China. 1988, 12-13.Google Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.