Detection, 3-D positioning, and sizing of small pore defects using digital radiography and tracking

Abstract

This article presents an algorithm that handles the detection, positioning, and sizing of submillimeter-sized pores in welds using radiographic inspection and tracking. The possibility to detect, position, and size pores which have a low contrast-to-noise ratio increases the value of the nondestructive evaluation of welds by facilitating fatigue life predictions with lower uncertainty. In this article, a multiple hypothesis tracker with an extended Kalman filter is used to track an unknown number of pore indications in a sequence of radiographs as an object is rotated. Each pore is not required to be detected in all radiographs. In addition, in the tracking step, three-dimensional (3-D) positions of pore defects are calculated. To optimize, set up, and pre-evaluate the algorithm, the article explores a design of experimental approach in combination with synthetic radiographs of titanium laser welds containing pore defects. The pre-evaluation on synthetic radiographs at industrially reasonable contrast-to-noise ratios indicate less than 1% false detection rates at high detection rates and less than 0.1 mm of positioning errors for more than 90% of the pores. A comparison between experimental results of the presented algorithm and a computerized tomography reference measurement shows qualitatively good agreement in the 3-D positions of approximately 0.1-mm diameter pores in 5-mm-thick Ti-6242.

1 Introduction

Radiographic inspection is frequently used within the manufacturing industry to detect and characterize defects in a wide variety of structures. The characterization generally consists of determining the defect type, size, and position within the structure. The size together with its distance to other defects and to the surface are known to affect the fatigue life of the structure[1]. This results in three issues: firstly, the size is of interest to measure in itself; secondly, the size is also of interest since assuming no interaction between the defects, the smallest defect that can be detected with high probability, will be a parameter limiting the predicted fatigue life; and thirdly, the distance which is of interest is in three-dimensional (3-D). However, only the distance in two-dimensional (2-D) is available from conventional projection radiography, and the distance in the third dimension therefore has to be assumed. These three issues are encountered, for example, with components built by laser-welded thin lightweight alloys. Lightweight titanium alloys are extensively used within the aero industry, and laser welding of these alloys are increasing. However, the combination of lightweight alloys and laser welding, including welding methods similar to it, has been shown to result in the formation of clusters of small submillimeter pores in the melted zone of the weld[2, 3]. The pores are small (the average diameter is ≲ 0.4 mm), and as isolated defects, their effect on fatigue life is considered negligible. However, if they cannot be considered as isolated defects, depending on their size, they might no longer be negligible. Therefore, there is a need to detect small, low contrast-to-noise ratio (CNR) defects and to measure their size and 3-D positions.

The 3-D positioning has been solved by two fundamentally different approaches in the literature. The first approach is based on reconstructing the whole bulk volume using computerized tomography (CT). However, a complete set of projections over the whole 180° rotation is needed, and some structures, for example planar structures, are therefore difficult to reconstruct. In certain situations, the problem with an incomplete rotation set can be solved by means of limited view tomography[4]. The second approach is based on not reconstructing the whole volume but rather by focusing on reconstructing the 3-D position of each defect, referred to as point reconstruction methods[5]. In point reconstruction, the 3-D position of the defect is calculated using the defect projection coordinates in the image plane for a few rotations and/or translations.

The problem of detecting low CNR defects and of measuring their size has instead received attention within the more general automatic weld inspection analysis field. The automatic analysis field approach is in general to segment out any possible defect[6] and then characterize it[7–9], for example, by its type (lack of fusion or crack etc). A merge between the segmentation part of the general automatic analysis and the 3-D point reconstruction has been shown to result in a high probability of detecting true defects and a low probability of detecting false defects, especially for low CNR defects[10]. After segmenting out the defect indications in the detector plane, there will be more false defects than true defects detected due to low CNR. The true defects will form paths in the detector plane as the object is rotated or translated, while the false defects will not. The performance has been improved by adding the classification step and removing the need for prior knowledge of the setup geometry in[11]. If the 3-D positions of the defects are not needed, they can instead be used implicitly as in[12] where the defects are tracked during constant translation to yield a computationally less expensive algorithm.

The currently presented solutions to the coupled low CNR and 3-D point reconstruction problem all have in common that the defect needs to be detected in all rotation projections. This need is difficult to fulfill as the defect CNR decreases. Furthermore, the defect can also fail to be detected in some rotation projection due to extreme X-ray interactions. These extreme interactions will eventually occur, though with low probability, due to the inherent statistical nature of radiographic inspection. However, this need of full detection is removed in the solution proposed here. Instead of being formulated as a vision system problem and solved by epipolar geometry as in[10, 11], it is explored using general tracking theory[13]. In tracking theory, the state (3-D position) of an object (defect) is tracked by assigning measurements (indications) to it as time increases (rotation). In general, the measurements do not have to be present in all time points for the object to be successfully tracked. Therefore, a tracking theory approach will take advantage of the path of defect indications in the detector plane without making it a limitation by demanding the defect to be detected in all rotations. The main value of this work is that it considers the coupled low CNR and 3-D point reconstruction problem from a general tracking theory point of view.

The defect size is measured by considering the change in intensity over the defect indication compared to the background together with an intensity calibration, a procedure considered conventional in radiography. However, a systematic methodology to set up optimal parameters for the algorithms, including this last one, is proposed. The methodology consists of using design of experiment (DOE), robust design, and synthetic radiographs.

The outline of this article is as follows. First, the radiographic inspection procedure together with the algorithm is described. This is followed by an explanation of how simulated radiographs are used to set up the algorithm and to evaluate its performance. The result of the performance evaluation using the synthetic radiographs is then presented. This is followed by a description of the experimental setup and the experimental qualitative results.

2 Algorithm and inspection procedure

The radiographic inspection setup geometry and the procedure is illustrated in Figure1. Common setup geometry parameters such as source to rotation point x coordinate (R _x ), source to detector distance (SDD), and object to detector distance (ODD) are defined in the figure. The intended inspection application is to detect, position, and size submillimeter pore defects in thin (≈ 5 mm) laser-welded titanium. The laser weld geometry parametrization[14] is also indicated in Figure1. The parameters are nominal thickness (T), weld width (B), enforcement radius (R₀) and thickness (δ₀), and undercut radius (R₁) and thickness (δ₁).

Figure 1

Radiographic inspection method and cross section of the weld.

The image analysis of the radiographs faces two main difficulties, which are illustrated with synthetic radiographs. Firstly, as can be seen in Figure2, large-scale variations over small pixel length scales in the projected weld geometry dominate the gray scale variation compared to the pore indications shown in Figure3. The pore indications in Figure3 are some 20 to 30 times smaller than the large-scale variations in Figure2. Secondly, as indicated in Figure3, the CNR of the pore indications are low. For pores at the size of the detector pixel size, the contrast is highly sensitive to the pore size. The contrast is approximately proportional to radius³ or even worse depending on its size compared to the detector pixel size. Furthermore, the pore indication is small in terms of affected number of detector pixels.

Figure 2

Weld geometry line profiles. Two line profiles at 0° and 25° rotation taken from simulated radiographs perpendicular to the welding direction. Both profiles are arbitrarily scaled and offset for illustrative purposes.

Figure 3

Pore indication line profiles. Line profiles from simulated radiographs representing the low and high Ψ_cnr. In total eight pores are indicated by dotted lines at radii (0.055, 0.065…0.125 mm). The intensity is approximately 20 to 30 times less than the the intensity in Figure2.

The algorithm is divided into the following three tasks: segmentation, tracking, and size measurement. The segmentation output is a set of coordinates of pore projection indications in the radiographs from the different rotations. This list is input to the tracker, which solves for the 3-D coordinates at the same time as it filters out false pores. The pores proposed as true pores are then paired up with their projection coordinates and used as input to the size measurement algorithm. It should be noted that the concept is to retrieve what is interesting, the pore indications, rather than to first make a low-noise radiograph.

2.1 Segmentation

The segmentation should output a set of 2-D pixel coordinates representing the mass centers of all pore indications in each of the radiographs. It is preferred to have a high true-positive rate, a low false-positive rate, a high precision in the mass centers, and a high uniqueness. Where a high uniqueness is defined as each pore coordinates is only listed once among the coordinates, three different segmentation methods (see Figure4) have been evaluated: a radial symmetry based, a cross correlation, and an energy weighted cross correlation. To address the uniqueness, a merger algorithm common to all methods is applied as a final step.

Figure 4

Segmentation overview. Overview of the different segmentation methods of the algorithm.

2.1.1 Radial symmetry

The idealized pore will be projected with circular symmetry in the intensity pixel values around the mass center point when the intensity is otherwise constant. Therefore, a high level of circular symmetry is expected to be a good measure of a pore projection center. In[15], a measure of local symmetry based on normalized axial moments is derived and implemented as a discrete symmetry transform (DST). The DST is calculated as given in[15, 16] according to

\begin{array}{l}\text{DST}(i,j)=1-\sqrt{\frac{{\sum }_k(T_k{(i,j)}^2)}{n_{\text{rs}}}-{\left(\frac{\sum _k{T}_k(i,j)}{n_{\text{rs}}}\right)}^2},\end{array}

(1)

\begin{array}{ll}{T}_k(i,j)& =\frac{1}{T_{\text{max}}}\sum _{(l,m)\epsilon C_r}|(i-l)sin(\frac{k\pi }{n_{\text{rs}}})\\ -(j-m)cos(\frac{k\pi }{n_{\text{rs}}}){|}^{l_{\text{rs}}}\times I(l,m).\end{array}

(2)

The sum in T _k (i,j) is summed over the indexes which are on the boundary of the pixelated circle C _r of radius r_rs with pixels centered in (i,j). In total, n_rs number of local axial moments T _k at the order l_rs are calculated and summed over with k = 0,…,n_rs-1 at each i,j. The maximum value of T _k is then used as the normalization factor T_max. The transform will look for symmetry around the axis with slope given by k π/n_rs, that is, axes in the case of n_rs > 1.

In[15], the DST is only applied at points of nonuniformity; here, the nonuniformity stage is changed to identify potential pore projection center points instead. The following differential expression is proposed to identify a symmetric peak around the second index j with possible radius ≈ r_rs:

\begin{array}{ll}E(i,j)& =[I(i,j+r_{\text{rs}})-I(i,j+r_{\text{rs}}+1)]\\ +[I(i,j-r_{\text{rs}})-I(i,j-r_{\mathit{\text{rs}}}-1)],\end{array}

(3)

where the direction of index j is perpendicular to the weld geometry. E will be large when slopes are opposite around (i,j) and, for example, identical to zero for a perfect line. As a final step, the DST matrix is convoluted with a Gaussian kernel G with standard deviation σ_rs resulting in a correlation image:

C_{\text{rs}}(i,j)=(E(i,j)\times \mathit{\text{DST}}(i,j))\star G.

(4)

This procedure is run i_rs number of times for different r_rs, with the result averaged. It should be noted that when the number of axial moments is set to 1 (n_rs = 1), the correlation image will only depend on E(i,j) since DST(i,j) = 1 for all i,j.

2.1.2 Cross correlation

If a mathematical model of the pore projection can be derived, such a model can be correlated against the image to find correlation maximums and hence the locations of the pore indications. The same linear X-ray attenuation model as in[14] is used but with the detector approximated as ideal and the X-rays approximated as parallel. The 2-D projection of the pore with radius r_cc pixels centered in c is approximated as

K(i,j)\sim e^{2\mu \sqrt{r_{\text{cc}}^2-[{(i-c)}^2+{(j-c)}^2]}}-1,

(5)

where μ is the linear attenuation per pixel for the surrounding material. Furthermore, the equation is only valid for{(i-c)}^2+{(j-c)}^2\le r_{\text{cc}}^2.

The detector model used for the synthetic radiographs will smooth the pore projection indications due to its point spread function. This smoothing will also be present in real radiographs. However, due to the idealized detector model used, this effect is not included in Equation 5. Furthermore, the buildup factor (fraction of scattered to direct radiation) is assumed to be approximately constant over the area of the projected pore. This has been shown to hold for this application in[14]. Finally, probably the least valid assumption is that the background intensity profile, in this case dominated by the weld geometry, is changing little compared to the change due to the pore. This assumption is not valid, and therefore, geometry-related intensity changes must be reduced. This reduction is done using a simple local median with radius r_me,I^{\star }(i,j)=I(i,j)-{\text{Median}}_{k,l\epsilon r_{\text{me}}}I(i\pm k,j\pm l). As a last step, the normalized cross correlation is calculated at each pixel against the model proposed in Equation 5 according to the standard equation:

\begin{array}{ll}{C}_{\text{cc}}(i,j)& =\\ \frac{{\sum }_{(l,m)}K(l,m)\times I^{\star }(i+l,j+m)}{\sqrt{{\sum }_{(l,m)}{(K(l,m)-〈K〉)}^2\times I^{\star }{(i+l,j+m)}^2}},\end{array}

(6)

where the summation is over all indexes within a square of size s_cc and 〈〉 denote average. A value close to 1 will indicate a high similarity to K.

A variation of the cross correlation method is also considered and referred to as energy weighted cross correlation. The cross correlation is scaled with the energy in the intensity according to

C_{\text{ene}}(i,j)=C_{\text{cc}}(i,j)\times \sum _{(l,m)}{I}^{\star }{(i+l,j+m)}^2,

(7)

where the indexes l,m is given by K(l,m) ≠ 0.

2.1.3 Mean shift merging

The last step of the segmentation is a merger algorithm, which intends to make the coordinates unique. The algorithm is based on the mean shift algorithm[17]. The mean shift algorithm is generally used to find high-density regions (clusters) in multidimensional spaces. Here, it is used to find the centers of pore indication clusters in 2-D.

In more detail, the merger iteratively selects the pixels in the correlation image with values larger than C _T . For each pixel above C _T , the mean shift algorithm is restarted and iterated until it has converged into a local maximum in density. At each iteration, the mass center within a radius r_ms is calculated and compared with the previous mass center, the mean shift vector. In the next iteration, this next proposed mass center is used. This is repeated until the solution is considered as having converged, with the mean shift vector changing less than δ_ms percent. The converged coordinates are incrementally averaged in bins of 1-pixel sizes. As a consequence, indications closer than ≈ r_ms will not be resolvable.

2.2 Tracking

The tracker should not only invert the 3-D to 2-D X-ray projection for the defects to solve for the 3-D physical coordinates of the defects in the weld but at the same time also handle the high false-positive rate from the segmentation algorithm. Formally, a tracker tracks a state X of a single or multiple objects by assigning measurements Y to different objects as time increases (data association). The measurement is predicted, and the state of each object is updated when a new measurement is assigned to it (filtering). The measurements are the set of pore projection coordinates in the detector plane given by the segmentation.

The filtering step consists of measurement prediction and object state vector update. It is often handled by a member of the Kalman filter family (see for example[18]). In this article, the extended Kalman filter (EKF) is used since the measurement function Y = h(X) is nonlinear. The nonlinearity is rather low, and the EKF is assumed to be sufficient. The filter state variables are the 3-D coordinates p = (p _x ,p _y ,p _z ) of each of the pores and the rotation angle ω. The mass center coordinates m of the projection of an ideal pore following the notations in Figure5 are given by

\begin{array}{l}{\mathbf{m}}_y=\frac{\text{SDD}[T_{x}sin\omega +T_{y}cos\omega +R_y-S_y]}{T_{x}cos\omega -T_{y}sin\omega +R_x}+S_y,\\ {\mathbf{m}}_z=\frac{\text{SDD}[p_z-S_z]}{T_{x}cos\omega -T_{y}sin\omega +R_x}+S_z,\\ T_x=p_x-R_x,\\ T_y=p_y-R_y,\end{array}

(8)

where m is in millimeters and is translated and scaled into the detector plane before it is used. These equations are then linearized in the EKF framework, and a standard Kalman filter approach is applied. The measurement errors are assumed to be Gaussian with a diagonal covariance matrix. The diagonal elements are denoted R for the pore indication coordinates and R _ω for the angle. The states will typically oscillate weakly, an effect which can be modeled by a nonzero state covariance matrix. However, it is set to zero since the oscillations are assumed to be negligible compared to the measurement errors. Being linearized, the EKF is sensitive to the initial state. However, in some cases an initial state close to the true state can be derived from two measurements; therefore, the state initiation is delayed until measurements from two rotations are present. Allowing rotation only around one of the euclidean base vectors, one can solve the two equations in Equation 8 for the two unknown coordinates (p _x ,p _y ). Additionally, p _z is approximately known from the rotation axis limitations. The state initiation is accepted if G_{x 0} < p _x  < G_{x 1} and otherwise rejected. The gates G_{x 0} and G_{x 1} are approximated from the setup geometry. The initial state could also be derived using the formalism of vision systems, relaxing the demand on rotation axis, though this approach has not been takenhere.

Figure 5

Setup geometry. The pore mass center with position vector p is rotated with an angle ω around the z-axis with respect to the rotation point (R _x ,R _y ,0). The pore is projected on the detector plane at m = (SDD,m _y ,m _z ) (see also Figure1).

One data association method suitable for the situation is the multiple hypothesis tracker (MHT) originally presented in[19]. It is suitable when the number of objects to track is not known a priori and/or when the clutter density is high. The implementation derived is very similar to the ‘Structured Branching’ MHT as described in[20]. The assumptions made are that each object generates at most one measurement but each measurement can originate from many different objects. The last assumption accepts overlapping pore projections in some of therotations.

In the nomenclature of tracking, as the object changes its state in some partly known way, the measurements belonging to it forms a track and the track can be used to solve for the state. The main idea of the MHT is that it computes many possible tracks from different combinations of measurements and selects only the most probable. The different combinations can be kept in a tree, with the path from each leaf to the root representing one solution. The probability for each track (object) to be a true positive is, if it is not properly normalized, referred to as its score. The score is constructed by the summation over all nodes, one at each rotation k, for each track as log likelihoods according to

\begin{array}{l}{\text{score}}_k={\text{score}}_{k-1}+\mathrm{\Delta }\text{score},\end{array}

(9)

\begin{array}{l}\mathrm{\Delta }\text{score}=-d_0^2,\end{array}

(10)

\begin{array}{l}\mathrm{\Delta }\text{score}=S_{\text{mht}}\times ln\left[1/\sqrt{det(\text{RS})}\right]-d^2,\end{array}

(11)

where Equation 10 refers to the case of no measurement (the pore indication is not detected in the radiograph) and Equation 11 refers to when a measurement is present. RS is the measurement residual covariance matrix, which increases with R, R _ω , and the covariance matrix calculated in the Kalman filter step. S_mht is a scaling parameter to weight the two terms. Further, d² is a normalized statistical distance indicating how far away the measurement is from the prediction with respect to the uncertainty indicated by RS. See[13] for an in-depth discussion on the constants and the scores.

In the last paragraphs of this subsection, (row X) will be used to refer to row number X in the following pseudocode of the implementation:

Adding new nodes to the tree is done with great care for the tree not to grow too large. The gates G _z and G _y are applied on the two detector coordinates (row 5). If the difference between the previous measurement and the new one is within the gates, a new node for the measurement is added (row 6). There is always a node added for the no measurement case (row 7). The no measurement case represents the hypothesis that the measurement was not detected in the current rotation.

Apart from the gating, a pruning of the tree is also conducted. At the pruning (row 9), the leaves with 〈score〉 = score _k /k ≥ T₁ are accepted and kept while the rest are rejected and pruned. In addition, a maximum limit of five leaves per root is applied, where the leaves with the highest 〈score〉 are kept. These two constants are chosen to yield reasonable run times. The pruning is skipped for those trees which have less than three rotations since two rotations are required for deriving the initial state alone.

At the first rotation, all measurements lead to new track roots, but at later rotations, only the measurements which are far away from any current track prediction are added (row 10). The distance to other track threshold is given byd_{min}^2>d_{\text{upd}}^2, where d_min is the minimum distance to any track prediction for the measurement at the current rotation. This is done to assure that those measurements which are likely not to belong to any existing track are identified as new roots.

At the decide stage (row 11), the single most probable track (pore) from each tree root is selected and either accepted or rejected. The most probable track is the leaf with the highest 〈score〉. In addition, the following conditions must be met in order for the selected tracks to be accepted: 〈score〉 ≥ T₂, G_{x 0} < p _x  < G_{x 1}, and it should have less than i_lost number of no measurement nodes.

As a final step (row 12), the tracks (pore proposals) which have states close to each other are combined. This combine step is required since the creation of new tracks might lead to multiple pore proposals originating from the same physical pore. The distances between all pair of solutions are verified not to be closer than δ _u (in millimeters); if so, the highest 〈score〉 solution is chosen. This is iterated until there are no solutions closer than δ _u (in millimeters). The solution will, in most cases, not be unique but is believed to represent reality just as good as any of the other solution. As a consequence, δ _u becomes a resolution limiting parameter.

2.3 Size measurement

Two possible approaches to measure the size of a pore is either by its projection spatial size or intensity. A projection spatial size approach is assumed to have lower resolution in size than an intensity based for pore sizes at the order of the detector pixel size. Therefore, an intensity-based approach is used, where a scalar depending on the intensity is constructed to correlate with the pore size. Different scalar proposals are constructed in three steps. Firstly, the natural logarithm is optionally applied to the radiograph. Secondly, the weld geometry background (see Figure2) is removed by subtracting the local median of size r_me from the original radiograph as

I^{\star }(i,j)=I(i,j)-{\text{Median}}_{k,l\le r_{\text{me}}}I(i\pm k,j\pm l).

(12)

Thirdly, a sum U is constructed for the pore indication centered at (i₀,j₀) according to

U=\sum _{i=i_0,j=j_0\pm r_{\mathit{\text{siz}}}}F[I^{\star }(i,j)],

(13)

where the index j is along the direction perpendicular to the weld direction. Three functions F are evaluated: linear F = I^⋆, absolute value F = |I^⋆|, and square F = (I^⋆)². The sum U is then averaged over the radiographs at different rotations. The proposed method will yield relative pore sizes; to get real pore sizes, a calibration step is required, which is not elaborated on in this article.

3 Setup and performance evaluation with simulations

As has been shown in the previous sections, there is a set of algorithm control parameters, which need to be set for the algorithm to work. An iterative DOE approach in which parameters are screened, typically not full factor experimental matrices, and frozen out if not important for the calculated performance response has been used. The algorithm is made self-evaluating (see Figure6) by simulating the inspection. The inspection simulation output, a rotational sequence of simulated radiographs, is input to the algorithm, and its performance is evaluated for a number of such simulated inspections.

Figure 6

Algorithm evaluation overview. Overview of the evaluation of the whole algorithm.

A robust design framework[21] is used where the factors affecting the algorithm are separated into noise and control factors. A noise factor consists of two sets of min and max values for a list of radiograph sequence simulation control factors. Each set of min and max values represents one level of the noise factor. As can be seen in Table1, three different noise factors at two levels (NF low and high) are used: Ψ_cnr, Ψ_geo, and Ψ_det. The Ψ_cnr represents the inherent statistical nature of radiographic inspection. It consists of the two main CNR controlling parameters, D and S, in the X-ray system model. They control the ratio of high to low spatial frequency noise in the radiographs and the overall noise level. The Ψ_geo represents process variation in the weld geometry. The high level reproduces a variation similar to the weld class with medium geometry requirements. Finally, the Ψ_det consists of the width of the detector point spread function, with Lorentzian shape. The X-ray detector is not further specified apart from its pixel size, which is set to 0.05 mm. For a detailed discussion on the noise parameters and the X-ray model together with its restricted validity range, see[14, 22]. The performance evaluation should be interpreted as a performance indication due to the rather unspecified X-ray setup, with its advantage of a potentially high degree of generalization, and the simplified synthetic radiograph model.

Table 1 Radiograph simulation parameters and noise factors

The simulation of a single inspection of a single weld with pores consists of a sequence of radiographs taken at different rotations. The sequence is denoted as rscan for rotational scan. The rotation angles, amplitude and count, as given in Table1 are chosen for the inspection procedure to be industrially feasible and without any other optimization. A large amount of rscans is then pre-generated and grouped together in sets having the same noise factor settings. These groups will have different amount of inherent variation. The control factors belonging to a noise factor all have their values set to random numbers. The random numbers are drawn from uniform distributions with ranges set to either NF low or NF high as indicated in Table1. For example, Ψ_geo equals H indicate that for those rscans the width parameter (B) on the source side has been uniformly random-sampled within 2.6 to 3.4 mm and so on. Finally, for each rscan, a unique set of pore positions and sizes are randomly sampled from a uniform distribution (see Table1). The position distribution is limited by a minimum allowed interspacing between the pores.

The algorithm control parameters are varied in an experiment matrix followed by a performance measurement. For each matrix row, the algorithm is run for a small number of different rscans to get some variation in the input. This is essential since the main difficulty is to handle the variation in the input, not a limited number of special cases.

The setup of the algorithm control parameters is separated and suboptimized with the segmentation and the tracker screened one at a time. Two responses are used to set up and compare the different segmentation methods. The overall aim is to find as many pores as possible but even more so to know that no pores larger than some defined size is missed. In addition, the tracker will be able to handle some missing measurements for each pore (i_lost). One might argue that a good measure of the segmentation performance is the averaged worst case scenario. Therefore, the rotational probability of a hit is used as one of the responses and defined as

{\text{POH}}_{\text{rot}}\equiv {〈min({\text{POH}}_p)〉}_{\text{rscans}},

(14)

where POH _p  = TP/N with TP being the true-positive count for pore p over all the N rotations, and POH_rot × N is the number of rotation radiographs a pore is detected in. Another relevant response is the measure of the cost of detecting the TPs in terms of the number of false positives per rotation FP_rot ≡ 〈〈FP _r 〉_rotations〉_rscans, where FP _r is the total false-positive count for each rotation when TP is maximized.

For both the above responses, each pixel output by the merger procedure, which is the last step in the segmentation algorithm, must be classified as either a TP or FP. The merger procedure is set to retrieve at most C _n number of the highest valued pixels from the correlation image, referred to as candidate pores. In addition, the ground truth is known since the expected pixel coordinates for each pore can be calculated using Equation 8. To associate each true pore projection with one candidate, a simple nearest neighbor approach with no re-sampling is used. First, the real euclidian distances between each true known pore and all C _n candidates are calculated. This is followed by iteratively selecting the pair with the smallest interspacing distance until all p true pores are associated with one-pore candidates. The associated pairs might not be unique, but the method is simple and fast and is believed to be a good measure on the average. Finally, to be considered a TP, the distance to the nearest true pore should be less than 1.2 pixel; the threshold is chosen to be slightly larger than 1 pixel.

The tracker is set up and evaluated by setting the last threshold T₂ high enough to ensure that all candidates that made it through the tree pruning are present. Each pore candidate consists of its real 3-D coordinates, which are compared with the known exact positions using the same nearest neighbor approach as in the segmentation evaluation. The candidate is then considered a TP if the distance to the closest true pore is less than a predefined threshold (0.25 mm). If at this stage all pores are found (TP = P), a measure of the relative performance of the settings used in the algorithm is the separation of the scores for the TP and FP classifications. The concept of sensitivity, a measure of separation, is introduced in this context as

\text{Sensitivity}=\frac{{〈\text{score}〉}_{\text{TP}}-{〈\text{score}〉}_{\text{FP}}}{\sqrt{s_{\text{TP}}^2+s_{\text{FP}}^2}},

(15)

where the averages are over all pores in all rscans at the same time, and the s denotes the standard deviation of each of the groups. This is equivalent to the statistic used in common statistical hypothesis testing; note also its similarity to the Taguchi signal to noise ratio (see for example[21]). Two critical assumptions made are that the standard deviation is a good measure of the relative spread in the data and that the score distribution shape is approximately constant. One possible cost for finding a lot of the pores and maximizing sensitivity is the precision in their 3-D positions. Therefore, the error in positioning is added as a response. The error is defined as the distance between the true 3-D position (input to the simulation) and the one retrieved as a result from the tracker. The spread in the errors is then accounted for when constructing the scalar response by approximating and setting it to the 97th sample quantile[23] of the error in position using all the TPs.

Both the decision on TP and FP at the segmentation setup and the state model in the Kalman filter is based on a simplified but very similar X-ray interaction model as the model used for synthetic radiographs. Therefore, both the segmentation and tracker evaluation could potentially contain an inverse crime[24], but they do not since statistical noise is added to the radiographs.

3.1 Probability of a hit

A common method to quantify the performance of an NDE system is to use the methodology of probability of detection (POD). The standard POD against defect size plot is constructed by keeping as much as possible constant while changing the characteristic size of the defect. In order to get a high confidence in the POD, it is necessary to have many well-defined defects with the same size. However, to produce well-defined defects is both expensive and time consuming; the same is true for simulations, which can be computationally expensive. Different schemes to handle this have been developed; for example,[25, 26] is frequently used within the aero industry. Assumptions are made on the POD curve shape and underlying statistical distributions in order to facilitate an effective approximation of the confidence intervals, assumptions which are unrealistic for this automatic NDE system. Others[27] try to approximate the confidence intervals with the use of sophisticated DOE methods combined with less strict assumptions on the NDE system. However, in this article, the POD and the 95% lower confidence interval (POD95) methods of[28] is used.

In[28], the hit/miss formulation is used, where each defect is either detected (hit) or not detected (miss). As already noted in the previous section, in the present work, a hit is defined as detecting and 3-D positioning of a pore to an accuracy of 0.25 mm. Anything else is a miss or a false positive. This threshold is set to less than the minimum pore interspacing (see Table1) but large enough to be a relevant length scale. The probability of a hit is used and defined as POH = TP/P, and used to discriminate it from the usual meaning of the POD. Assuming that each pore has the same POH as all the other pores in the same range interval, the binomial distribution can be used as in[28]. The main advantage in using the binomial approximation is that the confidence intervals can be estimated from TP and P using standard methods and implementations.

In order to get a high confidence in the POH using few defects, the optimized probability method[28] has been used. The method is outlined here for completeness (see[28] for details). Iteratively, for one radius range at a time, a set of POHs at a lower confidence level of 95% (POH95) is calculated by including one lower radius interval at a time into the calculation. For each radius range, the POH95 is set to the largest of these POH95s. A critical assumption is that the true detectability increases with increased defect size. Curves produced using the optimized method will in general extend the high POH region into lower sizes but at the same time be conservative.

The false detection rate FDR = FP/(FP + TP) is used as a measure of false alarms. This is believed to be relevant since the pore distribution output from the tracker and in the future input to the fatigue life model will consist of FP + TP number of pores.

3.2 Size measurement

A robust scalar measure of the error in size measurement is used to compare different size measurement methods. The ground truth for the size of each pore is known together with the projection coordinates in each radiograph output from the tracker algorithm. The error measure is constructed by first calculating the U as given in Equation 13 for each pore in each rotation in all rscans. Pairs of a known radius together with a calculated U is produced. The radius is discretized into a number of radius intervals [r - Δr,r], where Δr is a parameter indicating the required resolution. For each radius interval, the range of U is calculated [U₀,U₁]. This is followed by selecting the set of all radius R having U₀ ≤ U ≤ U₁, in order to calculate the new radius interval range as R₊ = max(R) - r and R_- = min(R) - r. Finally, the scalar measure of the relative performance E = E₊ + E_- is calculated by summing over N radius intervals r _i according to

\begin{array}{l}{E}_k=\frac{1}{N\mathrm{\Delta }r}\sum _{r_i=1}^N|R_k|,\end{array}

(16)

where k = ±. A high performance is indicated by E being close to 1. It should be noted that the method makes no assumptions on the shape of the correlation between U and the radius and also accounts for the outliers.

4 Simulation results

This section is both an example of the setup and evaluation methodology presented in the previous section and a description of the simulation results. In the setup and evaluation, the algorithm is divided into three parts and performed in the following order: the segmentation, the tracker, and the size measurement. Each part is optimized and evaluated with all the other parts of the algorithm fixed at optimum performance.

4.1 Segmentation

All three segmentation methods are individually optimized in order to select the one with the highest performance. For each segmentation method and for each control parameter row in the experimental matrix (configuration), all four combinations of the (L) and (H) of the noise factors Ψ_geo and Ψ_cnr are evaluated. The best result for each method and noise factor combination is presented in Table2, where the best result is defined by maximizing POH_rot, followed by minimizing FP_rot if there is more than one configuration with the maximum POH_rot. The best configuration chosen this way will typically be different for each of the four different noise factor combinations. The main conclusion from Table2 is that the cross correlation with energy term method have the lowest performance (POH_rot × 6 = 2.8 at Ψ_geo = H,Ψ_cnr = L), and it is therefore excluded from further considerations.

Table 2 Segmentation methods screening

The definition of the best configuration is then extended to include the rule that the best configuration for each segmentation method should be the same for all noise factor combinations. As indicated in Table3, the radial symmetry method still performs better in all noise factor combinations than the correlation method. Common control parameters to both methods are the merger settings r_ms = 2.8 px and δ_ms = 2%. The best configuration for the cross correlation method is r_me = 4 px, s_cc = 5 px, and r_cc = 0.1 mm. The best radial symmetry method configuration is r_rs = 1 px and 2 px, i_rs = 2, n_rs = 1, l_rs = 1, and σ_rs = 1.55 px. As can be seen, in the optimal radial symmetry method configuration, the DST is not active but only the symmetry differential operator E.

Table 3 Segmentations methods with constant control parameters

A more detailed performance evaluation is conducted on the radial symmetry method, which was the best segmentation method. In Table4, the effect on POH_rot and FP_rot when extending the pore radius range down to 0.055 mm and changing the Ψ_cnr levels can be seen. The CNR for the 0.055 mm pore is roughly CNR ≈ 1 ± 1 for Ψ_cnr = L^′ and CNR ≈ 7 ± 1 for Ψ_cnr = H^′. The segmentation is evaluated using the same experimental matrix as used in Table3, and the optimum configuration is verified to be the same. Furthermore, the conclusion is that for the current radius range, the change in POH_rot is mainly due to the change in CNR and the noise spatial frequency characteristics (different Ψ_cnr), rather than the change in spatial size of the 2-D indication.

Table 4 Radial symmetry segmentation at different CNR

4.2 Tracking

The tracker part of the algorithm is optimized and evaluated with the segmentation control parameters fixed. The two optimized responses are the sensitivity and the 97th quantile of the positioning error. First, an iterative approach using DOE screening and space filling experimental matrices is conducted. The number of rows in the experimental matrix (configurations) for sets 1, 2, and 3 in Table5 are 15, 40, and 60. Each configuration is evaluated on eight rscans. Since C _n is held fixed and POH monitored to be close to 1, the clutter density can be considered constant. The gates are also fixed at G _z  = 4 mm, G _y  = 30 mm, G_{x 0} = 565 mm, and G_{x 1} = 579 mm. In addition, at most, two measurements are accepted to be lost (i_lost = 4). The results of the iterative process can be seen in Table5. As can be seen in Figure7 in combination with Table5, R can be frozen out and S_mht is then used to maximize the sensitivity.

Table 5 Tracker DOE iterations

Figure 7

Tracker parameters R and S _mht . Correlations found in set 3 as the result of sets 1 and 2 in Table5. Each control factor or their combination is normalized by translation and scaling.

In order to explore which noise factor with the highest effect on the sensitivity, the tracker is evaluated with the noise factors set to difficult one at a time and once for all of them. For each noise factor combination, the tracker is evaluated using two different levels of S_mht and C _T . As can be seen in Figure8, it is the CNR (Ψ_cnr) that has the largest effect on the sensitivity, not the width of the detector point spread function (Ψ_det) nor the process variation in the weld geometries (Ψ_geo).

Figure 8

Sensitivity for different noise factors. Sensitivity is shown for different combinations of noise factors and two levels on S_mht and C _T .

In order to set a value on S_mht, a full factor experiment on S_mht and C _T is executed on a set of rscans with the most difficult noise factors combined (Ψ_cnr = L, Ψ_geo = H, Ψ_det = H). As can be seen in Figure9, approximately 5 is a good value on S_mht. Furthermore, the 97th percentile of the error in positioning is within 0.131 to 0.138 mm.

Figure 9

Sensitivity against S _mht at different C _T . Sensitivity as the whole algorithm is swept for different C _T and S_mht. The data is based on Ψ_cnr = L, Ψ_geo = H, and Ψ_det = H and each point in the plot is based on 20 rscans.

As a final setup and optimization step, the effect of different minimum radii in the pore radius distribution on the performance is explored. Three sets of rscans with different minimum pore radii 0.055, 0.085, and 0.1 mm (maximum radius constant at 0.13 mm) are generated. The Ψ_cnr settings are changed to S = 24,000 and D = 160. The results based on four rscans can be seen in Figure10. The higher the sensitivity, the less sensitive to T₂ (robust) are the POH and FDR and the lower the FDR for a given POH is expected to be. The effect can be seen in the figure as the different behaviors of the two innermost lines (different S_mht) where the T₂ values of the dashed line has been translated and re-scaled for visual comparison.

Figure 10

POH and FDR against T ₂ . Radius refers to minimum radius. Only the FDR for the smallest radius is shown; the remaining ones are similar.

For the purpose of performance evaluation, two new sets of rscans are generated with minimum pore radius 0.055 mm and nonstandard high and low Ψ_cnr given by S/D = 50,000/270 and S/D = 30,000/200. The algorithm configuration is the same in both cases with S_mht = 5, C _T  = 1.87, and constant T₂. The resulting POH95 plotted against pore radius is given in Figure11, where the Ψ_cnr = H is based on 50 rscans and Ψ_cnr = L is based on 200 rscans. As can be seen, the POH95 is higher for the Ψ_cnr = H compared to Ψ_cnr = L. Furthermore, the convergence of the POH for two radius ranges from Figure11 as can be seen in Figure12. Convergence is considered as being reached in both cases since other uncertainties due to the simplified synthetic radiograph model and the unspecified general X-ray setup are assumed to be higher.

Figure 11

POH95 against radius. POH95 is calculated using the optimized probability method[28]. The radius intervals are approximately 0.005 mm and the radius taken as the max radius in the interval. The FDR is less than 1%.

Figure 12

Convergence of POH. POH against number of rscans for two different radius ranges (R). The range is given as the average radius over the rscan subset and the data is the same as in Figure11.

Finally, the performance in terms of the error in positioning is evaluated. As can be seen in Figure13, with a q-plot of the error, it is less than 0.05 mm for 80% of the pores in the Ψ_cnr = L case. In addition, out of the 358 pores which were found in Figure13, 10 were missing one measurement and 3 were missing two measurements.

Figure 13

Error in positioning. Quantile plot for error in positioning for the radius = 0.055 to 0.13 mm case. The graph is based on 50 rscans of the Ψ_cnr = L case in Figure11.

4.3 Size measurement

The results for the optimization and selection of the optimum size measurement method are presented in Figure14. The best configuration is chosen to be F = I², pre-logarithm on image, r _siz  = 2 px, and r_me = 10 px. In addition, the performance is evaluated in terms of resolution in radius. The resolution is indicated in Figure15, which shows the real true radius (known) plotted against the measured radius. However, the X-ray model used does not include scattered radiation, which is assumed to lower this sizing resolution even more.

Figure 14

Optimization of the sizing method. Optimization of error E₊ in Equation 16 on the sizing procedure; data is based on 12 rscans containing pores of radius 0.055 to 0.13 mm and using the Ψ_cnr = L settings used in Figure11. The radius intervals are based on Δr = 0.0075 mm.

Figure 15

Sizing error. Real radius ranges for each radius r representing the range [r-0.0075 mm,r], selected using the same algorithm as when calculating R₊ and R_-. Function F = (I^⋆)², r_me = 10 px, and I^⋆ = lnI are applied on the radiograph. Datasets are the same as in Figure11.

5 Experimental setup

The experimental radiographic setup is the same as already illustrated in Figures1 and5. It is a conventional radiographic setup with a motorized rotational stage. The X-ray source is a General Electric ISOVOLT 450M2 (Fairfield, CT, USA) with a focus size of 0.4 mm. The radiographic setup parameters are held fixed at a tube voltage of 120 kV, current of 5.8 mA, and exposure time of 29 s.

The detector is a high-resolution (pixel size 0.0135 mm) digital detector optimized for high-energy (450 keV) applications (see[29] for details). It is an indirect detector with a scintillator connected to a CCD camera. The scintillator-camera connection is a bent fiber optic bundle. The bundle introduces detector plane spatial distortions with smooth spatial dependence; however, no corrections are applied.

Four pre-processing steps are applied to the raw radiographs. First, the dark-field radiograph, which is the zero exposure detector intensity level, is subtracted from the radiograph. Second, a flat-field correction is applied, in which each pixel is divided by its value at exposure of a homogeneous similar body as the one to inspect and multiplied by the average flat-field value. The flat-field correction is necessary since each pixel responds slightly individual to radiation.

Some pixels respond to radiation which is so different compared to the rest that it is required to classify them as noninformation-bearing pixels, the bad pixels. The algorithm chosen for bad pixel handling is that of[30] with some minor changes. More sophisticated algorithms have been proposed (see for example[31]), but in this article, a simple bad pixel handling is assumed to be sufficient. Each pixel in the flat field is compared to the median of its neighbors (15 × 15 px kernel). If the absolute deviation from the median is larger than 2.9 standard deviations, it is classified as a bad pixel. Typically, around 3% are classified as bad (fixed bad pixel map); this is expected and is caused by the special kind of scintillator used. As a third pre-processing step, each pixel labeled as bad is then substituted in the current radiograph by the median value of its neighbors (5×5 px kernel). Often, there are clusters of bad pixels, and of course, any indication at the same size and inside such a cluster would be averaged away. In the fourth and final pre-processing step, the pixels are averaged 2 × 2 into pixel size 0.027 mm (binned) to increase the CNR at the cost of resolution.

The aim of this work is not to find the most optimal radiographic setup and settings. However, a higher CNR (for example, due to a microfocus X-ray source, different X-ray energy or exposure time) was indicated (see Section 4) to increase the algorithm performance. Therefore, an experiment on both a high and a low CNR, labeled CNR + and CNR -, is conducted. In the CNR + case, the average over eight exposures at each rotation angle is used; the average is taken before the first pre-processing step. In the CNR - case, four exposures are instead averaged.

The reference measurement of the sample is made with a commercial CT metrology system, Carl Zeiss Metrotom 800 (Oberkochen, Germany). The sample is a titanium alloy laser weld (Ti-6242, Ti₆Al₂Sn₄Zr₂Mo), which is cut out from a large plate into a 5 × 10 × 30 mm volume. The sample volume is reconstructed with a voxel size (discretized volume size) of 0.018 mm. For large objects, consisting of more than a few voxels, the accuracy and precision in their mass center 3-D position is typically subvoxel. However, for the small objects (the pores) in this article, precision and accuracy are assumed instead to be at the order of the voxel size[32].

6 Experimental results

The radiographic image quality is indicated using a DIN62AL 10 ISO 16 (INTECH NDE, Edmonton, Alberta, Canada) wire penetrameter. The quality is shown in Figure16 with the penetrameter on the source side of a homogeneous Ti-6242 plate. A faint linear indication of the smallest wire (0.1 mm) can be seen in the CNR + case and the second smallest wire (0.125 mm) in the CNR - case. The detectability of this penetrameter does not indicate the detectability of the pores’ 1:1 ratio. However, the overall quality of the radiograph is visualized.

Figure 16

Image quality of the radiographs. The DIN62AL 10 ISO 16 wire penetrameter on an ≈ 4.87-mm-thick Ti-6242 plate. Both CNR + and CNR - are shown, and the radiographs are cropped to approximately 150 × 220 pixels.

A single rotation radiograph for both CNR cases of the actual weld is shown in Figure17. The two boxes indicate the selected region of interest, which is held constant during the analysis at approximately 900 × 240 pixel. Horizontal line profiles over three different pore indications from Figure17 is shown in Figure18. The CNR is approximately around 2.

Figure 17

Experimental radiographs. Experimental radiographs of the weld for both CNR cases. The region of interest is also indicated by the boxes.

Figure 18

Pore indication line profiles. Three typical line profiles over the pore indications in Figure17 along the horizontal direction.

The setup geometry as indicated in Figures1 and5 is derived using the same procedure, settings, and calibration sample as in[33]. In short, a block of plastic with five ball-bearing steel balls (diameter 0.5 mm) cast inside at different depths and y-z positions is measured with a CT metrology system. The sample is then inspected with the same radiographic setup as will later be used for the algorithm. The steel balls are detected, and an evolutionary algorithm is used to derive a setup geometry parameter set from the known steel ball interspacing positions in 3-D. For details, see[33]. A setup geometry was derived and verified to give approximately 0.1-mm maximum positioning errors. The error was approximated by rerunning the analysis discarding the prior knowledge of the steel ball 3-D positions. Referring back to Figure5, the derived setup geometry parameters used are R _x  = 649.9421 mm, R _y  = 8.4612 mm, R _z  = 12.9290 mm, S _x  = 0.0000 mm, S _y  = 10.8245 mm, S _z  = 13.3204 mm, and S D D = 717.5830 mm. The object to detector distance ODD≈25 mm; it is however not required by the algorithm and therefore only manually approximated.

The algorithm parameters are held fixed for both the CNR cases. The radial symmetry method is used, and its parameters are the same as the ones used to produce Table4. The merger parameters are also the same as those used in Table4 except for the number of pore indications to retrieve in each radiograph, which is C _T  = 600. Most of the tracker parameter values are the same as the frozen ones in Table5, except for the final score (probability) threshold T₂ = 1.8, the intermediate score threshold T₁ = -1,000, the score when no measurement is foundd_0^2=70, the measurement covariance R = 3.5 × 10^-5 mm², the threshold for merging 3-D positions δ _u  = 0.08 mm, and the weight when comparing prediction to measured indication S_mht = 3. The δ _u had to be lowered since the pore interspaces in the real samples were lower than assumed.

The nominal thickness of the plate is measured with a mechanical caliper and compared to both CT and the algorithm in this article (abbreviated DR). For both the CT and DR cases, the thickness is measured using three points on one side of the surface to construct a plane; the perpendicular distance to a single point on the other side of the plate is then taken as the thickness. For the DR case, the points consisted of 0.5-mm high-tolerance ball-bearing steel balls, which the algorithm was set up to detect and position. The algorithm parameter values were the same as the for the calibration procedure described in[33]. The result of the three different thickness measurements are 4.86 ± 0.02 mm for caliper, 4.93 mm for CT, and 4.73 mm for DR. This indicates an overall magnitude of the error in positioning.

The pore reference sample (ground truth) is created by manual pore detection in the CT visualization program, Volume Graphics. Additionally, the diameter and 3-D position of each detected pore are also measured manually in the visualization program. The DR pore 3-D position coordinate system is then manually translated and rotated to match the six largest pores from the CT measurement, a procedure known as registration. The positioning errors between the two point sets (DR and CT) are found to be approximately ≤0.1 mm, which is consistent with the validation of the derived setup geometry above. Furthermore, the pore radius range in the reference sample is found to be 0.03 to 0.065 mm, where the smaller radius range POD is assumed to be less than 1.

In Figures19,20,21,22, the two DR cases, CNR + and CNR -, are compared to the CT reference measurement. The pore sizes are scaled with an arbitrary number, which is different for CT and DR. In the DR case, the size of each each pore is derived using the same sizing control parameters as used in Figure15 before scaling. Note that the sizes of the pores and aspect ratios between the axes are scaled for visualization, not to reproduce correct scaling. The pore 3-D positions measured with the proposed algorithm (DR) agree qualitatively well with the CT reference.

Figure 19

Pore depth ( x ) against weld length ( z ) position. The depth coordinate x against weld length coordinate z of the pores for DR CNR - and CNR + and CT at two different lower radius thresholds. The weld root enforcement height is δ₀ ≈ 0.5 mm (caliper), and it is located on the detector side at high high x coordinates. Dashed lines indicate the base plate surfaces.

Figure 20

Pore lateral ( y ) against weld length ( z ) position. The pore lateral coordinate y against weld length coordinate z for DR CNR - and CNR + and CT at two different lower radius thresholds.

Figure 21

Pore depth ( x ) against weld length ( z ) position at different CT radius thresholds. The pore depth coordinate x against weld length coordinate z for both DR CNR + and CT. The radius (r) represents two different lower thresholds in the CT results.

Figure 22

Pore lateral ( y ) against weld length ( z ) position at different CT radius thresholds. The pore lateral coordinate y against weld length coordinate z for both DR CNR + and CT. The radius (r) represents two different lower thresholds in the CT results.

A qualitative POH measurement is indicated in Table6. Each DR pore is manually matched to one of the CT reference pores and classified as detected, or if no CT reference pore is close enough (see Figures19,20,21,22), it is classified as a false positive. The information in Table6 should be seen as a rough estimate of where the POH starts to decrease, in this case, at the diameter of approximately 0.1 mm. In addition, as expected, the POH is higher for the CNR + than for the CNR - case. As for the false positives, it can be seen in Figures21 and22 that many pores can be mistaken for false positives unless smaller pores are included in the reference measurement. The number of false positives are low when the small pores (r ≥ 0.03 mm) are included. The two pores in the DR cases at x,y,z ≈ 2,11,18 mm can be seen as grooves on the surface of the sample. This adds up to in total one false positive in the CNR + case at (x,y,z) ≈ (2.5,11,18) mm. Finally, among the pores found in Table6, three missed one measurement in the CNR + case and two in the CNR - cases. The conclusion is that the algorithm can detect pores without requiring them to be detected in all projections.

Table 6 Detected pores for both DR and CT

7 Conclusions

In this article, an algorithm has been derived to handle the detection, positioning, and sizing of submillimeter-sized pore defects in thin laser-welded titanium inspected with radiography. The algorithm has been set up and pre-evaluated on synthetic radiographs using a DOE methodology. In addition, it has been qualitatively evaluated on real experimental welds. The algorithm is based on general tracking theory, in contrast to the previous solutions in literature, which are based on a vision system approach. In addition, it does not require the defects to be detected in all rotation projections. This relaxed demand on detection in all projections is important for detecting low CNR defects with radiographic inspection.

The qualitative experimental comparison shows good agreement between the 3-D positions found using the proposed algorithm and the computerized tomography reference measurements. In addition, performance evaluation on both the synthetic and the experimental radiographs indicate that the probability of a hit increases with CNR; hence, as the hardware performance used for inspection will improve in the future, the performance of the algorithm will too. The synthetic and the experimental radiographs both indicate a low false-defect detection rate. However, the defect size measurement part of the algorithm could not be experimentally verified due to very low resolution in the reference measurements.

In the future, a quantitative experimental benchmark of the proposed algorithm and its inspection procedure needs to be conducted. Furthermore, in order to detect and position the defects which are not detected in all projections more efficiently, other trackers or modifications to the multiple hypothesis tracker used here could be interesting to explore.