Skip to main content

Recent multiscale AM-FM methods in emerging applications in medical imaging


Amplitude-modulation frequency-modulation (AM-FM) decompositions represent images using spatially-varying sinusoidal waves and their spatially-varying amplitudes. AM-FM decompositions use different scales and bandpass filters to extract the wide range of instantaneous frequencies and instantaneous amplitude components that may be present in an image. In the past few years, as the understanding of its theory advanced, AM-FM decompositions have been applied in a series of medical imaging problems ranging from ultrasound to retinal image analysis, yielding excellent results. This article summarizes the theory of AM-FM decompositions and related medical imaging applications.

1. Introduction

In the field of computer aided detection and diagnostics (CAD), recent advances in image processing techniques have brought a wide array of applications into the field. Many existing CAD methods rely on fixed basis functions based on wavelet decompositions [1] and Gabor filters [2]. Amplitude-Modulation Frequency-Modulation (AM-FM) methods [36] represent an emerging technique that shows great promise in this area.

Multidimensional AM-FM models and methods provide us with powerful, image and video decompositions that can effectively describe non-stationary content. They represent an extension to standard Fourier analysis, allowing both the amplitude and the phase functions to vary spatially over the support of the image, following changes in local texture and brightness.

To explain some of the advantages of AM-FM methods, we begin with the basic AM-FM model. In the 2D model, we expand an input image I(x, y) into a sum of AM-FM harmonics using:

I ( x , y ) = n = 1 M a n ( x , y ) cos φ n ( x , y ) ,

where a n (x, y) denote slowly-varying instantaneous amplitude (IA) functions, φ n (x, y) denote the instantaneous phase (IP) components, and n = 1, 2,..., M indexes the different AM-FM harmonics. In (1), the n th AM-FM harmonic is represented by a n (x, y) cos φ n (x, y).

With each phase function, the instantaneous frequency (IF) vector field is defined by φ n ( x , y ) . Here, the AM-FM demodulation problem is defined as one of determining the IA, IP, and IF functions for any given input image.

AM-FM decompositions provide physically meaningful texture measurements. Significant texture variations are captured in the frequency components. For single component cases, IF vectors are orthogonal to equi-intensity lines of an image, while the IF magnitude provides a measure of local frequency content. In (1), by using AM-FM components from different scales, we can produce IF vectors from different scales, at a pixel-level resolution [4, 5].

Since AM-FM texture features are provided at a pixel-level resolution, AM-FM models can be used to segment texture images that are difficult to model with the standard brightness-based methods [7]. On the other hand, using just histograms of the IF and IA, we can design effective content-based image retrieval systems using very short image feature vectors [8, 9].

In summary, the advantages of AM-FM methods include [10]: (i) they provide a large number of physically meaningful texture features, over multiple scales, at a pixel-level resolution, (ii) the image can be reconstructed from the AM-FM decompositions, (iii) based on the target application, different AM-FM decompositions using different frequency coverage can be designed, and (iv) very robust methods for AM-FM demodulation have been recently developed (see some recent examples in [4]).

AM-FM decompositions can be extended to represent videos using:

I ( x , y , t ) = n = 1 M a n ( x , y , t ) cos φ n ( x , y , t ) ,

where each AM-FM function has been extended to be a function of both space and time. The original phase-based modeling approach was provided in [11] and was recently extended and improved in [12, 13].

AM-FM decompositions can also be used to reconstruct the input images, allowing us to evaluate their effectiveness on different parts of the image. For continuous-space image decompositions, AM-FM reconstruction examples can be found in [14], while [4, 5] give several recent, robust multiscale examples for both images and videos. AM-FM transform examples were shown in [15], while multidimensional orthogonal FM transforms were demonstrated in [16].

An early example of the use of frequency-domain filtering to target a particular application can be found in the fingerprint examples in [17]. More recently, [18] provided a tree-growth application, where inter-ring spacing was used to design filterbanks that cover a specific part of the spectrum, so as to recover tree ring and tree growth structure from very noisy image inputs. For general images, Gabor filterbank approaches were investigated in [3]. Similarly, for general images and videos, dyadic, multiscale decompositions were introduced in [4].

A summary of AM-FM methods is provided in Section 2. Results are presented for several medical imaging applications in Section 3. Finally, conclusions and future work are presented in Section 4.

2. Methods

2.1. Multidimensional AM-FM demodulation methods

Recent interest in the development of multidimensional AM-FM methods can be traced to early work on speech signal models by Jim Kaiser. In early work, Dr. Kaiser developed on algorithms for estimating the energy of ID signals in [1921]. This led to the development of the ID and 2D energy separation algorithm (ESA) as described in [2224]. Maragos et al. continued with the ID AM-FM work previously mentioned for ID application in [22, 23, 25, 26]. Early work on multidimensional energy separation methods appeared in [27]. AM-FM demodulation based on Gabor wavelets appeared in [28, 29]. Early research on the use of multidimensional energy operators continued in [24, 30]. In what follows, we begin with an introduction to multidimensional energy operators.

In 2D, image energy is estimated using the Teager-Kaiser operator given by [30]:

Ψ { I } ( x , y ) I ( x , y ) 2 - I ( x , y ) 2 I ( x , y ) ,

where 2 = ∂2/∂x2 + ∂2/∂y2 denotes the Laplacian operator. The IA and IF estimates are obtained using:

φ ^ x ( x , y ) Ψ { I / x } Ψ { I } ,
a ( x , y ) Ψ { I } Ψ { I / x } + Ψ { I / y } ,

where φ ^ y , is estimated by replacing the x-derivative by the y-derivative in (4). To eliminate sign ambiguities, the Teager-Kaiser operator can select the candidate IF for which the image gives the largest projection (e.g., see [31]). Recently, Kokkinos et al. showed a related accurate demodulation method using energy operators in [32] by computing all necessary derivatives by convolving with derivatives of Gabor filters, as opposed to using finite differences. The IA estimates are corrected by dividing by the magnitude response of the Gabor filter at the estimated IF (see [32] for details).

Analytic image methods for AM-FM demodulation are based on providing a Hilbert-based extension of the ID Hilbert-based demodulation approach. Here, the basic idea is to simply apply the ID Hilbert operator along the rows (or the columns). The fundamental advantage of this approach is that it preserves the 2D phase and magnitude spectra of the 2D input image. In fact, implementation involves taking the 2D FFT of the input image, removing spectral frequency with a negative row-frequency component, multiplying the result by 2, and taking the inverse 2D FFT. Given the conjugate symmetry of 2D images, the removal of two frequency quadrants does not result in the loss of any spectral information. Furthermore, it can be shown that for single-component AM-FM signals, this can lead to exact demodulation. In practice though, we replace derivatives by finite differences. We will further elaborate on this method in Section 2.2. For early work on this approach we refer to Havlicek's dissertation [3]. Havlicek et al. [3, 3341] presented the first results for TV-dimensional signals using the quasi-eigenfunction approximation (QEA) method (see Section 2.2).

Both ESA and Hilbert-based methods share the use of a filterbank prior to AM-FM demodulation. The basic idea is to use a filterbank to be able to separate out among different AM-FM components. AM-FM demodulation is then applied at the output of each channel filter. Here, it is important to note that in the event that two AM-FM components fall within the same channel filter, the filterbank approach will not allow us to separate them. Here, new algebraic approaches should be considered.

The filterbank generates AM-FM demodulation outputs for each channel. Both ESA and Hilbert-based methods select estimates from a dominant component. For ESA, the dominant component is selected based on an energy criterion. In QEA, the dominant component is often selected based on the maximum IA estimate. Here, please note that a single channel is selected over the entire filterbank. In a multiscale approach, instead of selecting dominant components over the entire filterbank, we select the dominant channel from a collection of channels. The basic idea is to define scales based on the frequency magnitude. The most popular approach is to define low, medium, and high frequency scales (see examples in [4]).

Another robust approach for computing AM-FM estimates based on a quasi-local method was developed in [4244] for 1D signals. This methodology was extended to digital images in [5, 45]. Furthermore, in [5], we have a comparison of ESA, Hilbert-based, and the quasi-local methods for a variety of 2D AM-FM signals. From the comparisons, we note that the choice of the filterbank can have a dramatic effect on the estimation. Flat passbands tend to help with the IA estimation. On the other hand, Gabor-based filterbanks are easy to design and implement and they can perform very well on IF estimation.

Related with the Hilbert based approach, Larkin et al. [46, 47] introduced the phase quadrature transform by isotropically extending the Hilbert transform in 2D. Here, instead of applying the Hilbert transform along the rows or the columns, the spiral-phase quadrature transform is applied along all directions. It is equivalent to applying the Hilbert transform along each radial direction. Here, a single component AM-FM signal is convolved with

s ( x , y ) = i ( x + i y ) 2 π ( x 2 + y 2 ) 3 / 2 = i exp ( i θ ) 2 π r 2 ,

with a frequency response given by

S ( u , v ) = u + i v u 2 + v 2 = exp [ i ϕ ( u , v ) ] .

Thus, the quadrature-phase transform does not alter the magnitude of the AM-FM signal. On the other hand, the same cannot be said about the phase. The phase information is not longer preserved. By examining (7), we can see that this is especially problematic for high frequencies (H).

Alternatively, Felsberg and Sommer proposed an n D generalization of the 1D analytic signal based on the Riesz transform, which is used instead of the Hilbert transform [48, 49]. This is termed the monogenic signal in [48]. Felsberg and Sommer introduced the following filters in the frequency domain [50]:

H 1 ( u 1 , u 2 ) = j u 1 u 1 2 + u 2 2 H 2 ( u 1 , u 2 ) = j u 2 u 1 2 + u 2 2 .

with spatial impulse responses given by

h 1 ( x , y ) = - x 2 π ( x 2 + y 2 ) 1 . 5 h 2 ( x , y ) = - y 2 π ( x 2 + y 2 ) 1 . 5 .

The local amplitude of the filter response corresponds to a quantitative measure of a structure (including the contrast) and the local phase corresponds to a qualitative measure of a structure (step, peak, etc.) [50]. The monogenic signal f M is then defined as a 3D vector formed by the signal f(x, y) with its Riesz transform f = ( h * f ) with (h) = (h, h2), using

f M ( x , y ) = ( f , h * f ) ( x , f ) .

The phase and magnitude of the monogenic signal f M are then taken as the phase and magnitude of the AM-FM signal. This is also extended to multiple scales in the multiresolution framework of [51], and into scale-space in [49]. Other related work on image demodulation based on the Riesz transform extension has been reported by Mellor, Noble, Hahn, Felsberg, Sommer and collaborators in [48, 5257]. In (8), it is important to note that while the input signal is 2D, the generated monogenic signal is actually 3D. This was done to extend the ID analytic properties to 2D. On the other hand, it is also clear that these 2D convolutions will also alter the phase of the 2D input AM-FM signal. In fact, as can be seen in the fingerprint example of Figure eleven of [51], the estimated amplitudes contains the ridges. In contrast, in the ESA fingerprint examples of [6], the ridges are modeled as a Frequency-Modulation process. This is a fundamental difference in the different approaches considered here.

Similar to AM-FM methods, we mention the work by Knutsson et al. for representing local structures on phase using tensors [5860]. Furthermore, the complex Wavelet transform provides an extension to the discrete wavelet transform that is related to the 2D Hilbert-space extension [6165].

2.2. Multiscale AM-FM methods

In this section, we provide more details on the use of multiple-scales in AM-FM demodulation. Many of the concepts introduced in this section are shared by the AM-FM demodulation methods described in Section 2.1.

We consider multiscale AM-FM representations of images given by

I ( x , y ) n = 1 M a n ( x , y ) cos φ n ( x , y ) ,

where n = 1, 2, ..., M denote different scales [4, 5]. In (9), a continuous image I(x, y) is a function of a vector of spatial coordinates (x, y). A collection of M different scales are used to model essential signal modulation structure. The amplitude functions a n (·) are always assumed to be positive.

AM-FM models non-stationary image content in terms of its amplitude and phase functions [6]. The aim is to let the frequency-modulated (FM) components cos φ n (·) capture fast-changing spatial variability in the image intensity. For each phase function φ n (·) we define the IF, φ n (·). in terms of the gradient:

φ n ( x , y ) = φ n x ( x , y ) , φ n y ( x , y ) .

The IF vector φ n (·) can vary continuously over the spatial domain of the input signal.

We generalize the concept of scale by considering AM-FM demodulation over a collection of bandpass filters. This is clearly depicted in Figure 1. For real-valued images, we only need two quadrants as shown in Figure 2. Note that by applying the ID Hilbert operator along the columns, the upper two frequency quadrants will be set to zero. On the other hand, if the monogenic signal is computed here, we will keep the entire frequency spectrum and generate a 3D signal as given in (8).

Figure 1
figure 1

Block diagram of the 2D multiscale AM-FM demodulation. Dominant AM-FM components are selected over different image scales. The bandpass filter selector (upper left) is used to define the bandpass filters that correspond to each scale. The dominant AM-FM component is selected using the maximum amplitude at every pixel (see lowest block).

Figure 2
figure 2

Four-scale filterbank used. (a) Complete frequency spectrum of the filterbank. (b) Zoom on the low frequency bandpass filters.

The effect of either operation is to generate a complex-valued signal estimates of the form: a(x, y) exp ((x, y)), where the IA and phase functions will hopefully approximate the input AM-FM components. Then, a collection of bandpass filters is used to isolate the individual AM-FM components [3].

The basic assumption here is that different AM-FM components will be picked up by different bandpass filters at the same image region. In other words, given any local image region the assumption is that the corresponding AM-FM components will be separated by the use of different filters in the interbank.

In discrete terms (see Figure 1), given the digital input image I(k1, k2) (where k1 and k2 are the discrete versions of x and y, respectively), the application of an analytic extension generates an "analytic signal" of the form: I AS (k1, k2). This 2D extension is processed through a collection of bandpass filters (to be described in the Section 2.3) within the desired scale. Each processing block will produce the IA, the IP, and the IF in both x and y directions. At the output of each block, the IA and the IP are estimated by simply taking the magnitude and the phase of the output.

In dominant component analysis (DCA), for each pixel approach, the best AM-FM demodulation estimates are selected from the bandpass filter that produces the largest IA estimate. For energy-based approaches, the dominant channel is selected based on the maximum energy estimate. Hence, the algorithm adaptively selects the estimates from the bandpass filter with the maximum response. This approach does not assume spatial continuity, and allows the model to quickly adapt to singularities in the input signal (both high and low frequency changes).

A robust method for estimating the IF was produced by Murray et al. [4, 5]. Here, for estimating the first IF component for low-frequencies, four estimates are generated using:

φ ^ 1 ( k 1 , k 2 ) = 1 n 1 arccos Ī A S ( k 1 + n 1 , k 2 ) + Ī A S ( k 1 - n 1 , k 2 ) 2 I A S ( k 1 , k 2 ) ,

where Ī A S ( k 1 , k 2 ) = Î A S ( k 1 , k 2 ) / Î A S ( k 1 , k 2 ) , and n1 represents a variable displacement that varies from 1 to 4. Variable spacing, local linear phase (VS-LLP) produces the most accurate of the four estimates by considering the condition number of the arccos(·) function [4, 5]. Similarly, the approach is repeated for the second direction. It turns out that VS-LLP reduces to the earlier method of QEA for the special case when n1 = n2 = 1 (see [3] for QEA).

2.3. Filterbank design

For computing the AM-FM estimates, the AM-FM components need to be isolated from (9) using a multiscale filterbank. Here, the basic idea is to isolate different AM-FM components over different frequencies (see [66] for details). Figure 2 (a) depicts the frequency support for a dyadic filterbank decomposition.

In Figure 2, the low pass filter (LPF, with label number 1) has frequency support in [-π/16, π/16] radians for both the x and y directions. The filters in the highest frequencies (filters from 2 to 7 in Figure 2), have a bandwidth of π/2 for both x and y directions. The bandwidth is decreased by a factor of 0.5 for each added scale. In Figure 2 (b), a closeup that shows the low frequency filters is provided.

The scales are provided using a collection of bandpass filters as: (i) LPF, (ii) very low frequencies (VL), (iii) low frequencies (L), (iv) medium frequencies (M) and (v) H as given in [4], When the structures are well defined in sizes, the measurements can be related with specific instantaneous wavelengths described in number of pixels [67, 68].

For example, for classifying lung opacities in pneumoconiosis (to be fully described in Section 3.3), the correspondence between opacity size and instantaneous frequencies using eleven different combinations of scales (CoS) is shown in Table 1. The pneumoconiosis images that were analyzed in the study have a resolution of 300 dpi. This resolution means that along 0.5 mm we have 6 pixel samples. From the ILO standard [69], the rounded opacities labeled as p, q and r, are in the sizes up to 10 mm. With the knowledge of the sizes, the lengths can be related with the corresponding frequencies. These relationships are summarized in Table 2. Based on the frequency ranges for each rounded opacity type, the cut-off frequencies of the bandpass filters are set up to fit the opacities in them by each scale. Then, a four-scale filter-bank is designed such that each scale will be related with the frequency ranges described in Table 2.

Table 1 CoS used for computing the dominant AM-FMfeature parameters
Table 2 Relationship among ILO standard grades for rounded opacities and the size in mm, size in pixels and range in frequency

2.4. Post-processing methods

2.4.1. Histogram features

AM-FM features can be summarized using appropriately chosen histograms. Here, we provide a summary of how this can be accomplished.

For each of the 11 CoS (see Table 1), a 96-bin feature vector that contains the AM-FM histograms is created. Each AM-FM histogram has the information of (32-bin each): (i) the IA, (ii) the IF magnitude, and (iii) the IF angle. Thus, each image produces 11 different feature vectors, each one corresponding to one of the 11 different CoS.

Since the AM-FM estimates of neighboring pixels could be affected by noisy estimates, a 9 × 9 median filter is often applied to the output of the IA and IF estimates (see [4, 5, 14]). We refer to [70] for methods that can be used to adaptively set the median filter window size.

Due to the ambiguity of the IF vectors (coming from cos φ(k1, k2) = cos(-φ(k1, k2))), the IF estimates are mapped to two frequency quadrants:

  • Instantaneous frequencies φ n x ( x , y ) 0 and φ n y ( x , y ) 0 are kept as they are.

  • Instantaneous frequencies φ n x ( x , y ) 0 and φ n y ( x , y ) <0 are kept as they are.

  • Instantaneous frequencies φ n x ( x , y ) <0 and φ n y ( x , y ) <0 are mapped to φ n x ( x , y ) and φ n y ( x , y ) .

  • Instantaneous frequencies φ n x ( x , y ) <0 and φ n y ( x , y ) 0 are mapped to φ n x ( x , y ) and - φ n y ( x , y ) .

Then, the IF histograms are computed.

For the IF angle, given by arctan φ n y ( x , y ) / φ n x ( x , y ) , the histogram is centered around the maximum value using:

[h] ← histogram [IF angle using l bins] {Where h is a column vector.}

H ← [hhh]. {Triplicate version of h.}

iLocation of the maximum value of h.

h ←Hl+i-l/2+l:l+i+l/2.

As last step, all the computed histograms (IA, IF magnitude, and IF angle) are normalized such that the cumulative sum of each one is equal to 1. Thus, the bins will now correspond to the probability density function (PDF) estimates.

2.5. Partial least squares classification of AM-FM features

The histograms of AM-FM features can be used in classification applications. Here, we introduce a promising classifier based on Partial least squares.

Partial least squares (PLS, [71]) is a linear regression method. Here, classification is expressed as a regression problem is given by

y = X β + ε ,

where y is a n × 1 vector of the classification variables, X is a n × p matrix of the extracted AM-FM features, β is a p × 1 vector of regression weights, and ε is a n × 1 vector of residuals. The least squares solution to estimating β is given by the normal equations β = (XTX)-l (XTy).

In most of medical imaging applications, there are much more features than images (p <n), and AM-FM features in X are highly correlated. Thus, XTX canbe singular or nearly singular and a unique solution to the normal equations will not exist. PLS reduces X to a lower dimensional subspace (k p). The first step is to factor X as X = TL, where T is an orthogonal n × k matrix of T-scores and L is a k × p matrix of factor loadings. The T-scores matrix are used to find a threshold for classification as outlined in [72],

2.6. Morphological segmentation

AM-FM features can also be used for image segmentation. A simple Bayesian classification scheme based on the IA and IF magnitude was demonstrated in [7].

The segmented AM-FM regions can be very noisy. Segmentation region denoising can be achieved via the use of an alternating sequential filter (ASF) as given in [73]. The noise in the segmented image is assumed to be present due to small structural components that do not belong to the abnormal regions. Under this assumption, it can be shown that the optimal morphological filter will be an ASF defined by:

M ( ( ( ( ( ( ( I S B ) B ) 2 B ) 2 B ) ) n B ) n B ) ,

where I s is the binary segmented image, B is the structural element; nB is the result of n dilations of B by itself; the standard open operation and • the standard close operation.

The value of n needs to be chosen so that the smaller, noisy structures are removed, while the larger structures are preserved. Once the order of the morphological filter is calculated, the segmented image is denoised using (13).

3. Applications in medical imaging

We provide a selected list of AM-FM biomedical imaging applications in Table 3 (see [10] for an earlier version of this table). Here, note that there is a number of medical applications based on energy operators. More recently, medical imaging applications have benefited from the use of multiscale methods summarized in Section 2.2.

Table 3 Select list of AM-FM applications in Biomedical Imaging

An interesting application of AM-FM models in motion estimation was presented by Fleet and Jepson [11]. In [11], Fleet and Jepson use a 3D Gabor interbank to estimate optical flow motion at pixel-level resolution. Their approach motivated the later application of a harmonic phase (HARP) method in [74].

In one of the earlier applications in electron microcopy, a simple Bayesian method using the IA and the IF was used, in conjunction with morphological filtering to provide segmentations of different abnormalities over 26 images [7]. Elshinawy et al. [75, 76] demonstrated the reconstruction of breast cancer images using AM-FM components.

Boudraa et al. [77] introduced a new cross-energy operator and used the operator to demonstrate the functional segmentation of dynamic nuclear images. Maragos et al. provide an important application for ultrasound spectroscopy in [78], where AM-FM models are used for improving Doppler ultrasound resolution. Vector-valued based AM-FM demodulation is given in [79] by Alexandratou et al. Multiscale AM-FM methods were applied to chest radiographs [67, 68], ultrasound images of the carotid artery [8083], image retrieval in ophthalmology by Acton et al. [84], retinal image classification [8, 85, 86], and electron microscopy [7].

We also have some related 1D medical signal applications by different research groups. Relevant 1D AM-FM methods appear in [8790]. Medical applications include the classification of surface electromyographic signals in [80], and the analysis of brain rhythms in electroencephalograms [91].

More general (non-medical) AM-FM applications in tracking include the work reported by Prakash et al. [92, 93] and Mould et al. [94]. Recent AM-FM texture analysis research is also reported by Kokkinos et al. [95] and Tay et al. [96]. Recently, Belaid et al. [97] presented an ultrasound image segmentation method based on the monogenic signal and quadrature filters [49].

In the rest of this section, we provide recent examples from the application of multiscale AM-FM decompositions to biomedical imaging. While the applications are focused on the particular models described in Section 2.2, it is important to note that new applications are expected, especially from methods based on multidimensional energy operators and methods based on the monogenic signal.

3.1. Optical flow motion estimation

One of the earliest applications of AM-FM methods comes from the application of phase-based method in estimating optical-flow motion. Here, we use the term "optical-flow motion" to differentiate it from actual object motion. We begin with a formulation of optical flow motion and proceed to describe the phase-based approach.

Let the image intensity in a digital video sequence be denoted by I(x, y, t). After a sufficiently small time interval Δt, the intensity at (x, y) will move to a point (x + Δx, y + Δy). Here, we are assuming that the image intensity is preserved: I(x + Δx, y + Δy, t + Δt) = I(x, y, t).

A Taylor-series expansion at starting point (x, y) gives:

I ( x + Δ x , y + Δ y , t + Δ t ) = I ( x , y , t ) + Δ x I x + Δ y I y + Δ t I t + h . o . t .

where h.o.t refers to higher-order terms. Next, if we ignore the higher order terms, divide by Δt, and then take the limit as Δt → 0, we get the Optical flow constraint equation (OFCE):

I x x t + I y y t + I t = 0 .

To formulate the motion problem, define the motion velocities u and v using: u(x, y, t) = ∂x/∂t and v(x, y, t) = ∂y/∂t. This leads to a new form of the OFCE given by:

I x u + I y v + I t = 0 .

In (14), at every pixel, we have a single equation in two unknowns. This forces us to add additional constraints. For example, a continuity constraint requires that the solution keeps the spatial integral of u x 2 + u y 2 + v x 2 + v y 2 small over the solution small. With this additional constraint we end up with two equations for two unknowns per pixel.

Fleet and Jepson [11] proposed the use of a single-component AM-FM model for modeling digital video based on

R ( x , y , t ) = a ( x , y , t ) exp ( j φ ( x , y , t ) ) .

For estimating the AM-FM components, Fleet and Jepson used a 3D Gabor interbank. Then they estimate the IF using

φ ( x , y , t ) = I m [ R * ( x , y , t ) R ( x , y , t ) ] a 2 ( x , y , t ) ,

where * denotes the complex conjugate, and refers to the spatial-gradient. The IF is then used to estimate the velocity components that are along the direction of the the IF using

( u , v ) projected on IF = - φ t ( x , y , t ) φ ( x , y , t ) φ ( x , y ) φ ( x , y , t ) ,

where the first parenthesis gives the velocity magnitude, while the second parenthesis provides the unit vector defined in the direction of the IF. To provide for a method of identifying accurate motion estimation, Fleet and Jepson require that the estimated IF is within the range of the estimating filter. This is expressed as

φ ( x , y , t ) - ( k i , w i ) τ σ k

where (k i , w i ) is the peak tuning frequency of the i th filter, σ k is the standard deviation of the filter's amplitude spectrum, and τ is a threshold used to reject unreliable estimates of instantaneous frequencies. Similarly, for the amplitude, they require that the local signal amplitude must be as large as the average local amplitude, and at least 5% of the largest response amplitude across all the filters at that frame. When either one of these two conditions is not met, the method does not provide velocity estimates.

To recover the full velocity components, Fleet and Jepson apply a local linear fit to the projected velocities. This is accomplished by fitting a local linear model over 5 × 5 neighborhoods:

( u ( x , y ) , v ( x , y ) ) = ( α 0 + α 1 x + α 2 y , β 0 + β 1 x + β 2 y ) ,

where the local velocity estimate at the central pixel is given by (α0, β0).

This study represents an important early application of an AM-FM model to motion estimation. It provided the motivation to the recent work in estimating cardiac motion from MRI (see HARP method in [74]). We expect to see the emergence of new, multiscale AM-FM methods that would be applied to motion estimation from medical video.

3.2. Retinal image analysis

Diabetic retinopathy (DR) and age-related macular degeneration (AMD) are two retinal diseases that present particular characteristics (lesions) on retinal photographs which can be used in early detection and/or classification of the disease. DR and AMD affect individuals in their most productive years of life. Early detection that leads to prevention of vision loss, alone, will lead to significant decrease in risk of early vision loss.

According to the National Eye Institute, DR is one of the leading causes of blindness among working-age Americans, while macular degeneration is a leading cause of blindness among older Americans [98]. It has been shown that regular comprehensive eye exams and timely treatment can lead to improved outcomes and reduced loss of vision. However, to screen the tens of millions at risk for DR would tax the health-care system beyond capacity. Results of an automatic DR screening system based on AM-FM exhibit strong promise in addressing this problem.

AMD is the most common cause of visual loss in the United States and is a growing public health problem. One third of Americans will develop AMD in their lifetimes. To detect AMD, retinal images are graded using the age-related eye disease study (AREDS) protocol for human grading. AM-FM has been used to develop an automated system for characterizing pathological features on these images.

3.2.1. Diabetic retinopathy

Multiscale AM-FM decompositions have been used to characterize retinal images with DR. and to screen out patients with different levels of the disease. Figure 3 shows an image of a retina with DR and one of its IA image.

Figure 3
figure 3

Example of a retinal image with DR. Note the exudates shown with a white square. (a) Original Retinal Image. (b) IA using CoS 4. (c) Thresholded Image of (b) showing large IA values. A valid range for the pixels between [15-30%] of the maximum value was used for determining the thresholded image.

The types of lesions that need to be detected by the system are shown in Figure 4. Agurto et al. [9, 85, 99] presented the extraction of AM-FM features for classification of DR images using 376 images from the MESSIDOR database [100].

Figure 4
figure 4

Lesions typical of a DR retinal image. A) Microaneurysm B) Exudates C) Hemorrahages D) Neovascularization.

These 376 images were processed with AM-FM and were divided in two groups for training and testing purposes. Table 4 shows the number of images per risk in each group. The most severe level of DR is grade 3, in which 15 or more microaneurysms or 5 or more hemorrhages are present. For grade 2, the number of microaneurysms in the retina is between 5 and 15 and for grade 1 the retinal images contain between 1 and 5 microaneurysms. The grade 0 corresponds to diabetic patients with no apparent DR in the image. It is important to mention that the images not only contain red lesions such as microaneurysms and hemorrhages, but also contain bright lesions such as exudates and cotton-wool spots.

Table 4 Images correctly classified per risk

After all the images were processed with AM-FM, they were divided in regions of 100 × 100 pixels avoiding the optic disc. Feature vectors were calculated using the moments in each region: mean, standard deviation, skewness, and kurtosis. Finally, the information per image is the input for the PLS algorithm were the algorithm is trained to classify normal images vs abnormal images.

The results obtained with this approach show an area under the ROC curve of 0.84 with best sensitivity/specificity of 92%/54%. Table 4 (last column) shows the number of images correctly classified per risk level [9]. On this table we can see a large percentage of the high-risk patients are being classified correctly.

3.2.2. Age-related macular degeneration

Barriga et al. [86] have developed a system based on AM-FM to generate multiscale features for classifying pathological structures, such as drusen, on a retinal image. Drusen are subreti-nal deposits characteristic of AMD. Figure 5 shows an AMD patient with soft and hard drusen present (white spots on the fundus). The standard high-resolution AREDS images were downloaded from the Wisconsin Fundus Photo Reading Center's website [101]. A certified ophthalmic technician (reader) selected the retinal features. For these studies, only areas that contain retinal background, vessels, soft drusen, and hard drusen were used. After the reader selected the features, regions of interest (ROI) of 40 × 40 pixels were extracted from the image.

Figure 5
figure 5

Retinal image from AMD patient from the AREDS standard database.

After calculating the AM-FM histograms, principal components analysis (PCA) was applied to the matrix containing histograms of the 120 ROIs (30 for each retinal structure) for three different AM-FM features: IA, IF magnitude (|IF|), and IF angle. Finally, the Mahalanobis distances between the different retinal structures based on PCA decomposition were calculated. These distances are shown in Table 5.

Table 5 Mahalanobis distance between retinal features: Retinal Background (RB), Hard Drusen (DRH), Soft Drusen (DRS), and Vessels

The numbers in Table 5 represent the standard deviations separating the histograms of the retinal structures. The entries under hard drusen (DRH) and soft drusen (DRS) show that they are significantly differentiated from other structures in the retina. A distance of 3 standard deviations represents a classification accuracy of more than 90%. The most interesting entry is the Mahalanobis distance between the two drusen types (2.8 standard deviations). Though still considered a high classification rate (85%), this demonstrates the challenge not only to the algorithm, but for the grader in unequivocally assigning drusen to one class or another.

3.3. Pneumoconiosis in chest radiographs

The chest radiograph is an essential tool used in the screening, surveillance, and diagnosis of dust-related respiratory illness resulting from silica and coal dust, asbestos, and a variety of other dusts that can lead to disease. The standardized method used by the international labor organization (ILO, [69]) for interpreting the chest radiograph for inorganic dust-induced diseases or pneumoconioses has been widely utilized for the past five decades.

Two limitations of the current ILO system are the intra- and inter-interpreter variability and the time consuming process of interpreting large numbers of radiographs taken for screening and surveillance programs. Murray et al. [67, 68] have developed a technique based on AM-FM that detects the level of pneumoconioses based on the nodule formation on the lungs.

Currently, the US remains 6th in the world for pneumoconiosis and interstitial lung disease (ILD) or pulmonary interstitial fibrosis as large numbers of workers continue to be exposed to dust in their work environment. Coal workers' pneumoconiosis (CWP) is identified by a specific pattern of X-Ray abnormalities and a history of exposure to coal dust. The chest radiograph is the single most useful tool for clinically evaluating both occupationally related and non-occupational chronic lung diseases. The chest radiograph is an essential tool used in the screening, surveillance, and diagnosis of dust-related respiratory illness resulting from silica and coal dust, asbestos, and a variety of other dusts that can lead to disease.

An AM-FM method for grading chest radiographs according to the ILO standards is presented in [67, 68] by Murray et al. Related work, by different research groups, can be found in [102108]. First, a logarithmic transformation to the images is applied to improve the contrast of the X-Rays [109]. Then, the AM-FM estimates of each X-Ray image is encoded using their histograms as described in the Section 2.4.1 using the tuned bandpass filters and scales showed in Tables 1 and 2. The final classification was computed using the linear regression method PLS (described in Section 2.5).

The results obtained by the system were excellent (area under the ROC curve, AUC = 1.0). However, the significance of the results should be debated based on the small database used. The authors plan as future work the testing of the system using a much bigger database. The authors produced classification results that were significantly better than those presented in [102, 103, 107].

3.4. Carotid artery ultrasound images

Cardiovascular disease (CVD) is the third leading cause of death and adult disability in the industrial world after heart attack and cancer. Of all the deaths caused by CVD among adults aged 20 and older, an estimated 6 millions are attributed to coronary heart disease and to stroke, with atherosclerosis as the underlying cause [8082, 110].

A method based on image analysis of ultrasound images of carotid plaques that can differentiate between the stable plaques that tend to remain asymptomatic and the unstable ones that eventually produce symptoms has the potential to refine the basis for surgery and spare some patients from an unnecessary costly operation which itself carries a risk of stroke.

AM-FM methods are being applied for characterizing and analyzing plaques in ultrasound images. Christodoulou et al. present in [81] their investigations for the AM-FM characterization of carotid plaques in ultrasound images. In [82], Loizou et al. present how to use AM-FM features for describing atherosclerotic plaque features. In what follows, we described the basics of their approaches.

The extraction of features characterizing efficiently the structure of ultrasound carotid plaques is important for the identification of individuals with asymptomatic carotid stenosis at risk of stroke. Christodoloulou et al. present how to characterize the carotid plaques using AM-FM features in [81]. They use as basic descriptors the information about the AM-FM estimates similar as in the Section 2.4.1: the IA, IF magnitude and IF angle histograms. In addition to the AM-FM features, the authors compute Spatial Gray Level Dependence Matrices (SGLDM) computing the following texture measures: (i) angular second moment, (ii) contrast, (iii) correlation, (iv) sum of squares: variance, (v) inverse difference moment, (vi) sum average, (vii) sum variance, (viii) sum entropy, (ix) entropy, (x) difference variance, (xi) difference entropy and (xii) information measures of correlation. Also, the Gray Level Difference Statistics (GLDS) are computed: (i) contrast, (ii) angular second moment, (iii) entropy and (iv) mean. The final step applied by the authors corresponds to the statistical k-nearest neighbor (KNN) classifier implemented for different values of k (k = 1, 3, 5, 7, 9, 11, 13). The authors applied their method to a database of 274 carotid plaque ultrasound images divided as 137 symptomatic and 137 asymptomatic. Using the AM-FM estimates, the authors got a classification success up to 71.5% when the three AM-FM estimates and k = 5 were used. This result was better than using the textures features SGLDM and GLDS that gave 68.2%.

Loizou et al. [82] present in an AM-FM analysis as an application for investigating the intima media complex (IMC), media layer (ML) and intima layer (IL) of the common carotid artery (CCA). This study represents the first study for the IMC, IL and ML. Clinically, the intima-media thickness (IMT) is used as a validated measure for the assessment of atherosclerosis, that causes enlargement of the arteries and thickening of the artery walls. The authors use the same AM-FM histograms as in Section 2.4.1 but computed using only horizontal-oriented bandpass filters. Thus, the authors use similar scales like those described in Table 1 but using only the filters numbered as (see Figure 2a): 1 (LPF), 2, 5 (in the H scale), 8, 11 (in the M scale) and 14 and 17 (in the L scale). The reason for the selection of these horizontal-oriented filters is related with the horizontally elongated nature of the atherosclerotic structures. As the last step for the statistical analysis, the authors use the Mann-Whitney rank sum test in order to identify is there are significant differences (SD) or not (NS) between the extracted AM-FM features. The authors use p < 0.05 for significant differences and comparison between different age groups.

Their study, performed on 100 ultrasound images, reveals that the IA of the media layer decreases with age. The authors state that the decreasing of the IA is maybe related to the reduction in calcified, stable plaque components and an increase in stroke risk with age. In terms of the IF, the median of this AM-FM estimate of the media layer increases suggesting fragmentation of solid, large plaque components that also increases the risk of stroke.

4. Conclusions and future work

We have provided a summary of recent AM-FM applications in medical imaging. In the coming years, we expect that there will be a variety of new applications. Here, it is important to note that medical imaging applications come from a variety of AM-FM methods and models.

The advantages of considering AM-FM models come from their ability to describe image structures over different frequency scales. For example, in retinal images, structures such as drusen and microaneurysms have a well-defined shape that get captured by the higher-frequency scale AM-FM filters. On the other hand, vessels and hemorrhages are better represented in the lower frequencies. In chest radiographs, nodules can also be well represented using the AM-FM features. The combination of feature extraction and classification using PLS has produced robust systems for the analysis of chest radiographs, where an automatic grading system for pneumoconiosis has been demonstrated on a limited-size database. For DR, a screening system has been developed and it is currently being tested on an extensive database for FDA approval.

Future work on AM-FM models will undoubtedly yield new methods and new applications. We expect to see new applications from at-least three separate approaches: (i) applications from AM-FM methods associated with multidimensional energy operators, (ii) applications from multiscale AM-FM decompositions based on Hilbert-based methods, (iii) applications associated with methods associated with the monogenic signal, and (iv) applications from new AM-FM methods that are currently being developed.


  1. Unser M, Aldroubi A, Laine A: Guest editorial: Wavelets in medical imaging. IEEE Trans Med Imag 2003, 22: 285-288. 10.1109/TMI.2003.809638

    Article  Google Scholar 

  2. Osareh A, Shadgar B: Retinal vessel extraction using gabor filters and support vector machines. Commun Comput Inf Sci 2009, 6: 356-363. 10.1007/978-3-540-89985-3_44

    Article  Google Scholar 

  3. Havlicek JP: AM-FM image models. The University of Texas at Austin; 1996.

    Google Scholar 

  4. Murray Herrera VM: AM-FM methods for image and video processing. University of New Mexico; 2008.

    Google Scholar 

  5. Murray V, Rodriguez P, Pattichis MS: Multiscale AM-FM demodulation and image reconstruction methods with improved accuracy. IEEE Trans Image Process 2010, 19(5):1138-1152.

    Article  MathSciNet  Google Scholar 

  6. Pattichis MS, Bovik AC: Analyzing image structure by multidimensional frequency modulation. IEEE Trans Pattern Anal Mach Intell 2007, 29(5):753-766.

    Article  Google Scholar 

  7. Pattichis M, Pattichis C, Avraam M, Bovik A, Kyriacou K: AM-FM texture segmentation in electron microscopic muscle imaging. IEEE Trans Med Imag 2000, 19(12):1253-1257. 10.1109/42.897818

    Article  Google Scholar 

  8. Murray V, Pattichis M, Soliz P: New AM-FM analysis methods for retinal image characterization. Asilomar Conference on Signals, Systems and Computers 2008.

    Google Scholar 

  9. Agurto C, Murray V, Barriga E, Murillo S, Pattichis M, Davis H, Russell S, Abramoff M, Soliz P: Multi-scale AM-FM methods for diabetic retinopathy lesion detection. IEEE Trans Med Imag 2010, 29(2):502-512.

    Article  Google Scholar 

  10. Pattichis M: Multidimensional AM-FM models and methods for biomedical image computing. the 34th IEEE Annual International Conference of the Engineering in Medicine and Biology Society 2009.

    Google Scholar 

  11. Fleet DJ, Jepson AD: Computation of component image velocity from local phase information. Int J Comput Vision 1990, 5(1):77-104. 10.1007/BF00056772

    Article  Google Scholar 

  12. Murray V, Pattichis MS: AM-FM demodulation methods for reconstruction, analysis and motion estimation in video signals. In IEEE Southwest Symposium on Image Analysis and Interpretation. Volume 0. IEEE Computer Society, Los Alamitos, CA, USA; 2008:17-20.

    Google Scholar 

  13. Murray V, Murillo SE, Pattichis MS, Loizou CP, Pattichis CS, Kyriacou E, Nicolaides A: An AM-FM model for motion estimation in atherosclerotic plaque videos. 41st Asilomar Conference on Signals, Systems and Computers 2007, 746-750.

    Google Scholar 

  14. Havlicek J, Tay P, Bovik A: Handbook of Image and Video Processing. Elsevier Academic Press, Ch. AM-FM Image Models: Fundamental Techniques and Emerging Trends; 2005:377-395.

    Chapter  Google Scholar 

  15. Pattichis M, Bovik A: AM-FM expansions for images. Proc European Signal Processing Conference 1996.

    Google Scholar 

  16. Pattichis M, Bovik A, Havlicek J, Sidiropoulos N: Multidimensional orthogonal fm transforms. IEEE Trans Image Process 2001, 10(3):448-464. 10.1109/83.908521

    Article  MathSciNet  MATH  Google Scholar 

  17. Pattichis M, Panayi G, Bovik A, Shun-Pin H: Fingerprint Classification using an AM-FM model. IEEE Trans Image Process 2001, 10(6):951-954. 10.1109/83.923291

    Article  MATH  Google Scholar 

  18. Ramachandran J: Image analysis of wood core using instantaneous wavelength and frequency modulation. University of New Mexico; 2008.

    Google Scholar 

  19. Kaiser J: On a simple algorithm to calculate the 'energy' of a signal. International Conference on Acoustics, Speech, and Signal Processing 1990, 1: 381-384.

    Article  Google Scholar 

  20. Kaiser J: On teager's energy algorithm and its generalization to continuous signals. In IEEE Digital Signal Processing Workshop. New Paltz, NY; 1990.

    Google Scholar 

  21. Kaiser J: Some useful properties of teager's energy operators. IEEE International Conference on Acoustics, Speech, and Signal Processing 1993, 3: 149-152.

    Google Scholar 

  22. Maragos P, Kaiser JF, Quatieri TF: On amplitude and frequency demodulation using energy operators. IEEE Trans Signal Process 1993, 41(4):1532-1550. 10.1109/78.212729

    Article  MATH  Google Scholar 

  23. Maragos P, Kaiser JF, Quatieri TF: Energy separation in signal modulations with applications to speech analysis. IEEE Trans Signal Process 1993, 41(10):3024-3051. 10.1109/78.277799

    Article  MATH  Google Scholar 

  24. Maragos P, Bovik AC: Image demodulation using multidimensional energy separation. J Opt Soc Am A 1995, 12(9):1867-1876. 10.1364/JOSAA.12.001867

    Article  Google Scholar 

  25. Maragos P, Quatieri T, Kaiser J: Speech nonlinearities, modulations, and energy operators. International Conference on Acoustics, Speech, and Signal Processing 1991, 1: 421-424.

    Google Scholar 

  26. Maragos P, Kaiser J, Quatieri T: On separating amplitude from frequency modulations using energy operators. IEEE International Conference on Acoustics, Speech, and Signal Processing 1992, 2: 1-4.

    Google Scholar 

  27. Maragos P, Bovik A, Quatieri T: A multidimensional energy operator for image processing. SPIE Symp Visual Commun Image Processing 1992.

    Google Scholar 

  28. Bovik A, Gopal N, Emmoth T, Restrepo A: Localized measurement of emergent image frequencies by gabor wavelets. IEEE Trans Inf Theory 1992, 38(2):691-712. 10.1109/18.119731

    Article  Google Scholar 

  29. Bovik A, Maragos P, Quatieri T: Am-fm energy detection and separation in noise using multiband energy operators. IEEE Trans Signal Process 1993, 41(12):3245-3265. 10.1109/78.258071

    Article  MATH  Google Scholar 

  30. Maragos P, Bovik A: Demodulation of images modeled by amplitude-frequency modulations using multidimensional energy separation. IEEE International Conference on Image Processing 1994, 3: 421-425.

    Article  Google Scholar 

  31. Pattichis MS: AM-FM transforms with applications. University of Texas at Austin; 1998.

    Google Scholar 

  32. Kokkinos I, Evangelopoulos G, Maragos P: Texture analysis and segmentation using modulation features, generative models, and weighted curve evolution. IEEE Trans Pattern Anal Mach Intell 2009, 31(1):142-157.

    Article  Google Scholar 

  33. Havlicek J, Havlicek J, Bovik A: The analytic image. Image Processing, 1997. Proceedings, International Conference on 1997, 2: 446-449.

    Google Scholar 

  34. Havlicek J, Havlicek J, Mamuya N, Bovik A: Skewed 2d hilbert transforms and computed am-fm models. Image Processing, 1998. ICIP 98. Proceedings. 1998 International Conference on 1998, 1: 602-606.

    Google Scholar 

  35. Havlicek J, Harding D, Bovik A: Discrete quasi-eigenfunction approximation for am-fm image analysis. Image Processing, 1996. Proceedings, International Conference on 1996, 1: 633-636.

    Google Scholar 

  36. Havlicek J, Harding D, Bovik A: Extracting essential modulated image structure. Signals, Systems and Computers, 1996. 1996 Conference Record of the Thirtieth Asilomar Conference on 1996, 2: 1014-1018.

    Google Scholar 

  37. Havlicek J, Bovik A: Multi-component am-fm image models and wavelet-based demodulation with component tracking. Image Processing, 1994. Proceedings. ICIP-94., IEEE International Conference 1994, 1: 41-45.

    Google Scholar 

  38. Havlicek J, Bovik A, Maragos P: Modulation models for image processing and wavelet-based image demodulation. Signals, Systems and Computers, 1992. 1992 Conference Record of The Twenty-Sixth Asilomar Conference on 1992, 2: 805-810.

    Article  Google Scholar 

  39. Havlicek J, Harding D, Bovik A: The multicomponent am-fm image representation. Image Process. IEEE Trans 1996, 5(6):1094-1100.

    Google Scholar 

  40. Havlicek J, Pattichis M, Harding D, Christofides A, Bovik A: Am-fm image analysis techniques. Image Analysis and Interpretation, 1996., Proceedings of the IEEE Southwest Symposium on 1996, 195-200.

    Google Scholar 

  41. Havlicek J, Harding D, Bovik A: Reconstruction from the multi-component am-fm image representation. Image Processing, 1995. Proceedings, International Conference on 1995, 2: 280-283.

    Google Scholar 

  42. Vakman D: Signals Oscillations and Waves: A Modern Approach. Artech House, Boston; 1998.

    Google Scholar 

  43. Vakman D: On the Analytic Signal, the Teager-Kaiser energy algorithm, and other methods for defining amplitude and frequency. IEEE Trans Signal Process 1996, 44(4):791-797. 10.1109/78.492532

    Article  Google Scholar 

  44. Girolami G, Vakman D: Instantaneous frequency estimation and measurement: a Quasi-Local Method. Measurement Sci Tech 2002, 13(6):909-917. 10.1088/0957-0233/13/6/312

    Article  Google Scholar 

  45. Rodriguez P, Pattichis M: New algorithms for fast and accurate am-fm demodulation of digital images. IEEE International Conference on Image Processing 2005, 2: II1294-7.

    Google Scholar 

  46. Larkin KG, Bone DJ, Oldfield MA: Natural demodulation of two-dimensional fringe patterns. I. General background of the spiral phase quadrature transform. J Opt Soc Amer A: Opt Image Sci Vision 2001, 18(8):1862-1870. 10.1364/JOSAA.18.001862

    Article  Google Scholar 

  47. Larkin KG: Natural demodulation of two-dimensional fringe patterns. II. Stationary phase analysis of the spiral phase quadrature transform. J Opt Soc Amer A: Opt Image Sci Vision 2001, 18(8):1871-1881. 10.1364/JOSAA.18.001871

    Article  Google Scholar 

  48. Felsberg M, Sommer G: The Monogenic Signal. IEEE Trans Signal Process 2001, 49(12):3136-3144. 10.1109/78.969520

    Article  MathSciNet  Google Scholar 

  49. Felsberg M, Sommer G: The monogenic scale-space: A unifying approach to phase-based image processing in scale-space. J Math Imag Vision 2004, 21: 5-26.

    Article  MathSciNet  Google Scholar 

  50. Felsberg M, Sommer G: A new extension of linear signal processing for estimating local properties and detecting features. In 22. DAGM Symposium Mustererkennung 2000, 195-202.

    Google Scholar 

  51. Unser M, Sage D, Van De Ville D: Multiresolution monogenic signal analysis using the riesz-laplace wavelet transform. IEEE Trans Image Process 2009, 18(11):2402-2418.

    Article  MathSciNet  Google Scholar 

  52. Mellor M, Hong BW, Brady M: Locally rotation, contrast, and scale invariant descriptors for texture analysis. IEEE Trans Pattern Anal Mach Intell 2008, 30(1):52-61.

    Article  Google Scholar 

  53. Grau V, Becher H, Noble J: Registration of multiview real-time 3-d echocardiographic sequences. IEEE Trans Med Imag 2007, 26(9):1154-1165.

    Article  Google Scholar 

  54. Felsberg M, Köthe U: GET: The connection between monogenic scale-space and gaussian derivatives. In Scale-Space 2005. Volume 3459. Springer LNCS; 2005:192-203.

    Google Scholar 

  55. Hahn SL: The analytic, quaternionic and monogenic 2-D complex delta distributions, report 3, Warsaw University Technology, Institute of Radioelectronics, Nowowiejska. 2002.

    Google Scholar 

  56. Bülow T, Sommer G: Hypercomplex signals - a novel extension of the analytic signal to the multidimensional case. IEEE Trans Signal Process 2001, 49(11):2844-2852. 10.1109/78.960432

    Article  MathSciNet  Google Scholar 

  57. Kothe U, Felsberg M: Riesz-Transforms vs. Derivatives: On the relationship between the boundary tensor and the energy tensor. In Scale Space and PDE Methods in Computer Vision. Edited by: Kimmel, R, Sochen, N, Weickert, J. Springer; 2005:179-191. vol. 3459 of LNCS

    Chapter  Google Scholar 

  58. Knutsson H, Westin CF, Andersson M: Representing local structure using tensors ii. In Image Analysis. Edited by: Heyden, A, Kahl, F. Springer Berlin/Heidelberg; 2011:545-556. vol. 6688 of Lecture Notes in Computer Science

    Chapter  Google Scholar 

  59. Knutsson H, Loglets AM: Generalized quadrature and phase for local spatio-temporal structure estimation. In Image Analysis. Edited by: Bigun, J, Gustavsson, T. Springer Berlin/Heidelberg; 2003:107-108. vol. 2749 of Lecture Notes in Computer Science

    Google Scholar 

  60. Knutsson H: Representing local structure using tensors, Tech. rep. Computer Vision Laboratory, Linkoping University; 1989.

    Google Scholar 

  61. Selesnick I, Baraniuk R, Kingsbury N: The dual-tree complex wavelet transform. Signal Process Mag IEEE 2005, 22(6):123-151.

    Article  Google Scholar 

  62. Kingsbury N: Complex wavelets for shift invariant analysis and filtering of signals. Appl Comput Harmonic Anal 2001, 10(3):234-253. 10.1006/acha.2000.0343

    Article  MathSciNet  MATH  Google Scholar 

  63. Kingsbury N: The dual-tree complex wavelet transform: A new technique for shift invariance and directional filters. In 8th IEEE DSP Workshop. Bryce Canyon; 1998:319-322.

    Google Scholar 

  64. Kingsbury N: Image processing with complex wavelets. Phil Trans Royal Soc Lond A 1997, 357: 2543-2560.

    Article  Google Scholar 

  65. Olhede S, Metikas G: The monogenic wavelet transform. IEEE Trans Signal Process 2009, 57(9):3426-3441.

    Article  MathSciNet  Google Scholar 

  66. Murray V, Rodriguez VP, Pattichis MS: Robust multiscale AM-FM demodulation of digital images. IEEE International Conference on Image Processing 2007, 1: 465-468.

    Google Scholar 

  67. Murray V, Pattichis MS, Davis H, Barriga ES, Soliz P: Multiscale AM-FM analysis of pneumoconiosis x-ray images. IEEE International Conference on Image Processing 2009, 4201-4204.

    Google Scholar 

  68. Murray V, Pattichis MS, Soliz P: Retrieval of X-Ray images with different grades of opacities using multiscale am-fm methods. Asilomar Conference on Signals, Systems and Computers 2009, 12-16.

    Google Scholar 

  69. International Labour Office: Guidelines for the use of ILO International Classification of Radiographs of Pneumoconioses. Geneva, Switzerland; 1980.

    Google Scholar 

  70. Hwang H, Haddad R: Adaptive median filters: new algorithms and results. IEEE Trans Image Process 1995, 4(4):499-502. 10.1109/83.370679

    Article  Google Scholar 

  71. de Jong S: SIMPLS: An alternative approach to partial least squares regression. Chemometr Intell Lab Syst 1993, 18(3):251-263. 10.1016/0169-7439(93)85002-X

    Article  Google Scholar 

  72. Barker M, Rayens W: Partial least squares for discrimination. J Chemometr 2003, 17(3):166-173. 10.1002/cem.785

    Article  Google Scholar 

  73. Schonfeld D, Goutsias J: Optimal morphological pattern restoration from noisy binary images. IEEE Trans Pattern Anal Mach Intell 1991, 13(1):14-29. 10.1109/34.67627

    Article  Google Scholar 

  74. Osman NF, McVeigh ER, Prince JL: Imaging heart motion using harmonic phase mri. IEEE Trans Med Imag 2000, 19(3):186-202. 10.1109/42.845177

    Article  Google Scholar 

  75. Elshinawy M, Chouikha M: Using am-fm image modeling technique in mammograms. Micro-NanoMechatronics and Human Science, 2003 IEEE International Symposium on 2003, 2: 660-663.

    Google Scholar 

  76. Elshinawy MY, Zeng J, Lo S-CB, Chouikha MF: Breast cancer detection in mammogram with am-fm modeling and gabor filtering. Proc 7th International Conference on Signal Processing 2004, 3: 2564-2567.

    Google Scholar 

  77. Boudraa AO, Cexus J-C, Zaidi H: Functional segmentation of dynamic nuclear images by cross-if b -energy operator. Comput Methods Progr Biomed 2006, 84(2-3):146-152. medical Image Segmentation Special Issue 10.1016/j.cmpb.2006.09.002

    Article  Google Scholar 

  78. Maragos P, Loupas T, Pitsikalis V: Improving doppler ultrasound spectroscopy with multiband instantaneous energy separation. Proc 14th International Conference on Digital Signal Processing 2002, 2: 611-614.

    Google Scholar 

  79. Alexandratou E, Sofou A, Papasaika H, Maragos P, Yova D, Kavantzas N: Computer vision algorithms in DNA ploidy image analysis. In Proc of the SPIE Imaging, Manipulation, and Analysis ofBiomolecules, Celss, and Tissues. Edited by: Farkas DL, Nicolau DV, Leif RC. SPIE; 2006:180-190. vol. 6088 of IV

    Google Scholar 

  80. Christodoulou C, Kaplanis P, Murray V, Pattichis M, Pattichis C: Classification of surface electromyographic signals using AM-FM features. In 19th International Conference on Artificial Neural Networks. Limassol, Cyprus; 2009.

    Google Scholar 

  81. Christodoulou CI, Pattichis C, Murray V, Pattichis M, Nicolaides A: AM-FM representations for the characterization of carotid plaque ultrasound images. 4th European Conference of the International Federation for Medical and Biological Engineering 2009, 22: 546-549. 10.1007/978-3-540-89208-3_130

    Article  Google Scholar 

  82. Loizou CP, Murray V, Pattichis MS, Christodoulou CS, Pantziaris M, Nicolaides A, Pattichis CS: AM-FM texture image analysis of the intima and media layers of the carotid artery. In 19th International Conference on Artificial Neural Networks. Limassol, Cyprus; 2009.

    Google Scholar 

  83. Loizou C, Murray V, Pattichis M, Pantziaris M, Pattichis C: Multi-scale amplitude modulation-frequency modulation (am-fm) texture analysis of ultrasound images of the intima and media layers of the carotid artery. IEEE Trans Inf Tech Biomed 2011, 15(2):178-188.

    Article  Google Scholar 

  84. Acton S, Soliz P, Russell S, Pattichis M: Content based image retrieval: The foundation for future case-based and evidence-based ophthalmology. Proc IEEE International Conference on Multimedia and Expo 2008, 541-544.

    Google Scholar 

  85. Agurto C, Murillo S, Murray V, Pattichis M, Russell S, Abramoff M, Soliz P: Detection and phenotyping of retinal disease using AM-FM processing for feature extraction. Asilomar Conference on Signals, Systems and Computers 2008, 659-663.

    Google Scholar 

  86. Barriga E, Murray V, Agurto C, Pattichis M, Russell S, Abramoff M, Davis H, Soliz P: Multi-scale AM-FM for lesion phenotyping on age-related macular degeneration. In IEEE International Symposium on Computer-BasedMedical Systems. Albuquerque, New Mexico; 2009.

    Google Scholar 

  87. Ezzat T, Bouvrie J, Poggio T: AM-FM demodulation of spectrograms using localized 2d max-gabor analysis. Proc IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2007, 4: IV-1061-IV-1064.

    Google Scholar 

  88. Gianfelici F, Turchetti C, Crippa P: Multicomponent AM-FM demodulation: The state of the art after the development of the iterated hilbert transform. IEEE International Conference on Signal Processing and Communications ICSPC 2007, 1471-1474.

    Google Scholar 

  89. Wang N, Ambikairajah E, Celler B, Lovell N: Feature extraction using an AM-FM model for gait pattern classification. Proc IEEE Biomedical Circuits and Systems Conference BioCAS 2008, 25-28.

    Google Scholar 

  90. Li H, Fu L, Li Z: Fault detection and diagnosis of gear wear based on teager-huang transform. Proc Inter-nationalJoint Conference on Artificial Intelligence JCAI '09 2009, 663-666.

    Google Scholar 

  91. Nguyen DP, Barbieri R, Wilson MA, Brown EN: Instantaneous frequency and amplitude modulation of EEG in the hippocampus reveals state dependent temporal structure. Proc 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society EMBS 2008, 1711-1715.

    Google Scholar 

  92. Prakash R, Aravind R: Modulation-domain particle filter for template tracking. Proc 19th International Conference on Pattern Recognition ICPR 2008 2008, 1-4.

    Google Scholar 

  93. Senthil PR, Aravind R: Invariance properties of AM-FM image features with application to template tracking. Proc Sixth Indian Conference on Computer Vision, Graphics & Image Processing ICVGIP '08 2008, 614-620.

    Google Scholar 

  94. Mould N, Nguyen C, Havlicek J: Infrared target tracking with AM-FM consistency checks. Proc IEEE Southwest Symposium on Image Analysis and Interpretation SSIAI 2008 2008, 5-8.

    Chapter  Google Scholar 

  95. Kokkinos I, Evangelopoulos G, Maragos P: Texture analysis and segmentation using modulation features, generative models, and weighted curve evolution. IEEE Trans Pattern Anal Mach Intell 2009, 31(1):142-157.

    Article  Google Scholar 

  96. Tay P: AM-FM image analysis using the hilbert huang transform. Proc IEEE Southwest Symposium on Image Analysis and Interpretation SSIAI 2008 2008, 13-16.

    Chapter  Google Scholar 

  97. Belaid A, Boukerroui D, Maingourd Y, Lerallut JF: Phase-based level set segmentation of ultrasound images. IEEE Trans Inf Tech Biomed 2011, 15(1):138-147.

    Article  Google Scholar 

  98. Klonoff DC, Schwartz DM: An economic analysis of interventions for diabetes. Diabetes care 2000, 23: 390-404. 10.2337/diacare.23.3.390

    Article  Google Scholar 

  99. Agurto C, Pattichis M, Murillo S, Murray V, Abramoff M, Russell S, Barriga E, Davis H, Soliz P: Detection of structures in the retina using am-fm for diabetic retinopathy classification. 2009 Meeting of the Association for Research in Vision and Ophthalmology 2009.

    Google Scholar 

  100. TECHNO-VISION Project: MESSIDOR: methods to evaluate segmentation and indexing techniques in the field of retinal ophthalmology.[]

  101. AREDS, Areds database[]

  102. Soliz P, Pattichis MS, Ramachandran J, James DS: Computer-assisted diagnosis of chest radiographs for pneumoconioses. In SPIE. Volume 4322. Edited by: Sonka, M, Hanson, KM. SPIE; 2001:667-675.

    Google Scholar 

  103. Pattichis M, Pattichis C, Christodoulou C, James D, Ketai L, Soliz P: A screening system for the assessment of opacity profusion in chest radiographs of miners with pneumoconiosis. IEEE Southwest Symposium on Image Analysis and Interpretation 2002, 130-133.

    Chapter  Google Scholar 

  104. Kondo H, Kouda T: Detection of pneumoconiosis rounded opacities using neural network. Joint 9th IFSA World Congress and 20th NAFIPS International Conference 2001, 3: 1581-1585.

    Google Scholar 

  105. Kondo H, Kouda T: Computer-aided diagnosis for pneumoconiosis using neural network. IEEE Symposium on Computer-Based Medical Systems 2001, 467-472.

    Google Scholar 

  106. Pattichis M, Cacoullos T, Soliz P: New models for region of interest reader classification analysis in chest radiographs. Pattern Recogn 2009, 42(6):1058-1066. digital Image Processing and Pattern Recognition Techniques for the Detection of Cancer 10.1016/j.patcog.2008.09.021

    Article  Google Scholar 

  107. Pattichis M, Muralldharan H, Pattichis C, Soliz P: New image processing models for opacity image analysis in chest radiographs. IEEE Southwest Symposium on Image Analysis and Interpretation 2002, 260-264.

    Chapter  Google Scholar 

  108. Van Ginneken B, Ter Haar Romeny BM, Viergever M: Computer-aided diagnosis in chest radiography: a survey. IEEE Trans Med Imag 2001, 20(12):1228-1241. 10.1109/42.974918

    Article  Google Scholar 

  109. Pattichis M, Ramachandran J, Wilson M, Pattichis C, Soliz P: Optimal scanning, display, and segmentation of the international labor organization (ILO) X-ray images set for pneumoconiosis. IEEE Symposium on Computer-Based Medical Systems 2001, 511-515.

    Google Scholar 

  110. Loizou CP, Murray V, Pattichis MM, Seimenis I, Pantziaris M, Pattichis CS: Multiscale amplitude-modulation frequency-modulation (am-fm) texture analysis of multiple sclerosis in brain mri images. IEEE Trans Inf Tech-nol Biomed 2011, 15(1):119-129.

    Article  Google Scholar 

  111. Pitris C, Kartakoullis A, Bousi E: AM-FM techniques in the analysis of optical coherence tomography signals. J Biophotonics 2009, 2(6-7):364-369. 10.1002/jbio.200910023

    Article  Google Scholar 

  112. Kartakoulis A, Bousi E, Pitris C: AM-FM analysis of optical coherence tomography signals. In Optical Coherence Tomography and Coherence Domain Optical Methods in Biomedicine XIII. Volume 7168. Edited by: Fujimoto, JG, Izatt, JA, Tuchin, VV. SPIE; 2009:71681M.

    Chapter  Google Scholar 

  113. Kartakoullis A, Bousi E, Pitris C: AM-FM techniques in optical coherence tomography. In Optical Coherence Tomography and Coherence Techniques IV. Volume 7372. Edited by: Andersen, PE, Bouma, BE. SPIE; 2009:73720U.

    Chapter  Google Scholar 

  114. Rodriguez VP, Pattichis MS: Nested random phase sequence sets: a link between AM-FM demodulation and increasing operators with application to cardiac image analysis. Proc 6th IEEE Southwest Symposium on Image Analysis and Interpretation 2004, 196-200.

    Google Scholar 

  115. Rodriguez VP, Pattichis MS: Real time AM-FM analysis of ultrasound video. 45th Midwest Symposium on Circuits and Systems 2002, 1: I-216-19.

    Google Scholar 

  116. Rodriguez VP, Pattichis M, Goens M, Abdallah R: M-Health: Emerging Mobile Health Systems, Springer US. 2006, 491-507.

    Chapter  Google Scholar 

  117. Gill D, Gavrieli N, Intrator N: Detection and identification of heart sounds using homomorphic envel-ogram and self-organizing probabilistic model. Proc Computers in Cardiology 2005, 957-960.

    Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Victor Murray.

Additional information

Competing interests

V. Murray, M.S. Pattichis, and P. Soliz are co-inventors on a relevant, pending patent on the use of AM-FM methods in medical imaging applications.

V. Murray and M.S. Pattichis declare no competing interests in the research.

E.S. Barriga and P. Soliz work with VisionQuest Biomedical, LLC. who is interested in the commercialization of AM-FM technology in medical image analysis applications. Publication of this manuscript represents positive publicity to VisionQuests commercial endeavours.

Authors' contributions

VM worked on the computational examples, worked on all aspects of the writing, and coordinated the writing efforts of the paper. MSP worked on the overall structure and organization of the document, helped with the literature review section, and edited the technical sections of the paper. ESB worked on the literature review of medical imaging applications, in particular for diabetic retinopathy and age-related macular degeneration. PS provided technical guidance on the AM-FM applications reported in the manuscript, including diabetic retinopathy, age-related macular degeneration, and pneumoconiosis.

Authors’ original submitted files for images

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Murray, V., Pattichis, M.S., Barriga, E.S. et al. Recent multiscale AM-FM methods in emerging applications in medical imaging. EURASIP J. Adv. Signal Process. 2012, 23 (2012).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: