To explain HMMs [19], we should describe Markov models first. A Markov process is a timevarying random phenomenon for which the Markov property holds. We will mathematically define the Markov property but in words, a stochastic process with the Markov property, or memorylessness, is one for which conditional on the present state of the system, its future and past are independent. The term Markov chain is used to mean a Markov process which has a discrete (finite or countable) statespace. Suppose that for a Markov model, we have

Observable states: 1, 2, ..., N.

Observed sequence: q _{1}, q _{2}, ...,q _{
t
}, ..., q _{
T
}
Firstorder Markov property means:
P\left({q}_{t}=j{q}_{t1}=i,{q}_{t2}=k,\dots \right)=P\left({q}_{t}=j{q}_{t1}=i\right)
(4)
A Markov model can be recognized by its state transition matrix T:
\left[{t}_{ij}\right]
(5)
where
{t}_{ij}=P\left({q}_{t}=j{q}_{t1}=i\right)\phantom{\rule{1em}{0ex}}1\le i,j\le NN
(6)
with the constraints
\left\{\begin{array}{c}\phantom{\rule{1em}{0ex}}{t}_{ij}\ge 0,\phantom{\rule{1em}{0ex}}\forall i,j\\ {\sum}_{j=1}^{N}{t}_{ij}=1,\phantom{\rule{1em}{0ex}}\forall i\end{array}\right.
(7)
An HMM is a statistical Markov model in which the system being modelled is assumed to be a Markov process with unobserved (hidden) states. In a regular Markov model, the state is directly visible to the observer and therefore the state transition probabilities are the only parameters. In an HMM, the state is not directly visible but the output that depends on the state is visible. Each state has a probability distribution over the possible output tokens. Therefore, the sequence of tokens generated by an HMM gives some information about the sequence of states. Note that the adjective 'hidden' refers to the state sequence through which the model passes, not to the parameters of the model; even if the model parameters are known exactly, the model is still 'hidden'.
Suppose for an HMM, we have

Set of states: 1, 2, ..., N.

Set of outputs: 1, 2, ..., M.

state transition matrix T:
\left[{t}_{ij}\right]
(8)
where
{t}_{ij}=P\left({q}_{t}=j{q}_{t1}=i\right)\phantom{\rule{1em}{0ex}}1\le i,j\le NN
(9)
An HMM can be described by the last three parameters: λ = (T, B, π).
In this article, we employ HMM to study temporal behavior of kspace lines. To this end, we consider outputs of clustering step as the set of outputs of a HMM (O). We want to estimate λ such that P(O  λ) is maximized. Fortunately, there is BaumWelch algorithm to solve this problem. As explained later, we use the estimated T to decide about including or not including a kspace line in the sampling pattern.
Iterative thresholding reconstruction
We will use the iterative thresholding reconstruction algorithm described in [16, 17] for image reconstruction from undersampled kspace data. The Landweber thresholding algorithm was originally presented in [20]. For the sake of completeness, we present a brief description of the reconstruction algorithm here.
In a discrete setting, assume an N × N image, f(x, y), and its discrete Fourier transform, F(u, v) = Ғ{f (x, y)}, where Ғ denotes the Fourier transform. Define a sampling mask, S, acting on the Fourier transform of the image, F(u, v),
S\left(u,\nu \right)=\left\{\begin{array}{cc}\hfill 1\hfill & \hfill F\left(u,\nu \right)\phantom{\rule{2.77695pt}{0ex}}\text{sampled}\hfill \\ \hfill 0\hfill & \hfill F\left(u,\nu \right)\phantom{\rule{2.77695pt}{0ex}}\text{not}\phantom{\rule{2.77695pt}{0ex}}\text{sampled}\hfill \end{array}\right.
(12)
The relationship between the incomplete set of observed samples, G, and the image, F, can be expressed by the above sampling mask as follows:
where · denotes the pointwise product of S and F.
Clearly, the objective is to reconstruct the image, f , from the available Fourier domain observations, G, while the sampling mask, S, is also known. The reconstruction algorithm can be summarized as below:

1.
initialization (min. energy reconstruction): n = 0, F ^{n}= F ^{0} = S · G

2.
f ^{n}= Ғ ^{1}{F ^{n}}

3.
n++

4.
maintain the constraint: h ^{n}= S _{
γ
}(f ^{n 1})

5.
recovery of the original observations: F ^{n}= G + H ^{n} S · H ^{n}

6.
convergence? Yes: exit; no: go to 2
Here, superscripts denote the iteration number. Sγ denotes soft thresholding operation defined as:
{S}_{\gamma}\left(g\right)={\Sigma}_{i}{S}_{\gamma}\left(\u3008g,{\psi}_{i}\u3009\right){\psi}_{i}
(14)
where
{S}_{\gamma}\left(x\right)=\left\{\begin{array}{cc}\hfill x+\frac{\gamma}{2},\hfill & \hfill x\le \frac{\gamma}{2}\hfill \\ \hfill 0,\hfill & \hfill \leftx\right\le \frac{\gamma}{2}\hfill \\ \hfill x\frac{\gamma}{2},\hfill & \hfill x\ge \frac{\gamma}{2}\hfill \end{array}\right.
(15)
and {ψ_{
i
}} is a set of orthonormal basis functions for the sparse domain. Thresholding, S_{
γ
}, is essentially a sparsitypromoting operation. In brief, this operation promotes sparsity in each iteration.
Proposed algorithm
As mentioned, increasing MRI speed by undersampling ktspace may decrease SNR of the reconstructed images. Similarly, the MRI acquisition time may be decreased by acquiring undersampled kspace data. This, however, may result in loss of resolution or aliasing reconstruction artifact in the spatial domain. Nevertheless, MR images often have sparse or compressible representations in appropriate transform domains such as wavelets. In fact, one may be able to use this sparsity as an a priori reconstruction constraint to interpolate missing kspace data. Also, use of CS in MRI does not need any specific hardware changes. Therefore, in this research, we focus on increasing MRI speed by CS.
To this end, we address two problems involved in applying CS to MRI: (1) how to choose the undersampling pattern of k space; and (2) how to reconstruct an image from the undersampled data. We use the data, itself, to learn the undersampling pattern and apply an iterative thresholding method for reconstruction from undersampled kspace data.
Our proposed method is expressed in the following main stages and the flowchart in Figure 1 illustrates its steps.
For the odd points in the cardiac cycle,

Sample the upper half of k space.

Reconstruct the images corresponding to each coil channel.

Process the available k space trajectories to specify the ones that should be sampled.
In the above steps, we gather and use k space of odd time points to reconstruct the corresponding images and determine sampling pattern of even time points.
For the even time points in the cardiac cycle,
To clarify details of the mentioned stages, we should note the following.

We sample the central lines in addition to the upper half of k space. The resulting mask covers 62.5% of all lines. According to the MRI techniques, 5/8 or 62.5% of all lines is a standard fraction for half scan.

We reconstruct images using undersampled k space data by iterative thresholding, as described before.

Images corresponding to the multichannel coil are computed using the images of each coil channel and sumofsquares rule [21, 22].

K space corresponding to each image is obtained by 2D Fourier transform.
As stated previously, we study temporal behavior of k space patterns to decide about selecting or not selecting them in the sampling pattern. To this end, we employ fuzzy cmeans clustering and HMM's as intermediate tools. We cluster features extracted from each kspace lines and then consider these class labels as states of some HMM's. Finally, we estimate state transition matrices of these HMM's and use them to decide about selecting or not selecting specific kspace lines in the sampling pattern.
The processing stage is described below.

Feature extraction of kspace patterns: Mean, standard deviation, maximum, and median of each trajectory are extracted.

Clustering the extracted features: Clustering method is fuzzy cmeans and the number of clusters is about ¼ of the available trajectories. Clustering is done for the available trajectories of each coil channel. It is possible to accelerate dynamic MRI, 2 to 8 times dependent on the type of MRI [4]. Thus, we assume that every 4 (between 2 and 8) kspace lines in the time direction, can be clustered into one cluster. We choose fuzziness factor equals to 1.125.

Class label rearrangement: After clustering, the resulting labels for each coil channel may be different. Thus, we have to rearrange these labels such that the clusters with similar centers have the same label. Class label rearrangement is automatic. In the fuzzy clustering section, we first apply fuzzy clustering to the extracted features from kspace corresponding to the entire coil. In this way, we obtain cluster centers and class labels for each kspace line. After that, we apply fuzzy clustering to kspace of each coil channel. Clusters of the entire coil and clusters of each coil channel should have same labels if they have the nearest centers. We compare cluster centers of coil channels with cluster centers of the entire coil and change them according to the above rule, if necessary.

HMM training: The above clusters are assumed as outputs (O) of Markov models. We estimate parameters of HMM, λ = (T, B, π), such that a posteriori probability of outputs, P(O  λ), is maximized. This is done using BaumWelch algorithm. We use the resulting transition matrices to decide about selecting or not selecting each kspace line.

Selecting or not selecting a kspace trajectory (line) to be sampled in the even points of the cardiac cycle: A steady Markov chain has a transition matrix near an identity matrix. Thus, we select the trajectories that have transition matrices different from an identity matrix. Such trajectories are expected to have more changes than the others and are more useful for improving the reconstruction quality.