Neural network-based clustering for agriculture management

Abstract

Remote sensing images have been used productively for land cover identification to accurately manage and control agricultural and environmental resources. However, these images have often been interpreted interactively due to the lack of effective automated methods. We propose such a method using self-organizing maps (SOM) based spectral clustering, for agriculture management. By combining the powerful aspects of the SOM (adaptive vector quantization in a topology preserving manner) and of the spectral clustering (a manifold learning based on eigendecomposition of pairwise similarities), the proposed method achieves successful results, as shown on three study areas with images from RapidEye (a recent constellation of satellites with a specific focus on agricultural applications).

Introduction

Since the very beginning of the 1990s, one of the main application areas of remote sensing in agriculture in Europe has been the control of area-based agricultural subsidies given to the farmers in the framework of the common agricultural policy (CAP) [1]. For many years, this activity, also referred to as controls with remote sensing (CwRS), has consisted in checking whether the area and the crop type declared by the farmer are in line with the real situation as identified mainly from satellite remote sensing data. In the early years, when the identification of crop types was very important, both computer aided photo-interpretation methods (CAPI) and automatic classification procedures were involved in CwRS. Recent changes in the CAP decreased the importance of crop-type identification during the control procedure, while increasing the importance of the detection of anomalies related to the compliance with the rules defined as good agricultural and environmental conditions (GAEC). While CwRS covers the control aspect of the CAP management there is another important element: the land parcel identification system (LPIS). The LPIS is the so called reference system for locating and identifying each agricultural parcel in the EU that is declared in the farmers’ annual applications. The LPIS has to be as up to date as possible in order to reflect the real situation on the ground inside the LPIS parcels in terms of the correct quantification of the maximum eligible area in each LPIS parcel. For identification of these parcels, the use of digital geographic information system (GIS) techniques has been compulsory since January 2005, by a legislation change in 2000 [2].

The EU member states set up different strategies to update their LPIS, mainly by acquiring new very high resolution (VHR) orthoimage, but also including some field checks. In addition to their efforts to keep the LPIS updated, the member states have to assess the quality of this system for agricultural parcels annually. For LPIS assessment, they often use current year VHR orthoimages that are acquired annually in the frame of CwRS on a sample basis (control zones), or they acquire new airborne or spaceborne orthoimages. The actual inspection is usually done by the CAPI of these VHR images. However, automated methods are of great importance for fast and accurate assessment, because automated detection of problems in LPIS can guide yearly updates, reducing the risk of paying sanctions due to the inappropriate identification. Our study addresses this need for an automated LPIS quality check, based on land cover identification. We choose high resolution image provided by the recent RapidEye constellation (launched in 2008, with a specific target of agricultural applications) due to its considerable potential to determine the land cover encompassing a whole country, thanks to its spatial and spectral resolution (additional Red Edge band) and daily overpass [3].

For land cover identification from remotely sensed image, both supervised and unsupervised methods are considered in previous research [4, 5]. To accurately determine class boundaries, supervised (classification) algorithms require labeled training samples that can only be obtained from field inspections and through manual expert labeling [6], whereas unsupervised (clustering) methods depend on pixel similarity, determined according to some criteria [7]. For annual control of the LPIS, we focus on unsupervised methods, due to the fact that supervised methods need costly field inspections and significant processing time to collect the necessary training samples throughout the whole country.

A successful unsupervised method that is common for cluster extraction from remotely sensed images is the use of self-organizing maps (SOM) [8–12]. The SOM is an artificial neural network introduced by Kohonen [13] based on observed properties of neural maps. It maps a high-dimensional data space onto (often) a lower-dimensional lattice (usually 2-D or 3-D) in a topology preserving manner (as much as possible with the given rigid lattice), while providing a faithful vector quantization of the data space [14]. This enables informative SOM visualizations of high-dimensional spaces, facilitating interactive extraction of clusters (see [14, 15] and references therein). In addition, automated methods, which often use hierarchical agglomerative clustering (HAC) [16–19], also exist for fast and accurate SOM based clustering. Moreover, a recent study [20] introduced another HAC that exploits the SOM properties by introducing a similarity measure (CONN) based on detailed local density distribution to achieve an accuracy higher than the accuracy obtained by distance-based approaches.

Apart from remotely sensed images (which are often large datasets), spectral methods [21–23], which exploit pairwise similarities of data instances using eigendecomposition of a distance-based similarity matrix, are becoming popular for clustering, due to supporting empirical studies, their ability of extracting irregularly-shaped clusters, and their easy implementation [24]. However, they are infeasible for large datasets, due to their high computational complexity and memory requirement. Therefore, several studies [25–27] select data representatives (either sampled randomly or by a vector quantization method) and apply spectral clustering to those selected representatives. The approach of partitioning randomly selected samples (data representatives) in [25] is used in few studies exploiting spectral clustering for remotely sensed data [28, 29].

Our contribution in this study is to utilize both the SOM properties and the advantages of spectral clustering, by obtaining the data representatives (prototypes) with the SOM and clustering the SOM prototypes by spectral partitioning using a local density-based similarity CONN instead of a distance-based similarity, for agriculture monitoring. We develop an LPIS quality assessment based on this SOM based spectral clustering, and show that it is successful using three test zones in Hungary. We also compare the proposed method to other SOM based HAC methods and also to neural gas [30] based clustering. The rest of this article is organized as follows: Section ‘Neural networks for unsupervised clusteringment’ explains the clustering methods (HAC and spectral clustering) for these neural networks. Section ‘Study area and images’ describes three test zones and the remote sensing images used in this study. Section ‘Results and discussions’ discusses the clustering performance for these zones, and Section ‘Conclusions’ concludes the article.

Neural networks for unsupervised clustering

Self-organizing maps

The self-organizing map (SOM) [13] is composed of neural units ordered on a (usually 2-D) fixed lattice, where each unit has an associated weight vector with the same dimensionality of the data space. The weight vectors of the SOM units are adapted to become quantization prototypes of the data samples, by an iterative learning process composed of three steps: competition, cooperation, and adaptation. For a D-dimensional dataset $\mathcal{ℳ}$ and an SOM grid $\mathcal{G}$ (most frequently a 2D rectangular grid) with N neural units, where w _i is the D-dimensional weight vector associated with the neural unit i, the sequential SOM learning can be summarized as follows:

• 1. (competition) randomly select a data sample $v\in \mathcal{ℳ}$ and find its best matching unit (BMU) i whose w _i satisfies

  $$\begin{array}{cc}\parallel v-w_i\parallel \le \parallel v-w_j\parallel & \forall j\in \mathcal{G}\end{array}$$

  (1)

• 2. (cooperation) excite the grid neighbors j of the BMU i determined by a neighborhood function h_i,j(t) (usually defined as a Gaussian around the BMU i, based on the grid distance between neural units i and j),

• 3. (adaptation) adapt w _i and its grid neighbors using

  $$w_j(t+1)=w_j\left(t\right)+\alpha \left(t\right)h_{i,j}\left(t\right)(v-w_j(t\left)\right)$$

  (2)

where α(t) is a learning parameter decreasing with time.

These steps are repeated until either a predefined error criterion or a maximum number of iterations is reached. After learning, adapted weight vectors of neural units (the SOM prototypes) produce a Voronoi tessellation of the data space, where each prototype is the geometric center of its Voronoi polyhedron (receptive field) where it is the BMU for the data samples. Thanks to the SOM rigid grid and cooperative adaptation, the SOM produces a topology preserving mapping of the data space onto a (lower-dimensional) fixed grid on the prototype level. Namely, the prototypes neighbors on the grid are ideally neighbors in the data space (their Voronoi polyhedrons share an edge) and vice versa. The SOM learning also provides a faithful representation of the data distribution on the prototype level, which can be controlled by a magnification parameter with slight changes in the learning algorithm [31].

Topology preserving mapping onto a lower-dimensional grid enables interactive cluster extraction using various informative SOM visualization schemes (see [14, 15] and references therein). Despite the success of the interactive process [9, 10, 12], practiced knowledge is usually necessary to evaluate visualized SOM information, which in turn makes it difficult for inexperienced users, and time consuming even for the experienced users. This necessitates automated methods for fast and effective SOM segmentation, especially for applications that require processing many large datasets, such as monitoring environmental and agricultural resources, relying on remotely-sensed data acquired on a yearly basis. Automated methods, described in Section “Automated SOM clustering for LPIS assessment”, exploit how the SOM prototypes quantize the dataset by determining prototype similarities (such as distances, neighborhood relations, density distribution) with respect to data space. Due to the rare consideration of the SOM grid in automated methods, a neural network paradigm with no rigid grid, neural gas [32], described in the following section, is also used to obtain prototypes for comparison in terms of clustering accuracies. Even though the neural gas is not a topology-preserving mapping (contrary to the SOM), it is shown to reach relatively small quantization errors [32, 33].

Neural gas

The neural gas [32] has also a learning algorithm based on finding the BMU unit w _i (as in Equation (1)) and adaptation ofw _i and its neighbors. Contrary to the SOM which forces a grid layout of the neural units, the neural gas defines the neighbors using distance ranks (${\rho }_{w_j}$s) of neural units (w _j s) to the presented data sample v, which are calculated at each learning step. Noting that ${\rho }_{w_i}=0$ for the BMU w _i , the neighborhood function h _λ (w _j ) is constructed by ${\rho }_{w_j}$ and a characteristic decay λas $h_{\lambda }\left(w_j\right)=exp(-{\rho }_{w_j}/\lambda )$. Then adaptation rule for neural gas is

$$w_j(t+1)=w_j\left(t\right)+\alpha \left(t\right)h_{\lambda }\left(w_j\right)(v-w_j(t\left)\right)$$

(3)

with α(t) ∈[0,1]decreasing with time t.

Automated SOM clustering for LPIS assessment

Hierarchical agglomerative clustering for SOMs

Due to the fact that hierarchical agglomerative clustering (HAC) can find arbitrary cluster shapes with appropriate criterion for cluster similarity, and can suit high-dimensional data which are often hard to describe with parametric models, it is often preferred for SOM clustering [11, 16–18, 20, 34]. Each SOM prototype is considered as a singleton cluster and two clusters that are the most similar according to a predefined (dis)similarity criterion are merged iteratively until a predetermined number of clusters is obtained. A common approach is to use a criterion based on (Euclidean) distances between SOM prototypes, such as centroid linkage in [16, 18] and Ward’s measure in [17, 34]. Since any similarity measure solely based on the distances between SOM prototypes underutilizes available SOM knowledge such as data topology and data distribution, recent studies merges distance and density information. Brugger et al. [35] uses a recursive flooding of a Gaussian surface based on pairwise distances and receptive field sizes of SOM prototypes, resulting in partitionings similar to that of k-means clustering. Wu and Chow [19] evaluates similarity by single linkage and density distribution at the separation boundaries using a cluster validity index in [36], producing good partitionings for datasets with well-separated clusters.

A recent study [20] proposes CONN linkage, which is average linkage with CONN similarity based on detailed local density distribution, instead of traditional distance based similarity. CONN, originally proposed in [14] for informative SOM visualization, is a symmetric matrix, showing pairwise similarities of the SOM prototypes. Each pairwise similarity, CONN(i,j), is

$$\mathrm{CONN}(i,j)=\left|{\mathrm{RF}}_{\mathrm{ij}}\right|+\left|{\mathrm{RF}}_{\mathrm{ji}}\right|$$

(4)

with RF _ij is that portion of RF _i (receptive field of w _i ) where w _j is the second BMU, and |.| is the cardinality of the set. Therefore, CONN(i j)not only indicates neighborhood relations of prototypes with respect to the dataset but also indicates how data samples are distributed within their receptive fields with respect to the neighboring prototypes, providing a density information more detailed than on the prototype level. Consequently, CONN linkage is shown to outperform distance-based linkages for several real datasets including a remote sensing image [20]. In addition, since CONN does not depend on SOM grid structure, it can be used as a similarity measure for prototypes obtained by any other quantization method (such as neural gas, k-means).

Spectral clustering for SOMs

Similarly to HAC, spectral clustering (SC) can extract arbitrary shapes and can be easily implemented with high accuracies, as supported by empirical studies [24]. Contrary to HAC, SC is principally a manifold learning based on eigendecomposition of a similarity matrix, aiming at changing data representation to easily capture submanifolds (i.e., clusters). Being associated with relaxed optimization of graph-cut problems, by a graph Laplacian matrix, L, various methods exist for SC [21, 22, 37]; however no clear advantage exists among them as long as a normalized L is considered [23, 38]. Referring to [23, 39] for detailed overview on different methods, we briefly explain the method in [22] utilized for this study.

Let G = (V S)be a weighted, undirected graph, nodes (V ) represent N samples (prototypes in this study) $\mathcal{W}=\{w_1,w_2,\dots ,w_N\}$ to be clustered, and S, a N × Nsimilarity matrix, defines edges. A common way to construct edges is to define pairwise similarities based on the (Euclidean) distances,

$$s(i,j)=e^{-\frac{\left|\right|w_i-w_j\left|\right|}{2{\sigma }^2}}$$

(5)

with a decaying parameter σto be determined properly, either by experimentally finding the optimum σ value [22] or by an automated setting of σ(different σ _i for each prototype w _i , changing the denominator to 2 σ _i σ _j ) [28, 40, 41]. The latter is done by definingσ _i as the distance to the k th nearest neighbor ofw _i , introducing another parameter (k) to be set by the user.

Let D be the diagonal matrix denoting the degree of N nodes where d _i  = ∑ _j s(i j). Then the Laplacian matrix, L, is constructed in various ways depending on the approach for graph-cut optimization [23, 39]. Ng et al. [22] define a normalized Laplacian matrix, L_norm, based on S and D,

$$L_{\text{norm}}=D^{-1/2}S{D}^{-1/2}.$$

(6)

Then, K clusters are extracted using K eigenvectors associated with the K greatest eigenvalues, by the following algorithm [22]:

• 1. Calculate the similarity matrix S (Equation (5)), its degree matrix D, and normalized Laplacian, L _norm (Equation (6))

• 2. Find the K eigenvectors { e₁,e₂,…, e _K } of L_norm, associated with the K greatest eigenvalues {λ₁,λ₃,…,λ _K }

• 3. Construct the N ×  K matrix E = [e₁e₂…e _K ] and obtain N×K matrix U by normalizing the rows of E to have unit norm, i.e., $u_{\mathrm{ij}}=\frac{e_{\mathrm{ij}}}{\sqrt{\sum _k{e}_{\mathrm{ik}}^2}}$

• 4. Cluster the N rows of U with the k-means algorithm into K clusters.

Recently, we utilize this algorithm as an SOM clustering method [42], using similarity matrices calculated either by σ or local σ _i . This approach often outperforms HAC with the distance-based linkages or with CONN linkage, for synthetic and real datasets [42]. However, a σ or k value (to determine local σ _i ), specific to the dataset, is required to be set optimally [42, 43]. Contrary to the distance-based similarity requiring user-set parameters, CONN similarity can be advantageous for SC due to its construction using intrinsic data details without any parameter, its sparse nature by definition, and previous studies [20, 44] showing its outperformance. Therefore we modify the algorithm above by replacing S (Equation (5)) with CONN (Equation (4)).

Proposed method for the LPIS assessment

The proposed method aims to find the anomalies in the LPIS. Based on the SOM based spectral clustering described in previous section and the current LPIS, the method first finds a land cover mapping (with a predetermined number of clusters) in an unsupervised manner, then constructs an eligibility mask by checking whether clusters are eligible or ineligible according to the current LPIS. The difference between the resulting eligibility mask and the LPIS indicates possible anomalies in the system. A step-by-step explanation of the proposed method for the LPIS assessment is below:

• 1. Set the number of neural units, N, and the number of clusters, K.

• 2. Train the SOM with N units to obtain the prototypes (Section ‘Self-organizing maps’).

• 3. Construct the similarity measure CONN for the SOM prototypes (Equation (4)).

• 4. Obtain K clusters of the SOM prototypes by spectral clustering with CONN (Section ‘Spectral clustering for SOMs’).

• 5. Assign the cluster label of prototypes to the data samples in their corresponding receptive fields.

• 6. Use current LPIS (eligible-ineligible) to determine eligibility of each cluster, i.e., if the majority of the data samples in a cluster is eligible in LPIS, then that cluster is eligible, and vice versa.

• 7. Determine the areas where the resulting eligibility mask and LPIS have different labels.

The proposed method is automated, given that N and k are known a priori. Since the SOM is used as an intermediate quantization of the remote sensing images, and k can be determined from the LPIS, setting of N and k is not a limitation for this study.

Study area and images

For automated LPIS assessment with the proposed method, we use study areas in Hungary. According to the rules related to the direct support schemes for farmers under the Common Agricultural Policy (CAP) Hungary applies the Single Area Payment (SAPS) scheme. For SAPS the eligibility criteria are defined in Council Regulation (EC) No. 73/2009 of 19 January 2009, Article 124: the eligible agricultural area under the single area payment scheme shall be part of the total area taken up by arable land, permanent grassland, permanent crops and kitchen gardens. We select three test zones from the northern central part of Hungary (Figure 1). Zone1 is a transition area between the Hungarian Great Plain and the hilly area of the Northern Hungarian low mountain range, while Zone2 is between the two hills (Mátra and Bükk) of the Northern Hungarian low mountain range. Zone3 covers the geographic area called Cserhát. These zones are selected due to the availability of new airborne very high resolution (VHR) orthophotos with 50 cm ground sampling distance, acquired in 2010 for the same regions. The synergy between the RapidEye data and the new VHR orthophotos makes it possible to compare the clustering results with the VHR orthophotos of the same year. The systematic update of the LPIS using the new VHR orthophotos is carried out by the Hungarian administration parallel to the current project so the results of the “manual” update can also be used to evaluate the results obtained by the proposed method.

Figure 1

Location of the test zones in Hungary. Each test zone is a 24 km × 24 km square region. For each zone, a 5-band (blue, green, red, red-edge and near-infrared) 4800×4800pixel RapidEye image is used for LPIS assessment with the proposed method.

The first zone, Zone1, is dominantly agricultural (64%eligible), with mostly arable lands but with an intensive presence of vineyards. The zone covers partly two wine-growing regions: Mátraalja and Eger. The north-west corner of the zone partially covers the forests of Mátra Hills. The mid-west part of the zone is dominated by the outcrop lignite mines that are partially recultivated (covered by soils to be able to grow some vegetation). Figure 2a shows a false color composite of the RapidEye image for this zone acquired on 10 July 2010. Contrary to Zone1, the second zone, Zone2, is dominantly non-agriculture (35%eligible). The zone partially covers Mátra Hills (West), Bükk Hills (East), and Heves-Borsod Hills (North). The agricultural land is represented mainly by pastures and fodder crops with a limited number of arable parcels. A false color composite of the RapidEye image for the zone acquired on 10 July 2010 is shown in Figure 3a. The third zone, Zone3, is balanced between agriculture and non-agriculture (45%eligible). The non-agricultural land cover is mainly forest stretching west, south and north-east from the center of the zone. The RapidEye image for the zone was acquired very early in the vegetation season, on 3 April 2010. As the false color composite on Figure 4a shows, the forests are still leafless. The areas appearing as yellow, orange and light brown are winter crops (winter wheat, rapeseed), pastures and alfalfa. Coniferous woods appearing as dark brown on the false color composite are very distinct on the early spring image. Zone3 was selected to test whether the proposed method can be applied to images acquired very early in the vegetation season.

Figure 2

Zone1: (a) False color composite (RGB bands correspond to near-infrared, red-edge, and red bands, respectively), (b) Resulting LPIS mask obtained by spectral clustering with CONN similarity (SC_CONN) and k=30 clusters. White: eligible lands detected as eligible; Blue: ineligible lands detected as eligible; Red: eligible lands detected as ineligible; Black: ineligible lands detected as ineligible. Blue and red indicate possible anomalies in the LPIS according to the automated assessment. The outlined regions are discussed in the text

Figure 3

Zone2: (a) False color composite (RGB bands correspond to near-infrared, red-edge, and red bands, respectively), (b) Resulting LPIS mask obtained by SC_CONN and k=30, for LPIS assessment. White indicates eligible lands detected as eligible, blue stands for ineligible lands detected as eligible, red indicates eligible lands detected as ineligible, and black is for ineligible lands detected as ineligible. The outlined regions are discussed in the text.

Figure 4

Zone3: (a) False color composite (RGB bands correspond to near-infrared, red-edge, and red bands, respectively). This image is acquired in early spring, before vegetation growth. (b) Resulting LPIS mask. White: eligible lands; blue: ineligible lands detected as eligible; red: eligible lands detected as eligible; black: ineligible lands.

RapidEye images are used in this study, based on their successful applications in previous studies: [3] shows that they can help effectively determine land parcels which are in good agricultural condition (GAC) and are potentially applicable for payments in Bulgaria in the frame of the Common Agricultural Policy of the European Union (EU); [45] shows their use in agricultural applications, especially during the controls of agricultural subsidies. In addition, with its background mission, RapidEye aims at covering Europe at least once a year cloud free at 5 m resolution during the vegetation season. Thanks to its spectral resolution (including red-edge band) at a spatial resolution of 5 m, and its large area coverage, RapidEye can provide a huge opportunity to detect LPIS anomalies at the EU level, if an automated method—robust enough to be applicable from the Mediterranean to Northern continental regions of the EU and using images acquired on different dates throughout the vegetation season—can be developed.

Results and discussions

For the LPIS assessment with the SOM based clustering, first we obtain the SOM prototypes by Matlab SOMtoolbox (developed by Helsinki University of Technology) using a 50×50rectangular lattice with sequential learning and Gaussian neighborhood. We also train a neural gas with 2500 prototypes using default learning parameters in the SOMtoolbox. We select the number of prototypes (2500) to be of $O\left(\sqrt{n}\right)$ where n is the number of data samples (pixels in the 4800×4800 image of each zone, covering an area of 24km×24km). Then we cluster these prototypes by spectral clustering with CONN similarity (SC_CONN). For comparison, we also use spectral clustering with distance-based similarity (SC) and hierarchical agglomerative clustering (HAC) with average and CONN linkages. We set the maximum number of clusters, k = 30, according to the different land use and vegetation types declared in the zones, whereas we also choose two smaller values (k = 10 and k = 20). Based on our experiments, we also experience that using a k value greater than the number of existing natural clusters in the zone (number of declared vegetation types in the LPIS) would not help achieve higher accuracies. Since the expected number of clusters (k) is determined from the LPIS information a priori, setting k in advance does not represent a limitation here. Then we assign the clusters as eligible/ineligible using LPIS, based on the assumption that LPIS has to be (mostly) correct even though it may have anomalies due to changes in land cover or land use.

Tables 1, 2, and 3 show the resulting accuracies (% of pixels correctly identified as eligible or ineligible over all pixels) for three zones, different methods and three k values. The proposed SC of SOM prototypes with CONN similarity achieves the best accuracies for all three zones (82.9% for Zone1; 83.9% for Zone2; 81.2%for Zone3). The decreases in accuracies with decreasing k values indicate that a high k (30) is still necessary for better delineation of the boundaries, even though the accuracies are calculated based on eligibility/ineligibility (i.e., 2 clusters). Surprisingly, despite having smaller quantization errors than the SOM [33], NG based clustering produces slightly lower accuracies than SOM based clustering, for this study.

Table 1 Accuracies for different clustering methods with different number of clusters (k): Zone1

Table 2 Accuracies for different clustering methods with different number of clusters (k): Zone2

Table 3 Accuracies for different clustering methods with different number of clusters (k): Zone3

To evaluate the clusters obtained by different methods, we also calculate the purity of the clusters. The purity of a cluster is the ratio of the most frequent class in that cluster. For our 2-class (eligible/ineligible) case, purity(i) = max(r_eligible(i),r_ineligible(i)), wherer_eligible(i) andr_ineligible(i) are the ratios of eligible and ineligible pixels in the clusterC _i , respectively. Note thatr_eligible(i) = 1 − r_ineligible(i). Ifr_eligible(i) ≥ 0.5, the clusterC _i is eligible, otherwise it is ineligible. The closer ther_eligible(i)(and hence purity(i)) to 0.5, the more mixed the clusterC _i is. The r_eligible(i) of k = 30 clusters obtained by different methods are shown in Figure 5. The clusters extracted by SC_CONN have relatively low confusion: for examplefor Zone1, r_eligible(i) > 0.7 (eligible) or r_eligible(i)<0.3 (ineligible) whereas other methods have a few clusters within 0.6 > r_eligible(i) > 0.4. For all zones, the average purities, given in Table 4, indicate that the resulting partitions obtained by SC_CONN are purer than those obtained by other methods. They also show that density-based CONN similarity and SC produce purer partitions than distance-based similarity and HAC, correspondingly.

Figure 5

Ratio of eligible pixels in each cluster: (a) for Zone1 (b) for Zone3. Compared to other methods, SC_CONN produces purer clusters, for all zones. (We note that clusters (indexed from 1 to 30) obtained by different methods do not have a one-to-one correspondence

Table 4 Average purity for k = 30clusters for each zone

For visual evaluation, Figures 2, 3, and 4 show the false color composites for three zones and the resulting LPIS evaluation masks. These masks significantly diminish the necessary interactive process to find anomalies in the LPIS only to those blue and red areas. Some reference parcels in the LPIS are originally set as ineligible thanks to the additional ancillary information (such as land use or no subsidy applications within the respective LPIS parcel for 4 years), even though they are eligible with respect to the land cover (spectral information). Therefore those regions—such as grasslands within woodlands for which no subsidy claims were launched (examples are those blue regions outlined by the green rectangles) or in non-agricultural regions (a recultivated mining region in Figure 2 and a deforested area in Figure 3, outlined by black rectangles), small fields along the roads (outlined by black dashed rectangles)—are extracted as eligible in the resulting masks. Grassland or vegetation within the urban areas (blue regions within red rectangles) are extracted as eligible due to the same reason and also due to the spectral mixing because of 5 m spatial resolution. However urban regions can be removed easily by additional thematic data. An eligible crop (maize) is incorrectly captured as ineligible due to its spectral resemblance to the woodlands (ineligible) when it is grown. The spectral resemblance can be seen in the false color composite in Figure 2: reddish regions at the top-left are woodlands whereas those within black ellipses are maize. Clouds and their shadows (outlined by yellow ellipse in Figure 2) may result in extracting eligible regions as ineligible.

Tables 5, 6, and 7 show the confusion matrices of the produced masks in Figures 2, 3, and 4, correspondingly. They also show producer accuracies, PA (percentage of correctly clustered eligible/ineligible pixels over all eligible/ineligible pixels in the LPIS), and user accuracies, UA (percentage of correctly clustered eligible/ineligible pixels over all eligible/ineligible pixels in the mask). Since Zone1—which has mostly eligible pixels (almost 2/3 of all pixels)—has an image acquired in the middle of the vegetation season, agricultural vegetation and grasslands can be easily delineated from woodlands and water bodies, resulting in high PA (94.3 %), compared to other zones. Due to the same reason, urban vegetation—which may not be differentiated using only spectral values, but requires land use information—is confused, resulting in a low PA for ineligible areas. Zone2 has also an image in the middle of the vegetation season, but it has mainly ineligible pixels (mainly woodlands). PA for both eligible and ineligible areas are over 80 %. Despite having an image in the early vegetation development, PA for Zone3 are also acceptable, providing help for further evaluation of those incorrect labeling to detect anomalies in the LPIS.

Table 5 Confusion matrix of SOM based spectral clustering with CONN similarity, for Zone1

Table 6 Confusion matrix of SOM based spectral clustering with CONN similarity, for Zone2

Table 7 Confusion matrix of SOM based spectral clustering with CONN similarity, for Zone3

Figure 6 shows an example region (from Zone1) on how the resulting LPIS mask can guide for the LPIS assessment. Figure 6a is a false color composite of the 5-band RapidEye image (used for clustering) with the current LPIS. Figure 6b shows the mask automatically obtained by the proposed method: white/black stand for eligible/ineligible regions which require no adjustments whereas red/blue indicate possible anomalies in the LPIS. For example, the red area labeled with “A” is eligible in the LPIS, however it should be ineligible according to the mask. In this case the detected anomaly corresponds to a forest parcel (as it can be visually identified from the RapidEye false color composite in Figure 6a and from the VHR image in Figure 6c), which is not eligible under the single area payment scheme applied in Hungary and therefore should be part of the LPIS ineligible mask. Similarly, the areas marked with “B” also show real anomalies detected: forest patches along a watercourse (visually identifiable both from RapidEye and VHR image) that should have been marked as ineligible in the LPIS mask. Another type of anomaly is in the parcel “D” which is ineligible in the LPIS but the mask recommends it to be eligible. Visual interpretation of both RapidEye and VHR images suggests that this parcel may indeed be agricultural according to its land cover. For the evaluation of mask-based anomalies, Figure 6d shows a newer LPIS updated using interactive interpretation of VHR image and ancillary data. (We note that this newer LPIS is partly available only for some regions). Both A and B are in line with the manual update, however D and its surrounding are excluded (made ineligible) because no subsidy application has been submitted for those parcels within the previous 4 years, despite their eligible land cover. A similar area that is detected potentially eligible, but marked as ineligible in the LPIS is the region B in Figure 7.

Figure 6

An example anomaly detection in LPIS for a selected region in Zone1. The LPIS is overlaid on the figures: red lines show the boundaries of the LPIS physical blocks whereas cyan lines are the boundaries of eligible/ineligible regions with cyan dotted areas are ineligible (a) RapidEye false color composite (RGB bands are NIR-Rededge- Red bands, respectively) (b) The resulting LPIS mask obtained by the proposed SC_CONN method. Red/blue regions show possible anomalies in the LPIS (c) VHR orthophoto with 0.5 m spatial resolution, acquired in 2010 by the National Administration in Hungary for the LPIS update with visual evaluation (d) The same LPIS mask in (b) but with the LPIS updated by computer aided photo-interpretation. A, B, and D are real anomalies with respect to the land cover (D is made ineligible according to additional ancillary data), where C (maize parcels) is a false-alarm because of the spectral resemblance due to the 5 m spatial resolution.

Figure 7

A false-alarm example (from Zone1) for LPIS anomaly assessment, due to mono-temporal image. (a) RapidEye false color composite (RGB are NIR-Rededge- Red, respectively) (b) The LPIS mask produced by the proposed method. The red areas on the upper-half of the scene are temporary waterlog due to heavy rains, producing an incorrect labelling as ineligible, even though those are eligible

Some anomalies detected by the proposed method may be false alarms due to 5 m spatial resolution, mono-temporal image, the use of only multispectral image with no additional ancillary information, and land cover type. For example, due to the spectral resemblance of the maize (eligible) to the woodlands (ineligible) in 5 m resolution image during the vegetation season, two maize parcels (marked with “C”) in Figure 6 are detected as ineligible even though they are eligible. This resemblance is reduced in the VHR image, as it can be seen in Figure 6: they are similar to agricultural vegetation in Figure 6c while they are more similar to woodlands in Figure 6a. Another false alarm example is a temporary waterlog in Figure 7 (the area marked with “A”), which is detected as potentially ineligible, even though it is actually eligible; yet the mono-temporal spectral information is insufficient in those cases. Spectral information may also be insufficient to differentiate small difference in the radiometric response of grassland and forest in early April. Therefore, grassland (“B” in Figure 8) may be incorrectly detected as ineligible. Decisions on the eligibility of pasture parcels similar to the one marked with “A” in Figure 8 are difficult not only using very high resolution orthophotos (Figure 8c,d) but also on the ground. To decrease the false alarms and achieve even more precise determination of the anomalies for the entire country, further interactive evaluation of the resulting mask or additional decision rule-based classifier exploiting other available information (such as land use, farmer declarations, VHR image, etc.) is necessary despite the success of the proposed clustering method based on mono-temporal spectral image.

Figure 8

False-alarm examples (from Zone3) for LPIS anomaly assessment, due to land cover type and mono-temporal image. The LPIS is overlaid: red lines are physical blocks whereas blue lines are boundaries between eligible/ineligible areas, with dotted areas are ineligible. (a) RapidEye false color composite (NIR-Rededge- Red are assigned to RGB) of 3 April 2010. (b) The resulting LPIS mask obtained by the proposed method. (c) VHR orthophoto (0.5m spatial resolution), acquired in 2010. (d) The region outlined by black rectangle in (c) to show the details of the grassland (“B”) and the grassland with encroachment of bushes and small trees (“A”)

Conclusions

For agriculture management (at large cartographic scale) by mono-temporal multi-spectral remote sensing images, we develop a method using self-organizing maps (SOM) based spectral clustering (SC). By providing two consecutive mapping (SOM: a topology-preserving mapping together with an adaptive vector quantization; SC: a manifold learning based on eigendecomposition) and local density-based similarity, the proposed method outperforms both other SOM based and neural gas based clustering methods, for three test zones in this study. It can be an effective tool to reduce the extensive time required for interactive computer-aided photointerpretation for precise delineation of eligible/ineligible agricultural regions. However, in addition to mono-temporal multi-spectral image, other ancillary data—which can be exploited by decision rules—may be necessary for fine tuning of the resulting eligibility mask.