EURASIP Journal on Applied Signal Processing 2005:15, 2585–2594 c ○ 2005 Hindawi Publishing Corporation On-board Data Processing to Lower Bandwidth Requirements on an Infrared Astronomy Satellite: Case of Herschel-PACS Camera

This paper presents a new data compression concept, "on-board processing," for infrared astronomy, where space observatories have limited processing resources. The proposed approach has been developed and tested for the PACS camera from the European Space Agency (ESA) mission, Herschel. Using lossy and lossless compression, the presented method offers high compression ratio with a minimal loss of potentially useful scientific data. It also provides higher signal-to-noise ratio than that for standard compression techniques. Furthermore, the proposed approach presents low algorithmic complexity such that it is implementable on the resource-limited hardware. The various modules of the data compression concept are discussed in detail.


INTRODUCTION
Infrared (IR) astronomy requires dedicated data compression for economical storage and transmission of the large data volume regarding the limited budget and resources available for space missions [1,2]. In fact, this is most demanding for space observatories where images are generated in different domains with higher resolution and therefore larger dimensions. This yields to an important increase in terms of data volume and bit rate. Furthermore, telemetry capabilities did not follow the same performance increase. Therefore, compression becomes a requirement for communication systems in charge of storage and/or transmission of the data.
Infrared detectors consist, as a rule, of fewer pixels than those for visual range, but the design of multisensor instruments leads to even higher data volumes. If multiple detectors are operated in parallel to support multispectral or even hyper-spectral imaging, then the data volumes multiply. Furthermore, small spacecrafts are usually used for deep space missions. They are characterized by being restricted to low budget and consequently a low data rate. Therefore, although many applications exist, which generate or manipulate astronomical data [3,4,5,6], transmitting image information still faces a bottleneck such that this constraint has stimulated advances in compression techniques for astronomy [7,8]. However, the proposed techniques are often ad hoc and sometimes not appropriate for infrared data. For example, in [8], the listed methods involve filtering of information, which is not considered to be of use, by means of object recognition methods that face the background estimation problem to guarantee not to destroy information. Furthermore, this lessens the interpretability of the results and limits the extension of the method to nonimage data structures. Indeed, thermal infrared (mid and far infrared) imaging is a measure of heat. To capture this energy, a complex instrumentation is used such that the detectors are cooled down to few kelvins. Therefore, a composite signal (source + background) will result. The source is considered as the object heat (observed target). The background is the environment heat whose amplitude is usually far higher than that of the observed target. To capture the infrared image of the wished target, one has to integrate several images over time (usually hours, depending on the wavelength), which is called integration time. Therefore, the infrared image acquired at time "t" has no object structure, which makes compression task challenging, such that it has to ensure that the relevant signal (observed source) is not lost during compression. Furthermore, infrared image acquisition is susceptible to heavy particles (glitches) that might on one side disturb the signal accuracy, changing the electronic characteristics (e.g., detector responsivity), and on the other side, it might increase the signal entropy, and thus decrease the compression efficiency. Figure 1 illustrates a typical infrared image, as observed by the telescope GEMINI [9]. Figure 1a shows the raw image at 1500 second integration time. One interesting object (galaxy) in the image after a postprocessing can be found in Figure 1b. Figure 1c shows the relevant image for the astronomy expert after removing the noise and the stripping artifact due to the instrument electronic. The challenge of data compression is to preserve as much information from the image as possible such that the relevant image structure (e.g., Figure 1b) can be reconstructed.
Indeed, no real study has been performed for IR astronomical data compression apart from the use of waveletbased compression techniques [7,8]. Generally, IR data are collected on-board an observatory (satellite) that can overload downlink bandwidth and on-board memory resources rapidly. Therefore, a significant research effort has to be invested in analyzing the performance of the compression algorithms in terms of results quality and complexity. Such an analysis forms the basis for optimizing the algorithms, and also for determining whether a given algorithm is appropriate for the application at hand. Basically, data compression is a matter of modeling. The more information can be derived from it, the less information has to be transmitted. This paper is concerned with recognizing the best-suited technique for improving the efficiency of bandwidth-limited transmission channels in case of IR space astronomy. We propose a new concept, "on-board processing," which addresses both aspects of data quality and complexity [10]. The photodetector array camera and spectrometer (PACS) [11] is one of the three instruments operating on board the Herschel space observatory (HSO) [12] foreseen to be launched in early 2007. Our task in the framework of the PACS consortium concerns the reduction of the data collected on-board, to fit the available telemetry. This task is of special importance because of the extreme compression ratio (up to 40!) dictated by the combination of a high raw data rate with a relatively low telemetry rate available for an L2-orbit space mission. An on-board processing scheme combining data reduction with lossless compression algorithms for high compression ratio is presented in this paper. This paper is structured as follows. In Section 2, the challenges of a compression method are presented. We present the problem statements and the characteristics of the astronomical data, from PACS, in Section 3. In Section 4, the descriptions of the proposed approach, on-board processing, and its modules are given. Experimental results of the application of this reduction concept are given in Section 5. We conclude with a short summary.

COMPRESSION CHALLENGES
The major concern of a compression method is to recognize and remove all redundancy in order to reduce the data traffic over the transmission channel. The performance of a data compression method for infrared astronomy can be evaluated using the following relevant parameters: (i) the compression ratio versus the reconstruction error; (ii) the complexity of the method.
The first criterion (compression ratio) points out the capability of the method to find and remove the redundancy in the data. The more redundancy is removed the better compression ratio is achieved.
The reconstruction error defines the quality of the data after reconstruction. The results quality criteria, which can be retained for estimating the merits and performances of a compression method, in case of astronomy, are visual aspect, signal-to-noise ratio, detection of real and faint objects, object morphology, astrometry, and photometry. Although the upper criteria are very important to design a compression method, the complexity of the algorithm is of bigger importance because it defines the feasibility of the method. As the algorithm has to be run on-board a space observatory computer, then, the implementation of the method has to be part of the design of the method. In this paper, we treat the case of PACS where the photoconductors and bolometers are used as detectors. We focus in this paper on the photoconductors case. When such detectors are receiving infrared photons from an astronomical or internal source, the current I at their output is proportional to the number of photons falling on the detector. As this signal has generally very low amplitude [1], the signal preamplification and sampling are required. Since the output voltage should stay within a quite limited range, a voltage reset pulse is applied in addition to a sample pulse after sampling a number of desired voltages.
An illustration of 1-dimensional signal for selected pixels from Herschel-PACS Camera [11] is given in Figure 2. All plots represent the detector output voltage in bit values (yaxis) against time (x-axis). We can see different signal behaviors that depend on the detector setting and responsivity.

HERSCHEL-PACS CHARACTERISTICS
HSO will be equipped with a 3.5 m Cassegrain telescope and house three instruments inside its superfluid helium cryostat covering the spectral range between 55 and 670 µm. The three instruments are built by different European consortia with international cooperation with international cooperation as listed in Table 1.
PACS will conduct dual band photometry and imaging spectroscopy in the 55-210 micron spectral range. The instrument consists of two 25 × 18 Ge:Ga photoconductor arrays for spectroscopy, read out at 256 Hz and two bolometer arrays with 32 × 16 and 64 × 32 pixels for photometry, read out at a frequency of 40 Hz. In both modes, a high raw data flow of up to 4 Mbit/s is generated. This is far above the nominal telemetry downlink bandwidth, which is restricted to 120 kbps, due to the L2-orbit of the spacecraft in about four times the moon's distance.
When the photoconductors are receiving IR photons from an astronomical or internal source, the voltage V at its output will increase as a function of time. The incoming photons excite charge carriers into the conduction or valence band. The voltage increase is proportional to the current through the detector which is in turn proportional to the number of photons falling on the detector. In the case of the PACS instrument, the cold readout electronics (CRE) preamplifies and samples the photo-currents generated in the detector. There will be typically 8 to 1024 samples on each ramp.
An ideal bolometer, by definition, is a device that detects all the radiation falling on it. A typical detector consists of a small chip of the doped material supported by very thin wires which act as electrical conductors for the measurement of its resistance and at the same time connect it to a heat sink with a certain thermal resistance which has to be chosen in advance according to the background level of radiation that is expected to strike it. The doping level of the material is chosen to provide an optimum sensitivity of resistance to temperature at around its operating temperature, which is 0.3 K. The analog signal is buffered, and amplified at 300 K stage, then oversampled at the multiplexer stage while being converted to digital signal.
The main challenge is the high data rate of the instrument. The raw data stream in spectroscopy consists of 2 × 26 × 18 channels, so a total of 936 channels. With a readout rate of 256 Hz we get a sampling rate of 239616 samples/s. Conversion of this analog data stream by means of a 16 bit ADC yields the maximum data rate of the raw data stream of 3744 Kbits/s. The data stream in photometry consists of 10 × 16 × 16 channels, so a total of 2560 channels. With maximum readout rate of 40 Hz, we get a raw data stream of 1600 Kbps after ADC oversampling.
For science data transmission, different modes are foreseen. In PACS prime mode, the maximum data rate is limited to 120 Kbps, while it is limited to 60 Kbps when PACS and SPIRE share the downlink bandwidth in parallel mode. In the burst mode, the maximum data rate is up to 300 Kbps. Hence, a minimum compression ratio of 40 is required 1 in prime mode. In addition to that, the detectors are continuously exposed to high energy cosmic particles inducing a disturbance (glitches) of the readout voltage, which decreases the signal-to-noise ratio and hence the data accuracy level. In the sequel, we assume the characteristics of the detector and the signalsto be as follows: signal-to-noise ratio ≈ 600-11000, glitch rate = 10 s/pixel, glitch tails < 0.5 second, detector output = 16 bits, significant bits = 14 bits.
The maximum possible compression rate we could obtain by a lossless compression (i.e., the original measurements can be recovered) can be computed as follows. A compression ratio of 16/14 is obtained by eliminating nonsignificant bits via spatial and temporal redundancy reduction. An additional compression factor of 4 is obtained by calculating the slope of the sub-ramp, which has to be given at least with the accuracy of the S/N. Therefore, 16 bit for the slope are sufficient. A further lossless compression of the signal is not possible because it contains basically the noise of the telescope, which is by definition, incompressible. This noise cannot be eliminated because we would loose the   astronomical signal. Therefore we can achieve a lossless compression factor of 4.57. Since lossless compression is impossible at such rate, we have to perform on-board processing.
In the next section we describe our compression concept in detail.

DATA COMPRESSION CONCEPT
This section reviews the basic concept for PACS on-board processing to achieve the desired downlink data rates. Figure 3 presents the different software modules. We will consider the case of photoconductors, which is challenging  in term of compression rate. Similarly, the bolometers case can be treated. First, the data packet received from the focal plane unit (FPU) is grouped into a set of reset interval measurements (useful time). Each one is called ramp. It contains measurement samples during one reset interval.
The compression concept can be coarsely divided into four modules.
(1) Header compression. Header or control data represent the observation configuration, the detectors setting, and the compression parameters. They are set by ground engineers that are responsible for running the planned observation. The control data are transmitted to PACS within the daily telecommunication period, executed by the detector and mechanic controllers according to the prescribed commanding sequence, and routed again to ground engineers as header information of the science observation data. This header is generated at science readout rate. The goal of this module is to compress the control data (header) lossless as much as possible such that the limited bandwidth can be fully exploited for the science data. In what follows, this module is not described further.
(2) Integration. The integration part of the approach performs the on-board data reduction. The basic idea is that in order to achieve the high compression ratio we have to integrate several samples on-board. Since, a ramp maybe effected by glitches, we have to ensure that we do not integrate over this ramps. This is done in the glitch detection module. (3) Lossless compression. The lossless compression part of the approach consists of the temporal and spatial redundancy reduction and the entropy coder. (4) Raw data selection. This module is responsible for transmitting selected detector data without compressing them. The main reason for this module is to check the performance of the compression software on ground. In what follows we will not describe this module further.
In the following subsections we describe the individual modules in detail, especially the integration and glitch detection part.

Detector selection
This module performs the data selection according to the detector tables. The detectors selection tables consist of model sets stored on-board depending on the object to observe or on the detector status. For instance, pixels that represent the object of interest are selected and data from others are discarded. Furthermore, bad pixels (e.g., dead pixels, saturated pixels, etc.) may be deselected and data from those pixels could be discarded.

Preliminary processing
This module is used to transform the received signal to the appropriate form (e.g., linearization of the ramps). In fact, this is used to reduce the noise (pick up and cross talk noise) in the data. It uses the infrared detectors characteristics where blind pixels (not exposed to the light) are used. They are used as reference for the correlated pick-up noise. A correlation matrix between the blind pixels on the reference lines and the actual pixels is used to remove the correlated noise.

Ramp fitting
Ramp fitting is one of the crucial steps of the proposed onboard processing concept. In this paper, we will only consider linear ramps. Indeed, an extension to nonlinear ramps could be easily done when analytic model of the ramp is available, or the fitting can be performed over a small part of ramps (sub-ramps), typically 4 samples, such that nonlinear ramps are also considered, if nonlinearity is above the 4 samples. The ramps are fitted to the sensor readings in order to obtain the flux. We consider the samples belonging to a ramp given where s is the unknown slope, t are the known instants of sampling, o is the unknown offset, and η is a vector of random variables with distribution of every element assumed to be N(0, σ), characterizing the noise process. In order to obtain the parameters of interest, this equation has to be solved in a robust manner. We have following options.

Least squares solution
The least squares solution can be easily calculated in analytic form, and is optimal with respect to the Gaussian noise process. However, this solution is not appropriate in case of outliers (i.e., glitches), therefore, glitch detection module has to ensure the outliers removal before the fitting. Figure 4 shows an example where least squares fitting is successively performed over every 4 samples of a ramp (4-sample sub-ramp). The result of the ramp fitting are slopes and the offset of the sub-ramps, and for each sample on the ramp we have a flag if it is an outlier or not. If it is not an outlier, we have in addition a residual value. This is the input to the glitch detection module.

Glitch detection
Since ramp fitting/averaging and on-board integration might be performed, we have to ensure that we do not integrate over invalid sensor readings (i.e., glitches). The detection of such events will be performed in the glitch detection module. The glitch detection will be done at the individual sample level "intrinsic deglitching" as well as at ramp level "extrinsic deglitching" and by considering subsequent ramps.

Intrinsic deglitching
This is done by the residual and offset information calculated by the ramp-fitting module. All ramps/sub-ramps where an outlier has been detected will be discarded.

Extrinsic deglitching
In this case we have to take into account the difference in slope between two subsequent sub-ramps. If two subsequent slopes differ more than 2σ we have an indication of a glitch.
All ramps which are affected by glitches are discarded. Since we might only have four points per sub-ramp, it does not make sense to take those parts of the sub-ramp into account which are not affected by glitches.
Another critical issue is glitch tails detection. Since the behavior of the detector might change for some time after it has been hit by a glitch, this is a critical issue. At the moment the concept foresees to discard all samples within a fixed time interval when a glitch has been detected. However, in the future we will investigate also methods such that this can be detected automatically.

Integration
The integration module will perform on-board integration of the sensor readings in order to achieve the desired compression ratio. This is the lossy compression part of the software. Special emphasis has to be paid in order to guarantee integration over the right readings-synchronized with the positions of the chopper-and not to integrate over ramps affected by glitches. Thus, the integration process first determines whether to discard all data of an integration block if there is a lack of confidence in at least some of the samples. Then slope data of a number of successive ramps within the same chopper position will be added, if they are free of glitches.

Lossless compression
The previous modules represent the lossy, that is, reduction, part of the PACS data reduction/compression system. The further modules constitute the lossless, that is, compression, part. To perform the high compression rate required, several compression iterations should be applied.

Preprocessing
After the integration we have a sequence of arrays we call it frames (i.e., A t , where A ∈ R 16×25 is an array of integrated slopes at time t). Since temporarily and spatially adjacent measurements will be similar, we can use this fact for further data reduction.

Temporal redundancy reduction
We calculate ∆ t+1 = A t − A t+1 · · · ∆ t+n = A t − A t+n . If subsequent frames are similar |∆ t+i | |A t+i |, 1 ≤ i ≤ n, therefore we can gain in the compression ratio encoding A t and ∆ t+i , 1 ≤ i ≤ n.

Spatial redundancy reduction
After the temporal redundancy reduction, spatially neighboring values in ∆ t+i should be similar (in the ideal case they are zero), therefore we can gain additional compression by encoding the difference of neighboring pixels.

Entropy coding
Redundancy reduction as outlined above should have reduced the magnitude of pixels values as much as possible. This fact makes it possible to have assumptions about the distribution of the data, which is a prerequisite for efficient lossless compression. Generally, the astronomical images have uniform background, while observing the same target for long duration. Therefore, the data packet related will contain many identical sample values. The redundancy reduction is suitable to optimize the data packet size. A combination between the RZIP [13] and arithmetic encoding [14] algorithms is performed for further compression ratio. The RZIP algorithm is especially developed for PACS data entropy coding. See [13] for further details. Since arithmetic encoding is a standard algorithm, we will not describe it further.

EVALUATION OF THE ON-BOARD PROCESSING CONCEPT
The on-board processing concept is evaluated in this section, on a theoretical basis and on NGC1808 IR image from the infrared space observatory (ISO) mission.

Data reduction evaluation
We first consider how many ramps have to be integrated in order to achieve the desired compression ratio of 40. As we have explained in Section 3 with lossless compression, we can achieve only a compression ratio of 4.57. The additional compression factor of 7.8 has to be gained by integration of ramps or fitting bigger sub-ramp length (e.g., 8 samples per sub-ramp). Therefore, we have to integrate over 8 ramps 2 . The next thing to consider is the potential loss of scientific data. Of course, the glitch detection will not be 100% correct. We can quantify the potential loss of scientifically valid data by the glitch detection rate and the number of ramps that will be integrated. Assuming a glitch rate of every 10 s/pixel, with a glitch tail of 0.5 s we get a probability of p glitch = 1/20 that a ramp is affected by a glitch. Then we can calculate the potential loss of scientific data p loss by where n is the number of integrated ramps and p det is the glitch detection efficiency.  Table 2 lists the potential data loss for various numbers of integrated ramps for different glitch detection rates.
A glitch detection rate of more than 95% seems feasible, therefore the potential data loss will be around 1%-3%. In fact, it will be lower because in the above calculations we have assumed for simplicity that a glitch and its tail are independent events, which is not true. In fact, if the glitch is detected, we have also detected its tail. In addition, we have assumed that when we do not detect a glitch, all integrated measurements will be lost. In fact if we miss a small glitch and integrate over it, this just decreases the signal-to-noise ratio. Another thing we have not considered is false negative rate, that is, we discard a ramp even if it is not affected by a glitch, this will of course also lead to a loss of scientific data. But this can be directly estimated. In addition, this has no effect on the other data. From these considerations, one can see that the desired compression ratio can be achieved with minimal loss of scientifically valuable data.

Quantitative results
For performance evaluation, the proposed approach, onboard processing, is compared with JPEG 2000 [15], ZIP [16], and RAR [17] methods, in terms of compression ratio (CR), processing time, and memory usage, on NGC 1808 raw infrared images (1032 frames) from the ISO mission. Figure 5a depicts the resulted image after JPEG 2000 compression of individual raw images for CR = 6, while Figure 5b shows the on-board processing result for CR = 15. This JPEG 2000 implementation [15] uses EBCOT algorithm (embeded block coding with optimized truncation) [18] for the quantization of the wavelet coefficients and the binary arithmetic coder as backend entropy codec. The white vertical line represents the column 24 with dead pixels, detectors that were lost during the mission. The quality loss is  observed compared to the resulted image ( Figure 5b) from our proposed approach, which is due to the performed quantization by means of the EBCOT algorithm. For better error display, both reconstructed images are plotted as a 1D signal in Figure 6. On the x-axis, the pixel indexes (1-1024) are represented while pixel values, for both reconstructed images (JPEG 2000 and our approach), are depicted on the y-axis. It is shown in this figure the approximation error due to the EBCOT quantization.
Compression results comparing the above listed methods are reported in Table 3 for comparison purpose. All methods have been run on a 450 MHz Pentium PC with Windows Nt 4, for identical comparison platform, although the on-board processing is dedicated for embedded hardware (DSPs) for space applications. It is noted the highest compression ratio for faster processing time needed by the on-board processing for the reduction of NGC1808 raw images compared to those with the generic compression methods. Our approach makes use of the IR astronomy signal characteristics and the limited resources for better fit the compression needs to the available resources.

CONCLUSION
In this paper, a novel on-board data processing concept is proposed, that was dedicated for the Herschel-PACS mission of the European Space Agency (ESA). We have described the key modules like ramp fitting and glitch detection in detail. Our concept combines lossy and lossless compression; the presented method offers a high compression ratio with a minimal loss of potentially useful scientific data. It also provides higher signal-to-noise ratio than that for standard compression techniques. In [19] we illustrate the feasibility of the method presented in this paper on data from Infrared Space Observatory (ISO).

ACKNOWLEDGMENT
This work is supported by a grant from the Federal Ministry of Science, Innovation and Transport and the Austrian Space Agency.