The CNES (French Space
Agency) and DLR (German Space Agency) project MERLIN is a future integrated
path differential absorption (IPDA) lidar satellite mission that aims at
measuring methane dry-air mixing ratio columns (

Methane (

The Methane Remote Sensing Lidar Mission (MERLIN – website:

MERLIN's active measurement is based on a space-borne integrated path
differential absorption (IPDA) lidar. Just like a differential absorption
lidar (DIAL), MERLIN's IPDA lidar uses the difference in transmission between
an online pulse with a frequency accurately set in the trough of several

Laser frequency positioning of the online and offline laser beams. The online frequency is positioned in the trough of one of the methane absorption line multiplets. The offline frequency is positioned so that the methane absorption is negligible.

The MERLIN measurements require a well-defined processing chain that ensures the
final performance of the mission. The processing chain is divided into four
levels. Level 0 (L0) consists of raw data (backscattered signals and
auxiliary data), and level 1 (L1) processes the vertically resolved products and
the differential absorption optical depth (DAOD) values for both individual
calibrated signal shot pairs and for a horizontal averaging window. Level 2
(L2) computes the

To reach a usable precision, space-borne IPDA lidar missions often require an
averaging of measurements along the orbit's ground track (Grant et al., 1988).
This process of averaging data horizontally is a general concern for IPDA
lidar missions. The data processing of the NASA Active Sensing of

Principle schematics of the MERLIN IPDA lidar measurement. The
lidar emits two laser beams with slightly different wavelengths
(

The non-linearity of the equation relating calibrated signals and DAOD in combination with both the statistical noise inherent to any measurement and the varying geophysical quantities (altitude, pressure, reflectivity) of the sounded scene increases the relative systematic error (RSE or bias) and impairs measurement accuracy. Werle et al. (1993) describe RRE reduction when averaging signals using the concept of Allan variance. Up to an optimal integration time, measurement variance reduces because the measurement is dominated by white noise. For greater integration times, the estimation is biased due to drifts inherent to the measurement systems. The aim of the present article is not to correct biases caused by real system drift but to correct biases that are caused by the non-linearity of the IPDA lidar measurement equation.

MERLIN must reach an unprecedented precision and accuracy on

Section 2 gives an overview of the IPDA measurement and MERLIN data processing. Section 3 defines and compares biases of several averaging schemes (described below) and suggests correction algorithms. Section 4 presents a comparative evaluation of these averaging schemes and associated bias correction procedures using modelled scenes based on real satellite data. And finally, in Sect. 5, the results of the simulation are described and a “best approach” algorithm (i.e. the least biased on tested scenes) is proposed for the MERLIN processing chain.

MERLIN active measurement is based on a short-pulse IPDA lidar. The column content of methane between
the satellite and a “hard” target (ground, vegetation, clouds, etc.) is
retrieved by measuring the light that is reflected by the scattering surface,
which is illuminated by two laser pulses with a slight wavelength difference.
Figure 2 schematically shows the principle of the nadir-viewing space-borne
lidar MERLIN. The pulse-pair repetition rate is 20 Hz, and the sampling
distance is 350 m considering a ground spot velocity of about 7 km s

When the offline and online radiation reach the photodetector (Avalanche Photo Diode), it is converted to photoelectrons and to an electrical current. The measured raw signal obtained is the sum of the lidar signal and a background signal that is produced by background light, detector dark current and electronic offset. This background signal must be estimated to be removed from the raw signal. In the presence of measurement noise, when the SNR is low, this process of background signal removal can lead to a negative estimated lidar signal.

For the sake of conciseness, we introduce for any variable

As previously mentioned, in order to reach the targeted 1 % relative
random error on

As will be seen in the following sections, the noise that affects the
measurement is one of the factors that induce the averaging bias on the
retrieved methane mixing ratio. The noise originates from the detector
noise, shot noise and speckle noise. In the case of MERLIN system, the
dominant noise is the detector noise which is considered to be normal as it
is mainly thermal noise. Then, due to the high number of photons within the
signal (approximately 10

In the following sections, we will use triangular bracket notation to denote the
arithmetic sample mean

We are interested in the retrieval of the column-integrated methane
concentration on a 50 km horizontal section along the satellite track. This
quantity will be hereafter denoted

The discrete form of Eq. (3) is

There are several ways to average the

Averaging schemes and characteristics of their biases.

There are two main causes of bias on the retrieved

Effect of the non-linearity on the DAOD distribution for a low
reflectivity (0.016 ice and snow cover). Panel

The first scheme, AVX, directly averages the column mixing ratios of methane. Every shot is impacted by the statistical bias developed in Sect. 3.3.1. Furthermore, since a column with a high total molecular content and another with fewer molecules would count the same in the averaged mixing ratio, the uniform weighting of methane concentrations leads to the creation of a bias that is called geophysical bias of type 1, described in Sect. 3.4.1.

The second scheme, AVD, computes the ratio of the mean DAOD and the mean IWF. It is also impacted by the statistical bias (cf. Sect. 3.3.1). However, this scheme takes into account the fact that every column does not present the same molecular content as DAOD and IWF are averaged separately. Thus, it is not impacted by geophysical bias of type 1.

The third scheme, AVS, averages signals before computing relative
transmissions, DAOD and

The fourth scheme, AVQ, averages transmissions before computing average DAOD
(and average

In the following sections, for each averaging scheme of Table 1 (except averaging of quotients), we will quantify separately the statistical bias and the geophysical biases and will in the end combine them in order to determine the total bias for various scenarios.

The averaging of columns (either

Under the normality assumption, Eq. (10) can be decomposed into three terms:

The assumption that the calibrated signals follow a normal distribution does
not rigorously hold when the DAOD is computed. Indeed, over dark surfaces
(low reflectivity), the SNR may happen to be so low that either one or both
calibrated signals

To correct the bias due to the non-linearity of the IPDA lidar equation, the
SNR must be estimated. Once done, the bias correction scheme would either
need to estimate the bias directly from the approximate Taylor expansion
formula of Eq. (15) or estimate the bias using Eq. (13) and a
numerical computation of Eq. (16). Typically, for MERLIN
observations, the error made by using the Taylor expansion of Eq. (15)
instead of Eq. (16) is lower than 1 ppb on the

Statistical bias induced by measurement noise. The online and offline SNRs drive the value of the statistical bias. The blue line is derived from the integration of the truncated normal distribution (Eqs. 16 and 13). The orange line is the Taylor development (Eq. 15), only valid when reflectivity is high enough (i.e. high SNR). The expected bias computed from a simple Monte Carlo simulation (yellow dots) shows that the integration approach is the most accurate. For reflectivity values of 0.1 (vegetation cover), integration (blue) and Taylor development (orange), it differs by about 1 ppb (cf. Table 2 for some values).

Error on the statistical bias estimation by using the Taylor expansion instead of using a truncated normal distribution (cf. Fig. 4).

The third averaging scheme defined on Table 1, AVS,
averages online and offline calibrated signals separately. The
corresponding estimator of the average DAOD is written

Considering an arithmetic averaging for both AVX and AVD schemes yields
different results, since the former scheme averages concentrations and the
latter averages quantities that are proportional to number of molecules of
methane. Whereas the AVX scheme computes the arithmetic mean of

On the contrary, the AVD scheme averages the extensive properties of DAOD and IWF separately. Thus, when the DAODs are averaged, the molecule amount is preserved such that the AVD scheme is not affected by a type 1 geophysical bias.

Once the bias induced by the random nature of the measurement has been
subtracted, the resulting estimator is still biased by the effects of
horizontal variations of geophysical quantities. Indeed, using Eq. (22), we
are left with

According to Eq. (28), we notice that the AVS scheme, corrected for type 2 geophysical bias, computes an average DAOD weighted by the off-signal strength. Since the main cause of variation of the offline received power is the variation of surface or hard-target reflectivity, the transmissions associated to brighter scenes count more in the average than the transmissions of darker scenes. The AVS scheme averages the measurements in such a way that a greater weight is given to high SNR signals. Consequently, this DAOD estimate is more precise (lower standard deviation) but also biased. This bias is called type 3 geophysical bias and will be defined in Sect. 3.5.

In Sect. 3.3 and 3.4, the statistical and geophysical biases on DAOD have
been derived. Here we are interested in translating biases on DAOD to biases
on

For the AVD scheme, the DAODs are arithmetically averaged with a uniform
weight. Hence, the IWF must be averaged in the same fashion. A shot-by-shot
DAOD bias according to Eq. (13) translates into a statistical bias on

For the AVX scheme,

The three averaging schemes and their associated biases will be tested on
scenes modelled from real satellite data in terms of geophysical properties.
For this purpose, we are interested in simulating the calibrated signals

SPOT-5 was a CNES satellite launched in 2002 and operated until 2015 (Gleyzes
et al., 2003). Amongst the five spectral bands of the High Resolution
Geometric (HRG) instrument, it has a spectral band in the short-wave infrared
domain (1.55 to 1.7

Data sets' resolution characteristics.

Three sites have been selected to be representative of topographic variability; they are located in the neighbourhood of three French cities: Toulouse, Millau and Chamonix. The different characteristics of the three samples are described in Table 4. Figures 5 and 6 show the variation of surface pressure and relative variations of reflectivity along the averaging scheme. Toulouse presents a medium variation of geophysical parameters (altitude and thus surface pressure), Millau presents a high variation and Chamonix a very high variation. Figure 7 shows the global cumulative distribution of standard deviations of altitude of SRTM database worldwide. We notice that 67 % and 97 % of the scenes present a lower altitude standard deviation than the one considered on the Millau and Chamonix data, respectively.

Surface pressure of the three scenes from the data sets. Toulouse, Millau and Chamonix present medium, high and very high variability, respectively (cf. Table 4).

Relative variations of reflectivity of the three scenes from the data set. Toulouse, Millau and Chamonix present medium, high and high) variability, respectively (cf. Table 4).

Characteristics of the data used for the simulation.

For sensitivity study purposes, the reflectivity relative variations from the SPOT-5 data set are multiplied by a reference mean reflectivity that can be chosen to obtain the usable scene reflectivity. Four mean reflectivity values will be considered: 0.1 (vegetation), 0.05 (mixed water and vegetation), 0.025 (sea and ocean) and 0.016 (ice and snow).

Global cumulative distribution of the standard deviation of altitude obtained on SRTM. A total of 46 %, 67 % and 97 % of SRTM boxes present a lower standard deviation than the Toulouse scene, Millau scene and Chamonix scene, respectively. The three scenes are representative of medium, high and very high variations of altitude.

The pressure grid

The methane volume mixing ratio,

Finally, the weighting functions are calculated, as described in Eq. (4),
from methane absorption cross sections and meteorological data
(

The aim of the simulation is to compare the biases of the estimated

Global description of the simulation. Data sets (blue) are
described in Sect. 4.1. Signals and the IWF
computation (orange) are described in Sect. 4.3.
Averaging strategies performed and their related bias corrections (green)
are described in Sect. 4.4 and
Table 4. Target

Computational details about averaging schemes and bias evaluation.

In order to estimate the bias, the computation of an average column-integrated methane concentration

In order to assess the performance of averaging schemes and bias correction
algorithms, the standard deviation and mean of the difference

The typical standard deviation can also be evaluated from the sample and is approximately 22 ppb for the typical case (mean reflectivity of 0.1).

Once the scene parameters are defined on the 50 km averaging window and the
atmosphere is modelled, the online and offline calibrated signals must be
simulated. We first have to compute the deterministic values of the
calibrated signals without noise and simulate the random noise that affects
them. The values of the signals are determined by the scene reflectivity
(for both online and offline signals) and by the atmospheric transmission
(online signals only). From the weighting functions, the methane field and
the pressure field, we compute the reference DAOD, denoted

Then, Gaussian random noise has to be added to the values of the signals. It
is computed from the SNR that depends on the number of photons reaching the
detector (i.e.

Online and offline SNR computed from reflectivity according to instrument characteristics.

The

The simulation tested the three averaging schemes described in Sect. 3.2: AVX (Table 1, line 1), AVD (Table 1, line 2) and AVS (Table 1, line 3). Table 5 details the computational steps used for averaging, statistical bias evaluation and geophysical bias evaluation for the three schemes. For the AVX and AVD schemes, as explained in Sect. 3.3.1, signal couples with at least one negative calibrated signal must be discarded to compute the shot DAOD. However, since signals are averaged first for the AVS scheme, the probability that one of the averaged signals is negative is extremely small. Thus, no negative calibrated signal discarding is needed for the AVS scheme.

Resulting bias (in ppb) for the AVD scheme after noise induced bias correction.

Resulting bias (in ppb) for the AVS scheme after noise induced bias correction and geophysical induced bias correction.

Concerning statistical bias evaluation, an SNR estimation is needed. It is directly estimated from instrument parameters and online and offline calibrated signal strength. Once the SNR is estimated, as described in Sect. 3.3, there are two options to evaluate the statistical bias either using the Taylor expansion approximation or the numerical integral of a truncated normal distribution. Contrary to AVS, where Taylor expansion and the numerical integral make a negligible difference, for AVX and AVD, it is better to use the numerical integral as it is more accurate, and this is what is done here.

Type 1 geophysical bias, that affects the AVX scheme, is already compensated
by weighting the average

The first results presented here are the respective biases of each averaging
scheme without any bias correction. Figure 10 shows the bias on the average

Bias before correction for the three studied averaging schemes
(red dotted lines: targeted bias

For the AVS scheme on Toulouse and Millau scenes, where there are medium to
high variations of geophysical quantities, the bias is contained in the

On the contrary, the bias of the AVD and AVX schemes is not affected by the
geophysical variations but is mainly driven by the measurement noise, which
essentially depends on the scene's mean reflectivity. As shown in Sect. 3.4.1, the AVD scheme with uniform weighting and
the AVX scheme weighted by the integrated weighting function (

Without any correction and for the typical reflectivity, the AVS scheme is less biased than the AVD and AVX schemes. However, as we have seen in previous sections, there are ways to estimate the biases and to correct them. The following section will show the results after estimation and correction of the bias induced by the measurement noise.

As explained in Sect. 3.4, the random nature of the measurement associated
with the non-linearity of the measurement equation implies that the
estimation of the

Residual bias after statistical bias correction for the three
studied averaging schemes (red dotted lines: targeted bias

We see that the biases of the AVD and AVX schemes are significantly reduced (absolute value decrease by 85 % to 90 %) on every scene. The residual bias is caused by the fact that the SNRs are estimated from the noisy calibrated signals so that the estimation of the bias is not perfectly accurate. This implies that the calibrated signal outcomes from the lower part of the distribution lead to a high error on the estimated bias. This effect could be slightly compensated if, instead of discarding all the negative or null calibrated signals (extremely rare for a reflectivity value of 0.1 over 150), we discarded calibrated signals higher than a strictly positive threshold (e.g. 0.01, not shown). This would lead to a better correction and thus a lower bias, but at the cost of discarding more single-shot observations.

Bias (in ppb) of several averaging biases before any bias correction schemes are applied on the three scenes for four reflectivity values.

For the AVS scheme, as the signals are averaged first, the equivalent SNR is
very high (

Taking into account the correction of the bias induced by the measurement noise, the AVS scheme still presents a lower bias on Toulouse and Millau scenes than the bias of the AVX and AVD schemes. However, on the Chamonix scene, where the geophysical variations are very high, the AVX and AVD schemes are less biased than AVS.

The biases induced by the variation of the geophysical parameters (cf.
Sect. 3.4) does not affect the AVD scheme, as the
additive properties of DAOD and IWF are averaged separately. The variation
of the IWF affects the bias of the AVX scheme and has already been corrected
by introducing the

Residual bias after noise induced bias and geophysical variation
induced bias corrections for the three studied averaging schemes (red dotted
lines: targeted bias

Figure 12 shows the residual bias after the corrections of the statistical bias induced by the measurement noise and the variations of geophysical parameters (cf. Sect. 4.4 and Table 5). We notice that the residual bias for the AVS scheme is considerably reduced when the average weighting function is weighted by the offline calibrated signal strength. Furthermore, the iterative estimation of the bias converges at the first iteration of Eqs. (26) to (31).

Once geophysical biases are subtracted, the three scenes present a low bias.
The mean residual bias on the three scenes for the AVD and AVX schemes is
approximately

All results presented above are computed for scenes with a mean reflectivity
of 0.1, which roughly corresponds to vegetation cover. For the purpose of
choosing the least biased algorithm to compute average

First, as seen in Table 6 (AVD scheme), the Taylor bias correction does not succeed in quantifying the bias on any of the four mean reflectivity values. The uncertainties are too high and prevent quantitative analysis of the results. This is due to the fact that there are some calibrated signals that are really close to zero and for which the SNR is underestimated; thus the bias (and standard deviation) is overestimated. This could be mitigated by the choice of a higher threshold of the usable calibrated signal before the computation of the DAOD (not shown). The results when using the integral bias correction on AVD are more physical. However, they also show an over estimation of the bias, especially for low reflectivity values. In every case for the AVD scheme, the bias threshold is exceeded.

Table 7 gives the results of the robustness of the AVS
scheme to decreasing reflectivity. Unlike the AVD scheme, the AVS scheme,
when all corrections are made, presents satisfying results for all
reflectivity values, and in every scene the biases remain contained into the
threshold interval of

To summarize, the best algorithm to limit the bias for MERLIN processing algorithms is clearly the AVS scheme, with an average IWF weighted by the offline calibrated signal strength and both corrections of the geophysical bias and the bias induced by the measurement noise (either Taylor or integral bias correction). On every scene and for all expected reflectivity values, this algorithm is compliant with the averaging bias specifications of the MERLIN mission. Note that this conclusion holds in the case where all the 150 shots are considered; in the case of a partially cloudy window where only a subsample of clear sky shots are averaged, the AVS will still be the best averaging scheme, but the performance will be decreased.

The French–German space-borne IPDA lidar mission MERLIN will measure the
average integrated column dry-air mixing ratio of methane (

Three averaging schemes have been studied: averaging of

The three schemes are sensitive to the bias induced by the measurement noise
even if AVS is far less impacted for the typical reflectivity. This bias can
be corrected by a formula introducing the estimated SNR on the measured
signals if the SNR is high enough. The bias due to the variation of
geophysical parameters does not affect the AVD scheme because it directly
averages the desired additive quantities. On the contrary, the AVX scheme
must average the concentration weighted by the integrated weighting function
(IWF) in order to average a molecule number instead of averaging
concentrations. The third scheme AVS measures the average

These averaging schemes and their bias corrections have been tested on scenes modelled from real satellite data in terms of altitude, surface pressure, weighting functions and relative variations of reflectivity. The three scenes present interesting characteristics, as they show different geophysical variations that could impact averaging biases. Besides, the signals and random noise are simulated from geophysical parameters and instrument parameters.

The simulation shows that the lowest biases are obtained for the AVS scheme
using appropriate bias corrections and averaging weights. Furthermore, this
scheme is robust to low reflectivity values unlike the AVX and AVD schemes,
which are highly sensitive to the accuracy of the SNR estimation. The best
scheme, AVS, is compliant with the allocated averaging bias requirements
(RSE) of 0.06 % (1 ppb for a

A continuation of this study could evaluate the sensitivity of a poor (unprecise or biased) estimation of the SNR on the estimation of the bias due to measurement noise for low reflectivity values. Furthermore, the use of the lidar simulator and processor suites, currently in development at the LMD, could be beneficial to the evaluation of the biases, and more specifically of the averaging biases, on a wider scale (many scenes, atmosphere types, etc.).

SPOT-5 data can be accessed at

The simulation of calibrated signals requires a noise model. The signal distribution is considered to be Gaussian first, because the number of photons that reaches the photodetector is high enough for the Poisson distribution to be considered as Gaussian. Secondly, the system is limited by the detector noise that is mainly thermal noise, which is normally distributed.

The calibrated signals are produced using a pseudorandom number generator.
The expected values of the calibrated signal distributions,

The number of photons is computed from the reflectivity and the atmospheric
transmission. In the standard case (reflectivity of 0.1 sr

The first term of the denominator corresponds to the detector noise, the second to the shot noise and the third to the speckle. Note that the speckle term has been neglected in this article, whereas both detector noise and shot noise have been considered, as they are dominant compared to speckle noise.

The averaging of quotients estimates the average of the shot-by-shot two-way
transmissions

CP and YT designed the simulation algorithms while YT handled its implementation with the support of FG. MW developed theoretical aspects such as the averaging scheme definition or the iterative geophysical bias correction and supported the whole work. FM provided the data set used for the real scene model and the interpolated surface pressures. YT prepared the manuscript with contributions from all authors.

The authors declare that they have no conflict of interest.

This work was funded by CNES as part of the CNES and DLR project MERLIN. We thank Frédéric Chevallier (LSCE) for the kind support he provided to this work. The authors would also like to thank the following LMD collaborators working on the MERLIN project (in alphabetical order): Raymond Armante, Vincent Cassé, Olivier Chomette, Cyril Crevoisier, Thibault Delahaye, Dimitri Edouart and Frédéric Nahan. Edited by: Joanna Joiner Reviewed by: three anonymous referees