To develop an accurate measurement network of greenhouse gases, instruments in the field need to be stable and precise and thus require infrequent calibrations and a low consumption of consumables. For about 10 years, cavity ring-down spectroscopy (CRDS) analyzers have been available that meet these stringent requirements for precision and stability. Here, we present the results of tests of CRDS instruments in the laboratory (47 instruments) and in the field (15 instruments). The precision and stability of the measurements are studied. We demonstrate that, thanks to rigorous testing, newer models generally perform better than older models, especially in terms of reproducibility between instruments. In the field, we see the importance of individual diagnostics during the installation phase, and we show the value of calibration and target gases that assess the quality of the data. Finally, we formulate recommendations for use of these analyzers in the field.
The Integrated Carbon Observation System (ICOS) is a European
research infrastructure project that is currently reaching its operational
phase after a 5-
In the framework of ICOS, the ICOS Atmospheric Thematic Centre (ATC)
metrology laboratory (MLab hereafter) based at the Laboratoire des
Sciences du Climat et de l'Environnement (LSCE) is responsible for
testing every instrument within the ICOS atmospheric network. Here,
we present the results of tests for CRDS instruments that measure
In the first section, we describe the analysis technique and the
different models of instruments. Then, the protocols and the metrics
used to assess the performance of the instruments are defined. In the
last section, results for each species (
All the results presented in this study come from tests performed at
the manufacturer, at the MLab and in the field on CRDS analyzers
manufactured by the company Picarro, Inc. between 2008 and 2014. They
cover five different instrument models and up to four species
(
The CRDS technique can be described as follows. A laser source is used
to excite a measurement cell, which consists of a low-loss optical
resonant cavity composed of at least two concave high-reflectivity
mirrors. As the injected laser light propagates back and forth between
the mirrors, a portion of the light is retransmitted through the
mirror after each pass. A photosensitive detector located behind one
of the mirrors monitors the time decay of the laser light. The decay
(or ring-down) time depends on the cavity loss but also on the
presence of any absorber species inside the cavity. Thus, higher
concentrations of the target analyte molecule in the cavity correspond
to shorter ring-down times. This technique and the details for each
model are detailed in
The three models, named ESP1000, G1301 and G2301 measure
The 47 analyzers considered in this study. Their serial number,
model, ICOS number when attributed and period of purchase are indicated in the
first columns. If field results are presented here, the site as well as the
number of months the instrument was used in the span of this study (
Continued.
Continued.
Spectral fits of absorption data (black solid dots) for
The metrics defined in the study follow the International
Vocabulary of Metrology guidelines (VIM,
All instruments are tested before leaving the factory and
a certificate of compliance is provided with the instrument. For all
instruments and species (except
The continuous measurement repeatability
(CMR, called
precision in the certificate of compliance) is calculated as the
average over 30 h of 5 min interval SD of raw data (frequency about
0.5
For the first two generations of instruments (ESP1000 and G1301), the short-term drift was the peak-to-peak amplitude of the 5 min averaged data over 30 h. From the third generation on, the drift is defined as the peak-to-peak amplitude of the 50 min averaged data over 30 h. For CO, the drift is again defined differently as the peak to peak amplitude of the 5 min averaged data over 24 h.
The short-term repeatability (STR) is measured by cycling two gases at 10 min intervals for 2 h. The data from 8 min 50 s to 9 min 50 s are averaged and the SD of the averages calculated.
The accuracy against the factory internal scale was first evaluated with the mean of the raw data over 30 h then later over 30 min only. The results from this test are not discussed in this study.
Since the beginning of the MLab operation in 2008, the protocols and metrics used to evaluate the instruments have evolved. However, for our analysis, most of the data sets could be reanalyzed and the latest version of the protocols applied.
For the present protocol, the MLab keeps the instrument for about 1 month
to perform all tests using dry reference gases calibrated in agreement with
WMO scales, comparison to reference instruments and drying/humidifier system
to evaluate the sensitivity to water vapor content. A detailed report is
provided for each instrument. The cylinders are aluminum tanks (for the older
ones used only as test cylinders: LUXFER, UK aluminum alloy 6061 with VTI
Ventil Technik GmbH stainless steel valves, for all the newer ones: LUXFER,
UK aluminum alloy 6061 with Rotarex membrane valve (D200 type with PCTFE
seal) with brass or stainless steel body) with brass or stainless steel
pressure regulator from Air Liquide America Specialty Gases LLC (previously
Scott). They are either filled with dry natural air or with dry synthetic
air. The isotopic composition of these last cylinders is controlled to
correct for any bias compared to natural air. Indeed, as the CRDS instruments
are sensitive only to the major isotopologue (
For the first instruments, in order to develop meaningful tests, the
stabilization time for each instrument was evaluated in order to know
how long a cylinder had to be measured before being stable. This
stabilization period has been determined to vary between 5 and
15 min depending on the water content of the previous sample and the
length of its analysis (not shown). It also depends on the length of
the sampling lines and on the design of the whole system (dead volume,
flush volume, etc.), which is relatively uniform in the MLab but can
vary from site to site. These first tests have helped to define the
length of the measurement intervals for the now standard protocols.
Also, inlet pressure tests have been realized to evaluate the
influence of the inlet pressure onto the measurements. Results (not
shown) highlight the importance of having a difference below
0.4
As part of the latest MLab protocol, 11 criteria are now evaluated for each instrument. These are defined below. Target gases are used to evaluate most of the metrics except the calibration, linearity and comparison with reference instruments.
The continuous measurement repeatability is evaluated with the SD of the continuous measurements of a cylinder over 24 h as described above.
The short-term drift is defined as the peak-to-peak amplitude of the same measurements.
These two metrics are evaluated for different integration times
(typically, raw data, 1
The Allan deviation, which shows the stability as a function of the integration time and informs about the optimal integration time, is also calculated and provided in the synthesis report.
These three metrics are illustrated in the Appendix in
Fig.
The short-term repeatability is defined as the repeated measure of a sample over a short period of time (about 3 h). In the laboratory, a target gas is measured 10 times in 15 min sequences bracketed by 5 min of wet ambient air measurements. For each measure, only the last 9 min are averaged. The repeatability is then expressed through the mean and SD of these averaged measures.
The long-term repeatability (LTR) is comparable to the short-term repeatability but on
a longer timescale (3 days). In the laboratory, a target gas is
measured for 30 min bracketed by around 5 h of wet ambient air
over 72 h of total measurements. For each measure, only the last 10 min are averaged. The long-term repeatability is then expressed
through the SD of these averaged measures. Typically, several 3-day
exercises are performed and the results compared and aggregated at the
end of the 1-month duration of the instrument test period. In Fig.
For the latest instruments, since 2013, the temperature and pressure
dependencies have also been tested at the MLab. For the pressure, we plot
the target gas measurements realized during the long-term
repeatability test against the atmospheric pressure over several days
and evaluate the correlation between the two. For the temperature
dependence, the room temperature was until now varied using the room
air conditioning system and we plot the target gas measurements
against this varying temperature. Plans have been made to acquire
a temperature-controlled chamber. As for the pressure, the correlation
between the two is calculated. Two examples are shown in Fig.
An important test for the CRDS instruments is the water vapor correction evaluation. The applied factory correction is the same for all instruments even if not all of them have the same response to water vapor. It is also not always possible to measure only dry air when in the field. Over the years, different tests have been applied to evaluate this correction, from comparing two instruments measuring the same air, one with a drying system, the second without, to the latest tests that progressively increase the humidity of the measured gas stream. The complete methodology and the results of these tests will be treated in a separate publication.
For an operational network, it is crucial to report not only precise
but also accurate data linked to each other by a common scale. If each
instrument is usually calibrated at the factory, the calibration scale
used is not linked to the international WMO standards. Moreover,
a regular calibration allows us to correct for long-term drift in the
instrument. In the laboratory, at least four calibration sequences are
done to determine the calibration function that links the measured
values to the assigned values. Three to four standard gases are
measured one after the other at least four times for 30 min each
calibration sequence (each set of the four cylinders measurement is
hereafter called a cycle; see Fig.
The linearity of the instrument is also evaluated. For the first instruments,
the same cylinders as for calibration (four cylinders) were used. Then, two
cylinders (low and high concentrated cylinders; see Fig.
Finally, ambient air measurements from each instrument are compared
with other reference instruments maintained by the MLab. The MLab is
located in Gif-sur-Yvette, about 50
In the field, to estimate instrument performance, we use the
calibration and target gases. Usually, one or two target gases are
measured regularly for quality control purposes. If only one cylinder
is used, then this cylinder is a so-called short-term target and is
measured once to twice a day. If two cylinders are in use, then the
second one will be the long-term target and will be measured at the
same time as the calibrations, usually every 2–4 weeks. The
short-term target tank lasts about 1–2
For the stabilization time for one measurement, we select the last measurement interval of the last tank of the calibration sequence to avoid the influence of water from potentially humid ambient air samples and to ensure the flush and equilibration of the tank pressure regulators. We are indeed trying to look only at the performances of the instrument itself independently of the analysis chain setup. We calculate the minute averages within the interval and then the difference of the averages to the last minute of analysis.
For the stabilization time within one calibration sequence, we compare the average of the last 10–15 min of each interval for the last cylinder to the last measurement interval.
We also look at the stability of the instrument by looking at the evolution with time of the calibration equation and evaluate whether the periods between the calibration allow us to capture this evolution. Finally, we look at the evolution of the linear fit residuals to investigate the linearity of the instrument over time.
The target gases in the field are not measured continuously for 24 h. However, the short-term target is measured at least once a day for 20 to 30 min. Here, as an equivalent of the continuous measurement repeatability, we calculate the monthly average of the SDs of raw data over 1 min intervals.
For this value, we calculate the SD of the averaged target measurement intervals over 3 days as in the MLab; then we calculate monthly average of this number for graphical visibility.
By studying the drift of the calibration constants of the instruments
over time, we have an opportunity to study the behavior of this
population of instruments over time. The following quantities are
evaluated:
Allan deviation for
The first test in the MLab is the continuous measurement repeatability
measurement which allows us to draw the Allan deviation vs. the averaging
time and already gives good insight into the stability of the instrument.
In Fig.
Results for
Results for
Results for CO. First panel: continuous measurement repeatability (CMR) at the factory (as defined), at the MLab (on the raw data) and in the field (as defined). Second panel: short-term repeatability (STR) at the factory and the MLab. Third panel: long-term repeatability (LTR) at the MLab and in the field. Fourth panel: comparison with reference instrument, average difference at the MLab. The horizontal lines show the WMO compatibility goals.
In this section, we look at the performances of the groups of instruments
using some of the metrics defined in the previous sections. Results from the
factory, the MLab and the field sites are shown in Table
Finally, in terms of bias to the reference instrument, the average of the difference is usually within the WMO compatibility goals.
For all species, and especially
Whisker boxplots summarizing the CMR, STR and LTR tests for
CO pressure dependence before (left) and after (right) repair at the factory for CFKADS2084. Among the tested instruments, four others were showing a dependence higher than 4 ppb hPa
Calibration residuals for one set of cylinders measured on 14 different instruments. Each point is the average residual for each instrument for one calibration cylinder. For each cylinder, the reference concentration is indicated.
Daily average of data availability at each site (if 50 %
of data are valid for 1 day, then the day is valid; if less,
the day is invalid). Valid data are in color; invalid data are in
black (% are indicated in Table
Top:
Top:
Most of the instruments tested at the MLab show a limited sensitivity to
temperature and pressure. However, some instruments present a higher
dependence. In the case of the temperature changes, if these changes are slow
and within the range guaranteed by the manufacturer (10–35
In the case of atmospheric pressure variations,
To evaluate the linearity of the instruments, we have to be confident
in the assignation of our cylinders. In the past, we have used
different instruments (GC, Fourier transform infrared spectroscopy) and set of cylinders to assign the
values of the MLab calibration sets. We are still in the process of
reevaluating these values when possible. Here to assess whether the
amplitude of the observed residuals are due to the instruments or to
the calibration scales, we have plotted the residuals of each cylinder
for the 14 instruments that used the latest set of calibration
cylinders. In Fig.
Instrument failures, models concerned and solutions for the field instruments of this study.
Out of the 47 instruments tested at the MLab, we present results for
13 that have been installed in the field on sites instrumented
for at least 1 year (see Tables
The earliest data set begins in September 2008, and we end our study on
30 September 2014. As detailed in Table
In Fig.
In the next figures the data from these instruments are compiled. We
use first data from calibration gases. Data from the same site are in
the same shade of color. In Fig.
In the right panel of Fig.
For
In Fig.
To evaluate the drift, using the calibration equation, we calculate the raw
values of a virtual cylinder that once calibrated would have a fixed value
(390 for
Top:
Data showing the fractional change in
On the right panels of Fig.
In Fig.
In Fig.
We see that the fractional change is on the order of 0.001. In other
words, the drifts of the CRDS instruments are typically about 0.1 %
over a year. The magnitude of the fractional change is larger for
methane than for carbon dioxide, by about a factor of 2.7. For
Between 2008 and fall of 2014, 47 non-isotopic Picarro instruments
were tested at what is now the MLab. The goals of this work were
to give insight into the MLab testing procedures that are also applied
to the other instruments as well as to provide an evaluation of the
tested instruments. We show that over time the instruments tend to
have more reproducible performances. However, the first
instruments of a new model tend to differ from one another than the last instruments of the previous model. This conclusion holds for CO
even though its measurement is challenging. We also see that the
results from the factory, at the MLab and in the field generally agree
well with each other; in the case of the field, the performances
stay relatively unchanged over time. The laboratory test can then be
used to prioritize the location of the instruments according to their
performances and the needs of the stations before installation. This
also shows that, for instruments that could not be tested at the
laboratory, the field estimate could be an acceptable proxy if
measured on a long enough period of time using the same protocols as
described here. We can conclude that the instruments tested are well
designed for field study (with an average of more than 80 % of
valid data over the instruments tested in the field in this
study). The troubleshooting list provided in this study is
representative only of the observed failures for the tested
instruments. Within SNO-RAMCES, a troubleshooting logbook is developed
to allow every station to consult and add failures and solutions. This
could be extended to the whole ICOS network. We would also like to
add a short list of recommendations for use of these analyzers in the
field. These recommendations are most likely valid for other
instruments as well.
Instruments should be tested in the laboratory before being on
site. Indeed, some important tests such as the temperature, pressure and
water vapor dependence tests are only done at the laboratory. It is also
convenient to be able to verify the performances of an instrument with a
standardized and recognized protocol. Measurement interval duration should be at least 10 min to
allow for stabilization and should be in any case tested for the specific
setup of the station as we have shown that this seems to be mostly station-specific and not instrument-specific. Indeed, to be able to reach the WMO
comparison goals, we need biases as small as possible for every source of
bias. Here, we aim for a difference of less than 0.05 for CO Pressure difference between the different samples should not
exceed 0.4 Calibration sequences should have at least two cycles, but in most
cases this could be enough. Calibrations need to be run regularly to follow the instrument
and setup drift. Especially after each restart of the instrument,
calibrations have to be run. Each station setup being different, we cannot
recommend a specific frequency for calibration, but we recommend that during
the first 6 months these calibrations are run at least every 2 weeks.
Then after analysis of the data, the frequency should be optimized. Despite these findings, we highly recommend carrying out a thorough
test of the instrument at the station to take into account
specificities that would lead to a needed higher number of
calibration cycles or a longer interval time.
Abbreviations.
Short-term and long-term repeatability for the three species. First panel: short-term repeatability. Second panel: long-term repeatability.
CO pressure and temperature dependency. First panel: pressure
dependency. Second panel: temperature dependency. On the right of
the lower plot, the slope (I1), intercept (I0) and the coefficient
of correlation (
Schematics of the calibration procedure at the MLab. With a measurement interval time of 20–30 min, a full sequence lasts between 5 and 8 h.
Linearity test for
Comparison with the reference instrument for
We would like to thank the technicians in the field that carry out the regular maintenance and the station PIs who control the quality of the data. For AMS, we thank the French civil service volunteers, Olivier Jossoud, Delphine Combaz and Laurine Hégo. For BIS, we thank Yvan Rush, Patricia Salin and Christophe Martin from DGA. For IVI, we thank the GLK Danish navy and the Kommuneqarfik Sermersooq (Greenland). For LTO, we thank Palmer Yao, Ismael Coulibaly, Bi Danko Raphaël Zouzou and Adama Diawara from Abidjan-Cocody University. For MHD, we thank Gerry Spain, Mick Geever and Damien Martin from Galway University and Ireland's EPA and Simon O'Doherty from Bristol University. For OPE, we thank Sébastien Conil and Maxime Simon from ANDRA. For PUJ, we thank Tuomas Laurila, Juha Hatakka and Harri Portin from FMI. For PUY, we thank Jean-Marc Pichon, Mickaël Ribeiro, David Picard and Aurélie Colomb from LaMP. Finally, for TRN, we thank Eric Parmentié from IPGP.
We also would like to thank Lynn Hazan and Amara Abbaris, who develop and maintain the station database. This study was funded by the European Commission under the EU Seventh Research Framework Programme through ICOS (grant agreement no. 211574), and ICOS-INWIRE (grant agreement no. 313169). Edited by: D. Griffith