Ceilometer lidars are used for cloud base height detection, to probe aerosol layers in the atmosphere (e.g. detection of elevated layers of Saharan dust or volcanic ash), and to examine boundary layer dynamics. Sensor optics and acquisition algorithms can strongly influence the observed attenuated backscatter profiles; therefore, physical interpretation of the profiles requires careful application of corrections. This study addresses the widely deployed Vaisala CL31 ceilometer. Attenuated backscatter profiles are studied to evaluate the impact of both the hardware generation and firmware version. In response to this work and discussion within the CL31/TOPROF user community (TOPROF, European COST Action aiming to harmonise ground-based remote sensing networks across Europe), Vaisala released new firmware (versions 1.72 and 2.03) for the CL31 sensors. These firmware versions are tested against previous versions, showing that several artificial features introduced by the data processing have been removed. Hence, it is recommended to use this recent firmware for analysing attenuated backscatter profiles. To allow for consistent processing of historic data, correction procedures have been developed that account for artefacts detected in data collected with older firmware. Furthermore, a procedure is proposed to determine and account for the instrument-related background signal from electronic and optical components. This is necessary for using attenuated backscatter observations from any CL31 ceilometer. Recommendations are made for the processing of attenuated backscatter observed with Vaisala CL31 sensors, including the estimation of noise which is not provided in the standard CL31 output. After taking these aspects into account, attenuated backscatter profiles from Vaisala CL31 ceilometers are considered capable of providing valuable information for a range of applications including atmospheric boundary layer studies, detection of elevated aerosol layers, and model verification.
The works published in this journal are distributed under the Creative Commons Attribution 3.0 License. This license does not affect the Crown copyright work, which is re-usable under the Open Government Licence (OGL). The Creative Commons Attribution 3.0 License and the OGL are interoperable and do not conflict with, reduce or limit each other. © Crown copyright 2016
Ceilometer lidars are widely used to characterise clouds
(Illingworth et al., 2007). Sophisticated cloud base height detection is found to provide
reliable estimates, with multiple cloud layers identified
(Martucci et al., 2010). Although originally
developed as “cloud base recorders”, attenuated backscatter profiles from
ceilometers can also provide information on rainfall
(Rogers et al., 1997), formation and clearance of fog (Haeffelin et al., 2010), drizzle properties (when combined with cloud radar;
O'Connor et al., 2005), and for the study of
aerosols, including elevated layers of Saharan dust (Knippertz
and Stuut, 2014), biomass burning (Mielonen et al., 2013) or volcanic ash (e.g.
Marzano et al., 2014; Nemuc et al., 2014; Wiegner et al., 2012), and particles
dispersed within in the atmospheric boundary layer (ABL) (Tsaknakis et al., 2011). Using aerosols as a
tracer, boundary layer dynamics, including mixing height and the formation
of residual layers, can be inferred from ceilometer attenuated backscatter
observations (e.g Münkel et al., 2007; Stachlewska et al., 2012; Selvaratnam et al., 2015). As they can
operate automatically for long periods without maintenance or human
intervention even in extreme climates (Bromwich et
al., 2012), they are widely deployed operationally by national
meteorological services (NMS, e.g.
Although ceilometers are regarded as the most basic automatic lidars (Emeis, 2010), they detect the location and extent of aerosol layers and can be used to derive the aerosol backscatter coefficient, provided signal-to-noise ratio (SNR) is sufficient and a careful calibration is applied (e.g. Jenoptik CHM15K; Heese et al., 2010; Wiegner et al., 2014). Observations from ceilometers are highly valuable for the evaluation of numerical weather prediction (NWP) and air-quality models (Emeis et al., 2011b) and are increasingly used in forecast verification. Several NMS and research centres are currently evaluating the potential of using ceilometer profile observations for data assimilation (Illingworth et al., 2015).
This wide range of applications requires careful quality control of the
observed attenuated backscatter to ensure reliable data for analysis. The
European COST Action TOPROF (
Emeis et al. (2011a) report that attenuated backscatter from Vaisala CL31 ceilometers portrays structures in the ABL consistent with temperature and humidity profiles observed by radiosondes and a sodar RASS system. Initial evaluation of CL31 attenuated backscatter observations for quantitative aerosol analysis (Sundström et al., 2009) suggests accuracy might be sufficient in the ranges near the instrument if certain systematic artefacts found in the profiles can be removed or accounted for. McKendry et al. (2009) find that, under clear-sky conditions, the CL31 has the capability to “detect detailed aerosol layer structure (such as fire or dust plumes) in the lower troposphere” that is consistent with the aerosol structure detected by an aerosol research lidar (CORALNet-UBC). However, comparing a Vaisala LD40 and two CL31 ceilometers, Emeis et al. (2009) show that attenuated backscatter may vary distinctly between these sensors. The differences found cannot be explained by a lack of absolute calibration as they are manifested in vertical structures rather than as a simple offset. Instrument-specific signatures may have implications for the representation of ABL structures. Emeis et al. (2009) state “internally generated artefacts from the instrument's software” could play a role, but they refrain from providing further details. While software-related artefacts might contribute to the differences, the discrepancy between the attenuated backscatter profiles observed by the two CL31 sensors tested (Emeis et al., 2009) might also be explained by the hardware-related (electronic or optical) background signal. Recent work on a Halo Doppler lidar suggests such background signal features could be corrected for during post-processing (Manninen et al., 2016).
Incomplete optical overlap can be corrected for, but uncertainties may remain. Recent research shows, for example, that the overlap function of a Lufft CHM15K is slightly temperature dependent (Hervo et al., 2016). Due to the co-axial beam design, the full optical overlap for the CL31 is reached at low ranges (Münkel et al., 2009), which can be beneficial when studying meteorological processes in the lowest part of the atmosphere, such as fog, haze, or aerosols emitted at the Earth's surface. For example, in comparison to an LD40 which reaches complete overlap only at 200 m, the CL31 has an advantage in detecting low, stable layers (Emeis et al., 2009). Although Vaisala suggests that the attenuated backscatter profile is reliable down to the first range gate, Sokół et al. (2014) document a distinct local minimum in CL31 attenuated backscatter observations at the fourth range gate persisting throughout their entire observational campaign. As others have found artefacts in CL31 profiles below 70 m (e.g. Martucci et al., 2010; Tsaknakis et al., 2011), these lowest ranges are often excluded from analysis. Sundström et al. (2009) evaluate the applicability of CL31 observations for quantitative aerosol measurements and conclude that the artefacts in the range gates near the instrument are a major source of uncertainty. Van der Kamp (2008) smooths out systematic artefacts by strong vertical averaging; however, this removes the possibility of identifying any atmospheric features close to the surface.
Various techniques have been developed to infer the mixing height from the shape of the attenuated backscatter profiles from ceilometers (Emeis et al., 2008; Haeffelin et al., 2012). While detection algorithms vary, all methods exploit the fact that aerosol concentrations (and atmospheric moisture if boundary layer clouds are absent) are typically significantly higher in the ABL compared to the free atmosphere above. This causes a distinct decrease in attenuated backscatter at the boundary layer top, provided that the SNR is sufficiently large up to this height.
A series of studies have successfully used CL31 observations to detect mixing height (e.g. Münkel et al., 2007; van der Kamp and McKendry, 2010; Eresmaa et al., 2012; Sokół et al., 2014; Tang et al., 2016), often reporting an increased performance under convective conditions that ensure the backscattering aerosols are well dispersed. However, Eresmaa et al. (2012) report that fitting an idealised profile to the observed attenuated backscatter from a CL31 may be challenging where noise levels are high. As the CL31 operates with a very low-powered laser, its noise levels may be higher than that found for other ALC systems (cf. Jenoptik CHM15K; Haeffelin et al., 2012). Madonna et al. (2015) evaluate the profiling ability of several ALCs from different manufacturers (i.e. Jenoptik CHM15K, Vaisala CT25K, and Campbell CS135s) against a MUSA advanced Raman lidar during night-time. They conclude that the attenuated backscatter coefficient generally is in good agreement with the reference measurement for the CHM15K, while the CS135s shows good agreement only for small values and the CT25K tends to underestimate, which may be related to the overall lower SNR of the latter two sensors. If noise levels are too high within the ABL, as reported e.g. by Haeffelin et al. (2012) for a case study using a Vaisala CL31 ceilometer at the SIRTA site near Paris, the signal might not be sufficient to detect the top of the ABL. De Haij et al. (2006) apply an SNR threshold to restrict observations from a Vaisala LD40 ceilometer to be used for mixing height detection. Such filtering based on SNR diagnostics presents a useful tool to differentiate measurements containing significant atmospheric signal from observations dominated by instrument noise and atmospheric noise induced by solar radiation.
Neither the SNR nor the noise inherent in each profile is provided in the
output of ALCs. Xie and Zhou (2005) propose a method for
SNR calculations for lidar observations whereby the signal profile is
approximated by a linear fit to the readily averaged profile along set range
bins and assigning the deviations from that fit to the noise. Markowicz et al. (2008) apply this
method to observations of a Vaisala CT25K averaged over 200 s. These SNR values
indicate that the observations are only reliable within the ABL (absence of
clouds) and it is stated that an SNR
Despite the evidence that attenuated backscatter profiles are a complex data product that might have to be carefully evaluated before being used to draw conclusions on the probed atmosphere, no guidelines are available to ensure systematic QAQC. This study documents the important processing steps that should be considered when analysing attenuated backscatter profiles from Vaisala CL31. Observations from three ALC networks (Sect. 2) are used to illustrate relevant data processing aspects (Sect. 3). Depending on the firmware version, the CL31 instrument internal processing may introduce certain artefacts that should be accounted for if the attenuated backscatter is required for analysis. It is shown how the signal strength can be used for quality assurance (Sect. 4) and findings are summarised in the form of recommendations for the processing of CL31 profile observations (Sect. 5).
The Vaisala CL31 transmits a very short pulse of 110 ns (corresponding to an
effective pulse length of about 16.5 m; e.g. Weitkamp, 2005). The receiver uses an APD detector to record the returned signal. The instrument
oversamples the backscattered signal at a temporal rate, which corresponds to
the range resolution setting. The reported range
The spectral wavelength of the laser diode used in the Vaisala CL31 is
905
The detector of the CL31 responds to the backscattering of the laser pulse
from molecules, aerosols, rain drops, and both liquid and ice cloud
particles. It also responds to noise originating from both external (e.g.
daytime solar radiation) and internal (e.g. electronic) sources. The
hardware-related noise is larger than the Rayleigh signal associated with
clear air so that the latter is too small to be distinguished. Vaisala
states that the variance of the electronic noise signal is
range independent. The background light from solar radiation increases the
current through the APD, but as the amplifiers are AC coupled, the
relatively slowly varying solar signal (almost DC) does not get to the A/D
converter. (The AC-coupling time constant is 1 ms; i.e. the AC coupling
works as a high-pass filter with 159 Hz corner (
Internal averaging interval applied in different CL31 firmware
versions as a function of range
The Vaisala CL31 firmware has been modified over time along with certain
developments in the hardware, i.e. the receiver (CLR) and engine board (CLE)
where the internal processing takes place. These updates have resulted in
the creation of a range of firmware versions. For CLE311
Vaisala CL31 ceilometer specifications of sensor hardware, firmware, H2 noise setting, and resolution selected by the user. The term “H2” is discussed in Sect. 3.2.
Observations from three ceilometer networks (Table 2) are used in this study to illustrate aspects of the data acquisition and
processing of Vaisala CL31. The London Urban Micromet Observatory (LUMO;
Long-term observations are available from four CL31 ceilometers with different generations of hardware and various firmware versions. Over time, the LUMO network firmware versions have changed from the first LUMO sensor deployed in 2006 with version 1.56 (Table 2). Sensors A and B are the old hardware generation with the CLE311 board, as is the Met Office sensor W, while LUMO sensors C and D and the SIRTA sensor S have engine boards CLE321. For both sensors A and B the transmitter has been upgraded from CLT311 to CLT321 during their operation, and for sensor S the transmitter CLT321 was replaced by a spare part of the same generation. While the LUMO sensors are set to acquire data every 15 s with a vertical resolution of 10 m, data from the Met Office ceilometer have a resolution of 30 s and 20 m, and the SIRTA ceilometer captures data every 2 s with a range resolution of 5 m. Analysis presented here uses block averages over 30 s and 15 m of the SIRTA ceilometer data.
Range histograms for 24 h of observations from Vaisala CL31
ceilometers operating with firmware versions (1.56–2.03) on different
clear-sky days. Sensor ID in brackets (see Table 2
for settings, e.g. H2
The backscattered signal detected by an ALC generally consists of actual
signal contributions from atmospheric attenuation, the atmospheric background
signal associated with scattered solar radiation, and the instrument-related
background signal (Cao et al., 2013). Here, “background signal” is used to
describe systematic contributions from solar radiation or instrument
components (including hardware and software). The CL31 measurement design
accounts for the temporal
noise bias induced by
varying solar radiation by introducing a variable zero-bias level
(Sect. 2). The atmospheric background signal still
contributes to the noise in the profile. On average, the RCS (labelled “range and sensitivity normalised attenuated
backscatter” in CL31 output) is inherently corrected for the impact of
atmospheric background signal
The instrument-related background signal
Signal
For data collected with firmware 1.72 or 2.03, no cosmetic shift is
incorporated (
A suitable method to identify discrepancies in the profile shape is to create signal-range histograms (Fig. 1a) using 24 h (or more) of data. The most obvious effect revealed by the range histograms is a step change in the width of the distributions at 2400 m evident for all firmware versions, apart from 1.72 and 2.03. This step change is introduced by the averaging of the sampled signal that is applied internally by the instrument's firmware (Sect. 2, Table 1). The decrease in averaging time for range gates < 2400 m performed for earlier firmware increases the signal noise (see Sect. 4.2). Data acquired with version 1.72 or 2.03 are more consistent across all range gates as the whole profile is treated equally with an internal averaging interval of 30 s.
Long-term median vertical profiles of range-dependent,
instrument-related background
The range histograms (Fig. 1a) show the impact of
the incomplete background correction; i.e. instrument-related background
signal and cosmetic shift are not accounted for. Both cause a systematic
pattern in the observed profiles, illustrating the range dependence of the
background signal. The cosmetic shift is particularly strong for version
1.71. To capture both background effects, profiles are analysed during times
when atmospheric variations are expected to be small and instrument
conditions are stable (Fig. 2).
The night-time profile climatology (Fig. 2) reveals
a small temporal variability with a seasonal cycle (amplitude
For sensor B (Fig. 2b), the change in transmitter
from CLT311 to CLT321 also altered the profile of the background signal,
mainly by introducing a systematic pattern along the range. A wave-like
structure appears superimposed over the random noise for ceilometer B when
operating with transmitter CLT321 (Fig. 2b). A
similar effect is detected in observations from ceilometer C in general
(Fig. 2c) and to some extent in ceilometer S after
the transmitter was changed (Fig. 2e). Such
“ripple” patterns are introduced by a physical effect which overlays a
vertically alternating positive and negative bias on top of the signal
noise. While this wave-type bias tends to be similar for successive profiles
(regions with positive and negative amplitude overlap), it is not entirely
constant over the course of a day because it is slightly affected by
attenuation by clouds and ABL particles. As shown, this ripple is
sensor specific (e.g. higher frequency detected for sensor B than C;
Fig. 2b, c). While ripple may occur for ceilometers
with both CLE311
Assuming the actual information content related to atmospheric backscatter
is low above the ABL in the selected night-time profiles (i.e. signal
contribution is small cf. noise in the absence of clouds), the median
climatology grouped by firmware plus transmitter configuration
(Fig. 2, right) describes the background signal
composed of instrument-related background signal and potential cosmetic
shift (i.e.
The profiles for each sensor by firmware version
(Fig. 3a) show that the complete background signal
may be similar for sensors with the same generation of hardware (e.g.
profiles of A and B, both with CLE 311
The seasonality evident in the time series (Fig. 2c, d, left) is related to the laser heat sink temperature which is used to further classify the background signal of sensors A–D into three sub-classes (Fig. 3b, see legend). Profiles are only analysed above 2400 m as climatological measurements within the (sub-)urban ABL of London and Paris are inappropriate; if long-term measurements are available where ABL aerosol and moisture content are low (e.g. mountain sites), the climatology approach may provide valuable insights at lower range gates.
To evaluate whether the night-time climatology is a suitable basis to assess the
background signal profile, test measurements for four LUMO sensors with
recent firmware and hardware configurations are conducted (Fig. 3c). The ceilometer window is covered by a Vaisala termination hood to mimic full
atmospheric attenuation (i.e. only little signal is backscattered to the
receiver which is below the sensitivity of the detector). The recorded
signal should represent internal contributions (e.g. background signal)
only. To eliminate transient behaviour in the lowest range gates the hood
measurements are taken for 30 min periods. Later tests indicate observations
at range < 50 m may require about 1 h to settle to a characteristic
value (Fig. 4), which is in agreement with CeiLinEx
CL51 ceilometer termination hood measurements (Frank Wagner, DWD, personal
communication, 2015;
Average termination hood profiles are compared to night-time climatology profiles from the same laser heat sink temperature classes. For most sensors and firmware, the median night-time climatology agrees very well with the profile observed by the termination hood measurement (Fig. 3c). Only for ceilometer A (firmware 1.71) does the termination hood measurement have a slightly different shape, albeit with a similar order of magnitude. As there are no data available from the climatology approach for ranges below 2400 m, profiles are assumed to be constant up to this range. While this results in an obvious discrepancy between the climatology-derived background and the termination hood profiles (Fig. 3c), implications of this assumption are greatly reduced after range correction is performed (Fig. 3d–e). Although uncertainties remain regarding the profiles of background signal below a range of 2400 m, termination hood reference measurements give confidence that the night-time climatology measurements are not significantly influenced by backscatter from atmospheric particles and hence provide reasonable estimates of the background signal. This finding is extremely useful as it allows for the background signal of ceilometer sites that were operated in the past or that are difficult to access (e.g. termination hood measurements are unfeasible) to be evaluated based on the observed profile data alone.
Logarithm of range-corrected signal reported RCS
Vaisala states (firmware release note) that no deliberate cosmetic shift is
implemented in versions 1.72 and 2.03. Given that background signals from
the earlier release versions are much closer to 0 or even positive, it
can be concluded that there is no (or negligible) cosmetic shift in versions
1.56, 1.61, 2.01, and 2.02 and the complete background signal
For firmware with no significant cosmetic shift, the atmospheric
contribution to the background correction is negligible so that a static
correction over time can be applied defined by the night-time profiles:
As discussed, background profiles from sensors C and D have a small
temperature dependence (Fig. 3b); however, the
background signal of these sensors has an overall very small magnitude so
that this thermal effect is considered negligible in the proposed correction
(Eq. 3). Data with cosmetic shift (i.e. those
collected with firmware 1.71) show strong diurnal variations in the signal
background in response to background solar radiation. This indicates some
contribution of the atmospheric background is retained in observations from
this firmware version as the dynamic “zero-bias level” is effectively
different from 0 (Sect. 2). Because this is
performed internally by the firmware, the exact contribution of the
atmospheric background signal is not available for post-processing use.
However, it can be approximated by the average signal
The derived background correction
All ceilometers tested here have a non-zero background profile, which confirms analysis by the Met Office (termination hood measurements and case study analysis) giving a negative background for other CL31 sensors in their network (Mariana Adam, Met Office, personal communication, 2014–2015). This creates additional challenges when deriving the aerosol backscatter coefficient from such measurements (Mariana Adam, Met Office, personal communication, 2015). For firmware versions without (or negligible) cosmetic shift, the background signal consists solely of the instrument-related contributions which may be small. Implications of these instrument-specific variations might be limited for observations within clouds or in the ABL, where backscatter values tend to be large and mostly positive. However, the instrument-related background signal can reach significant values that may dominate any signal differences expected at the top of the ABL. The cosmetic shift in version 1.71 clearly affects observations within the ABL (Sect. 4.2). Note that the cosmetic shift and instrument-related background signal should be carefully evaluated before using noise for quality-control purposes, including absolute calibration and SNR calculations (Sect. 4).
For a given concentration of atmospheric scatterers (cloud, aerosol,
molecules), the strength of the backscattered signal returned to the
ceilometer telescope and detector decreases by the square of the range
The signal
When clouds are detected, the cloud signal is range-corrected using
Eq. (5) for range gates where cloud is determined
to exist. To create a fully range-corrected signal from such observations
for the whole vertical profile (according to
Eq. (5), i.e. as if run with the setting “Message profile noise_h2 on”) in the
absence of clouds, the scale factor needs to be reversed and the range
correction applied to the observations above
The range histograms of the range-corrected signal (Fig. 1b, c) illustrate the increase in signal variability with range. After applying the full range correction (Eq. 7) to observations from a CL31 operated with “Message profile noise_h2 off” (rightmost panel in Fig. 1), the variability of the signal is height-invariant above the ABL (Fig. 1a), while the expected increase is found in the range-corrected signal (Fig. 1b, c); i.e. it has the same signature as if it were recorded with the setting switched on.
The receiver field of view reaches complete optical overlap with the emitted laser beam at a certain distance above the instrument. This overlap depends on instrument design. Overlap correction functions can be applied to partly account for this effect, with dimensionless multiplication factors determined empirically (e.g. Campbell et al., 2002). The overlap correction may either be performed by firmware or during post-processing. Uncertainty remains for observations at the closest range gates (e.g. Vande Hey, 2014; Hervo et al., 2016).
Applying an optical overlap correction
Vaisala ceilometers have a single-lens, coaxial beam setup
(Münkel et al., 2009). For the CL31, complete optical
overlap is reached at about 70 m from the instrument
(Fig. 5) and an overlap correction is performed by
the firmware (i.e.
Although Vaisala suggests that the attenuated backscatter profile is reliable down to the first range gate, Sokół et al. (2014) document a distinct local minimum in CL31 attenuated backscatter observations at the fourth range gate persisting throughout their whole observational campaign. As others have found artefacts in CL31 profiles below 70 m (e.g Martucci et al., 2010; Tsaknakis et al., 2011), these lowest layers are often excluded during processing. As noted, van der Kamp (2008) smoothed out systematic features by strong vertical averaging, but this removes the possibility of identifying any atmospheric features close to the surface. Without correction, these artefacts may cause detection of significant gradients when examining profiles to diagnose mixing heights or top of the ABL. Artefacts in the first 70 m could be related to the incomplete optical overlap (Sect. 3.3) but are more likely associated with a hardware-related perturbation and a correction introduced by Vaisala to prevent unrealistically high values in the near range when the window is obstructed.
Given the primary function of cloud base height detection, Vaisala CL31 firmware addresses effects causing extremely high backscatter values outside of clouds. Under severe window obstruction (e.g. leaf on window), values in the first range gates can be unrealistically high. A correction is applied to restrict the backscatter profile in the ranges closest to the instrument. At times, this correction introduces extremely small values at ranges < 50 m that are clearly offset from the observations above this height. In addition to this artefact from the obstruction correction, for some sensors, backscatter values in the range of 50–80 m are slightly offset by a hardware-related perturbation. Both artefacts from the obstruction correction and hardware-related perturbation do not impact cloud detection, vertical visibility, or boundary layer structures (> 80 m). It is only for attenuated backscatter closer than 90 m that these artefacts need to be accounted for. The issues are not firmware specific apart from versions 1.72 and 2.03, in which the artefacts of obstruction correction and hardware-related perturbation have been mostly removed. These near-range artefacts are expected to be consistent in time for data collected with older firmware.
Manufacturer-deduced overlap function of Vaisala CL31 ceilometers using firmware versions 1.71, 1.72, 2.02, or 2.03 (older versions used an overlap function with 5 to 10 % lower overlap values). The function, applied in the lowest range gates above the instrument, is derived from laboratory measurements and field observations under homogeneous atmospheric conditions. During the production process, the applicability of the overlap function is verified for each unit. Due to the stable instrument conditions (e.g. low internal temperature variations), Vaisala expects no systematic variations of the overlap function. The error is stated to be below 10 %.
Median range-corrected signal reported RCS
To evaluate the effect of the obstruction correction and hardware-related
perturbation, profiles of the range-corrected reported signal in the lowest
90 m are normalised by the value at 100 m (RCS(
Based on the median climatological profiles (Fig. 6), a near-range correction is proposed to reduce the impact of the obstruction correction and hardware-related perturbation. Only profiles that roughly match the general shape of the climatology are corrected; i.e. if strong vertical gradients in the signal are observed (e.g. descending fog) the near-range correction is inapplicable. However, these near-range artefacts are usually small compared to the physical processes influencing the attenuated backscatter across the profile.
Given that all sensors tested have a distinct peak at a certain range gate (Fig. 6a) this peak is used to indicate whether a correction should be applied. The inverse approach could correct observations with a strong local minimum at the fourth range gate as reported by Sokół et al. (2014). The aim is to apply the near-range correction only to profiles with a pronounced peak value that appears physically unreasonable. First, the range gate with the peak is identified from the climatology (fifth range gate for LUMO sensors). Second, the peak strength is defined as the ratio of the range-corrected signal reported at this range gate to that reported at the adjacent gates (i.e. fourth and sixth for LUMO sensors). If both these peak-strength indicators of a given profile are at least 25 % as strong as the peak-strength indicators of the climatology profile, the values of this profile in the near range (< 100 m) are divided by the median climatology profile (Fig. 6b). Profiles affected by the obstruction correction, i.e. with clearly offset values in the first four range gates, are treated separately. If the first peak-strength indicator (i.e. the one below the peak) is at least 50 % as strong as the respective indicator of the climatology of this regime (Fig. 6c) and the value at the range gate of the peak is greater than the values in the two range gates above, the respective median climatology profile is used for the correction (Fig. 6c).
Correction functions can help to reduce the processing artefact due to the
obstruction correction and the hardware-related offset as demonstrated for
several case studies (Fig. 7; LUMO ceilometers A–D,
see Table 2). Observations taken with firmware
versions < 1.72 (for systems running with engine board plus receiver
combination CLE311
Observations from four Vaisala CL31 ceilometers from the LUMO network (Table 2) over the first 200 m range: (i, iii, v, vii) logarithm of the range-corrected signal reported RCS (a. u.); (ii, iv, vi, viii) as in (i, iii, v, vii) but after application of correction for near-range artefacts associated with the obstruction correction and a hardware-related perturbation (see Fig. 6). Sensors were operating with firmware 1.61 (A, B) and 2.01 (C, D) on (i, ii) 10 January 2014, (iii, iv) 15 January 2014, (v, vi) 6 January 2013, and firmware 1.72 and 2.03 on (vii, viii) 13 March 2016. White areas indicate values outside of the range of values selected (see colour legends). Note that data are not absolutely calibrated.
Vaisala introduced a correction for the near-range artefacts that proves efficient in dry conditions (Fig. 6d); however, if attenuation is increased due to hygroscopic growth, the peak at the fifth range gate is still evident in the normalised RCS profile (Fig. 7vii). Applying the near-range correction proposed for observations from earlier firmware version (as used for Fig. 7ii, iv, vi), the artefacts could still be removed (Fig. 7viii, ceilometers A and B), but it could also result in an overcorrection (Fig. 7viii, ceilometers C and D). Note that this approach can only be tested on sensors for which a historic dataset of measurements with older firmware versions is available to calculate the respective correction profiles (Fig. 6). Given that the near-range correction introduced by Vaisala in versions 1.72 and 2.03 is not sufficient in moist conditions with gradients along the profile (Fig. 7vii), it was proposed to Vaisala to remove their correction again so that the near-range correction can be applied during post-processing.
The range-corrected attenuated backscatter
The lidar constant
Given that noise is a critical component of the attenuated backscatter
recorded, data with values below a certain SNR are
unlikely to contain sufficient information about the state of the
atmosphere. Where high-resolution observations are obtained, rolling spatial
(along-range) and temporal averaging increases the signal contribution
relative to the noise. For every range gate
The quality of range-corrected attenuated backscatter can be evaluated by
comparison to the noise floor. The latter represents variations associated with
electronic and optical noise and noise introduced by the solar background
light. If no high cirrus clouds are present, it is assumed the signal from
the very highest range gates contains only noise (i.e. atmospheric signal
contribution is negligible). In this case, the noise floor
Observations from Vaisala CL31 sensor D
(Table 2) at top range gates (7410–7700 m) with
cirrus during the early evening on 1 February 2013:
Acceptance (%) based on Welch's
To ensure that profiles used for the calculation of
If RV
CL31 observations on 24 July 2012 (rows 1 and 2), 22 June 2014
(rows 3 and 4), and 29 June 2016 (row 5) from four sensors with firmware
version in brackets: A (1.61), B (1.71), C (2.02), and S (2.01); see
Table 2.
The SNR is calculated from smoothed, non-range-corrected attenuated backscatter
(Eq. 12) and the noise floor
The impact of averaging, background correction, and noise filtering on observations taken by different sensors and firmware versions is illustrated based on three case study days (Fig. 10): 24 July 2012 with clear-sky conditions comparing sensor A running with firmware version 1.61 (row 1) and to sensor C with firmware 2.02 (row 2), 22 June 2012 with some boundary layer clouds present comparing sensor B with 1.71 (row 3) and sensor C with firmware 2.02 (row 4), and 29 June 2015 with a few isolated medium- and high-level clouds showing observations from sensor S with 2.01 (row 5). The range-corrected attenuated backscatter reported (Fig. 10a) is quite noisy in all sensor observations and the evolution of the ABL is difficult to discern. When the moving average is applied (Fig. 10b), the signal contribution clearly increases so that aerosol layers can be identified visually. However, the contrast between ABL and the clear air above varies greatly with sensor and firmware version. While the ABL reveals distinct attenuated backscatter signatures for data from sensor C with firmware 2.02 (rows 2 and 4), the values in the free troposphere are elevated for sensors A (row 1) and S (row 5). This is explained by the different profiles of background signal inherent in these observations (Fig. 3a): sensor C has a small and slightly negative background signal which barely affects observations above the ABL, while both sensor A (firmware 1.61) and sensor S (2.01) have a positive background signal leading to an overestimation of signal below about 5000 m. Even more severe is the impact of the cosmetic shift inherent in observations from sensor B (1.71; row 3), which reduces the signal significantly even within the ABL. For the example shown, average values become negative below the boundary layer clouds so that no mixing height detection algorithm would be able to derive relevant statistics.
The described artefacts can mostly be accounted for by the proposed background correction (Eq. 1–4; Fig. 10c). While it can help to improve the contrast at the boundary layer top for sensors A (row 1) and S (row 5), it can revert the cosmetic shift in data from sensor B (row 3). In the latter case, the background correction can increase data availability. It should be noted that the systematic ripple effect (sensors B and C; see Sect. 3.1) becomes apparent after the background correction. Although the ripple is somewhat coherent and not truly random, affected areas above the ABL can still be successfully masked by the SNR filter (Fig. 10d). For all sensors, the statistical threshold helps to distinguish data with significant information content (compare Fig. 10c, d) so that quality can be assured for later applications (e.g. mixing height detection). Still, some significant noise may remain near the ABL top for the older generation of hardware running with firmware 1.xx (row 1 and 3). It can be concluded that data quality of sensors of the recent hardware generation, i.e. those operating firmware version 2.xx (here sensors C and S) are clearly superior to older generations (sensors A and B).
Ceilometers are valuable instruments with which to study not only clouds but also the ABL and elevated layers of aerosols. Vaisala CL31 sensors provide good-quality attenuated backscatter. While their cloud base height product might be readily useful, to understand the profiles of attenuated backscatter the user needs to be aware of the instrument model's specific hardware and firmware. The following sections summarise aspects useful to consider in the post-processing of CL31 ceilometer attenuated backscatter profiles.
By taking into account these instrument-specific aspects of the CL31 profile observations, data quality and availability can be improved. If data are collected according to best practice, as recommended (Sect. 5.3), issues are being corrected for in the post-processing (e.g. applying the proposed methods) and sensors are carefully calibrated, then the attenuated backscatter observations might prove useful for NWP model verification and evaluation, and potentially even for data assimilation.
Initial internal averaging of the sampled ceilometer signal is applied over selected time intervals that depend on the range and the user-defined reporting interval for firmware versions < 1.72, 2.01, and 2.02. Data acquired with firmware 1.72 or 2.03 are more consistent than earlier versions because the whole profile (at all range gates) is treated equally with an internal averaging interval of 30 s.
If the user-defined reporting interval is shorter than 30 s, consecutive profiles partly overlap in time and are hence not completely independent.
When averaging several profiles, a discontinuity is evident at around both 4940 and 7000 m for all sensors and firmware versions. These regions of increased noise are introduced by the data storage procedure of the firmware, which slightly changes its operating mode after a certain number of gates have been collected. Care should be taken when looking at gradients or statistics near these ranges.
Depending on firmware version, a “cosmetic shift” is applied to the attenuated backscatter profiles. This shift should be reversed before using any part of the profile for analysis. Of the firmware tested, the cosmetic shift appears to be negligible for all versions except for 1.71, in which a strong negative shift is applied to the observations.
In addition, a range-dependent, instrument-related background signal is inherent in the signal reported, altering the profiles systematically. The background signal values (instrument-related background signal plus potential cosmetic shift) tend to be either predominantly positive or predominantly negative in ranges below 6000 m and to switch sign between about 6000 and 7000 m.
Both the range-dependent instrument-related background signal and the cosmetic shift applied may cause issues for studying the ABL because signal differences expected at the ABL top may be obliterated or the signal reduced too strongly for successful mixing height detection.
Molecular scattering at the instrument wavelength is very weak, typically below the sensitivity of the instrument.
The CL31 measurement design accounts for temporal variations in solar radiation by introducing a variable zero-bias level so that the atmospheric background signal is inherently accounted for in the signal reported. However, solar radiation still contributes to the random noise.
In the absence of clouds, rain, or elevated aerosol layers, the recorded signal includes the instrument-related background signal plus potential cosmetic shift and noise associated with both the instrument (electronics and optics) and the solar background radiation. Instrument-related background signal and cosmetic shift should be carefully evaluated before using noise for quality-control purposes.
For some instruments, a “ripple” effect is detected that superimposes a
wave-type structure over the random noise. For the sensors evaluated, this
was found in two generations of the engine board plus receiver (CLE311
Vaisala instruments have a setting (“Message profile noise_h2 off”) that restricts the range correction to the signal in the lower part of the profile up to a set critical range. It is implicitly assumed that most data at ranges beyond this critical range contain only noise. If no clouds are present in the profile, then the signal is simply multiplied by a constant, range-invariant scale factor. Where clouds are detected, the signal is actually range-corrected as usual, but only for range gates where cloud is determined to exist.
Several artefacts may be found in the lowest range gates close to the instrument. The co-axial beam design of the CL31 ceilometer allows complete overlap to be reached at 70 m. Below this range, an overlap correction is applied internally by the sensor.
In addition to the overlap correction, Vaisala applies another correction to observations from the first few range gates to avoid exceptionally high readings when the ceilometer's view is obstructed (e.g. a leaf on the window). At times, this obstruction correction introduces extremely small values at ranges < 50 m that appear unrealistically offset from the observations above this height. Attenuated backscatter values may also be slightly offset in the range of 50–80 m which can be explained by a hardware-related perturbation.
Although CL31 output is labelled as attenuated, range-corrected backscatter, the absolute calibration might not be accurate enough for use in meteorological research. The stratocumulus or liquid cloud calibration (O'Connor et al., 2004) can be used to determine the instrument-specific lidar constant based on external properties. This allows absolute calibration to be performed during post-processing. Data gathered with instruments that cause a strong electronic background signal and/or with firmware that applies the cosmetic shift (version 1.71) should be corrected for these effects before the calibration is applied. Note that absolute calibration is included for completeness but is not addressed here.
To create a fully range-corrected signal from data gathered with the setting “Message profile noise_h2 off” for the whole vertical profile (as if the setting “Message profile noise_2 on” had been used) in the absence of clouds, the scale factor needs to be reversed and range correction applied to the observations above the critical range.
A climatology of night-time profiles can be used to determine the background correction that is required to account for the instrument-related background signal and potential cosmetic shift. A comparison with termination hood measurements proves the nocturnal climatology accurately describes the background signal. Thus, the two can be considered equivalent and the background correction can be determined through either termination hood reference measurements (e.g. if profile observations are not available for a long time) or the climatology approach (e.g. if historical data are analysed or ceilometer site access is difficult). No reliable information can be derived from the climatology technique below about 2400 m given the presence of the ABL. However, termination hood profiles show the magnitude of the instrument-related background signal below this range is rather small after range correction so the profile can be assumed as range invariant in this region.
The cosmetic shift has temporal variation through the day, indicating that an influence of the atmospheric background is retained during internal processing. This effect can be accounted for in the background correction by including an offset based on average observations in the top range gates.
The artefacts related to obstruction correction and hardware-related perturbation are mostly accounted for in versions 1.72 and 2.03, but small effects remain under situations with considerable attenuated backscatter. Hence, removal of this correction in the next firmware update to allow for consistent corrections to be applied during post-processing was recommended to Vaisala. Data from earlier versions (and probably later versions) need to be corrected during post-processing if observations from the near range are to be analysed; a correction procedure has been proposed based on climatological statistics of well-mixed atmospheric profiles.
As the noise in the background-corrected signal is independent of range, it can be determined using the top-most range gates in the profile where the contribution of real atmospheric scattering is negligible in the absence of high cirrus clouds. The noise floor, taken as the mean plus standard deviation of the background-corrected signal (before range correction is applied), is calculated across moving time windows over those top range gates with the mean generally negligible after background correction. Regions containing significant aerosol or cloud should be excluded. They are efficiently masked based on the relative variance.
To increase the SNR, a moving average is calculated for
the non-range-corrected attenuated backscatter across set windows in range
and time. The relation of these smoothed statistics and the noise floor
defines the SNR which may be used to mask observations where the noise exceeds
the actual information content of atmospheric signal. A suitable SNR threshold
to distinguish the signal from the noise region is estimated based on
Welch's
Data quality and SNR of sensors running with engine board CLE321, receiver
CLR321, and firmware version 2.xx are superior to those of the old
generation (CLE311
In response to results presented here and discussions within the TOPROF
community, Vaisala released two recent firmware versions: 1.72 for sensors
running with older generation hardware (engine board CLE311 and receiver
CLR311) and 2.03 for sensors running with newer-generation hardware (CLE321
Assuming the sensors evaluated in this study are representative of CL31
ceilometers in general, the following recommendations are made for the
operation of Vaisala CL31.
Operate with the setting “Message profile noise_ h2 on”. A reporting interval (temporal resolution) of at least 15 s is recommended
(despite down to 2 s being possible). It is advised to operate sensors with engine board CLE321 If only older hardware (CLE311 The instrument-related background signal should be carefully evaluated for
all sensors and firmware versions. This can be achieved based on night-time
climatology statistics or termination hood measurements. Correction of the
range-dependent background signal may improve the contrast between the ABL
and the clearer air above. For data gathered with firmware 1.71, the cosmetic shift can be corrected
based on a combination of background signal profile estimates and average
attenuated backscatter across the profile's top range gates in the absence
of cirrus clouds. If information close to the sensor (< 100 m) is of interest,
near-range artefacts should be corrected in historical data collected with
firmware versions 1.54, 1.61, 1.71, 2.01, or 2.02. This correction might
generally not be necessary for data gathered with firmware 1.72 or 2.03;
however, it was found to yield some improvement under moist conditions. Given the impact of both hardware and firmware on attenuated backscatter
profiles from CL31 ceilometers, any publication of such data should clearly
state relevant details on hardware generation and firmware versions used, if
any changes to the setup were made during the measurement period analysed,
and post-processing undertaken.
Data of Vaisala CL31
ceilometers used for this study can be accessed from the different sources:
Data from the LUMO network can be requested online (Grimmond, 2016). Data from the Meteo France ceilometer at the SIRTA site can be downloaded from
SIRTA (2016), using the “Download tool” menu, selecting “search by
instrument”, “backscatter LIDAR” category and finally “Ceilometer CL31”. Met Office data are available from the Centre for Environmental Data
Analysis (Met Office, 2015).
Simone Kotthaus is involved in maintaining the LUMO measurement network, performed all analysis, developed correction procedures, and wrote the main parts of the manuscript. Ewan O'Connor was involved in the development of the background correction, the calculation of SNR, and discussion on range correction and provided useful comments to the manuscript. Christoph Münkel provided information on sensor specifics and internal processing procedures, wrote parts of the manuscript, and provided the TOPROF firmware versions tested in this study. Cristina Charlton-Perez provided CL31 observations from the Met Office sensor, contributed to writing the manuscript, and was involved in discussions on sensor specifics, background correction, range correction, and SNR calculations. Martial Haeffelin provided CL31 data from SIRTA, was involved in discussions on background correction, near-range correction, SNR calculation, and terminology, and provided useful comments to the manuscript. Andrew M. Gabey developed the approach for calculating an appropriate SNR threshold. C. Sue B. Grimmond provided CL31 measurements from the LUMO measurement network, was involved in discussions on instrument specifics and all data processing aspects, and provided useful comments to the manuscript.
This study was funded by H2020 URBANFLUXES, NERC ClearfLo (H003231/1), NERC Airpro (NE/N00700X/1), NERC//Belmont TRUC (NE/L008971/1 G8MUREFU3FP-2201-075), European Cooperation in Science and Technology (COST) action “TOPROF”: ES1303, King's College London, and University of Reading. We thank KCL Directorate of Estates and Facilities, ERG/LAQN, and RGS/IBG for providing sites and other support and the many staff and students at University of Reading and KCL who contributed to data collection. Met Office, Meteo France, and the SIRTA observatory are kindly acknowledged for providing observations. We would like to acknowledge the useful discussions within the TOPROF community (especially with Mariana Adam, Met Office; Frank Wagner, DWD; and Maxime Hervo, Meteo Swiss). We thank all anonymous reviewers for their comments and helpful suggestions. Edited by: A. Sayer Reviewed by: four anonymous referees