The Cloud Aerosol Lidar and Infrared Pathfinder Satellite
Observations (CALIPSO) mission released version 4.00 of their lidar level 1
data set in April of 2014, and subsequently updated this to version 4.10 in
November of 2016. The primary difference in the newly released version 4
(V4) data is a suite of updated calibration coefficients calculated using
substantially revised calibration algorithms. This paper describes the
revisions to the V4 daytime calibration procedure for the 532 nm parallel
channel. As in earlier releases, the V4 daytime calibration coefficients are
derived by scaling the raw daytime signals to the calibrated nighttime
signals acquired within a calibration transfer region, and thus the new V4
daytime calibration benefits from improvements made to the V4 532 nm
nighttime calibration. The V4 calibration transfer region has been moved
upward from the upper troposphere to the more stable lower stratosphere. The
identification of clear-air columns by an iterative thresholding scheme,
crucial to selecting the observation regions used for calibration, now uses
uncalibrated 1064 nm data rather than recursively using the calibrated 532 nm data, as was done in version 3 (V3). A detailed account of the rationale
and methodology for this new calibration approach is provided, along with
results demonstrating the improvement of this calibration over the previous
version. Extensive validation data acquired by NASA's airborne high spectral
resolution lidar (HSRL) shows that during the daytime the average difference
between collocated CALIPSO and HSRL measurements of 532 nm attenuated
backscatter coefficients is reduced from
The Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP), on-board the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO) satellite, has been providing a near-continuous record of high-resolution vertical profiles of clouds and aerosols since the summer of 2006. Launched 28 April 2006, CALIPSO is an integral part of the NASA's Afternoon (A-Train) constellation, working in tandem with other Earth observing satellites to probe the nature and influence of clouds and aerosols on the global climate system (Winker et al., 2010). CALIOP is a dual wavelength, polarization-sensitive elastic backscatter lidar powered by an Nd : YAG diode-pumped laser that makes range-resolved measurements of the total backscatter intensity at 1064 nm and the 532 nm backscatter intensities in planes oriented parallel and perpendicular to the polarization plane of the transmitted laser beam (Hunt et al., 2009). Among the first of its kind, and having delivered the longest duration of continuous on-orbit operations, CALIOP provides unique insights into the vertical distribution, morphology and variability of clouds and aerosols (Chand et al., 2009; Vernier et al., 2011; Forbes and Ahlgrimm, 2014; Tan et al., 2016; Stephens et al., 2018).
Measured CALIOP signals are translated into meaningful atmospheric observations by proper calibration of the three receiver channels. Due to differences in signal-to-noise ratios (SNR) and intra-orbit thermal stability, CALIOP uses different techniques to calibrate the 532 nm daytime and nighttime measurements. The calibration procedure for the nighttime 532 nm parallel signals, which is the basis for all other calibrations, uses a high-altitude molecular normalization technique (Russell et al., 1979; Powell et al., 2009; Kar et al., 2018), in which calibration coefficients are determined by taking the ratio of the measured signal to the expected signal computed using an atmospheric model. This approach assumes that all constituents of the nighttime normalization region (i.e., including aerosol loading) can be accurately modeled or characterized. The same technique cannot be used during daytime, however, because the SNR is substantially lower due to the influence of the reflected solar background radiation. This is rectified by scaling the daytime to the nighttime calibration by using clear air attenuated scattering ratios, defined as the ratio between the measured attenuated backscatter and modeled molecular signal. These are measured and accumulated over identical altitude ranges and latitude bands during both daytime and nighttime. The fundamental assumption for the 532 nm daytime calibration procedure is that a persistent “calibration transfer region” can be identified where the aerosol loading remains diurnally invariant over relatively short periods of time (e.g., 7–10 days).
Calibration algorithms used in the version 3 (V3) series of L1 data products (Vaughan et al., 2018), released beginning in June 2009 are described in Hostetler et al. (2005); Powell et al. (2008, 2009, 2010); and Vaughan et al. (2010). Over the intervening years since the release of V3, several shortcomings have been identified in the 532 nm daytime calibration algorithm. First, the altitude of the V3 calibration transfer region was too low, and hence the assumed diurnal invariance for the 532 nm daytime calibration was often not satisfied, as noted in Powell et al. (2010) and described further in Sect. 2. Frequent cloudiness at tropical latitudes also limited the number of clear-air samples available at this altitude range. Second, identifying the cloud-free data segments needed to calculate the V3 daytime calibration coefficients was accomplished by repeatedly generating a subset of the lidar level 2 (L2) products, and this interdependency prohibited the independent calculation of the 532 nm daytime calibration coefficients. Finally, the calculation of the calibration uncertainty estimates in V3 failed to accurately include all error sources associated with the approach.
To account for these identified weaknesses in the V3 algorithm, the CALIPSO project completely redesigned the calibration architecture for the version 4 (V4) L1 data products. The 532 nm nighttime calibration has been updated to accommodate a change in the molecular normalization region in the stratosphere from 30–34 to 36–39 km (Kar et al., 2018). This change was based on a better understanding of the vertical distribution of stratospheric aerosols (Vernier et al., 2009). As for the 532 nm daytime calibration, which is the focus of this paper, the technique still relies on matching daytime and nighttime clear air scattering ratios, but with several crucial modifications that address the problems identified above. An overview of the V3 calibration procedure is provided in Sect. 2, followed by a detailed summary of the new V4 method in Sect. 3. Section 3 fully describes the updated assumptions and new techniques used in V4. Section 4 compares the V4 calibration coefficients against internally established scientific metrics used to assess the performance of the algorithm. Section 4 also updates a previous comparison between CALIOP V3 backscatter coefficients and extensive collocated measurements from the NASA Langley Research Center (LaRC) airborne high spectral resolution lidar (HSRL) (Rogers et al., 2011). Some concluding remarks are given in Sect. 5.
The V3 532 nm daytime calibration procedures transfer the 532 nm parallel channel nighttime calibration to the daytime measurements (Powell et al., 2010). Calibrating the daytime signals relative to the nighttime measurements is done for two reasons: (a) low SNR during the daytime prevents calibration using the high-altitude molecular normalization technique, and (b) thermally induced changes in the alignment of the laser transmitter with respect to the receiver produce substantial changes in the daytime calibration over the course of each daytime orbit segment. Transferring calibration from nighttime to daytime was accomplished in V3 by using latitudinally varying clear-air attenuated scattering ratios, accumulated for both day and night orbital segments, to derive correction factors associated with the along-track misalignments that occur during the daytime. For any daytime granule, the 532 nm daytime calibration coefficients were then computed as the product of a mean correction factor built from an accumulation of several days' worth of scattering ratios (discussed further in this section) and the mean 532 nm calibration coefficient from the previous nighttime granule (Powell et al., 2010).
The attenuated scattering ratios,
The nighttime and daytime clear-air attenuated scattering ratios in V3 are
calculated for “frames” of data within the calibration transfer region.
Each frame extends for 200 km along-track at an altitude of 8 to 12 km. The
8 to 12 km altitude range was chosen to be high enough to avoid substantial
diurnal variation of the aerosol loading in the lower troposphere and low
enough to provide relatively robust SNR, thus minimizing the influences of
solar background radiation on the daytime signal. The clear-air attenuated
scattering ratios in the V3 8–12 km calibration transfer region were assumed
to be diurnally invariant. Based on this assumption, initial estimates of
the mean attenuated scattering ratios,
Initial correction factor estimates,
To construct valid sets of correction factors, the data frames used in the
V3 daytime calibration procedure must consist entirely of “clear air” from
the top of the lidar return at 40 km down to the base of the V3 calibration
transfer region at 8 km. Note that, in this context, “clear air” does not
imply a pristine, aerosol-free molecular atmosphere. Instead, we impose the
requirement that no layers are detected within this region by the CALIOP
multi-resolution layer detection algorithm (Vaughan et al., 2009). Any
undetected residual aerosol is assumed to be diurnally invariant. For the
calibrated nighttime data, the required clear air frames can be identified
directly from the automatically generated L2 data products. But because the
layer detection algorithm requires calibrated L1 data for its operation,
determining daytime clear air frames requires an intermediate operation,
wherein the layer detection module is embedded in an iterated optimization
loop initiated using a coarse first approximation to the daytime calibration
scale factors. These scale factors are subsequently refined through
successive iterations. At each step, the required regions of “clear air”
are identified by applying the layer detection procedure to the current
estimate of the daytime attenuated scattering ratios, which are derived by
applying the most recent iteration of the daytime calibration scale factors.
The final correction factor curves are stored in internal ancillary lookup
tables. The 532 nm daytime calibration coefficients are derived by
multiplying these time-varying correction factors by the mean of the
previous granule's 532 nm nighttime calibration coefficient, as shown in
Eq. (5), where
V3.40 532 nm calibration coefficients between successive CALIPSO night–day–night orbits on 18 December 2016 from 20:15:31 to 22:40:41 UTC.
The fundamental assumption underlying the V4 532 nm daytime calibration scheme is the same one invoked in V3; i.e., the mean attenuated scattering ratios do not vary significantly during a diurnal cycle within a defined region in the atmosphere, and hence the mean uncalibrated daytime attenuated scattering ratios can be scaled to match the mean nighttime attenuated scattering ratios in this calibration transfer region. Here we list the major changes between the V3 and V4 daytime calibration algorithms. First, two new signal adjustments have been incorporated into the L1 processing, and these have a small but direct impact on the subsequently derived calibration coefficients. Second, the selection of the calibration transfer region has been changed so that a 400 K potential temperature isotherm in the lower stratosphere now defines the bottom of the vertical calibration range, replacing the fixed altitude cross-tropopause region used in V3. Third, a newly developed multi-granule averaging scheme compensates for the reduced SNR incurred by moving the calibration transfer region upwards. To further boost the SNR, rather than using the total signal (i.e., parallel plus perpendicular, as in Eq. 2), only the parallel channel is considered. Fourth, the V4 procedure uses a modified version of the L2 layer detection algorithm to search the uncalibrated 1064 nm channel backscatter coefficients for clear air regions, instead of using the calibrated 532 nm attenuated scattering ratios, as was done in V3. This eliminates the need for a multi-pass architecture, and has two major benefits. The revised V4 calculations are more transparent, making it easier for external data users to replicate and/or validate the calibration coefficients and uncertainties reported in the V4 L1 data products. Also, by eliminating the recursive process, the V4 scheme significantly reduces the number of processing steps required to generate the resultant data product. Fifth and finally, the V4 532 nm daytime calibration coefficients uncertainties are computed directly from the nighttime calibration coefficient uncertainties, using the calibrated 532 nm nighttime and uncalibrated 532 nm daytime attenuated scattering ratios. Increased accuracy in computing the 532 nm daytime calibration coefficient uncertainties for V4 was also important, as a better understanding of the key contributors to the overall error was crucial in driving the averaging decisions used to calibrate all three channels. Each of these five algorithm updates is discussed in detail in the subsections below.
The V4 calibration procedure applies two new corrections to the daytime signal prior to the calibration: an adjustment to remove photomultiplier (PMT) baseline shapes and an updated day-to-night gain ratio. The motivation and implementation of these two corrections are discussed in the paragraphs below.
The output of PMTs exposed to constant background light (e.g., sunlight
reflected from dense water clouds) typically increases with time after the
PMT is gated on, thus generating a signal-induced baseline shape that varies
as a function of the background light level. Prior to launch, the baseline
shapes for the CALIOP detectors were repeatedly measured in the laboratory,
and the magnitudes of the required signal adjustments were found to be quite
small relative to the atmospheric signals typically measured in the
troposphere. Consequently, because the prelaunch daytime calibration
strategy was simply to interpolate daytime calibration coefficients between
neighboring nighttime molecular normalizations (Hostetler et al., 2006;
Powell et al., 2008), baseline shape corrections were deemed to be
unnecessary and thus were not implemented. This assessment changed with the
V4 redesign of the daytime calibration algorithms. The V4 daytime
calibration relies on highly averaged daytime measurements in the
middle-to-lower stratosphere where the expected molecular signals are
substantially weaker, and hence biases due to baseline shape artifacts are
potentially significant. To mitigate these concerns, we used prelaunch
laboratory measurements together with post-launch extended background
measurements acquired periodically throughout the mission to characterize
the PMT baseline shapes:
The improvements achieved by the application of the shape correction are illustrated in Fig. 2, which shows a 7-day average (13–19 December 2011) of the daytime correction factors derived using the V3 algorithm as a function of orbital elapsed time for both the 8–12 km (V3 calibration transfer region) and an elevated 18–22 km region (an approximate V4 calibration transfer region for this time period). Figure 2a shows that there is a difference of up to 8 % in the correction factors between these two altitude levels, particularly in mid-latitudes where bright clouds generate higher background signals. This difference as a function of altitude should not occur. By applying the V4 baseline shape correction for this case in Fig. 2b, the correction factors, though slightly reduced, are now more similar. Although applying the shape correction causes an overall reduction in the apparent signal with increased altitude, the corrected signal more accurately corresponds to the atmospheric signal and eliminates systematic artifacts in the daytime calibration coefficients.
The 532 nm daytime correction factor for 13–19 December 2011
based on the V3 L1 algorithm. The correction factor is computed for both the
V3 calibration transfer region (8–12 km) and an elevated transfer region
(18–22 km) without
To prevent saturation of the digitizers by large daytime noise excursions, a fixed reduction in the detector gains is applied to all three channels during daytime operations. To compensate for these gain changes, the CALIOP calibration routine applies fixed day-to-night gain ratios to the daytime measurements. On-orbit performance metrics and routine built-in test system (BITS) measurements (Hunt et al., 2009) suggested that the 532 nm perpendicular and parallel day-to-night gain ratios needed to be increased by 0.65 % and 3.3 %, respectively. Though large, the 532 nm parallel adjustment has essentially no impact on the calibrated 532 nm daytime attenuated backscatter coefficients, because the gain increase is absorbed as a multiplicative factor into the calculated calibration coefficients. Similarly, because the V4 daytime calibration only uses the signals from the parallel channel, changes to the 532 nm perpendicular day–night gain ratios have no impact on the derived calibration, though they will ultimately yield a small increase in the 532 nm perpendicular and total attenuated backscatter coefficients reported in the L1 data products.
The selection of the calibration transfer region is subject to two competing interests. Diurnal variation in background aerosol should be minimized, which argues for a higher altitude since, to first order, aerosol concentrations and diurnal variability tend to decrease with height. However, absent any aerosol loading CALIOP's SNR also decreases with height, and obtaining an accurate calibration requires sufficient signal to overcome the daytime background noise due to sunlight. The V3 algorithm approach maximized SNR, as previously discussed, by choosing a calibration transfer region with a fixed base of 8 km and a constant depth of 4 km (12 km top). However, this altitude domain occurs in the tropical troposphere where the CALIOP signal is frequently attenuated due to persistent cloud cover, and therefore has a reduced number of clear-air samples. There is also the possibility of potential contamination by clouds and aerosols not identified by the L2 feature detection technique used to isolate clear air. In the extra-tropics, 8–12 km altitude range straddles the tropopause, where there is additional background aerosol variability caused by fluctuations in the tropospheric jet locations (Gettleman and Wang, 2015; Manney and Hegglin, 2018). By elevating the calibration transfer region from the near tropopause into the lower stratosphere, the V4 approach attempts to improve the fidelity of the clear-air attenuated scattering ratios by substantially reducing the possibility of any diurnal variability of background aerosol. Relocating to the lower stratosphere also minimizes the need for a robust feature detection algorithm to identify clear-air, as by definition this more stable region contains fewer cloud and aerosol layers than are found in the troposphere. The trade off, as already noted, is a reduction in SNR that dictates more sampling, which will be discussed in more detail in Sect. 3.3.
Rather than using a globally fixed geometric altitude range, the base of the
V4 calibration transfer region is located above the 400 K isentropic
surface, with a thickness of 4 km, thereby reflecting latitudinal
differences in the height of the lowermost stratosphere. Using an isentropic
surface to identify the stratosphere is beneficial because (1) it accounts
for latitudinal and seasonal changes in geopotential heights, and (2) potential temperature (
Two additional safeguards are used to avoid possible contamination of the clear-air attenuated scattering ratios. First, to both guard against features intruding into the lower stratosphere, and because the algorithm uses a climatological monthly mean 400 K surface as the lower limit, an additional altitude offset of 2 km is applied to further elevate the base of the calibration transfer region. Secondly, since the stratosphere is not entirely devoid of features, the algorithm employs a 1064 nm feature detection technique, as discussed in Sect. 3.4, to exclude cloud and aerosol layers from the calibration averaging scheme. In particular, the presence of undetected polar stratospheric clouds (PSCs) in the calibration transfer regions can introduce high biases into the calibration coefficient estimates. The potential impacts of feature contamination of the calibration transfer regions are discussed in detail in Sect. 4.2 and 4.3.
At the time of the V4 algorithm development and deployment, GMAO provided an updated meteorological reanalysis product, MERRA-2 (Modern-Era Retrospective analysis for Research and Applications, Version 2) (Gelaro et al., 2017), which includes Microwave Limb Sounder (MLS) temperatures and is a marked improvement over earlier GMAO-FPIT products. This new meteorological data was incorporated into the V4.10 L1 and L2 data products, but was not used to re-compute the 400 K altitudes used by the 532 nm daytime calibration algorithm to set the calibration transfer region base altitude.
Because the dominant source of noise during daytime operations is the solar
background signal, daytime SNR scales approximately linearly with signal
strength (Hunt et al., 2009). Moving the calibration region upward from a
midpoint of 10 km in V3 to a nominal midpoint of
Continuous operation of the V4 daytime calibration procedure is predicated on maintaining the instrument in “steady state” conditions. There have been, of course, numerous cases where these steady state conditions have been interrupted. In these cases, the data acquired before and after the change in instrument state is potentially quite different, and hence the calibration procedure needs to be restarted at the temporal boundary of the change. The most common changes occur when there are gaps in the data that exceed 24 h. These are chiefly due to data dropouts, failed downlinks, or commanding the satellite to protect the hardware from solar storms or other anomalies. State changes also occur when routine on-orbit maintenance is performed. These tasks include re-alignment of the laser, etalon scans, or BITS (Sect. 3.2), and are followed by a reboot of the calibration procedure. This reboot introduces a hard-boundary, in which the calibration averaging windows stop at a defined time. Following a reboot, the calibration coefficients are observed to remain quite stable. However, as expected, the calibration uncertainties increase, reflecting the lower numbers of samples used in the averaging.
The use of multi-orbit averaging also helps suppress the influence of the
unusually large noise excursions that can occur when the satellite passes
through the South Atlantic Anomaly (SAA), an area on the globe from roughly
90
Because only clear-air regions are used for the 532 nm daytime calibration, frames are excluded where features are detected. In the CALIOP L2 processing, cloud and aerosol layers are detected using an iterated multi-resolution averaging scheme, in which the measured 532 nm attenuated scattering ratios are compared to dynamically constructed, altitude-dependent threshold arrays (Vaughan et al., 2009). Large positive excursions in scattering relative to the computed thresholds are identified as features, and the spatial and optical properties of these features are subsequently used to discriminate clouds from aerosols (Liu et al., 2009, 2018), and then determine either cloud thermodynamic phase (Hu et al., 2009) or aerosol species (Omar et al., 2009; Kim et al., 2018). This same layer detection scheme was used in the V3 532 nm daytime calibration procedure to identify the presence of clouds and aerosols on a per-frame basis within the daytime calibration transfer region (Powell et al., 2010).
The V4 daytime calibration scheme takes a different approach. Layers are still detected using the same profile scanning engine that drives the L2 processing. However, instead of recursively searching the calibrated 532 nm attenuated scattering ratios, layers are detected using the uncalibrated 1064 nm signals. In conducting the search, the molecular backscatter contribution to the total 1064 nm signal is assumed negligible, and the expected molecular signal is set to zero. This assumption is reasonable because the large amount of dark noise from the avalanche photodiode (Hunt et al., 2009) is much larger than any molecular contribution in the stratosphere. Additionally, the search for layers is instead carried out at a single horizontal resolution of 200 km rather than using the iterated multi-resolution averaging scheme.
The 1064 nm threshold arrays for feature detection are constructed as
follows. First, the measured background variation, MBV, in the averaged
profile is computed using
A detailed example of this technique is shown in Fig. 3, where the new 1064 nm feature detection algorithm evaluates a fairly typical blended cloud/aerosol scene for a nighttime granule. The 532 nm total attenuated backscatter is shown in Fig. 3a and the vertical feature mask (VFM) is shown in Fig. 3b. Superimposed on Fig. 3b are the frames corresponding to the two calibration transfer regions: V3, indicated by red boxes between 8 and 12 km, and V4, indicated using green boxes which track 2 km above the 400 K isentropic surface. Clearly, the V4 calibration transfer region is well above all features detected within the vertical feature mask.
To further demonstrate the effectiveness of the new detection technique, the
calibration base altitude was lowered to 8 km, matching the base of the V3
calibration transfer region. The top altitude at which the 1064 nm technique
detected a feature is highlighted by a solid white line in Fig. 3b. The
detection follows the top of the cloud features, identifying a deep
convective cloud at 18
The
The mathematical approach used to derive the 532 nm mean attenuated scatter
ratios is fundamentally the same in both V3 and V4 (Powell et al., 2010).
Attenuated scattering ratios and uncertainty are averaged for frames of data
within the calibration transfer region. A “frame” is defined as 200 km
along-track and 4 km vertical segments of data. Along-track, the 200 km
resolution translates to 600 single shot (
Expanding Eq. (1), the nighttime attenuated scattering ratios
The daytime averaged uncalibrated scattering ratios,
The 532 nm calibration coefficients and their uncertainties for any given
daytime granule are initially computed on a fixed elapsed time grid that
spans from 0 s (referenced to the start of the daytime granule) to
3200 s with a resolution of 100 s. These coarse-resolution
calibration coefficients are then linearly interpolated, based on time, and
these interpolated values are applied to the 532 nm attenuated backscatter
measurements at the
Given the multi-day averaging needed to harvest the calibration data, as described in Sect. 3.3, time cannot be used, either elapsed or some other reference time, to aggregate day and night scattering ratios that span multiple orbits. The orbital transition point from day to night (i.e., the day–night terminator), by which CALIPSO designates granules as either daytime or nighttime, changes throughout the aggregation period. In order to properly account for this temporal drift, a reference latitude grid, independent of time, is built by mapping and interpolating the latitude of the daytime granule being calibrated onto the fixed elapsed time grid.
The parallel component of the 532 nm daytime calibration coefficient,
The daytime calibration uncertainty estimate,
Calibration coefficients derived from the day-to-night ratio of attenuated
scattering ratios can only be calculated within those portions of an orbit
in which both day and night observations are acquired. Because of the
illumination patterns in the polar regions during the solstice seasons,
there are no matching daytime and nighttime samples near the poles in summer
and winter. This seasonally recurring lack of high latitude matching day and
night data is accounted for by interpolating between the end points of the
neighboring daytime and nighttime calibration coefficient curves. Where
there are neither daytime and/or nighttime samples, the 532 nm daytime
calibration coefficient and coefficient uncertainty curves are linearly
interpolated as a function of orbital elapsed time anchored to the nearest
neighboring 532 nm nighttime calibration. Figure 4 shows an
example. In this orbit from July 2010, the day-to-night terminator occurs at
532 nm calibration coefficients for both daytime and nighttime measurements
are computed using only the parallel component of the backscattered signal.
The perpendicular channel measurements are calibrated relative to the
parallel channel using the polarization gain ratio (PGR), which quantifies
the relative gain between the two 532 nm detectors. Highly accurate PGR
values are measured directly using an onboard calibration procedure
described in detail in Hostetler et al. (2005), Hunt et al. (2009), and Powell
et al. (2009). The calibration coefficients for the perpendicular channel are
the product of the PGR and the parallel channel calibration coefficients;
i.e.,
Data latencies – i.e., the times between data acquisition and data product
delivery – have also changed between V3 and V4. The CALIOP V3 standard data
products are generated within 3 to 5 days from downlink, partially due to
the V3 calibration approach but also because the analyses require a number
of ancillary inputs that are not immediately available (Winker et al.,
2009). The latency for the V4 standard products is considerably longer.
Because V4 uses the MERRA-2 meteorological data rather than the GMAO FPIT
products, V4 standard products are typically not available until 6 to 10 weeks from downlink. However, the V3 expected products continue to be
available with 24–36 h from data acquisition (i.e.,
Performance of the 532 nm daytime calibration from 13 June 2006 to 31 December 2016 is shown in Fig. 5. Figure 5a shows the 532 nm daytime calibration coefficient anomalies while Fig. 5b shows the 532 nm daytime calibration uncertainty anomalies, both of which are normalized to their respective time-series mean. The figure shows gaps in the data record occurring over the course of the mission, the reasons for which are described in more detail in Sect. 3.3. From the start of the mission to 31 December 2016 there have been 138 distinct events that required calibration restarts, with 71 of these due to planned maintenance of the lidar. The others were due to unscheduled events when either the instrument was commanded to SAFE/OFF (leading to a period when no data was collected) or when data downlink issues caused delays that exceeded 24 h, and thus required a reboot of the calibration averaging, as described in Sect. 3.3. Also of note, the lidar switched from the primary laser to backup laser on 12 March 2009, resulting in a noticeable shift in the distributions of the calibration minimum between the two lasers.
Time series of
The impact of interpolating the high-latitude portions of the orbit, where
it is not possible to match daytime and nighttime attenuated scattering
ratios (Sect. 3.6), can also be seen in the distribution of the 532 nm
calibration coefficient uncertainties in Fig. 5b. The saw tooth seasonal
pattern of elevated uncertainty, greater than 1.5 times above the normalized
mean, directly corresponds to those areas of interpolation. Though the time
series of the uncertainty is fairly stable throughout the mission, there are
pockets at the mid-latitudes (
The ratio between the V4 and V3 532 nm daytime calibration coefficients is
shown in Fig. 5c. In general, the mid-latitude differences, corresponding
to 1200–1800 granule-elapsed seconds, show differences in the range of
The performance of the new calibration algorithm can be evaluated by
comparing zonal distributions of the day and night mean clear air attenuated
scattering ratios in different altitude regimes. Comparisons of the
calibration transfer region are presented first. Given that the daytime
calibration is scaled to the night-time, one should expect to see that the
daytime and nighttime attenuated scattering ratios should tightly follow
each other within this altitude band. Figure 6 confirms this
expectation. The red and blue curves show, respectively, mean daytime and
nighttime calibration coefficients as a function of latitude, with the
shaded areas around each curve delineating
From 60
Zonal clear-air attenuated scattering ratio (
The altitude region between 24 and 30 km is also examined. These altitudes
lie just above the top of the calibration transfer region used in the 532 nm
daytime calibration, but below the 36–39 km region used by the 532 nm
night-time calibration procedure. Thus, data within this region have not
been used in either of the calibration procedures. Figure 7 shows the
daytime–to–nighttime ratios of the clear air attenuated scattering ratios
measured in the 24–30 km region for both V3 (Fig. 7a) and V4 (Fig. 7b).
The same months and data filtering procedures used in creating Fig. 6 are
also used to construct Fig. 7. The V3 day-to-night ratios reveal high
daytime biases of up to 20 % in the mid-latitudes and 25 % in the
high-latitudes, with values above 1 consistently between
Day/night ratio of clear-air attenuated scattering ratio (
The V4 daytime calibration algorithm scans the uncalibrated 1064 nm measurements to ensure the presence of clear air down to the base of the calibration transfer regions. While the 532 nm channel is much more sensitive to the smaller aerosol particles that we expect to encounter most often in the stratosphere, the daytime calibration procedure does not require that we identify pristine air parcels. Instead, we need only identify and remove relatively robust, spatially varying, and temporally transient features – i.e., those layers that are not expected to persist uniformly across extended day–night cycles – and for this task the 1064 nm detection capabilities should be sufficient.
To establish the performance capabilities of our 1064 nm feature detection approach, 1 year of 532 nm daytime calibrations were regenerated using the more robust feature detection and clearing provided by the 532 nm detection methods of the L2 algorithm. In creating this second set of calibration coefficients, the V4 5 km merged layer product was used to identify those V4 calibration transfer regions where layers of any type are reported in the L2 data products, and regions identified as being feature-contaminated were excluded from the subsequent calculations. Like the L1 detection scheme, the L2 algorithm uses fixed frames of data, but with a maximum of 80 km rather than the 200 km horizontal averages used in L1. The L2 technique also employs multi-pass averaging (5, 20 and 80 km) and clearing to remove features detected at higher spatial resolutions prior to re-averaging and searching for features at coarser resolutions. The L2 532 nm algorithm implementation is thus capable of identifying features at much finer spatial scales than the 1064 nm version of the search routine implemented in L1.
Figure 8 shows the ratios of these two sets of 532 nm calibration
coefficients for the entirety of 2015, plotted as a function of latitude.
While some latitudinal deviation is seen, the mean value (i.e., the black
dashed line) varies by no more than
Ratio of the V4 532 nm daytime calibrations derived based on 1064 nm detection technique (new L1 algorithm) to the calibrations derived by applying 532 nm feature clearing for 2015 in black. Red error-bars indicate that the mean calibration uncertainty. The blue line is the ratio of V4 to V3 daytime calibrations.
Figure 9 provides a more detailed examination of the feature averaging and
detection characteristics of the layers identified by the L2 532 nm feature
detection algorithm and used in the re-calibration effort described in the
previous paragraph. The plot is segregated by the effectiveness of the 1064 nm technique to detect features relative to the averaging required
(i.e., 5, 20, or 80 km) to detect layers when using the 532 nm L2 detection
scheme. The distributions of detected and undetected L2 features are plotted
as a function of layer integrated volume depolarization (
The preponderance of the 1064 nm detection failures is seen in the bottom
left corners of the plots in the right-hand column of Fig. 9. These
features, for which both
Distribution of layer integrated volume depolarization ratio and
1064 nm integrated attenuated backscatter for all features contained in the
transfer regions used for calibration. In panel
Figure 10 provides an example wherein the 1064 nm feature detection
algorithm fails to identify legitimate features, and thus illustrates those
circumstances in which calibration accuracy can be degraded by high biases
in the nighttime attenuated scattering ratios. The orbit track begins just
southwest of Africa, transiting over the Southern Ocean and extending
into Antarctica at
The tops of the tropospheric clouds north of 57
From the beginning of the CALIPSO mission, the high spectral resolution
lidar (HSRL) group at NASA-LaRC has acquired an extensive series of
coincident airborne validation measurements. Following the release of the V3
L1 data set in April 2010, Rogers et al. (2011) conducted an in-depth
analysis comparing HSRL 532 nm attenuated backscatter coefficients measured
along the CALIPSO orbit track to the 532 nm attenuated backscatter
coefficients reported in the V3 CALIOP L1 data products. A major finding of
this work showed that the CALIOP V3 daytime attenuated backscatter data was
biased low with respect to the coincident HSRL data by
To characterize biases in the new V4 data set we replicated the Rogers study
using a slightly larger coincident data set that includes additional
overflights conducted since the original investigation. These include
flights over the Caribbean (19 August–28 September 2010), the DEVOTE
field campaign (4–8 October 2011), and flights over the
Azores (17 October 2012) and Bermuda (10–19 June 2014). In the
process of reproducing the Rogers et al. (2011) V3 results, a bug was
discovered in the code used to estimate the overlying two-way transmittance
differences between the two sets of measurements (see Appendix A in Kar et
al., 2018). Accounting for this error led to a small upward revision of the
daytime biases in the V3 data set, which we now estimate at
Bias of the daytime 532 nm attenuated backscatter measured between HSRL and CALIPSO for several over-flight campaigns between 2006 and 2014 broken by season and latitudes. The comparisons used both V3 (solid diamonds) and V4 (open circles) L1 data. Each point represents the mean and uncertainty of the HSRL-CALIPSO difference for each of the 62 flights conducted.
In this paper we have described the new procedures implemented in CALIOP's version 4 (V4) data release to better calibrate the 532 nm daytime measurements. Compared to version 3 (V3), the V4 updates deliver marked improvements in calibration accuracy and provide more realistic and comprehensive estimates of calibration uncertainties. The new V4 algorithm keeps the underlying approach that was used in V3, wherein the 532 nm daytime calibration coefficients are scaled relative to the 532 nm nighttime coefficients, which are calculated using the highly reliable high-altitude normalization technique. The simplified V4 calibration architecture reduces software coupling and increases cohesion by eliminating the need for multi-pass product generation cycle, which in turn enables a more direct computation of the calibration coefficients and their uncertainties. The V4 calibration performance meets pre-defined expectations established from internal science impact testing, and fully satisfies numerous day–night consistency metrics. Elevating the calibration transfer region, coupled with a revised feature detection scheme that uses the uncalibrated 1064 nm measurement, has greatly increased the probability that the attenuated scattering ratios used in deriving the calibration coefficients are computed within clear air regions and largely eliminated the diurnal aerosol loading artifacts seen in V3. Independent validation using collocated high spectral resolution lidar measurements shows a demonstrable improvement between CALIOP V3 and V4 daytime calibration, with the mean daytime bias between the two sets of measurements being reduced from 3 % to approximately 1 %.
The following CALIPSO data products were used in this study: the V3.01
CALIPSO level 1 profile product (Vaughan et al., 2018; NASA Langley Research
Center Atmospheric Science Data Center;
All co-authors have contributed to the paper, and the order in which they are listed is primary author's best estimate as to their level of contribution.
The authors declare that they have no conflict of interest.
This article is part of the special issue “CALIPSO version 4 algorithms and data products”. It is not associated with a conference. Edited by: James Campbell Reviewed by: Zhien Wang and J. Yorks