Interactive comment on “ Single-footprint retrievals of temperature , water vapor and cloud properties from AIRS ”

The authors describe a 1-D variational retrieval of temperature, atmospheric constituent gases, and cloud properties from AIRS observations (AIRS-OE) and co-located operational MODIS cloud property retrievals. Multiple retrieval strategies are employed based on the availability of MODIS cloud properties and their sub-pixel characteristics within an AIRS FOV. The paper fits well within the scope of AMT and provides a sufficient contribution to scientific progress in the field of remote sounding from hyperspectral IR measurements.


Introduction
An advantage of hyperspectral nadir measurement in the thermal infrared over the microwave is higher vertical resolution of retrieved temperature and water vapor.Operational instruments such as the Atmospheric Infrared Sounder on the EOS Aqua platform (AIRS;Aumann et al., 2003), the Infrared Atmospheric Sounding Interferometer on Metop-A and -B (IASI; Blumstein et al., 2004) and the Cross-track Infrared Sounder on the Suomi National Polar-orbiting Partnership (NPP) platform (CrIS; Han et al., 2013) provide global radiance data for assimilation into weather forecasting and reanalysis models and profile retrievals for process studies.Perhaps the largest complication for global retrievals (and assimilation) using infrared spectra is near-ubiquitous, highly variable cloud absorption and scattering in the instrument field of view (FOV).This is illustrated in Fig. 1, which shows sample brightness temperature spectra observed by AIRS in nine adjacent footprints from level 1b data (that is, radiance data calculated from the raw counts on the AIRS detectors).Most of the variation between the spectra is from differences in cloud-top temperature and cloud optical depth, and to a lesser extent, cloud particle radius as seen on the AIRS footprint.Two general approaches have been used in obtaining profile retrievals from cloudy infrared spectra.The first was "cloud clearing," which means the temperature and trace gas fields (including water vapor) are treated as constant across adjacent thermal infrared footprints, and only the cloud field varies.A level 2 cloud-free infrared spectrum is calculated from these cloudy level 1b spectra and then used for profile retrievals over a larger field of regard (FOR; e.g., Susskind et al., 2011).Cloud clearing simplifies (and speeds) calculations, as the forward model does not incorporate scattering or absorption by clouds.However, by combining retrieval footprints and assuming constant non-cloud quantities, cloud clearing can mask significant horizontal gradients, particularly in water vapor, which can have a much shorter horizontal length scale than temperature, especially at low latitudes and midlatitudes (e.g., Kahn and Teixeira, 2009).
The other approach that accounts for clouds includes cloud absorption and scattering in a retrieval forward model.Advances in efficient cloud-scattering algorithms and ever increasing computing power hold the promise of incorporating explicit cloud effects in forward models for routine operational retrievals.Several methods and software packages have been developed for the calculation of outgoing radiance in the presence of clouds: optimal spectral sampling (OSS; Moncet et al., 2015), The Havermann-Taylor Fast Radiative Transfer Code (HT-FRTC;Havermann et al., 2006), the principal component-based radiative transfer model (PCRTM; Liu et al., 2005), discrete ordinate algorithm for radiative transfer (DISORT; Laszlo et al., 2016), vector linearized discrete ordinate radiative transfer (VLIDORT; Spurr, 2006), the community radiative transfer model (CRTM; Han et al., 2005), the Radiative Transfer for the Television Infrared Observation Satellite (TIROS) operational vertical sounder algorithm (RTTOV;Saunders et al., 2013), the standalone AIRS radiative transfer algorithm (SARTA) coupled to a parameterization of clouds for longwave scattering in atmospheric models (PCLSAM; SARTA two-slab; DeSouza-Machado et al., 2017) and, as used in this work, a SARTA with delta-four-stream approximation (SARTA-D4S; Ou et al., 2013).
While incorporating cloud effects usually makes for a more complicated and computationally expensive radiative transfer calculation, the horizontal resolution of retrieved species can be improved compared to cloud-cleared results.Several methods have been employed for direct use of cloudy infrared spectra in atmospheric retrievals.Among them are combining channel radiances into "super-channels" using empirical orthogonal functions (Liu et al., 2009), neural networks (Blackwell, 2005) and parameterizations of frequency-dependent nonscattering optical depths (Kulawik et al., 2006a).Recently, DeSouza-Machado et al. (2018) presented an optimal estimation scheme using their SARTA two-slab forward model.Here we describe a new retrieval scheme, an optimal estimation retrieval system (AIRS-OE) that can use the level 1b radiances of single AIRS footprints, without cloud clearing for the retrieval of temperature profiles (T atm ), H 2 O volume mixing ratio profiles, skin temperatures (T sfc ), effective cloud optical depths over an AIRS field of view (τ eff ), cloud-top temperatures (T cldtop ) and effective particle radii (r eff ).CO 2 and O 3 profiles, while not the primary constituents examined or validated here, are also retrieved to improve the spectral fitting and temperature results.The goal is to improve the horizontal resolution of retrievals by using the less processed level 1b AIRS infrared radiance (at ∼ 13.5 km nadir horizontal resolution) rather than the ∼ 45 km resolution level 2 cloud-cleared radiance produced by the currently operational version 6 (AIRS V6) retrieval algorithm (Susskind et al., 2003(Susskind et al., , 2014)).Additional Table 1.A priori covariances used for retrievals.
differences of AIRS-OE from the current AIRS V6 retrieval include the following: 1. Cloud optical depth, effective particle radius and cloudtop temperature are now explicitly in the forward model and are retrieved along with temperature and water vapor profiles.A priori cloud information for cloud retrieval is calculated from the MYD06 data set of the Moderate Resolution Imaging Spectroradiometer (MODIS) instrument, co-located on the EOS-Aqua platform (Platnick et al., 2003(Platnick et al., , 2017)).
2. For a priori temperature profiles, skin temperature and water vapor profiles, we use European Center for Medium-Range Weather Forecasting (ECMWF) 6 h analyses linearly interpolated by time and space to the AIRS observation, where the AIRS V6 retrieval uses a neural network trained on AIRS radiances and ECMWF reanalyses (Blackwell, 2005).Emissivity is retrieved in AIRS V6.For AIRS-OE, emissivity is currently taken from monthly tables (over land) and formulae (over ocean) and not retrieved (see Sect. 3.2.3),although emissivity retrieval may occur in future versions.
3. In a departure from the singular-value-decomposition technique of AIRS V6, which does not use an a priori covariance, the new retrieval uses an optimal estimation scheme (e.g., Rodgers, 2000) similar to that of the nadirsounding Tropospheric Emission Sounder (TES; Bowman et al., 2006).
4. Retrievals of atmospheric constituents are made simultaneously rather than sequentially as in AIRS V6.
5. AIRS-OE uses only the thermal infrared data from AIRS and does not use ∼ 45 km nadir-resolution microwave data from the co-located Advanced Microwave Sounding Unit (AMSU).Profiles cannot be retrieved below IR-opaque clouds effectively covering a pixel.However, unlike AIRS V6, temperature, water vapor, etc. are not assumed to be uniform across the nine AIRS footprints in an AMSU field of regard (FOR).
In this paper, we give a very brief overview of the AIRS and MODIS instruments, and outline the retrieval and the information content analysis.Some sample cloud property results are presented, and we show an initial comparison with nearcoincident measurements by Cloudsat/CALIPSO.We then compare temperature, water vapor and relative humidity results to those of the operational AIRS V6 retrieval and with sets of near-coincident high-quality radiosondes.

Instrument and spectral data
The Atmospheric Infrared Sounder (AIRS) instrument is a thermal infrared grating spectrometer, with 2378 channels between 3.7 and 15.4 µm.In a sun-synchronous, polar orbit on the EOS Aqua satellite, AIRS delivers approximately 2.9 million spectral observations every 24 h.AIRS was designed for co-located measurements with the Advanced Microwave Sounding Unit (AMSU) microwave instrument, with nine AIRS observations (each with nadir horizontal resolution of ∼ 13.5 km) in a 3 × 3 grid over a single AMSU observation with a nadir horizontal resolution of ∼ 45 km (Aumann et al., 2003).AIRS was designed to provide global data on weather and climate processes and is a key antecedent to the IASI and CrIS spectrometers.
As noted, operational level 2 data from the Moderate Resolution Imaging Spectroradiometer (MODIS) are used as a priori data for cloud-top temperature, and during daytime observation, cloud optical depth and particle radius.Co-located with AIRS on EOS-Aqua, MODIS observes in 36 spectral bands from 0.4 to 14.4 µm with horizontal resolutions (depending on band) ranging from 250 m to 1 km at nadir.Details on the MODIS cloud optical properties in the MYD06 data set are found in Platnick et al. (2017).Prior to use, the MODIS cloud data are mapped and weighted over the AIRS footprint as described below in Sect.3.2.2.

AIRS-OE retrieval overview
Figure 2 shows a simplified block diagram of the AIRS-OE retrieval procedure.For convenience to the reader, blocks are annotated with the section numbers of this paper, where more detailed descriptions can be found.Input parameters are described below in Sect.3.1 to 3.5.A brief description of the retrieval itself is in Sect.3.6, followed by information content analyses (3.7) and quality control filters (3.8).

Optimal estimation cost function
The mathematical basis for optimal estimation retrievals is described by Rodgers (2000).Implementation is similar to that of TES (Bowman et al., 2006) with significant differences in the treatment of clouds.The retrieval algorithm minimizes the difference between an observed and a forwardmodeled radiance, subject to a quadratic constraint, through the cost function: where y is the vector of measured radiances, F( x, b) is the forward-model radiance, x is the full state vector, described below, b contains additional variables needed (but not retrieved) and observational metadata (e.g., scan angle) for calculating radiances, ẑ is the retrieval state vector, described below, z a is the a priori retrieval state vector, S −1 ε is the inverse radiance noise covariance, and S −1 a is the inverse covariance of the a priori z a .(Note that we accent a retrieved quantity with a caret, e.g., ẑ, to distinguish it from the "true" quantity, z.)The measurement error covariance, S ε , contains the radiance noise error covariance from the instrument.It can also contain other random radiance error sources, such as those from forward-model calculations, although these have not been included in this study.We discuss the measurement error covariance used below in Sect.3.3.The a priori state vector, z a , is also the first guess in a retrieval.The full state vector, x, has as many elements for each retrieved profile constituent (T atm , H 2 O, O 3 and CO 2 ) as there are layers in the forward model at or above ground, plus those for retrieved scalar quantities, T sfc , T cldtop , τ cld and r eff .There is a maximum of 100 layers in our forward model from the surface upwards, on its fixed-pressure grid with level pressures from 1100 to 0.1 mb (as described in Strow et al., 2003).The pressure layers are constructed as the log mean of the upper and lower pressures levels: P layer = P 2 − P 1 ln (P 2 /P 1 ) . (2) Following Bowman et al. (2006), the forward-model layer gridding must be fine enough for calculation of the observed radiance but is usually much finer than the vertical resolution of a retrieved profile.The retrieval state vector, z, has a reduced number of layers, which varies by constituent, to reflect a lower vertical resolution.(A maximum 42 layers are retrieved for T atm , 28 for H 2 O, 10 for CO 2 and 9 for O 3 .)The retrieved state vector, ẑ, is mapped to the full state vector, x, when the forward model is called to calculate a radiance: In our retrieval, the matrix M performs a piecewise linear interpolation by log pressure from the lower number of layers in ẑ to the higher number of layers in x.For gas profiles, τ cld , and r eff , logarithmic quantities are used in the state vector to ensure that their linear values always remain positive as input to the forward model during retrieval iterations.Retrievals for T atm , T sfc and T cldtop are linear quantities.The state vector thus usually contains both linear and logarithmic elements.Description and determination of the different elements of the cost function are described in the sections below.
3  2016) and are similar to those used by the Total Carbon Column Observing Network (Wunch et al., 2011).

MODIS cloud a priori information and mapping to AIRS footprint
For a priori data on T cldtop , τ cld and r eff in each AIRS observation, weighted averages of MODIS level 2 data are made over the AIRS spatial response function (SpatialRF).An average AIRS SpatialRF is first calculated for the observation (see Schreier et al., 2010).The SpatialRF varies by scan angle and spectral channel, and we make a simple average of the SpatialRF using only the channels used in the retrieval.Using formulae described by Pagano et al. (2015), the spatial response is mapped over the target scene with an approximate resolution of 0.5 km at nadir.MODIS data from the MYD06_L2 (Aqua) product are then mapped in the vicinity of the AIRS observation.These data have a horizontal resolution of about 1 km at nadir.The mapped AIRS SpatialRF is then spatially interpolated onto this MODIS horizontal mapping and normalized to sum to unity.
Once the AIRS and MODIS footprints are co-located, MODIS retrieval fields for 1 km cloud-top temperature, cloud optical thickness and cloud effective radius are extracted and mapped (as data are available).Figure 3 illustrates a sample MODIS T cldtop , τ cld and r eff field overlaid by the AIRS SpatialRF.Weighted averages and weighted standard deviations of the MODIS T cldtop and r eff are then calculated on the interpolated AIRS SpatialRF, excluding MODIS cloud-free pixels.Calculations are similar for τ cld , but cloudfree pixels are included in the averaging calculation with an optical depth of zero.From the MODIS Cloud_Mask_1km field, we extract and similarly map the cloud mask status (0 is undetermined, 1 is determined), cloud mask cloudiness (0 is confidently cloudy or fill if status flag is 0, 1 is probably cloudy, 2 is probably clear, 3 is confidently clear) and thin cirrus flags (0 is yes or fill if status flag is 0, 1 is no).The weighted averages of these flags over the AIRS scene are used to decide (a) whether the scene is clear; (b) that a cirrus cloud too thin for a confident MODIS retrieval of its cloudtop temperature is in the scene, but the retrieval by AIRS should be attempted using assumed cloud-top temperature a priori information; or (c) that a retrieval in a cloudy scene will be attempted using the averaged MODIS cloud data as a priori data.We categorize AIRS scenes as clear, thin cirrus or cloudy (noting that MODIS does not report τ cld and r eff at night) and set a priori cloud data accordingly.We describe the criteria for these bins in turn and the cloud a priori data selected for them.

Clear
An AIRS scene is treated as clear if a. the average for the cloud mask status flag is greater than 0.95, b. the average cloud mask cloudiness is greater than 2.5, and c. the average thin cirrus flag is greater than 0.9.
In this case, no cloud information is in the retrieval or full state vector, and the retrieval algorithm directly retrieves T atm , T sfc , H 2 O, O 3 and CO 2 .

Thin cirrus in daytime and at nighttime
A scene is considered to have thin cirrus but of unknown temperature if the average thin cirrus flag is ≤ 0.9 and either the following are true: a. cloud-top temperature cannot be calculated because of missing values, or b. both average cloud mask status > 0.95 and cloud mask cloudiness > 2.8.
In this case, we assume a default a priori T cldtop of 230 K.For daytime scenes, the initial τ cld is set to 0.1.For nighttime, the initial optical depth thickness is set to 1.For both day and night, we set the initial r eff at 40 µm.

Cloudy scenes in daytime and at nighttime
A scene is treated as cloudy (ice or water cloud) if the weighted average of the MODIS cloud-top temperature on the AIRS footprint can be calculated.The result is used as the a priori T cldtop .If τ cld can be calculated, it is used as the a priori value but is set to no less than 1 × 10 −3 .If a τ cld cannot be calculated during daytime, then the a priori τ cld is set to 1 × 10 −3 .(This lower limit was set to examine retrieval sensitivity for τ cld , which can depend not only on the true τ cld itself but also on the thermal contrast with the ground.Initial tests indicate that the lower limit for any sensitivity to τ cld is ∼ 0.005.)At nighttime, the a priori τ cld is 1.If the weighted average of the MODIS r eff can be calculated, it is used as the a priori value.If the average cannot be calculated or if it is nighttime, the a priori r eff is 40 µm.Lookup tables for cloud absorption and scattering parameters had a particle radii range from 5 to 85 µm; reported cloud absorption and scattering parameters outside this range rely on extrapolated parameters, and results may not be reliable.

Emissivity
Wavelength-dependent surface emissivities are input as fixed parameters and are not retrieved or modified; this may be revised in future versions of AIRS-OE.For ocean emissivity, we use the National Center for Environmental Prediction -Environmental Modeling Center (NCEP/EMC) Infrared Sea Surface Emissivity (IRSSE) formulae and coefficients (van Delst, 2003), calculated for channel frequency, view angle and wind speed, with the latter estimated from the ECMWF analysis (described above).Land emissivity is from the Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin -Madison Global Infrared Land Surface Emissivity Database (Seemann et al., 2008).These are monthly maps of land emissivity at 10 wavelengths from 3.6 to 14.3 µm, gridded spatially by 0.05 • .This spatial gridding is smaller than the AIRS footprint, so we spatially interpolate the emissivity to the coincident MODIS gridding (described in Sect.3.2.2 and as illustrated in Fig. 3).If the target scene is a mixed land-ocean surface, we calculate the ocean emissivities (as above) and use them to fill in the ocean parts of the MODIS grid.With the emissivities at the 10 wavelengths on the MODIS gridding, the AIRS SpatialRF is used to calculate weighted averages.Emissivities for each AIRS retrieval channel are then calculated by wavelength interpolation from these weighted averages.As the emissivity database does not extend prior to the calendar year 2003, we (arbitrarily) use the data from 2003 for observations made in 2002.

A priori covariances
In this initial evaluation of the retrieval described here, a priori covariances (S a in Eq. 1) are listed in Table 1.The covariances are ad hoc but guided by previous experience with AIRS and TES retrievals.We recommend caution in applying resultant errors, although they may still be useful in comparing results between retrievals.Note that H 2 O, O 3 , CO 2 , τ cld and r eff are retrieved as log e quantities, and the covariances of their logarithms are used.
Off-diagonal elements of the covariance matrices are created using assumed length scales: where σ 2 ij is the off-diagonal covariance for layers i and j , σ i , σ j are the square roots of the on-diagonal covariances, h i , h j are the estimated altitudes, and l is the assumed length scale.
The off-diagonal length scale for temperature and water vapor was kept low (0.5 km) as this tended to reduce the effect of unrealistic retrievals at layers below clouds adversely affecting retrievals above clouds.(This is discussed further in Sect.3.8) Covariance matrices are calculated individually for each constituent and then "stacked" into a larger matrix for use in the simultaneous retrieval.At present, we are not using covariances between constituents (say, between temperature and water vapor), but this will be investigated for use in later versions.
Measurement error covariance (S ε in Eq. 1) is taken directly from the radiometric noise reported in the AIRS level 1b product (see Pagano et al., 2003).This is reported channel by channel, without correlations between them.For this study, S ε is a diagonal matrix, and we have assumed the noise to be uncorrelated across channels, Gaussian and not scene dependent.A study by Tobin et al. (2007), using principal component analysis on AIRS radiance data, showed that the contribution from the correlated error can be significant depending on the detector array on the AIRS focal plane (see also Pagano, 2002).Channel radiance error can be dependent on the channel radiance in the shortwave above ∼ 2200 cm −1 , but these channels are not used in this study.Only a minor dependence of the radiance error on channel radiance is observed in the midwave (∼ 1200 to 1700 cm −1 ) and there is effectively no dependence for channels below ∼ 1200 cm −1 .Channels exhibiting non-Gaussian "popping" are flagged and excluded from the analysis of an observation (see Weiler et al., 2005).
As noted above in Sect.3.1, random errors in the calculated radiances from the forward model (briefly described below in Sect.3.4) are not added into the measurement error covariance as used in this study.The random errors from the "noncloudy" part of the forward model may be smaller than the noise error from the AIRS instrument for most channels (see, for example, Fig. 2 in DeSouza-Machado et al., 2017), but a significant random error may be introduced from the calculation of the cloud absorption and emission, and these require further investigation.It is planned that future versions of the algorithm will include estimates of the random error from the forward model, and the correlated (that is, offdiagonal) parts of the radiometric noise covariance.

Forward model
The forward model is the Standalone AIRS Radiative Transfer Algorithm (SARTA; Strow et al., 2003Strow et al., , 2006)), supplemented with a delta-four-stream (D4S) calculation for cloud transmissivity (Ou et al., 2013).This joint SARTA+D4S forward model has been used to retrieve ice cloud parameters from single-footprint AIRS observations (Kahn et al., 2014), and of three forward models tested, using SARTA+D4S produced the lowest biases in temperature and water vapor compared to coincident sondes in clear scenes.Here, we use this model to additionally retrieve water cloud properties.For ice clouds, scattering parameters are from Baum et al. (2007).For liquid water clouds, we use Mie-scattering parameters calculated using formulae from Mishchenko et al. (2002).
Within the SARTA+D4S forward model, SARTA calculates the cloud-free gaseous transmissivities for each pressure layer and retrieval channel given the temperature and gas profiles, emissivity, scan angle, etc.The D4S code calculates the cloud transmissivities for retrieval channels given a cloud-top temperature, optical depth and particle size.As each pressure layer in the forward model is assumed to be homogeneous, the gaseous transmissivity for each channel is multiplied with that of the cloud to produce a combined transmissivity within a single layer.Note that this assumes the cloud can be modeled to fit in one vertical layer, no matter how thick the cloud.The forward-model layer selected for this is the tropospheric layer, which has the lowest pressure, with an atmospheric temperature higher than or equal to the cloud-top temperature (or the next lowest pressure layer if the absolute difference between the cloud-top temperature and the layer temperature is smaller).
Only cirrus parameters are used at cloud-top temperatures below 253.15 K, while only Mie cloud parameters are used at temperatures above 273.15K. Between these temperatures, we use a sliding weight between Mie and cirrus-derived cloud.This approach may overestimate the amount of ice occurrence as the majority of cloud tops within the temperature range of mixed phase and supercooled clouds (233-273 K) is mostly liquid according to lidar estimates of phase (e.g., Hu et al., 2010).This transition of phase was tested by changing the lower boundary of this mixed-phase range in the retrieval from 253.15 to 233.15 K and reprocessing a test granule (no.44, 6 September 2002, described below).Changes were scattered and isolated in retrieved T atm , H 2 O and relative humidity (the calculation of which is described in Sect.3.7.3)and were largest in the boundary layer, with about 7 % of retrieved relative humidities changing by more than ±5 % at ∼ 900 mb.For clouds with a MODIS-derived a priori cloud-top temperature less than 273.15K, about 9 % of T cldtop changed by more than ±5 K, and 24 % of τ cld changed by more than ±10 %.However, at low to moderate optical depths ( 1) these changes tended to happen in regions where there was a high standard deviation in the MODISderived average cloud-top temperature over the AIRS FOV (> 20 K), which is more likely to contain a mixture of iceonly, liquid-only and mixed-phase clouds.Put simply, the clouds in such scenes are more complicated to model, and more investigation is warranted.
This forward modeling of temperature, trace gases and cloud properties, while computationally fast, is best suited for optically and geometrically thin clouds and may not be well suited to thick clouds or where significant cloud formations occupy different heights within an AIRS pixel.In this initial effort, these scenes often, but not always, produce retrievals with poor spectral fits (described below in Sect.3.7.4 and 3.8) and are filtered out in quality control.We discuss possible ways to improve the forward modeling of clouds in Sect.6, but we note here that in future versions it may be useful to include an effective Mie cut-off temperature as a retrieved parameter in the state vector.Put another way, minimization of the cost function, Eq. ( 1), would be used to modify the temperature range of the transition from supercooled water to ice clouds.This may more effectively model clouds that contain a mixture of cirrus and supercooled water droplets.

Retrieval channels
Table 2 lists the spectral channels used in AIRS-OE retrievals.This channel list is similar to that for AIRS V6, except only longwave channels were used (< 1650 cm −1 ).We found that using channels in the shortwave region of the AIRS bandpass would often result in retrievals not converging or producing unrealistic retrieval quantities.This may be partly related to the shortwave channel radiance noise error being too low in scenes where the radiance was high, as described above in Sect.3.3, with the algorithm failing to fit the calculated spectrum within the underestimated noise of these channels.This also may be related to errors in calculating outgoing radiation from reflected sunlight, which remains a challenge in the near-infrared, particularly in cloudy scenes because of uncertainties in the scattering/absorption ratio (Nakajima and King, 1990;Nakajima et al., 1991).
For O 3 , we do not use the 9.6 µm band as its inclusion often results in the retrieval failing to converge.(As noted by Kulawik et al. (2006b) for the TES retrieval, difficulty in finding a global minimum can happen when retrieving all species at once.)However, including ozone in the retrieval (through its weak absorption within the 14 µm CO 2 band) but not in-cluding the strong 9.6 µm band improves the overall fitting with fewer failed retrievals.Comparisons of H 2 O and T atm with validation measurements also improve (not shown).We therefore retrieve O 3 as an interferent gas solely to improve the fitting within the 14 µm CO 2 region, and these O 3 retrievals are not recommended for further study.

Retrieval by minimization of cost function
After setting the different elements of the cost function (Eq. 1) as described above, the retrieval is performed by iteratively minimizing the cost function by modifying the retrieval state vector ẑ with a combination Gauss-Newton and Levenberg-Marquardt solver.Formulae are described by Bowman et al. (2006), applying the algorithm of Moré (1978).(See also Sarkissian, 2001.)A simultaneous retrieval of τ cld , T cldtop , r eff , T sfc , T atm , H 2 O, O 3 and CO 2 is made.Convergence tests are as described in Sect.IV.B(2) of Bowman et al. (2006), setting the threshold value, ε, of 0.2.If a given retrieval cannot converge within a specified number of iterations, or both the level 2 norm of the trust region ( in Moré, 1978) and linearity measure (ρ) within the Levenberg-Marquardt solver fall below 10 −3 , the algorithm is halted and flagged as non-convergent.Converged retrievals are analyzed for information content and are quality-control (QC) checked as described below.

Averaging kernels
We assume that the retrieval is nearly linear in the vicinity of the solution, although we have not formally evaluated this assumption.The Jacobian is the matrix of derivatives of the outgoing radiance to changes in each element of the state vector and is calculated by finite difference for each retrieval iteration.The gain, G z , is a measure of the sensitivity of the retrieval, ẑ, to changes in the radiance: The gain is multiplied by the Jacobian to produce the averaging kernel matrix, A, which is a measure of the sensitivity of the retrieval vector, ẑ, to changes in the true state, z: This is a square matrix dimensioned n × n, where n is the number of elements of the state vector, and as described below, is useful for calculating the error covariance of the retrieval.Each element of the averaging kernel matrix is a measure of the sensitivity for one retrieved member of a state vector (ẑ i ) to the changes in the true value of that member (z i ) or to the true value of a different member (z j ).That is, Figure 4 shows a sample averaging kernel from a simultaneous retrieval taken during the daytime on 6 September 2002 at 18.1 • N, 133.8 • E, over the Pacific Ocean, south of Japan (within the same granule depicted in Fig. 1 of Kahn et al., 2014).A thin cirrus cloud is retrieved with a T cldtop equal to 220 K (∼ 155 mb) and a τ cld of 0.42.The axes indicate the retrieval pressure layers for each constituent and are not set on a regular altitude or pressure scale.T atm , T sfc and T cldtop are retrieved as linear quantities, but H 2 O, O 3 , CO 2 , and τ cld and r eff are retrieved as their natural logarithms and their partial derivatives are reported as such.Note that the color scale is restricted to emphasize the weaker sensitivities.The diagonal of the averaging kernel indicates the sensitivity of a constituent to changes in its true self, while (as will be seen) off-diagonals (within the same constituent) have information on the vertical resolution of the retrieval.
A row of the averaging kernel contains measures of the sensitivity of a single element of the state vector to changes in the true values of itself and other elements.Note how in this retrieval, looking across the row for T sfc , the retrieved surface temperature can be sensitive to changes (and errors) in the true surface temperature itself, the lower tropospheric temperature profile and the cloud-top temperature.There is little to no sensitivity to changes in H 2 O, CO 2 and O 3 .By contrast, most rows for CO 2 indicate retrieval sensitivity to changes in its true self and also changes in all other retrieved constituents.
A column of the averaging kernel contains measures of the sensitivity of the entire retrieval state vector to changes in a single true state vector element.For example, looking up the T sfc column, all constituent retrievals are affected somewhere (even if only weakly) by sensitivity to changes in the true surface temperature.By contrast, looking at the columns for CO 2 , changes in the true CO 2 tend to affect the retrieved CO 2 but not other constituents.
The averaging kernel can be subdivided to provide sensitivity data on scalar values (e.g., T sfc and individual cloud properties) or sensitivity and vertical resolution for individual profiles.Figure 5 shows the retrievals, individual averaging kernels and ancillary information for the T atm and H 2 O profiles (using the parts of the averaging kernel matrix that are ∂ Tatm /∂T atm and ∂ ln[ The leftmost panel shows the a priori data and retrieved temperature profile, along with the estimated error (discussed below).For clarity, the error is shown as a separate line (with a separate axis) rather than an error bar.The second panel shows the rows of the T atm averaging kernel, along with its row sums.An averaging kernel can be examined to better understand the sensitivities of the measurement and retrieval.Note, for example, that the temperature averaging kernel www.atmos-meas-tech.net/11/971/2018/ Figure 4. Sample averaging kernel from a simultaneous retrieval.A row of the averaging kernel matrix is a measure of the sensitivity of the retrieved value to changes in the true value of itself and other parameters, shown in the columns, assuming the retrieval is in a near-linear regime.A column indicates the sensitivity of the retrieved state vector to a change in the true value of a single retrieval parameter.The color scale has been limited to better show the weaker sensitivities.
rows are comparatively low in the boundary layer (750 mb), indicating lowered sensitivity to changes in the true temperature, but sensitivity increases at altitudes above this in the free troposphere.These changes in sensitivity affect the error (in the leftmost panel), which reaches a minimum in the region of about 380 mb.The row sum of an averaging kernel row is a rough but useful indicator of how much a retrieval relies on the data for its results (see Sect. 3.1.5 of Rodgers, 2000).A row sum near unity indicates that the retrieval at that layer relies mostly on the observed spectral data, while a value near zero indicates most reliance on the a priori data.
While the row sums are indicative of how much information came from the observation, they do not indicate vertical resolution.Indeed, a visual inspection of the temperature averaging kernel rows in the left panel of Fig. 5 shows that while the row sums are high in the region between about 300 to 100 mb, the widths of the peaks are much broader than those below at higher pressures.To estimate vertical resolution, we use a simple full width at half maximum calculation for each averaging kernel row using a Gaussian fit and assuming a 7 km scale height in converting pressure to altitude.(Other approaches are described in Sect.3.3 of Rodgers, 2000.)From this fitting approach, shown in the third panel of Fig. 5, the vertical resolution is about 1 to 1.5 km from the ground to about 300 mb, above which the resolution of the T atm retrieval quickly degrades.The fitting approach used here is not useful at pressures lower than about 200 mb where the rows of the T atm averaging kernel become much flatter and can be double-peaked when crossing the tropopause.The right three panels of Fig. 5 show the H 2 O a priori data, retrieval and error, averaging kernel with row sums and estimated vertical resolution.(As the logarithm of the H 2 O volume mixing ratio is retrieved, the upper and lower errors of the retrieval in linear space are slightly different.They are shown as separate lines rather than error bars for clarity.)As with T atm , the averaging kernel at pressures greater than 750 mb indicates lower sensitivity in the boundary layer.Sensitivity improves and is fairly constant between 750 mb up to about 200 mb, above which sensitivity decreases and effectively disappears at 100 mb.The rightmost panel of Fig. 5 shows the approximate vertical resolution.The flattened averaging kernels near the boundary layer lead to a local maximum in the vertical resolution of ∼ 3.5 km, seen at about 800 mb, but the averaging kernel rows become more sharply defined at lower pressures.This improved definition is reflected in the vertical resolution, which is about 1.8 km at 700 mb, and steadily increases above to a maximum of 4.3 km at 175 mb.
Note for an AIRS retrieval, an averaging kernel is scene dependent.Sensitivities at different layers depend on the amounts of trace gases present, the temperature lapse rate, the particulars of the cloud field, the view angle and the employed spectral channels.Scene dependence has been noted in studying averaging kernels and vertical resolution from the (version 5) AIRS operational results (Maddy and Barnet, 2008).Since the AIRS-OE retrievals are simultaneous and not sequential, the averaging kernel describes dependencies within and between retrievals of different constituents and can be used to more robustly calculate uncertainties as described below.

Error estimation
The smoothing error covariance measures the uncertainty in the fine structure of the retrieval due to the measurement's limited vertical resolution.However, as we have an averaging kernel from a joint retrieval, the smoothing error also indicates how the uncertainty in one retrieved constituent affects the uncertainty in another: The retrieval noise error covariance calculates the impact of the radiance noise on the retrieval: Again, as noted in Sect.3.3 and as used here, S ε is a diagonal matrix; future versions of the retrieval will include off-diagonal components and estimates of the forward-model random error to more accurately reflect the error covariance.With substitutions, these terms can be added to provide the covariance of the maximum a posteriori solution: with the square roots of the diagonal reported as errors for the state vector.Note also that the total retrieval error does not include any systematic errors from the forward model (e.g., those due to instrumental line-shape errors, spectral biases or other errors that are correlated across observations), although we note the SARTA model is tuned to better match outgoing radiances as calculated from coincident measurements and analyses (see Strow et al., 2006).We again emphasize that since our a priori covariances are ad hoc, caution should be observed in using the reported errors.
For this initial version of our algorithm, we have not implemented code to calculate the model parameter error, which contains the uncertainty from parameters that affect the retrieval but are not retrieved themselves (surface pressure, emissivity, scan angle, etc.): In this case, the total retrieval error covariance would be the sum of Eqs. ( 11) and ( 12): The addition of the model parameter error (Eq.12) is planned for future development.
For constituents retrieved in logarithmic space, the error reported for the i'th element, i , is the error in the logarithm of the retrieved value, ẑi , with the range [lower, upper] of the retrieval in linear space being exp ẑi − i , exp ẑi + i . (14)

Calculation of relative humidity and error
In calculating relative humidity (RH), we use the layer retrievals of temperature and water vapor.Equations (2.5) and (2.21) of Wagner and Pruß (2002) are used to determine saturation pressures of water vapor over liquid and ice.At temperatures between 253.15 and 273.15 K, we set saturation pressure as a sliding-scale-weighted average of those over ice and over water.The relative humidity error calculation uses recalculated RHs by adding the errors from the temperature and (separately) the positive linear value of the water vapor error (the right-hand side of Eq. 14).We report the relative humidity uncertainty as the root sum of squares of the differences between these recalculated relative humidities and the reported values.

Chi square fitting parameter
The chi square fitting parameter, χ 2 , is a goodness-of-fit statistic of how well a spectrum's radiance is fitted within the bounds of the radiance error: where N is the number of channels, and ε i is the radiance error in channel i.A χ 2 1 indicates a poor spectral fit to the observed radiance.While the χ 2 does not directly enter into the error characterization, it is used in quality control as described below.

Quality control (QC) filtering
Retrievals that do not meet the following three criteria are filtered out: 1. normal convergence within the maximum specified number of iterations, 2. Chi square fitting parameter, χ 2 < 3, and 3. retrievals in layers with T atm > (T cldtop -10 K) require a surface temperature averaging kernel > 0.6.
The first criterion is to avoid wasting computational resources on poorly converging retrievals.The second criterion is to avoid reporting profiles with poor spectral fits.This often happens under ice cloud conditions when the cloud optical depth is high ( 20); it is likely that the radiative transfer is incorrectly calculated because a cloud is assumed to fit in one vertical model layer while in reality, thick clouds extend over many model layers.Poor spectral fits can also often occur when there is a high standard deviation, 20 K, of the MODIS 1 km cloud-top temperature weighted over the AIRS spatial response function.Again, we suspect that this poor fitting is from limitations in our forward model, which is limited to one cloud layer; the radiative transfer calculation can be inadequate when there are several cloud tops at different temperatures within the AIRS footprint.
The third criterion is a means to remove layers of a profile that have unphysical values of the relative humidity calculated from the retrieved temperature and water vapor; these are usually in or near the boundary layer.We again note that a cloud's transmissivity is incorporated in only one layer of Atmos.Meas. Tech., 11, 971-995, 2018 www.atmos-meas-tech.net/11/971/2018/ the forward-model vertical pressure grid, no matter how thick the cloud.We hypothesize that this can lead to erroneous outgoing radiances for temperature and water vapor channels in regions at or below moderately thick clouds, which in turn, produces erroneous Jacobians and averaging kernels.However, surface temperature retrievals appear to more correctly give a low-to-zero averaging kernel under moderate-to-thick cloud optical depths.We therefore require that retrievals at layers below clouds must "see" the surface (determined by the T sfc averaging kernel having a minimum of 0.6).For retrievals above clouds for which the surface temperature averaging kernel is less than 0.6, an additional thermal contrast provided by a 10 K buffer between the cloud top and the lowest profile layer to pass quality control eliminates more unphysical retrievals.Most, but not all of the retrievals that produce unphysically high relative humidities are eliminated by this method, usually in the boundary layer.

Results
For an initial examination of cloud, temperature and H 2 O profile retrievals for this effort, we use results of an AIRS daytime granule (#44) over the subtropical western Pacific Ocean from 6 September 2002.This is the same granule examined and discussed in detail by Kahn et al. (2014) for AIRS V6 cloud products.This granule has a large mix of cloud types and weather regimes, including a tropical cyclone to the west, and a mix of low and mid-level clouds from the center to the east and to the south.For an AIRS granule, there are 90 observations on the cross-track and 135 observations along-track.Here, the yield of retrievals passing quality control for at least some layers is ∼ 56 % of 12 150 observations.In the following subsections, we compare our results to those of different algorithms operating on the same scenes.MODIS-avg retrievals are from the MODIS 1 km pixels (MYD06 data set) in a weighted average over the AIRS SpatialRF, as described above in Sect.3.2.2.

Cloud-top temperature, effective cloud optical depth and effective cloud particle radius
Figure 6 displays cloud-top temperatures (T cldtop ), effective optical depths (τ eff ) and effective particle radii (r eff ) from the a priori data (left column) and AIRS-OE retrievals (center column) from the granule described above, along with retrieval averaging kernels (right column).With a few exceptions, the a priori data are generated from co-located MODIS-avg retrievals.Retrieved quantities have similar fields to their a priori counterparts.As indicated by the averaging kernels, sensitivity is enhanced for ice clouds, which is likely because of the higher thermal contrast with the surface.For all clouds as they get thicker, their infrared radiation is more dominant in the window channels, producing a more confident retrieval.An examination (not shown) indi-cates that the T cldtop averaging kernels reach ∼ 0.5 at cloud optical depths of about 0.4 for ice clouds and between 1 and 2 for water clouds.(An averaging kernel of 0.5 indicates that roughly half the information of the retrieval is from the spectral data and half is from the a priori data.)We do note, however, that the retrievals can fail quality control at or near the thick center of the cyclone, likely for reasons described in Sect.3.4.

Comparison with CloudSat/CALIPSO
We make an additional "snapshot" comparison with (multilayer) cloud observations by the combined CloudSat cloud profiling radar (Stephens et al., 2002) and the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations lidar (CALIPSO; Winker et al., 2009), using the 2B-CLDCLASS-LIDAR product (Wang et al., 2013).Figure 7 compares AIRS-OE cloud-top retrievals with near-coincident Cloud-Sat/CALIPSO (CsC) observations, using the same date and region as in Fig. 2  Similarly to the (1 km footprint) MODIS retrieval on the same transect (see Fig. 2 of Wang et al., 2016), AIRS-OE retrieves cloud tops at a lower altitude than CsC for the thicker regions of the cirrus clouds (marked A and B in Fig. 7).This is similar to comparisons of AIRS version 5 cloud retrievals to CsC by Kahn et al. (2008), citing Holz et al. (2006) in how infrared retrievals of cirrus tend to place the cloud top 1 to 2 km or more below the physical cloud top.The retrieval does not pass quality control at a few points over the deep convective core, likely because radiance spectra above such deep clouds can be poorly calculated by our forward model.Significant differences in cloud-top altitude are notable along the thin cirrus between about 5 and 12 • (C), with retrieved optical depths of about unity or less.However, a close examination of the cloud-top temperature and cloud optical depth fields in the upper panels between 10 and 5 • N show that the CloudSat/CALIPSO transect is on the edge of a thin northsouth-aligned strip of ice cloud, and the centers of the closest AIRS observations to the CsC transect were 2 to 5 km apart, so some sampling bias may be present.Similarities and differences shown in Fig. 7 may be broadly similar to previous AIRS/MODIS/CloudSat/CALIPSO comparisons (e.g., Kahn et al., 2007;2008), but here we explicitly illustrate the feasibility of forward-modeling clouds in a hyperspectral IR retrieval, simultaneously with temperature and trace gases.ond column shows the initial guess for each retrieval quantity, calculated by linear interpolation in time and space and vertically by log(pressure) from 6 h ECMWF analyses.The third column shows the AIRS-OE retrievals passing the quality control criteria described in Sect.3.8.The fourth column shows the estimated error of the AIRS-OE retrieval.Note that H 2 O is presented in a volume mixing ratio, not a mass mixing ratio.Comparing AIRS-OE relative humidity to its a priori or AIRS V6 values shows significant local differences in relative humidity and many unphysically high RH values (> 100 %) throughout the region studied.We note, however, that the calculated error for the AIRS V6 relative humidity at this layer is fairly high, with a median of 28.7 % with an interquartile range (IQR, that is, the range between the 25th and 75th percentiles) of 26.0 to 31.8 %.The AIRS-OE relative humidity is biased high compared to AIRS V6, with a median bias of 8.5 % (IQR = −1.3 to 19.1 %).At 525 mb (Fig. 9), qualitative agreement for T atm , H 2 O and RH is improved compared to 918 mb across the ECMWF-derived a priori data, AIRS V6 and AIRS-OE.However, the AIRS-OE retrieval for H 2 O begins to depart from the a priori data to more closely resemble the AIRS V6 retrieval.For example, the a priori data shows a smaller region of higher water vapor to the east of drier air near 30 • N and 135 • E (A) while AIRS V6 and AIRS-OE show this region to be larger.The AIRS-OE median relative humidity error is 16.1 % (IQR = 12.0 to 19.9 %), smaller than that at the 918 mb layer.Regional biases in the H 2 O and RH of AIRS-OE compared to AIRS V6 can again be readily seen, although the median RH bias is reduced to 0.4 % (IQR = −4.4 to 6.7 %).

Temperature, water vapor and relative humidity profiles
At 321 mb (Fig. 10), the AIRS-OE temperature field shows a much broader region of cold air than either AIRS V6 or the a priori data (dark blue overlaid by A).However, the H 2 O and RH fields of AIRS V6 and AIRS-OE resemble each other more than the a priori data, and the additional horizontal resolution of AIRS-OE allows sharper boundaries to be seen between dry and wet regions.Although there are still many missing pixels in AIRS-OE, there is improved definition in the boundaries between lower and higher values of the H 2 O volume mixing ratio (B in upper left).For relative humidity, note that the region of humid air near 32 • N and 140 • E in AIRS V6 (C) is more fully resolved as three small but distinct regions in AIRS-OE.The median error of the AIRS-OE relative humidity is 13.1 % (IQR = 7.2 to 19.7 %).The median As a test of the algorithm's sensitivity to the infrared spectrum, Fig. 11 compares the 321 mb retrieval of relative humidity under different a priori assumptions.The left panels show the relative humidities calculated from T atm and H 2 O a priori data as interpolated from ECMWF analyses (as above), a climatology, and the neural-net a priori data of the current AIRS V6 retrieval scheme.The right panels show the AIRS-OE relative humidity retrievals from these different a priori data.In all cases, MODIS-avg data are used as cloud a priori data.While showing some minor differences, all retrieval relative humidity fields show a similar structure, indicating good sensitivity of the retrieval in the free troposphere over the ocean.
A rough estimation of the sensitivity as a function of pressure is shown Fig. 12, using averages of the row sums of the T atm and H 2 O averaging kernels from quality control-passed layers.The left panel of the Fig. 12 shows these profiles using ECMWF data as a priori data.(Results using climatology or the neural net, as in Fig. 11  above which it markedly decreases and is small across the 100 mb level.The right-hand graph shows yield increasing with altitude, as more retrievals become available above the cloud tops.It should be emphasized that the sensitivity of the retrieval of a constituent, as indicated by the averaging kernel and its row sums, is dependent not only on the strength of the Jacobians (Eq.5) but also on the size of the a priori covariance (Sect.3.1).As the a priori covariances used in this study are ad hoc, the sensitivity of the AIRS-OE retrieval (as illustrated by, say, Fig. 12) could change as the a priori covariances are refined in future versions.al. (2015).Figure 13 illustrates the location of the sondes matched to AIRS (launched within 3 h and 100 km of an AIRS observation), colored by the number of successful retrievals at 321 mb; anywhere from 3 to 105 successful AIRS-OE observations are achieved as matchup to a single sonde.
Whether an AIRS-OE retrieval is successful at a particular pressure layer depends to a large extent on the particularities of the cloud cover, and so the number of successful retrievals can vary widely for matched radiosondes.For example, of the 210 MAGIC sondes that were coincident with AIRS observations, an average of 41 ± 26 (1σ ) AIRS-OE successful retrievals were made for each sonde at 321 mb, while at 918 mb, an average 29 ± 25 retrievals were made (not shown).Simply taking a global average (or even a global median) of the differences between AIRS retrievals and sonde observations can introduce significant sampling biases, with clear or nearly clear areas overrepresented.We also found that average bias could often be significantly skewed by retrieval outliers -usually because the water vapor and relative humidity were unphysically high.Thus, to calculate an overall bias between AIRS-OE and the MAGIC sondes and compare this to the a priori and AIRS V6 results, we report the "median of the medians," calculated in this manner: a.For the ensemble of a priori and QC-passed retrievals for a single sonde, we calculate the median T atm bias, RH bias and relative bias in H 2 O (e.g., [AIRSsonde]/sonde) in %).
b.We do the same as above for AIRS V6 observations but only where a successful AIRS-OE retrieval is within an AIRS V6 footprint, and only layers that have an AIRS V6 quality control of 0 (best) or 1 (good).As there can be up to nine AIRS-OE retrievals within an AIRS V6 footprint, an AIRS V6 retrieval is only entered once when calculating the median to avoid overcounting.
Note that these criteria for matching up sondes and AIRS V6 observations are different from Kalmus et al. (2015), so caution should be taken when comparing these results with that work.Also, since the inclusion of an AIRS V6 profile in calculations depends on the QC result of an AIRS-OE retrieval, these data should not be taken as validation of AIRS V6 results.Note also that the AIRS V6 uses a different a priori value than AIRS-OE, and the AIRS V6 a priori value is not shown or compared here.Figure 14 illustrates the median temperature profile differences between the a priori, AIRS-OE and AIRS V6 retrievals for T atm , the relative difference for H 2 O and the difference in RH compared to the MAGIC sondes.Thin lines indicate the 25th and 75th percentiles of the distributions.For temperature, AIRS-OE (blue line) shows a negative bias of 0.65 K at the surface, increasing to a maximum of 0.9 K at 865 mb.The positive bias continues to about 400 mb, and is higher than either the a priori or AIRS V6 retrievals.Between 400 and 200 mb, the AIRS-OE retrieval is within 0.3 K, as are the a priori and AIRS V6 retrievals.For water vapor, the (global median) relative bias of AIRS-OE retrievals stays within 10 % of the sondes from the surface to about 800 mb, where it increases to about a 20 % bias at 525 mb, decreases to a 3 % bias at 321 mb and then increases again, as do the a priori and AIRS V6 retrievals.For the bias in relative humidity, except for a local minimum of −4 % at 840 mb, biases are positive and tend to be within 5 % up to 200 mb, but this good agreement may be partly due to compensating biases in temperature and water vapor.

Tropical Western Pacific (TWP), Southern Great
Plains (SGP) and North Slope of Alaska (NSA) sondes For these comparisons, we use AIRS observations co-located with high-quality radiosondes launched from ground-based sites of the U.S. Department of Energy's Atmospheric Radiation Measurement (ARM) Climate Research Facilities.Inclusion criteria and median bias calculations are similar to those for the MAGIC sondes (Sect.5.1). Figure 15 illustrates the median temperature biases, water vapor relative biases and relative humidity biases for the ECMWF analysisderived a priori profile, and the AIRS-OE and AIRS V6 re- trievals over the TWP site at Nauru, the SGP site in Oklahoma and the NSA site in Alaska.
For temperature, all three sites show AIRS-OE median biases for temperatures below ±1 K and median relative biases for H 2 O to mostly within ±15 %.At TWP, AIRS-OE generally shows larger biases compared to the a priori values for temperature but is similar to the ECMWF-derived a priori values for H 2 O and relative humidity from about 800 mb up to about 200 mb.At SGP, AIRS-OE shows a high bias of ∼ 0.7 K between the surface to 400 mb but it slightly improves on the a priori water vapor for most of the troposphere, except near 700 mb.AIRS-OE relative humidity biases compare well with the a priori data.At NSA, between about ∼ 800 and ∼ 220 mb, AIRS-OE temperature biases are low (less than ±0.25 K), as are water vapor relative biases (less than ±7 %).Relative humidities are improved compared to the a priori values from the surface to about 400 mb.While again we caution that the results in Fig. 15 should not be taken as validation for AIRS V6, we note that the singlefootprint retrieval AIRS-OE results generally compare well with the cloud-cleared spectra results of AIRS V6.

Summary and discussion
We have presented a new retrieval scheme for the AIRS instrument, AIRS-OE, which uses MODIS results as cloud parameter a priori data, and a forward model that incorporates cloud effects in its radiative transfer.As AIRS-OE directly uses level 1b spectra in retrievals, and not level 2 cloudcleared spectra as in AIRS V6, it improves the nadir horizontal resolution over AIRS V6 from ∼ 45 to ∼ 13.5 km.Focusing on cloud parameters, and temperature and water vapor profiles, we have presented some initial comparisons to currently operational AIRS version 6 products.The improved horizontal resolution has been shown to provide greater detail in water vapor and relative humidity fields in the free troposphere.As AIRS-OE rests on an optimal estimation framework and includes simultaneous retrieval of profiles and scalar variables, it has an information content analysis that operates both within an atmospheric parameter (e.g., uncertainties in the temperature profile) and across different atmospheric parameters (e.g., uncertainties in water vapor due to uncertainties in temperature).Initial comparisons against co-incident radiosondes indicate that retrieval biases for tem-perature and water vapor profiles are at least competitive with AIRS V6.Some aspects of this new retrieval need development.More realistic clouds in the forward model, with multiple cloud decks or clouds that extend over several model layers, will likely allow more footprints to be successfully analyzed.A better a priori τ cld at nighttime may be made by comparing brightness temperatures in the window channels to the a priori surface temperature, similarly to Kulawik et al. (2006a).Noting the work of DeSouza-Machado et al. (2018), it may be useful to combine MODIS and numerical weather prediction cloud data to determine an a priori profile for the cloudtop height, particle size, optical depth (or liquid/ice column) and horizontal and vertical extent of cloud in the AIRS FOV.Improved cloud results will hopefully better leverage and compliment the spatial coverage and horizontal resolution of MODIS, and the vertical precision and detail of Cloud-Sat/CALIPSO.A forward model incorporating scattering by dust and other aerosols (as previously done for AIRS by DeSouza-Machado et al., 2006, 2010) would open more regions for analyses.(See also Maddy et al., 2012.)Efforts can be made to better model the outgoing daytime radiance of the shortwave channels of AIRS (> 2200 cm −1 ) and improve the radiance noise error estimate, so we can more fully exploit the spectral range of the instrument, particularly near the 4 µm CO 2 band.An observation-based a priori covariance (including cross-species covariances), specific to region and season, would provide improved constraints and more realistic retrieval errors.Adding in retrieval linearity estimates, such as those described in Sect.5.1 of Rodgers (2000), would help to flag those retrievals where the error estimation may be suspect.A method of assigning ground-level snow and ice conditions to observations is needed -possibly from the colocated AMSU microwave instrument or other near-real-time data (see Pope et al., 2014).Addition of effective emissivity retrievals will likely be necessary to improve results over difficult regions, such as deserts or mountains.If the emissivity field is highly variable within the AIRS FOV (say, on a coastline), weighting by the MODIS cloud mask can help to determine an a priori emissivity as it affects the AIRS observation by taking cloud cover into account.The algorithm could be extended to better retrieve O 3 using its 9.6 µm band as well as retrieve CH 4 , CO and other gases within the AIRS bandpass.Successful implementation of these improvements may be challenging (or at least time consuming) but could prove useful not just to single-footprint retrieval from AIRS but also to other instruments such as CrIS and IASI.
Since the design stage of the AIRS instrument in the 1990s, increased computing power and advances in modeling cloudy spectra allow new approaches to utilize the high spectral resolution output of existing infrared sounders.The horizontal resolution gained by avoiding cloud clearing can provide more nuanced data for water vapor, especially where it is highly variable at smaller spatial scales.Even with the algorithmic deficiencies described, initial results presented here indicate that AIRS-OE retrievals on cloudy infrared spectra can compare well with operational AIRS V6 retrievals that require cloud clearing.Additional improvements in execution speed and data handling are needed before this work can become operational like AIRS V6.However, with some 16 billion AIRS infrared spectra since its launch in 2002, numerous opportunities are available for targeted studies with this new algorithm.

Figure 1 .
Figure 1.AIRS level 1b brightness temperature observations of adjacent cloudy spectra.Data are from daytime Granule 44, 6 September 2002.Average cloud-top temperatures and cloud optical depths are estimated from coincident MODIS L2 retrievals, averaged on the AIRS spatial response function (see Sect. 3.2.2 in text).

Figure 3 .
Figure 3. Sample MODIS fields of cloud-top temperature, cloud optical depth and effective particle size overlaid with the AIRS spatial response function (SpatialRF) interpolated to the MODIS grid.Data are daytime observations over ocean from 6 September 2002.For this example, the weighted averages over the AIRS SpatialRF are 211.3K for cloud-top temperature, 4.26 for optical depth and 29.5 µm for effective particle radius.

Figure 5 .
Figure 5. Sample retrieval profiles, errors, averaging kernels, row sums of the averaging kernels and approximate vertical resolutions for temperature (a) and water vapor volume mixing ratio (b).Each colored line in the averaging kernel panels is from a partial row of the averaging kernel (e.g., the rows of ∂ Tatm /∂T atm and ∂ ln[ Ĥ 2 O]/∂ ln[H 2 O], as seen in Fig. 4).
) (See Sect.V(B) ofBowman et al., 2006 and Sects.3.4  and 4.1 of Rodgers, 2000.)F. W.Irion et al.:  Single-footprint retrievals of temperature of Wang et al. (2016), who compared MODIS cloud classifications and CsC profiles.(As in Wang et al., 2016, the horizontal axis is ordered by decreasing latitude.)The upper two panels show AIRS-OE cloud optical depths and cloud-top temperatures from a daytime 31 July 2009 swath over the Pacific (latitude on the horizontal axis) with the CsC transect overlaid.The third panel shows the vertical extent of the clouds from CsC (in grey).Superimposed are the approximate cloud-top altitudes of the QC-passed AIRS observations (no more than one per AIRS cross-track) closest to the CsC transect.(Note that the distance between the center of an AIRS observation and the closest CsC observation can be up to 7.5 km).The AIRS observations in this panel are colored by the AIRS-OE retrieved cloud optical depth.The bottom panel again shows the CsC cloud layer and AIRS-OE cloud-top altitudes but colored by the AIRS-OE cloud-top temperature averaging kernel, which can be used as a measure of confidence in the AIRS cloud-top altitude.

Figure 8
Figure 8 presents maps of T atm (top row), H 2 O (middle row) and a calculated relative humidity (RH; bottom row) at the 918 mb layer for the 6 September 2002 granule discussed above.For comparison, the left column shows results from the operational AIRS V6 retrievals, interpolated by log(pressure) to the AIRS-OE retrieval layer.Note that the AIRS V6 retrievals used cloud-cleared radiances on the AMSU footprint, each point covering an area approximately nine times that of a single AIRS infrared footprint.The sec-

Figure 7 .
Figure 7.Comparison of AIRS-OE approximate cloud-top altitude with CloudSat/CALIPSO (CsC).(a) AIRS-OE cloud-top temperature with CsC transect.(b) AIRS-OE cloud optical depth.(c) CsC cloud layer altitudes (grey) with approximate AIRS-OE cloud-top altitudes, colored by retrieved optical depth.(d) Same as third panel but points colored to show cloud-top temperature averaging kernels.Data are from daytime observations, 31 July 2009 over the Pacific ocean.

Figure 8 .
Figure 8. AIRS V6 retrieval and AIRS-OE a priori data, retrievals and errors for temperature, water vapor and relative humidity (RH) at 918 mb for (daytime) Granule 44, 6 September 2002.

Figure 9 .
Figure 9. AIRS V6 retrieval and AIRS-OE a priori data, retrievals and estimated errors for temperature, water vapor and relative humidity (RH) at 525 mb for (daytime) Granule 44, 6 September 2002.
, were similar.)Error bars are 1σ standard deviations.The right panel shows the yield of quality control-passed retrievals.Temperature retrievals generally indicate a low-to-moderate sensitivity at the surface but increase with altitude to about 600 mb, then show high sensitivity upwards to 100 mb.Similarly, water vapor sensitivity is low to moderate at the surface but quickly increases with altitude.Sensitivity is high from about ∼ 850 to about 225 mb,

Figure 10 .
Figure 10.AIRS V6 retrieval and AIRS-OE a priori data, retrievals and errors for temperature, water vapor and relative humidity (RH) at 321 mb for (daytime) Granule 44, 6 September 2002.Note that unlike Figs. 8 and 9, water vapor volume mixing ratio is on a logarithmic scale.

5
Comparison with radiosondes 5.1 MAGIC campaign sondes AIRS-OE retrievals of temperature and water vapor are compared with radiosonde profiles from the ship-based Marine ARM GPCI Investigation of Clouds (MAGIC) campaign of September 2012 to October 2013.Sondes were launched from a Department of Energy Atmospheric Radiation Measurement mobile facility atop a container ship traveling between Honolulu and Los Angeles.An extensive description of the MAGIC field campaign and comparison of AIRS V6 and ECMWF ERA-Interim reanalysis water and temperature results with MAGIC have been reported by Kalmus et

Figure 11 .
Figure 11.AIRS-OE relative humidity (calculated from retrieved temperature and water vapor) at 321 mb using different a priori data.Panels (a), (c), (e) show the relative humidity calculated from the a priori data, while (b), (d) and (f) show the retrieval.Panels (a) and (b) uses an ECMWF analysis-derived a priori data linearly interpolated in time, space and log pressure to the AIRS observation.(c) and (d) uses a climatology.Panels (e) and (f) uses the neuralnet calculation (on the AIRS-AMSU footprint) that is used with the operational AIRS V6 retrieval.Note that while the different a priori data were used for temperature, skin temperature and water vapor, the same MODIS-derived cloud data (cloud-top temperature, cloud optical depth and cloud particle radius) were used as a priori data for the cloud retrievals.

Figure 12 .
Figure 12.Average row sums of temperature and water vapor averaging kernels (a) and percentage yield of quality control-passed retrievals (b) for (daytime) Granule 44, 6 September 2002.A priori data for temperature and water vapor were from ECMWF.Error bars are 1σ standard deviations.For clarity, some error bars are not shown.

Figure 13 .
Figure 13.MAGIC sonde launch locations that were matched to coincident AIRS observations.Points are colored by the number of QC-passed AIRS-OE retrievals of water vapor at 321 mb.AIRS observations were within 3 h and 100 km of the sonde launch.

Figure 14 .
Figure 14.AIRS-OE, AIRS version 6 and a priori profile biases from MAGIC radiosondes.(a) Median temperature difference (minus sonde).(b) Median relative difference in water vapor ([AIRSsonde]/sonde in %).(c) Median difference in relative humidity (in%).Thin lines represent the 25th and 75th percentiles.AIRS observations were within 3 h and 100 km of the sonde launch, and only those retrieval layers passing quality control were counted.In total, 8633 AIRS profiles were matched to 210 sonde profiles.See Sect.5.1 for description of data aggregation before calculation of medians.

Figure 15 .
Figure 15.AIRS-OE, AIRS version 6 and (AIRS-OE) a priori profile differences from radiosondes launched from the Tropical Western Pacific site (a), Southern Great Plains (b), and Alaska North Slope (c).Left column: median temperature differences (minus sonde).Middle column: median relative difference in water vapor ([AIRS -sonde]/sonde in %).Right column: median difference in relative humidity (in %).Thin lines represent 25th and 75th percentiles.AIRS observations were within 3 h and 100 km of sonde launch and only those retrieved layers passing quality control were counted.The a priori profile for AIRS V6 is not shown.See Sect.5.1 for description of data aggregation before calculation of medians.