Journal cover Journal topic
Atmospheric Measurement Techniques An interactive open-access journal of the European Geosciences Union
Journal topic
Atmos. Meas. Tech., 11, 3689-3715, 2018
https://doi.org/10.5194/amt-11-3689-2018
Atmos. Meas. Tech., 11, 3689-3715, 2018
https://doi.org/10.5194/amt-11-3689-2018

Research article 26 Jun 2018

Research article | 26 Jun 2018

# Comparisons of bispectral and polarimetric retrievals of marine boundary layer cloud microphysics: case studies using a LES–satellite retrieval simulator

Comparisons of bispectral and polarimetric cloud retrievals using LES
Daniel J. Miller1,3, Zhibo Zhang1,2, Steven Platnick3, Andrew S. Ackerman4, Frank Werner2, Celine Cornet5, and Kirk Knobelspiesse3 Daniel J. Miller et al.
• 1Physics, University of Maryland, Baltimore County, Baltimore, MD, USA
• 2Joint Center for Earth Systems Technology, University of Maryland, Baltimore County, Baltimore, MD, USA
• 3NASA Goddard Space Flight Center, Greenbelt, MD, USA
• 4NASA Goddard Institute for Space Studies, New York, NY, USA
• 5Laboratoire d'Optique Atmosphérique, Université des Sciences et Technologies de Lille, Villeneuve d'Ascq, France
Abstract

Many passive remote-sensing techniques have been developed to retrieve cloud microphysical properties from satellite-based sensors, with the most common approaches being the bispectral and polarimetric techniques. These two vastly different retrieval techniques have been implemented for a variety of polar-orbiting and geostationary satellite platforms, providing global climatological data sets. Prior instrument comparison studies have shown that there are systematic differences between the droplet size retrieval products (effective radius) of bispectral (e.g., MODIS, Moderate Resolution Imaging Spectroradiometer) and polarimetric (e.g., POLDER, Polarization and Directionality of Earth's Reflectances) instruments. However, intercomparisons of airborne bispectral and polarimetric instruments have yielded results that do not appear to be systematically biased relative to one another. Diagnosing this discrepancy is complicated, because it is often difficult for instrument intercomparison studies to isolate differences between retrieval technique sensitivities and specific instrumental differences such as calibration and atmospheric correction. In addition to these technical differences the polarimetric retrieval is also sensitive to the dispersion of the droplet size distribution (effective variance), which could influence the interpretation of the droplet size retrieval. To avoid these instrument-dependent complications, this study makes use of a cloud remote-sensing retrieval simulator. Created by coupling a large-eddy simulation (LES) cloud model with a 1-D radiative transfer model, the simulator serves as a test bed for understanding differences between bispectral and polarimetric retrievals. With the help of this simulator we can not only compare the two techniques to one another (retrieval intercomparison) but also validate retrievals directly against the LES cloud properties. Using the satellite retrieval simulator, we are able to verify that at high spatial resolution (50 m) the bispectral and polarimetric retrievals are highly correlated with one another within expected observational uncertainties. The relatively small systematic biases at high spatial resolution can be attributed to different sensitivity limitations of the two retrievals. In contrast, a systematic difference between the two retrievals emerges at coarser resolution. This bias largely stems from differences related to sensitivity of the two retrievals to unresolved inhomogeneities in effective variance and optical thickness. The influence of coarse angular resolution is found to increase uncertainty in the polarimetric retrieval but generally maintains a constant mean value.

1 Introduction

The cloud droplet size distribution (DSD) is an important microphysical property of liquid-phase clouds. Given the cloud water content, it largely determines the shortwave radiative effects of clouds (Twomey, 1977). It also plays a critical role in cloud-precipitation processes (Pruppacher and Klett, 1978). As a result, anthropogenic perturbation to the DSD could lead to a variety of cloud property changes with significant climate implications (Lohmann et al., 2007).

Many satellite-based techniques have been developed to retrieve cloud DSD properties from regional to global scales. These techniques typically infer DSD properties based on an assumed size distribution shape, characterized by an effective radius (re) and an effective variance (ve). One such retrieval method is called the bispectral total reflectance technique, hereafter referred to as the “bispectral technique”, which simultaneously retrieves cloud optical thickness (τ) and re from a pair of cloud reflectances, typically one in the visible to near infrared (VNIR) and the other in the shortwave infrared (SWIR) or midwave infrared (MWIR) spectral range (Nakajima and King, 1990b). This retrieval technique has been implemented for numerous satellite and airborne instruments, such as the Moderate Resolution Imaging Spectroradiometer (MODIS; King et al., 2003; Platnick et al., 2003, 2017), the Spinning Enhanced Visible and Infrared Imager (SEVIRI; Roebeling et al., 2006), and the Suomi National Polar-orbiting Partnership Visible Infrared Imaging Radiometer Suite (Suomi NPP VIIRS; Rosenfeld et al., 2014).

A second, fundamentally different, retrieval technique is the multi-angular polarimetric reflectance technique, hereafter referred to as the “polarimetric technique”. This retrieval requires multi-angular observations of the polarized reflectance in the cloudbow scattering region. In addition to re, the polarimetric technique can also retrieve ve (Bréon and Goloub, 1998). Global retrievals using the polarimetric technique were first demonstrated by the Polarization and Directionality of Earth Reflectance (POLDER; Deschamps et al., 1994) instruments operating from 1996 to 2013 on three different satellite platforms. The Aerosol Polarimetry Sensor (APS; Mishchenko et al., 2007) would have been the first spaceborne multi-angular polarimeter from the US to provide global aerosol and cloud property retrievals. Unfortunately, it was lost as a result of the satellite launch failure in 2011, which suddenly interrupted development of polarimetry-based remote sensing in the US. Recognizing the great potential of polarimetric techniques for aerosol and cloud remote sensing, NASA has invested heavily in recent years in the development of airborne polarimeters, such as the Research Scanning Polarimeter (RSP; Cairns et al., 1999), the Airborne Multi-angle Spectro-Polarimetric Imager (AirMSPI; Diner et al., 2013), and the Airborne Hyper-Angular Rainbow Polarimeter (Air-HARP; Martins et al., 2017). Moreover, several spaceborne missions are in development, such as the Multi-Angle Imager for Aerosols (MAIA; Liu and Diner, 2017); HARP; the Plankton, Aerosols, Cloud, ocean Ecosystem mission (PACE); and the Multi-viewing, Multi-channel, Multi-polarization Imaging mission (3MI; Marbach et al., 2013). Each of these missions will have a multi-angular polarimeter on board. In the foreseeable future, we may expect to have operational global retrievals of cloud droplet size distributions from both bispectral and polarimetric methods.

The bispectral and polarimetric remote-sensing techniques are the primary tools we have to obtain DSD observations on a global scale. It is therefore essential to identify and explain differences between the two techniques so we can better understand their respective advantages and limitations. A satellite retrieval intercomparison of POLDER and MODIS re retrievals by Bréon and Doutriaux-Boucher (2005) represented one of the first attempts to identify and understand the differences between the two techniques. The main finding from this study is that the bispectral MODIS retrieval of re (2.13 µm) (using the 2.13 µm SWIR band) is persistently 2 µm larger than the 150 km scale polarimetry-based POLDER retrieval over the ocean, despite a close correlation between the two. A variety of factors, from differences in sensitivity to cloud vertical profile to influence of cloud horizontal inhomogeneity, have been suggested to explain this difference. However, as pointed out by the authors, all these factors might contribute to the difference. It is difficult, if not impossible, to untangle them in observations and determine their relative importance. In addition, POLDER observations in this study were aggregated from the nominal 6 km spatial resolution to a much coarser 150 km resolution to achieve the angular resolution needed to resolve the cloudbow. The vast difference in spatial resolution (i.e., 150 km for POLDER and 1 km for MODIS) makes the interpretation of the 2 µm re difference between the two retrievals even more difficult.

A more recent study by Alexandrov et al. (2015) is based on observations from the recent sub-orbital Polarimeter Definition Experiment (PODEX) in 2013. In that study, the polarimetric re retrievals for marine stratocumulus decks off the California coast from the airborne RSP instrument are compared to collocated bispectral retrievals from the Autonomous Modular Sensor (AMS). Interestingly, the two retrievals are found to be in close agreement, with a correlation of 0.928 and negligible bias of less than a micron. Beyond the clear instrument differences of the Alexandrov et al. (2015) and Bréon and Doutriaux-Boucher (2005) studies, it is still unclear how well the bispectral and polarimetric retrievals should compare to one another and what situations might cause them to differ, raising numerous questions and motivating this study.

A great challenge facing these observational studies is the intertwining of various instrument- and scene-dependent factors that lead to retrieval differences. For example, the polarimetric and bispectral methods have different sensitivity to the cloud vertical profile, and at the same time they are also both affected by cloud horizontal inhomogeneity (Zinner et al., 2010; Zhang et al., 2012, 2016; Miller et al., 2016). It is difficult, if not impossible, to disentangle these factors based on observations alone. This study approaches the intercomparison of bispectral and polarimetric retrievals through a different route: rather than use observational remote-sensing data, synthetic retrievals are generated from large-eddy simulations (LESs) of clouds. Modeling radiative transfer in an LES scene to obtain total and polarized reflectances opens up the possibility of using the LES to perform synthetic bispectral and polarimetric retrievals. This retrieval simulator framework has proven to be a useful tool in other cloud remote-sensing studies (Miller et al., 2016; Zhang et al., 2012). Using this idealized simulation at high spatial resolution, we can attempt to parse the effects of unresolved sub-pixel inhomogeneity, spatial resolution, and angular resolution on the intercomparison of polarimetric and bispectral retrievals. The scale of the LES simulations ( 10 km) in this study prevents us from examining resolutions as large as the standard POLDER retrieval ( 150 km), but we are able to advance understanding of how spatial resolutions between 50 m and 1 km influence retrievals. These scales are suitable for airborne instrument comparisons, which certainly fall somewhere in this range. The use of a satellite retrieval simulator opens up two unique opportunities for developing and studying cloud microphysical retrievals: first, it provides the means to compare retrievals directly to LES cloud microphysics; second, it allows us to perform a retrieval technique intercomparison that is independent of instrument characteristics and other differences that complicate observational studies. This study focuses on three particular questions:

• How well do the bispectral and polarimetric retrievals perform relative to the LES fields used as input to the retrievals?

• How do the bispectral and polarimetric retrieval techniques compare to one another at high spatial resolution?

• How are the bispectral and polarimetric retrieval techniques sensitive to specific observational conditions (i.e., the influence of spatial and angular resolution)?

The rest of the paper is organized as follows: Sect. 2 provides a brief introduction to the theoretical basis of the two retrieval techniques; Sect. 3 describes the LES-based satellite retrieval simulations used in this study; the comparisons between the two techniques based on the LES cases are presented in Sect. 4, followed by summary and discussion in Sect. 5.

2 Background

## 2.1 Cloud microphysical and optical properties

In satellite remote-sensing DSDs are often described using theoretical distributions that fit well with in situ observations, in addition to being mathematically convenient (Deirmendjian, 1964; Tampieri and Tomasi, 1976; Martin et al., 1994; Miles et al., 2000). A popular theoretical DSD is the gamma distribution proposed by Hansen and Travis (1974):

$\begin{array}{}\text{(1)}& N\left(r;{r}_{\mathrm{e}},{v}_{\mathrm{e}}\right)\equiv {N}_{\mathrm{0}}C{r}^{\left(\mathrm{1}-\mathrm{3}{v}_{\mathrm{e}}\right)/{v}_{\mathrm{e}}}\mathrm{exp}\left[-r/\left({r}_{\mathrm{e}}{v}_{\mathrm{e}}\right)\right],\end{array}$

where the independent variable r is the cloud droplet radius, N(r) is the droplet size distribution, N0 is the droplet number concentration, and C is a normalization constant. The two distribution parameters are the effective radius (re) and the effective variance (ve) of the DSD:

$\begin{array}{ll}{r}_{\mathrm{e}}& \equiv \frac{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{Q}_{\mathrm{e}}\left(r\right){r}^{\mathrm{3}}N\left(r\right)\mathrm{d}r}{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{Q}_{\mathrm{e}}\left(r\right){r}^{\mathrm{2}}N\left(r\right)\mathrm{d}r}\\ \text{(2)}& & \approx \frac{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{r}^{\mathrm{3}}N\left(r;{r}_{\mathrm{e}},{v}_{\mathrm{e}}\right)\mathrm{d}r}{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{r}^{\mathrm{2}}N\left(r;{r}_{\mathrm{e}},{v}_{\mathrm{e}}\right)\mathrm{d}r}=\frac{〈{r}^{\mathrm{3}}〉}{〈{r}^{\mathrm{2}}〉},{v}_{\mathrm{e}}& \equiv \frac{\mathrm{1}}{{r}_{\mathrm{e}}^{\mathrm{2}}}\frac{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{Q}_{\mathrm{e}}\left(r\right)\left(r-{r}_{\mathrm{e}}{\right)}^{\mathrm{2}}{r}^{\mathrm{2}}N\left(r\right)\mathrm{d}r}{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{Q}_{\mathrm{e}}\left(r\right){r}^{\mathrm{2}}N\left(r\right)\mathrm{d}r}\\ \text{(3)}& & \approx \frac{\mathrm{1}}{{r}_{\mathrm{e}}^{\mathrm{2}}}\frac{{\int }_{\mathrm{0}}^{\mathrm{\infty }}\left(r-{r}_{\mathrm{e}}{\right)}^{\mathrm{2}}{r}^{\mathrm{2}}N\left(r;{r}_{\mathrm{e}},{v}_{\mathrm{e}}\right)\mathrm{d}r}{{\int }_{\mathrm{0}}^{\mathrm{\infty }}{r}^{\mathrm{2}}N\left(r;{r}_{\mathrm{e}},{v}_{\mathrm{e}}\right)\mathrm{d}r}=\frac{〈{r}^{\mathrm{4}}〉〈{r}^{\mathrm{2}}〉}{{〈{r}^{\mathrm{3}}〉}^{\mathrm{2}}}-\mathrm{1},\end{array}$

where $〈{r}^{n}〉={\int }_{\mathrm{0}}^{\mathrm{\infty }}{r}^{n}N\left(r\right)\mathrm{d}r$ is the nth moment of the DSD. For the spectral bands relevant to this study, the extinction efficiency (Qe) is approximately constant (i.e., Qe(r)∼ const. = 2). Thus, the relationships between re and ve can be conveniently reduced to relations between arithmetic moments of the DSD. The DSD plays an important role in defining the bulk optical properties of a cloud. The optical property libraries used in this study are based on single-scattering Mie calculations of monodisperse droplet optical properties that are averaged with respect to size, according to the gamma DSD (Wiscombe, 1979). In addition, these single-scattering optical properties are averaged with respect to wavelength over an instrument-specific spectral response function (based on MODIS bands in this study) and solar source functions (Planck blackbody function; Planck, 1914). The single-scattering bulk cloud optical properties are subsequently used to run radiative transfer calculations for the creation of the so-called bispectral reflectance look-up table (LUT). This LUT is made up of pre-calculated reflectances of plane-parallel and homogeneous (PPH) clouds over a high-resolution grid of combinations of τ, re, and ve. Here, τ is defined in terms of the DSD:

$\begin{array}{}\text{(4)}& {\mathit{\tau }}_{\mathrm{tot},\mathit{\lambda }}\equiv {\int }_{\mathrm{TOA}}^{\mathrm{0}}\left[{\int }_{\mathrm{0}}^{\mathrm{\infty }}{Q}_{\mathrm{e},\mathit{\lambda }}\left(r\right)\mathit{\pi }{r}^{\mathrm{2}}{N}_{\mathrm{0}}n\left(r\right)\mathrm{d}r\right]\mathrm{d}z.\end{array}$

## 2.2 Bispectral and polarimetric retrieval methods

The bispectral method retrieves τ and re simultaneously from a pair of observed cloud reflectances, typically using a combination of VNIR and SWIR/MWIR bands. The VNIR band, with relatively negligible liquid water droplet absorption, and the SWIR/MWIR band, where droplets are moderately absorptive, can be used to remotely infer τ and re because of this difference in sensitivity to multiple scattering (thickness) and absorption (droplet size). This method is usually implemented using a LUT, shown graphically in Fig. 1a, which has a fixed ve. Cloud reflectance in the VNIR band (centered around 0.865 µm) increases with τ (gray) for a fixed re, while the reflectance in the MWIR band (centered around 3.75 µm) decreases with re (colored) when τ is fixed. The retrieved properties are obtained by performing a two-dimensional inverse interpolation between observed reflectance and the τre grid. A notable characteristic of the bispectral LUT is that, when the optical thickness is low (τ < 3), the isolines of the LUT are more densely packed and less orthogonal, which results in reduced sensitivity and increased retrieval uncertainty (Werner et al., 2013). The bispectral technique is not particularly sensitive to ve, so typically a fixed value is assumed (e.g., ve= 0.1 in the operational MODIS retrieval, though it is kept as an error source in calculating pixel-level uncertainties). While different combinations of bands are used to perform the bispectral retrieval, in this study we focus on VNIR reflectances centered on 0.865 µm, with the second band selected from either a 2.13 µm centered SWIR band or a 3.75 µm centered MWIR band. There are consequences for the re retrieval depending on the particular set of bands selected. For example, a strongly absorbing SWIR/MWIR band limits penetration into the cloud, and as a result the retrieved re is vertically weighted toward the microphysics prevalent in the uppermost part of the cloud (Platnick, 2000).

Figure 1Demonstrations of the microphysical sensitivity of the bispectral and the polarimetric techniques. (a) features the bispectral LUT exhibiting sensitivity to re (colored isolines), due to absorption in the MWIR reflectance. The VNIR reflectances provide sensitivity to optical thickness (gray isolines). (b) and (c) demonstrate the sensitivity of polarimetric technique to re and ve, respectively. The supernumerary bow peaks of the polarized phase function (P12) shift and become narrower with increasing droplet size (re), whereas the peaks erode in magnitude for broadened droplet size distributions (ve).

Figure 2The optical and microphysical properties (τ, re, and ve) of the LES cases examined in this study. The panels are arranged such that each LES case appears row-wise and the different properties appear column-wise. Note that the vertical weighting functions used for the displayed re(VW) and ve(VW) correspond to single-scattering assumptions. Cloud-free masking in each of the images appears in gray. Refer to Sects. 2 and 3 for discussion and definition of each of these properties. Axis labels have been removed to enlarge each map, but the spatial dimensions of each scene are roughly 7 × 7 km (refer to Sect. 3 for the specific resolutions of each LES case).

For the polarimetric retrieval, the angular pattern of the linearly polarized reflectance1 is the source of sensitivity to cloud microphysical properties. Polarized reflectances are dominated by single scattering because multiple scattering induces depolarization. As a result, the single-scattering polarized phase functions (P12) shown in Fig. 1b and c are good approximations to the observed angular pattern of polarized cloud reflectances (Bréon and Goloub, 1998). These phase functions demonstrate the sensitivity of the polarimetric retrieval to both re and ve. As re increases in Fig. 1b, the supernumerary bow peaks (around a scattering angle of 142) become narrower and shift toward smaller scattering angles. In contrast, as ve increases in Fig. 1c the supernumerary bow peaks erode in magnitude, eventually smoothing out for broad DSDs (ve > 0.15). A consequence of this erosion of the supernumerary peaks is that the polarimetric retrieval has less sensitivity to both re and ve for very broad DSDs. The polarimetric retrieval does not significantly rely on multispectral information, although observations in several bands may help provide stronger observational constraints due to the shift in the supernumerary bows with changing wavelength (refer to Fig. 3 of Bréon and Goloub, 1998). The dominance of the single-scattering contributions to the polarized reflectance leads to cloud retrievals that represent microphysical properties with a mean penetration depth of τSS 0.5 and sensitivity that saturates for optical depths greater than  3 from the cloud top. The polarimetric retrieval is often based on a parametric curve-fitting retrieval algorithm like the one presented in Alexandrov et al. (2012b), although there are other techniques (e.g., the rainbow Fourier transform (RFT) technique of Alexandrov et al., 2012a, which can retrieve DSDs with arbitrary mathematical form). The parametric technique relies on a library of P12 curves with varying re and ve that are parametrically scaled and adjusted to fit the observed reflectance via a nonlinear least squares optimization procedure. This process yields the phase function that best matches the angular pattern of the observation, thus determining the re(pol) and ve(pol) retrieval. The polarimetric method described above does not result in a retrieval of τ; however, it can still be obtained by implementing a simplified variant of the bispectral τ retrieval. With simultaneous measurements of the total reflectance in a VNIR band and the re(pol) retrieval, a VNIR-only LUT curve can be used to perform a 1-D interpolation of the corresponding bispectral LUT curve for RVNIR(re(pol), τ).

Both bispectral and polarimetric techniques are susceptible to a variety of retrieval uncertainties. The main objective of this study is to understand how the retrieval uncertainties influence each technique and whether they can lead to deviation between the two techniques in terms of retrieval results. In this study, we focus on five major sources of retrieval uncertainty for both techniques:

1. Cloud vertical profile. In the operational retrievals, both bispectral and polarimetric techniques assume vertically homogenous clouds. However, clouds in reality often have significant vertical variability resulting from various processes (e.g., condensational growth, coalescence, sedimentation, entrainment). Deviation from the assumed profile gives rise to many questions. For example, how do we interpret the re and ve retrievals based on the homogenous cloud assumption? To what extent does cloud vertical profile influence the bispectral and polarimetric techniques? Note that Platnick (2000) developed a method utilizing the so-called “vertical weighting function” to interpret the re retrieval from the bispectral method for clouds with vertically varying re profile. Recently, Alexandrov et al. (2012b) took an approach that focused on the vertical weighting of the droplet size distribution to interpret the re and ve retrievals from the polarimetric technique. Miller et al. (2016) demonstrated the usefulness of this vertical weighting approach for understanding both bispectral and polarimetric re retrievals. In Sect. 4.1, we will apply a vertical weighting function for both techniques on the basis of the LES cloud fields, to help understand if cloud vertical structure could lead to significant differences between the two techniques.

2. Sensitivity to observational uncertainty. The uncertainties associated with observations of total and polarized reflectances can differ, indicating that uncertainty may also impact bispectral and polarimetric retrievals differently. Additionally, the two retrievals rely on a different number of uncertain observations: a pair of uncertain total reflectances (bispectral) as compared to numerous uncertain polarized reflectances (polarimetric). Furthermore the different algorithmic approaches, two-dimensional interpolation vs. nonlinear optimal curve fitting, introduce additional layers of complexity in terms of the impact of uncertainty. The impact of uncertainty on retrieval results for each method is highlighted and explored in Sect. 4.2.

3. Reduced sensitivity. It can be clearly seen from Fig. 1a that, when clouds are optically thin (τ < 3), the LUT for the bispectral retrieval becomes less orthogonal and the isolines of re become more densely packed. This reduction in sensitivity can lead to significant retrieval uncertainties in bispectral techniques for optically thin clouds (τ < 3). Similarly, the sensitivity of the polarimetric technique to re and ve is reduced when DSD becomes very broad (i.e., ve > 0.15), in which case the supernumerary bow features are barely distinguishable (Fig. 1c). In Sect. 4.3 we will investigate the impacts of the reduction of sensitivity on retrieval consistency between the two techniques.

4. Sub-pixel inhomogeneity. The impact of spatial resolution and unresolved sub-pixel cloud inhomogeneity on bispectral retrievals has been well studied (Zhang and Platnick, 2011; Zhang et al., 2012, 2016). An important conclusion from these studies is that the so-called plane-parallel homogenous bias (PPHB) can cause the bispectral technique to significantly overestimate re. In contrast, the sensitivity of the polarimetric retrieval to unresolved sub-pixel inhomogeneity and resolution has not been thoroughly studied. In Sect. 4.4 we will compare the impacts of sub-pixel inhomogeneity on bispectral and polarimetric techniques, and investigate whether it can cause deviation between the two techniques.

5. Angular resolution and sampling for polarimetric technique. In addition to spatial resolution, angular resolution and sampling are also important for the polarimetric technique. A coarse angular resolution may not be able to resolve the supernumerary bow features. Similarly, if the scattering angles corresponding to the supernumerary bows are not or only partially sampled, then the polarimetric technique may not have enough information content for retrieval. This issue will be discussed in Sect. 4.5.

3 Model and methodology

The satellite retrieval simulator implemented in this study is built around an LES model (DHARMA) with bin microphysics (Ackerman et al., 2004; Zhang et al., 2012; Miller et al., 2016). The LES provides freely evolving 3-D cloud microphysical properties, which are used as a reference when comparing to numerically simulated retrievals. The LES in this study adopts 25 droplet size bins to represent droplet size distributions (Ackerman et al., 1995). The optical properties of each size bin are computed by bulk-averaging Mie scattering properties over a highly resolved flat sub-bin droplet size distribution. The optical properties of each bin are provided as input to radiative transfer simulations based on the size distributions of the LES cloud fields. Vector radiative transfer calculations are performed using a polarized doubling–adding technique (PDA) to produce 1-D total and polarized reflectances at the horizontal resolution of the LES grid (described below) (De Haan et al., 1987). The sole consideration of 1-D retrievals avoids 3-D radiative effects and focuses this study on retrieval technique differences rather than on radiative processes. A future study will focus on the comparison of 3-D retrievals to these 1-D bispectral and polarimetric retrievals. The radiative transfer modeling in this work is performed for numerous solar zenith angles (SZAs = [20, 40, 60]), viewing zenith angles (VZAs = [70 : +70]), and a constant relative azimuthal angle (ΔΦ= 30). The VZA resolution results in a scattering angle (Θ) resolution on the order of 0.5. Reflectances in spectral bands (based on MODIS spectral response functions) are centered on 0.865, 2.13, and 3.75 µm wavelengths. Total reflectances in all bands are used to produce bispectral retrievals, whereas linearly polarized reflectances in the 0.865 µm band are used to produce polarimetric retrievals. Subsequently, bispectral and polarimetric retrievals are performed on the simulated reflectances to obtain re, ve, and τ retrievals. Bispectral and polarimetric retrievals are performed over a subset of observation geometries, with bispectral retrievals performed for VZAs = [50, 40, 30, 20, 10, 0, 10] and all SZAs. Meanwhile, the polarimetric retrievals are performed for a SZA = 20 and a range of VZAs = [0 : 27], which result in reflectances spanning scattering angles required to observe the primary and supernumerary bow features (i.e., Θ= [135 : 160]). Reflectances are also aggregated from the 50 m native LES resolution up to coarser 100, 200, 400, and 800 m horizontal resolutions to reflect the influence of different remote-sensing footprints. The retrievals in this study are also performed at all of the footprint resolutions. The bispectral LUT implemented in this study spans microphysical properties re= [2 : 30] µm in steps of 0.5 µm and ve= [0.01 : 0.11] in steps of 0.01. The τ retrieval in this study is anchored to the 0.865 band optical properties and spans τ= [0.1  :100] with 101 logarithmically spaced grid points. Including ve variability in the bispectral LUT allows for the comparison of standard MODIS-like retrievals (the ve= 0.1 LUT) to retrievals with other ve assumptions. The bispectral retrieval is then accomplished by performing a 2-D linear interpolation of the observed reflectances and inverting between the reflectance and retrieval space. For the polarimetric retrieval, the polarimetric phase function library spans re= [2 : 40] µm in steps of 0.25 µm and ve= [0.01 : 0.3] in steps of 0.01. The polarimetric retrieval implemented in this study is based on the approach of Alexandrov et al. (2012a), fitting the polarized phase functions in their Eq. (3) to the modeled polarized reflectances of the LES scene. The optimal parametric fit in the P12 library is determined by using a Levenberg–Marquardt nonlinear least squares algorithm. This optimal phase function is then used to identify the corresponding re(pol) and ve(pol) retrieval. As previously stated in Sect. 2.2, the polarimetric retrieval of τ is accomplished by using a constrained 1-D version of the bispectral LUT.

The LES cloud fields are used not only to drive the radiative transfer simulations but also to help interpret and understand the retrieval results. As mentioned in Sect. 2.2, it is not trivial to interpret the re and ve retrievals based on the homogenous cloud assumption when the cloud has significant vertical structure. To address this issue, for each LES column with detailed vertical profiles of DSD, we derive two reference variables, re(VW) and ve(VW), from the vertical weighted (VW) integration of the DSD profile. The vertical integration is weighted by a function to account for the penetration depth and multiple scattering of radiation in the corresponding wavelength associated with each particular retrieval. Thus, re(VW) and ve(VW) should be comparable to the retrieved re and ve from the simulated reflectance (Alexandrov et al., 2012b; Miller et al., 2016; Zhang et al., 2017). The method of vertical weighting in this study is described in detail in Sect. 2 of Miller et al. (2016); however in this study we have modified the vertical weighting function to account for multiple scattering. Motivated by the convenience and flexibility of the parametric approach proposed in Eq. (4) of Zhang et al. (2017), we implement a two-variable parametric vertical weighting function:

$\begin{array}{}\text{(5)}& W\left(\mathit{\tau }\right)=c{\mathit{\tau }}^{b}\mathrm{exp}\left[-a\mathit{\tau }\left(\frac{\mathrm{1}}{\mathit{\mu }}+\frac{\mathrm{1}}{{\mathit{\mu }}_{\mathrm{0}}}\right)\right],\end{array}$

where the new parameters a and b are introduced to account for the influence of multiple-scattering effects not originally considered in the vertical weighting function discussed previously in Miller et al. (2016). The parameter a scales the optical depth, modeling the enhanced transmission caused by multiple scattering, whereas the parameter b produces a peaked vertical weighting function associated with the expected penetration depth of the reflected light, and c is the normalization factor. Each of these parameters is strictly positive, and for a= 1 and b= 0 we obtain the original single-scattering vertical weighting used in Miller et al. (2016). For smaller values of a and larger values of b, the vertical weighting function extends deeper into the cloud, leading to droplet size distribution properties deeper in the cloud contributing more information to the vertically weighted value. For the polarimetric retrieval, a= 1 and b= 0 were selected due to the dominance of single scattering in polarized reflectances. In contrast, multiple scattering can significantly impact total reflectances. For total reflectances a single value of a and b was selected for each spectral band and observation geometry based on coefficients that fit best to numerically calculated vertical weighting functions based on the method presented in Eq. (4) of Platnick (2000)2. Generally, we found that a(3.75 µm) was larger than a(2.13 µm), as would be expected because of stronger absorption in 3.75 µm reducing transmission into the cloud. We also found that b was dependent on observation geometry (scattering angle), and b(3.75 µm) was less than b(2.13 µm) because multiply scattered light in the 2.13 µm band can penetrate deeper into the cloud before scattering back out. In addition to re(VW) and ve(VW), we also derive τLES for each LES column simply by integrating the extinction coefficient (for λ= 0.865 µm) from cloud bottom to cloud top. re(VW), ve(VW), and τLES are used as references in the retrieval and LES property comparison in Sect. 4.1 to understand the differences between the retrievals and the original LES fields. After obtaining re(VW), ve(VW), and τLES at the 50 m native LES resolution, they are aggregated to 100, 200, 400, and 800 m to help interpret the retrievals at these coarser resolutions. It is important to note that there is a subtle difference between directly aggregating re(VW) or ve(VW) and aggregating the DSD (i.e., N(r)) first and then deriving the corresponding re(VW) and ve(VW). The differences between the two methods are discussed in the Appendix. The main conclusion is that, although the two aggregation methods could be different in some hypothetical cases with unrealistically large variability in the unresolved microphysics, they are essentially equivalent for practical purposes. In this study, we simply aggregate re(VW) and ve(VW) from the native LES resolution of 50 m to obtain average values at the desired resolution (e.g., 800 m).

Three LES cases are the focus of this study. The first (referred to as “ATEX clean” hereafter) and second (“ATEX polluted”) cases are based on an idealized case study from the Atlantic Trade Wind Experiment (ATEX), with different aerosol loadings (Stevens et al., 2001). The ATEX cases are representative of a trade wind cumulus regime in which scattered cumuli rise into a thin, broken stratocumulus layer. The third case (referred to as “DYCOMS-II” hereafter), originally presented in Stevens et al. (2005), is an idealized setup based on clouds observed during the second research flight (RF02) of the Second Dynamics and Chemistry of Marine Stratocumulus project (DYCOMS-II) (Stevens et al., 2003). This case is representative of nocturnal marine stratocumulus under conditions of a dry inversion. The DYCOMS-II case has a domain size of 6.4 × 6.4 × 1.5 km (128 × 128 × 96 grid points), while each of the ATEX simulations has a domain size of 7.2 × 7.2 × 3 km (144 × 144 × 200 grid points). The horizontal grid spacing of these LES cases is fixed at 50 m, while the vertical grid is stretched, with a minimum spacing of 5 m near the surface and the capping temperature inversion to better resolve small-scale turbulence there. Further details of the model setup for the DYCOMS-II case are provided in Ackerman et al. (2009). The ATEX cases are updated model runs with increased spatial resolution that are similar to the cases discussed in Fridlind and Ackerman (2011). For each LES scene a snapshot of cloud microphysical and optical properties is saved every half hour after the first hour of each simulation, resulting in numerous cloud fields. A single time step of each of the cases was selected to be the focus of this retrieval study, each occurring  3 h into the simulation.

Table 1Mean values (μ) and standard deviations (σ, in parentheses) of various optical (τ and Hσ) and microphysical properties (re(VW) and ve(VW)) of the LES scenes examined in this study. Note that vertically weighted properties are listed for the polarimetric vertical weighting function and Hσ for nadir-viewing 0.865 µm reflectance. Cloudy pixels is defined using a threshold of τLES > 0.1.

The variability of cloud optical and microphysical properties in each of the LES cases is highlighted in Fig. 2 and Table 1. Spatial inhomogeneity of both optical and microphysical properties of these scenes is evident, with the ATEX polluted and DYCOMS-II cases exhibiting lower spatial inhomogeneity and the ATEX clean case being more broken and inhomogeneous. One method for quantifying the optical inhomogeneity of a cloud scene is to use the sub-pixel inhomogeneity index,

$\begin{array}{}\text{(6)}& {H}_{\mathit{\sigma }}\left(\mathrm{resolution}\right)=\frac{\mathrm{SD}\left[{R}_{i}\left(\mathrm{0.865}\phantom{\rule{0.125em}{0ex}}\mathrm{µ}\mathrm{m},\phantom{\rule{0.125em}{0ex}}\mathrm{50}\phantom{\rule{0.125em}{0ex}}\mathrm{m}\right)\right]}{\mathrm{mean}\left[{R}_{i}\left(\mathrm{0.865}\phantom{\rule{0.125em}{0ex}}\mathrm{µ}\mathrm{m},\phantom{\rule{0.125em}{0ex}}\mathrm{50}\phantom{\rule{0.125em}{0ex}}\mathrm{m}\right)\right]},\end{array}$

where the numerator and denominator are the standard deviation and mean of the native LES resolution (50 m) reflectances within a coarser-resolution pixel. Thus, the value of Hσ is computed for a coarser-spatial-resolution pixel (800 m in Table 1) using the highest-resolution nadir-viewing reflectances (50 m). The value of Hσ increases with increasing sub-pixel inhomogeneity, making it a useful measure for unresolved cloud variability. In addition to optical inhomogeneity, each of the LES scenes also has characteristically different microphysical properties. The average value of re(VW) of each scene varies, in part because of the initial background cloud condensation nuclei (CCN) in each particular case but also because of cloud top height variability. In these LES cases ve is spatially anticorrelated with τ and organized in a cellular structure – regions with higher τ tend to have smaller ve(VW), and regions with lower τ tend to have large ve(VW).

4 Results and analysis

## 4.1 Retrieval and LES property comparison

Before comparing the retrieval results from the two techniques to one another, we must first carry out a comparison of LES and retrieval properties to assess and understand the differences between the retrieval results and the original LES cloud fields at the native 50 m spatial resolution. This is a necessary sanity check that will help understand the accuracy and uncertainty of our retrieval routines. More importantly, this study will help to interpret the retrievals based on homogeneous cloud assumption when the LES cloud fields have naturally inhomogeneous vertical profiles. Note that the retrievals compared throughout the following sections are compared for all combinations of viewing and solar geometries indicated in Sect. 3.

Figure 3Joint histogram regressions of re and τ in all LES cases comparing the bispectral retrievals to the LES cloud microphysical properties. (a) and (b) are regressions of the bispectral re(2.13 µm) and re(3.75 µm) retrievals against the physical analogue re(VW). (c) and (d) are regressions of the bispectral τ(2.13 µm) and τ(3.75 µm) retrievals against the physical τ(LES). (e) and (f) display the regression of the bispectral re(2.13 µm) and re(3.75 µm) retrievals for only optically thick pixels (τ > 3). Note that in each panel the correlation is quantified with a linear correlation coefficient (R), and the black and white contours encompass 66 and 95 % of the population, respectively.

The bispectral retrieval comparison to LES properties in Fig. 3 depicts joint histograms of re and τ retrievals using both the 2.13 and 3.75 µm bands against the reference values derived from the LES fields, re(VW), ve(VW), and τLES. It is important to note that these joint histograms are presented as the logarithmic percent of the population, to emphasize deviations from the one-to-one line. Also, the mean regression biases reported throughout this study are stated relative to the plotted axes as ${\mathit{\mu }}_{\mathrm{bias}}=〈y-x〉$ and ${\mathit{\mu }}_{\mathrm{|}\mathrm{bias}\mathrm{|}}=〈‖y-x‖〉$ (i.e., x and y denoting x and y axes). The two bispectral re retrievals, re(2.13 µm) and re(3.75 µm), are in agreement with the LES ground truth (Fig. 3a and b) with good correlations, both exceeding 0.95. The biases between these two retrievals and the LES properties differ slightly. Compared to the LES, both re retrievals have relatively small sub-micron mean biases, and the mean absolute biases are also sub-micron. Additionally, it is important to note a limitation of this population: none of the LES scenes in this study have a mean cloud top re near 10 µm. To examine the two bispectral τ retrievals, τ(2.13 µm) and τ(3.75 µm) in Fig. 3c and d, we compare them in terms of percent differences, because the regression is so highly correlated (R > 0.99). A slight systematically high bias on the order of 2–5 % exists. The origin of this high bias is likely associated with deviations of the droplet size distribution from the assumed gamma distribution form. The LES size distributions sometimes exhibit longer large-droplet tails than the assumed form. As explained earlier in Sect. 2.2, the bispectral method suffers from a reduction of retrieval sensitivity when clouds are optically thin. Therefore, if we sample only LES columns that are optically thick (τ > 3), a substantial improvement in the regression correlations of the two re retrievals (Fig. 3e and f) is achieved. While not all retrievals in the τ < 3 population are biased, the majority of extreme retrieval bias outliers belong to this thin-cloud population. Even after this sub-selection of the data some outliers still remain. In particular, a small population of both re(2.13 µm) and re(3.75 µm) retrievals have biases exceeding re(VW) by as much as 20 µm. The cause of these outliers and some other differences between the retrievals and LES fields will be discussed in detail in Sect. 4.3.

Figure 4Joint histogram regressions of re, ve, and τ in all LES cases comparing the polarimetric retrievals to the LES cloud microphysical properties. (a) depicts the regression of the polarimetric re(pol) retrieval against the physical analogue re(VW), while (b) is sub-selection of the same regression for low ve. (c) depicts the regression of the polarimetric ve(pol) retrieval against the physical analogue ve(VW), while (d) is a sub-selection of the same regression for thick clouds (τ > 3). (e) depicts the regression of the polarimetric τ(pol) retrieval against the physical analogue τ(LES), while (f) is sub-selection of the same regression for low ve. Note that in each panel the correlation is quantified with a linear correlation coefficient (R), and the black and white contours encompass 66 and 95 % of the population, respectively.

The polarimetric retrieval comparison to LES properties in Fig. 4 depicts comparisons of the polarimetric retrievals, re(pol), ve(pol), and τ(pol). The re(pol) retrieval compares very well to re(VW) (Fig. 4a), with a regression correlation exceeding 0.98 and a mean bias of less than 0.1 µm. The quality of this retrieval comparison to LES properties also further supports the single-scattering definition of re(VW) for the polarimetric retrieval. In contrast, the polarimetric retrieval of ve(pol) reveals a regression against ve(VW) (Fig. 4c) that does not perform quite as well. In this case the regression correlation is much weaker (R= 0.71) with a mean bias of 0.011. While the mean bias is on the order of the ve LUT grid spacing, it is clear that the regression correlation is poor because of a systematic low bias for ve(VW) larger than about 0.15. It should also be noted that the increased concentration of ve(pol) retrievals at ve= 0.3 is a result of the boundaries of the retrieval space, ve= [0.01, 0.3], the upper limit of which is a consequence of the gamma distribution of Hansen and Travis (1974) becoming monotonic for ve 0.3. Comparing only the population with ve(VW)  0.15 (not shown here) results in an improved correlation of R=0.81 with negligible mean bias. The ve retrieval quality also depends on the assumption that LES droplet size distributions are accurately described using a single-mode gamma distribution. The DSDs in the LES sometimes deviate significantly from this assumption. In the context of the parametric polarimetric retrieval used in this study this is difficult to remedy or address. However, a different polarimetric retrieval – the RFT, introduced in Alexandrov et al. (2012a) – offers the possibility of retrieving an arbitrary droplet size distribution shape. The final retrieval product, τ(pol) (Fig. 4e), indicates that a more accurate a priori re and ve estimate has little impact on the retrieval of τ. As explained earlier in Sect. 2.2, the polarimetric method suffers a reduction of sensitivity when the DSD is broad, a finding that is consistent with previous work (Alexandrov et al., 2012b). This explains, for the re(pol) retrieval, why limiting the regression population to LES columns with ve(VW)  0.15 in Fig. 4b increases the correlation and decreases the absolute bias. This appears to be an indication of sensitivity to degradation of the supernumerary bow features for large ve, features that are necessary for reliable re(pol) and ve(pol) retrievals.

For ve(pol) we find that, by sampling LES columns that are optically thick (τ> 3), there is moderate improvement in the correlation and reduced biases (Fig. 4d). This improvement stems from the correlation between the population of optically thin clouds and high ve(VW) (Fig. 4f) that are found near cloud edges in the LES scenes. It should be noted that an increased τ does not implicitly lead to better polarimetric retrievals, but here it is observed to be a consequence of the aforementioned correlated relationship between DSD and optical properties.

## 4.2 Sensitivity to measurement uncertainty

The measurement uncertainties of total and polarized reflectances differ, leading one to expect that bispectral and polarimetric retrievals may have different sensitivities to uncertainty. Their relationships to uncertainty are further complicated by differences in retrieval approaches, namely interpolating two independent uncertain observations in a LUT (bispectral) or curve fitting through numerous observations that are each independently uncertain (polarimetric). Targeted uncertainties for cloud and aerosol remote sensing are δ DOLP = 0.5 % in degree of linear polarization and δI= 3 % in total reflectance (Knobelspiesse et al., 2012). A simple propagation of uncertainty analysis yields a polarized reflectance uncertainty of δQ= 2.5 % (in the principal plane). Using these uncertainties as a starting point, we can perturb the LES reflectances with uncorrelated random noise and perform retrievals that we can then compare to the original unperturbed retrievals. Note that, while the focus here is on uncorrelated randomly distributed noise, other sources of observational uncertainty exist and would need to be accounted for in the context of a specific instrument's uncertainty model. As the properties of particular instruments are not the focus of this study, we will focus on this more general uncertainty analysis.

For the bispectral retrieval, a randomly distributed reflectance perturbation within ±3 % was added to each LES reflectance. A histogram of the percent bias of bispectral retrievals of re induced by the addition of the reflectance uncertainty is shown in Fig. 5a. The mean and standard deviation of these bias distributions are stated, allowing us to interpret the results. First, the introduction of uncertainty has very little impact on the mean bias of bispectral re retrievals (on the order of 0.1 %). Second, the introduction of uncertainty results in a broad distribution of re retrieval biases with standard deviations of 5.44 and 4.02 % for the 2.13 and 3.75 µm retrievals, respectively. Together these two results indicate that biases associated with measurement uncertainty will not be systematic, with absolute variability on the order of 1 µm or less for droplet sizes below 20 µm (the most prevalent population in this LES study). The impact of observational uncertainty on all of the τ retrievals is the focus of Fig. 5b. The two bispectral retrievals, τ(2.13 µm) and τ(3.75), each have very minimal mean biases of 0.1 %. However, like the biases for the effective radius retrievals, the distribution of retrieval bias is broadened to standard deviations of 8.2 and 5.6 % for τ(2.13 µm) and τ(3.75), respectively. The polarimetric τ(pol) retrieval on the other hand, being methodically quite similar to the bispectral retrievals, exhibits a small systematic low bias of about 2.43 % as shown in Fig. 5b. The origin of this systematic bias is a known characteristic of single-band optical thickness retrievals and is clearly demonstrated in Fig. 1 of Marshak et al. (2006). The convexity of a single-band LUT curve produces low-biased retrievals for symmetrically distributed (or averaged) reflectances. The bias distribution also has a smaller variability (3.8 %) than the two bispectral retrievals, likely because the uncertainty in the SWIR/MWIR band also (weakly) influences the bispectral τ bias distributions.

Figure 5Histograms of the percent retrieval bias of retrievals based on perturbed reflectances stated relative to unperturbed retrievals. (a) displays retrieval biases for the bispectral re retrieval. (b) displays retrieval biases for the bispectral and polarimetric τ retrievals. Refer to the text for more information about the polarimetric re and ve retrieval biases.

The consequences of measurement uncertainty are markedly different for the polarimetric retrieval. This is a result of the polarimetric retrieval being a search for a similar curve in the phase function library, making the deviations in the magnitude of observations in any one angle less important when searching for the optimal curve – and therefore discrete re and ve combination. The discretely binned nature of the polarimetric retrieval makes description of bias distributions like the ones in Fig. 5 problematic. One way to describe how uncertainty in polarized reflectances influences polarimetric retrievals is to describe the population of retrievals that are unchanged and the population of retrievals that changed. After the introduction of random noise, 88.1 % of the polarimetric re(pol) retrievals were unbiased, with 9.1 % biased high by one grid point (+0.25 µm) and 2.7 % biased low by one grid point (0.25 µm). All together, these three populations accounted for the vast majority (99.9 %) of retrieval outcomes. The percent bias of the re(pol) retrieval had a mean of 0.06 % and a standard deviation of 0.78 %. These results agree with previous studies, for example the finding of Shang et al. (2015) indicating that the POLDER retrieval performed well as long as reflectance uncertainty was less than 10 %. It should be noted, however, that the sensitivity to uncertainty is also tied to the number of angular measurements available and the properties of the droplet size distribution. The polarimetric ve(pol) retrievals behaved similarly, with 85.2 % of all retrievals being unaffected, 12.7 % biased high by one grid point (+0.01), and 1.9 % biased low by one grid point (0.01). Again, these three populations account for the vast majority of (99.9 %) of retrieval outcomes. The percent bias of the ve(pol) retrieval had a mean of 1.14 %, consistent with a 0.01 bias and a standard deviation of 22.6 %. The greater tendency toward large biases for the effective variance is likely due to smoothing of polarized reflectance curves after the addition of uncertainty. The large majority of biases in the polarimetric retrieval of ve are coming from the population of ve near 0.1–0.15, where the supernumerary bow peaks are significantly eroded, and small shifts in the magnitude of reflectances at angles near these peaks could easily shift the retrieval to the next grid point.

Overall, the lack of strong systematic biases associated with uncertainty in the case of either retrieval supports an approach of neglecting the measurement uncertainty in further analyses. Of course, this requires acknowledging that biases that are below δre= 5, δve= 10, or δτ= 7 % in either retrieval are probably not as important because they likely are not detectable due to observational uncertainty.

Figure 6Joint histogram regressions of re retrievals for all LES cases comparing the bispectral and polarimetric techniques. (a) and (b) display the unfiltered regressions of re(pol) at 0.865 µm wavelength against the re(2.13 µm) and re(3.75 µm) bispectral retrievals, respectively. After introducing filters to these regressions to remove thin clouds (τ < 3) and broad droplet size distributions (ve > 0.15) (c and d), the retrieval intercomparison improves. (e) and (f) each replicate the results from the previous selection criteria but additionally provide bispectral retrieval in this regression with ve(pol) as an a priori for each retrieval. Note that in each panel the correlation is quantified with a linear correlation coefficient (R), and the black and white contours encompass 66 and 95 % of the population, respectively.

## 4.3 Retrieval comparison at high resolution

Figure 7(a) and (c) depict the standard bispectral LUT (light gray dashed lines) for both SWIR bands with the scattered reflectance points for the ATEX polluted LES case plotted on top. The scatterplot is colored by the bias between the bispectral retrieval and the physical reference (re(bispectral)re(VW)). Spatial maps of this bias are shown for context in (b) and (d). Note that some points are colored in black to indicate retrieval failure due to falling outside the LUT space. In addition to the standard LUT, an extended LUT including droplet sizes from 2 to 4 µm is included (black dashed lines), revealing an overlapping region of the two LUTs for smaller τ referred to as the “multiple-solution space”.

The origin of the broadly distributed high-biased bispectral retrievals in the small-droplet-size regime (re(pol) 5 µm) stems from the ATEX polluted case. A close examination of this case reveals that there are no bispectral retrievals below 5 µm, despite approximately 3.5 % of the scene being characterized by re(VW) < 5 µm4. This feature is a consequence of the bispectral LUT state space5, which spans a re range of 5–30 µm. In contrast, the polarimetric retrieval space spans 1–30 µm. The differences between these two LUT spaces is not so much a matter of decision-making but is more reflective of complexities of the bispectral retrieval for small re. To demonstrate this point, panels a and c of Fig. 7 depict the cloud reflectances from the ATEX polluted case within the respective bispectral LUT. It is obvious that the black isolines for τ and re increasingly overlap with the standard LUT space as τ decreases. In this region of the state space, there are multiple solutions for a single reflectance pair; one solution is representative of a small re (< 5 µm, extended LUT), while the other indicates a much larger re ( 5 µm, standard LUT). There is also a modest impact on τ, but due to the curvature of the LUT this impact is less severe. The overlapping region between the standard and extended LUT is referred to as the “multiple-solution space”, and the amount of LUT overlap is determined by both the observation geometry and the selected spectral bands. Depending on the optical thickness, the larger re retrieval may be significantly larger, because the extended LUT isolines cross numerous larger re isolines in the standard LUT. The associated bispectral retrieval bias, shown in Fig. 7b and d, highlights the conclusion that for optically thick clouds the bispectral re retrievals exhibit only moderate retrieval biases on the order of ±1 µm. However, for very thin clouds (near cloud edge) the retrieval bias can increase significantly. For some of these thinner clouds the retrievals also fall within the multiple-solution space, so it is possible to attribute the very large biases to the presence of ambiguous retrieval results. Furthermore, the multiple-solution space also provides an additional explanation for why the removal of optically thin (τ < 3) observations significantly improved the bispectral retrieval comparisons.

Figure 8Joint histogram regressions of τ retrievals for all LES cases comparing the bispectral and polarimetric techniques. (a) and (b) display the τ (2.13 µm) and τ (3.75 µm) retrievals, respectively. In each panel the quality of the correlation is quantified, and the black and white population density iso-contours are drawn surrounding 66 and 95 % of the data, respectively.

In contrast to the intercomparison of re retrievals, the τ retrieval intercomparison in Fig. 8 reveals very few differences between the bispectral and polarimetric techniques. This is not surprising, because the τ(pol) retrieval is simply an implementation of the bispectral technique with additional constraints on re and ve (as discussed in Sect. 2.2).

Figure 9Probability distributions of Hσ for the combined ATEX polluted and clean data sets at all coarsened spatial resolution (100, 200, 300, 400, 800 m).

Figure 10Joint histograms of retrieval biases (relative to each relevant vertically weighted LES property) with respect to Hσ for the combined ATEX clean and polluted data sets at all observation geometries and including all coarsened spatial resolutions (100, 200, 300, 400, 800 m). The color bar indicates percent occurrence. (a) and (b) depict the difference between the two bispectral re retrievals and the polarimetric retrieval, while (c) and (d) depict biases for the polarimetric re and ve retrieval against re(VW).

## 4.4 Sensitivity to unresolved spatial inhomogeneity

Unresolved spatial inhomogeneity influences the bispectral and polarimetric cloud retrievals in very different ways. Even for 100 % cloudy pixels these retrievals can exhibit sensitivity to sub-pixel inhomogeneity. This section focuses on the ATEX cases because they exhibit a broader distribution of Hσ, allowing us to highlight the impact of spatial inhomogeneity on retrievals. Spatial resolution and sub-pixel inhomogeneity index (Hσ) are inherently intertwined with one another. This is demonstrated in Fig. 9, where the broadening and shifting of the distribution of Hσ for increasingly coarsened spatial resolutions are clearly demonstrated using data from both the ATEX clean and polluted cases. In light of this relationship between resolution and inhomogeneity, the inclusion of data from all spatial resolutions together broadens our sampling of different inhomogeneity regimes. To that end, Fig. 10 combines all of the coarse-spatial-resolution data from the two ATEX cases into a single retrieval bias histogram. For the bispectral retrievals in Fig. 10a and b we compare them to the polarimetric retrieval, resulting in histograms that clearly show the two retrievals diverging from one another with increasing sub-pixel inhomogeneity, which tends to result in larger biases. In contrast, the polarimetric re(pol) retrieval in Fig. 10c does not appear to have a clear systematic bias. The ve(pol) retrieval in Fig. 10d tells a more complicated story: the median value of the bias is clearly close to zero, but there is a tendency toward low-biased retrievals with increasing inhomogeneity. It should be noted that the ve(VW) itself increases with increasing Hσ, which is presumably a consequence of the anticorrelation between τ and ve(VW). This might explain why for large values of Hσ, where the ve(VW) > 0.15 population is more common, there are more negative biases.

To further emphasize how unresolved inhomogeneity can influence these two retrieval techniques, we will highlight a particularly inhomogeneous pixel from the ATEX clean case at the coarsest resolution (800 m). Focusing first on the bispectral retrieval using the 2.13 µm SWIR band, the LUT scatterplot in Fig. 11a reveals that there is significant variability in the sub-pixel (i.e., 50 m) VNIR reflectances, indicated by a large value of the sub-pixel inhomogeneity index (Hσ= 0.5637). In contrast to the variability of VNIR reflectances, the microphysical properties are largely homogeneous in this 800 m pixel, indicated by the narrow distribution of sub-pixel re(VW)50 m (color of the points). The sub-pixel mean of re(VW)〉50 m= 19.71 µm agrees well with the mean of both sub-pixel retrievals, re(2.13 µm)〉50 m= 18.73 µm and re(pol)〉50 m= 18.92 µm. This combination of optical inhomogeneity and microphysical homogeneity leads to an average reflectance (indicated by the black star) for the 800 m pixel that falls significantly below the re= 20 µm isoline (i.e., the closest isoline to the mean sub-pixel retrievals). Thus, the coarse-resolution (800 m) reflectance results in an 800 m bispectral retrieval with re(2.13 µm)800 m= 23.62 µm, which is biased high by  4 µm. This effect is attributable to the well-documented PPH bias induced by the curvature of the bispectral LUT with respect to the optical thickness (Zhang and Platnick, 2011; Zhang et al., 2012, 2016). The PPH bias has a stronger influence on the 2.13 µm retrieval compared to the 3.75 µm retrieval (shown in Fig. 11b) because the curvature of the LUT space is more pronounced.

Figure 11(a) and (c) depict the bispectral LUTs and 50 m reflectances for the 2.13 and 3.75 µm bispectral retrievals, respectively, for a particularly inhomogeneous 800 m pixel. The scattered points correspond to 50 m reflectances, with color corresponding to re(VW), while the black star corresponds to the 800 m reflectance pair (the average of the 50 m data). The polarimetric reflectance distribution histograms in (b) and (d) address how the high-resolution (50 m) reflectance distribution influences the polarimetric retrieval at coarse resolution (800 m). The two curves (plotted with a 0.02 reflectance shift for clarity) are the 800 m observed reflectance (black dashed curve) and the 800 m retrieval (red solid curve). All of these figures include statistics on the high-resolution averages of physical properties and retrievals along with their coarse-resolution counterparts for comparison.

The polarimetric retrieval has a fundamentally different relationship to the unresolved sub-pixel inhomogeneity. This can be demonstrated with the sub-pixel polarized reflectance histogram in Fig. 11b. The reflectances in this figure have been binned by scattering angle to create a distribution of polarized reflectances for the 50 m sub-pixels within the selected 800 m pixel footprint. Within the plot there are also two curves, shifted in amplitude away from the histogram for clarity, that display the mean 800 m multi-angular polarized reflectance and corresponding 800 m retrieved polarized phase function (with appropriate fitting coefficients). Note that, while this histogram gives a sense of the variability of the magnitude and scale of the polarized reflectances, what ultimately matters for the coarse-resolution polarimetric retrieval is the relative shape of the 800 m averaged polarized reflectance curve. It is evident from this histogram and these curves that the mean angular position of the supernumerary bow does not shift, indicating that there is no significant difference between re(pol)800 m, re(pol)50 m, and re(VW)50 m. This agrees with previous studies on the impact of unresolved inhomogeneity on polarimetric re retrievals (Shang et al., 2015). In contrast, there is clear variability in the amplitude of sub-pixel polarized reflectances. This variability owes itself to both optical (τ) and microphysical inhomogeneity (i.e., ve(VW) > 0.15) within the coarse-resolution pixel. For thin clouds (τ < 3) the supernumerary bow amplitude is dependent on both τ and ve (Alexandrov et al., 2012b). With ve fixed, the polarized reflectance converges towards an asymptotic maximum for optically thick clouds (τ 3), a consequence of increasing depolarization due to multiple scattering. Similarly, for a fixed τ, reflectances corresponding to ve(VW) > 0.15 also produce decreased polarization in the primary and supernumerary bow features, as discussed in Sect. 2. Each of these effects reduces sensitivity to the cloudbow features; thus unresolved variability in τ and ve could influence coarse-resolution retrievals. For example, Shang et al. (2015) found that unresolved spatial inhomogeneity of τ and ve increased retrieval biases in ve(pol), while they were not able to discern a trend in retrieval biases in their study. However, in our case study featured in Fig. 11b we do not see a significant difference between coarse (ve(pol)800 m) and fine-scale (ve(pol)50 m) retrievals, but both retrievals are biased low relative to the mean LES property (ve(VW)50 m). This result was surprising, because both fine- and coarse-resolution retrievals were biased similarly. It appears as though coarse-resolution retrievals arrive at the same answer as the fine-scale retrievals through different processes. The average of fine-scale retrievals (that are systematically biased low) and the retrieval based on the average of fine-scale reflectances (which are reduced for reasons discussed above) results in a similar retrieval outcome. Unlike the bispectral retrieval, where retrievals differ from one another at different resolutions, the polarimetric retrieval seems to compare well to itself at both resolutions – even when it might be biased relative to the underlying microphysics of the physical scene. To examine this further, we performed polarimetric retrievals on sub-populations of the 50 m polarized reflectances within this 800 m pixel that omitted either the ve(VW) > 0.15 or τ < 3 from the population. Removing these thin or broad droplet size distributions from the high-resolution data set had little to no impact on either the coarse-resolution re(pol) or ve(pol) retrieval. From these results and the histogram in Fig. 10d it appears that the impact of spatial resolution on ve(pol) retrievals is largely a consequence of an unresolved anticorrelation between τ and ve rather than a feature directly related to spatial resolution.

Figure 12Angular-resolution sensitivity experiments examining polarimetric retrievals of re (a) and ve (b) for all LES scenes at the 800 m spatial resolution. The color and size of scattered points denote the angular resolution of each retrieval. The gray dashed lines denote the ±1 step in the LUT space of the polarimetric retrieval.

## 4.5 Sensitivity to angular resolution and sampling

The polarimetric retrieval requires high-resolution multi-angular data to resolve the supernumerary bow features. To test how angular resolution influences polarimetric retrievals, we examined coarse-spatial-resolution (800 m) re(pol) retrievals at different angular resolutions. Each angular resolution (i.e., changing angular step size) was also convoluted with shifting angular sampling (i.e., changing the initial angle). This convolution is necessary in order to account for all possible sets of scattering angle observations associated with each resolution. These coarse-resolution retrievals were then compared to the original high-angular-resolution retrieval. The results of this experiment (Fig. 12a) reveal that coarsening angular resolution does not systematically bias re(pol) retrievals, although angular resolutions exceeding 3 do result in a marked increase in retrieval variability (i.e., a constant mean bias but increased absolute bias). In contrast, Fig. 12b demonstrates that angular resolutions exceeding 3 lead to both high-biased ve(pol) and increased retrieval variability. An explanation for the origin of the observed degradation in retrieval accuracy above 3 angular resolution is demonstrated in Fig. 13a. Two different polarized phase functions with re= 15 µm and ve= [0.03, 0.2] (solid and dashed-dotted, respectively) are sampled at an angular resolution of 3.5 (indicated by the gray vertical lines). This resolution is coarser than the spacing between the supernumerary bow features. As a consequence, this particular angular sampling intersects these curves at nearly the same amplitudes. This degeneracy yields a relatively low cost function during the best-fit optimization step of the polarimetric curve-fitting retrieval algorithm, making it possible to obtain an inaccurate solution if this results in a cost-function minimum. The lack of observed differences between these two curves results in a lack of ve information, which could be exacerbated by observational uncertainty. However, under different angular sampling conditions, e.g., shifting the initial angle by a few degrees to the right, the supernumerary bow peaks of the low ve curve would be sampled and the similarity between the observations of these two curves would vanish. This example highlights an important feature of multi-angular polarimetry: observations at poor angular resolutions can suffer from increased biases depending on whether or not important angles are sampled. Generalizing this result requires determining the angular spacing of the supernumerary bow features for other re. Pursuing this, we find that decreasing cloud droplet size widens and dilates supernumerary bow features, making it easier to resolve supernumerary bow features at coarse angular resolution. The peak-to-peak distance of the supernumerary bow oscillations can be treated as the Nyquist frequency, or in this case Nyquist resolution. In signal analysis, a sampling resolution finer than the Nyquist frequency is required to appropriately resolve features of an oscillatory signal. The Nyquist angular resolution required for resolving the supernumerary bow oscillations changes with both re and λ according to the behavior illustrated in Fig. 13b. This analysis indicates that multi-angular observations in a shorter-wavelength spectral band would require finer angular resolutions. The Nyquist angular resolution for λ= 0.865 and re= 15 µm is 3, providing an explanation for the increased variability in re(pol) and ve(pol) LES retrievals at angular resolutions coarser than the Nyquist limit.

Figure 13(a) features the polarized phase functions re= 15 (red) at ve= 0.03 (solid) and ve= 0.2 (dashed). Gray dashed lines and circles indicate a 3.4 observation sampling of the phase functions. The Nyquist resolution is obtained by measuring the peak-to-peak distance of the supernumerary bow oscillations and dividing that distance in half. The Nyquist resolution changes as a function of re and λ as shown in panel b, where the gray vertical line highlights the Nyquist resolutions required for the re= 15 case.

Table 2The influence of unresolved microphysical inhomogeneity on polarimetric retrievals is explored in Shang et al. (2015). Their results are replicated here and compared to the arithmetic mean re (〈 re 〉), and the mathematical aggregation results (${r}_{\mathrm{e}}^{\prime }$ and ${v}_{\mathrm{e}}^{\prime }$) defined in Eqs. (7) and (8).

5 Summary and discussion

The analysis in this study, which features comparisons of fundamentally different passive cloud property retrieval techniques, is facilitated by comparisons to LES cloud fields used as input to the retrievals. At the native LES resolution (50 m) there are promising results for both the bispectral and polarimetric retrievals (with 1-D radiative transfer assumptions). For the bispectral retrieval, the LES comparison shows significant biases for retrievals of very thin clouds, as well as only small differences between the vertically weighted cloud properties in each of the two SWIR/MWIR bands (2.13 and 3.75 µm). Meanwhile, for the polarimetric retrieval, the comparison demonstrates that the re(pol) retrieval agrees well with the vertically weighted in situ properties of each LES scene. However, the ve(pol) retrieval exhibits persistent low biases due to a lack of retrieval sensitivity to very broad droplet size distributions (i.e., ve(VW) > 0.15). The optical thickness retrievals from both methods are effectively the same, with the caveat that the polarimetric technique performs the re(pol) retrieval as an a priori constraint on the τ retrieval space. Regarding τ, both bispectral and polarimetric retrievals were found to have a small systematically high bias on the order of 2–5 %.

The uncertainty in observed total and polarized reflectances was found to introduce only weak systematic biases in bispectral or polarimetric re retrievals (0.1 % or less). Similarly, the bispectral τ retrievals were also not systematically biased. In contrast, total reflectance uncertainty did produce a slight systematic bias of 2.43 % in the polarimetric τ(pol) retrieval that can be linked to the convexity of the single-band LUT used to perform the retrieval. This sort of bias could perhaps be accounted for by introducing a Taylor expansion correction similar to the one discussed in Zhang et al. (2016) in the context of unresolved inhomogeneity. Beyond these systematic biases, we found that the induced uncertainties in the bispectral retrievals were δre= 5 or δτ= 7 %. The influence of polarimetric retrieval is likely sensitive the polarimetric LUT grid spacing, but here we found uncertainties that were less than the bispectral retrieval of re, δre(pol) = 1 to 4 %, and δve(pol) = 10 to 20 %. In the context of the rest of our comparison studies, the lack of systematic biases and relatively small uncertainties allowed us to discuss retrieval behavior in the absence of uncertainty.

The retrieval intercomparison of polarimetric and bispectral retrievals in this study demonstrates that both techniques yield very similar results, especially when the most reliable populations of cloud properties are selected for each method (τ > 3 and ve around 0.1). While the physical principles and measurement requirements are vastly different, both retrieval techniques seem to be able to capture similar information about re and τ. These results agree with high-resolution airborne observations obtained during the PODEX and ORACLES field campaigns, where RSP and AMS microphysical retrievals are compared (Alexandrov et al., 2015; Knobelspiesse et al., 2017). These high-spatial-resolution field campaign observations indicate that the two retrieval techniques agree well to within the tolerances also found in the present study. The bispectral re retrievals are found to be moderately sensitive to ve in the 3.75 µm band, and less so in the less absorptive and more deeply penetrating 2.13 µm band. Coupling the retrieved ve(pol) to the bispectral re (3.75 µm) retrieval led to slight improvements in the re(pol) and re(VW) comparison. It should be noted that for MODIS cloud products the bias due to the ve= 0.1 assumption does not substantially impact the re retrieval compared to other sources of bias (i.e., cloud inhomogeneity or 3-D radiative effects). In addition, the MODIS Collection 6 cloud product includes uncertainty estimates associated with the ve assumption. The intercomparison of the bispectral and polarimetric τ retrievals indicates that the two produce very similar results. This was to be expected, as the polarimetric technique also uses a bispectral LUT approach to derive τ. When the results from the two methods diverge, the observations tend to be related to the thin-cloud regimes.

The presence of a multiple-solution space in the bispectral LUTs, where small droplet sizes (re < 5) have the same reflectance as larger droplets, was shown to induce numerous outliers, resulting in a significant high bias in the bispectral retrievals for both re and (to a lesser extent) τ. This multiple-solution space likewise impacts the MODIS operational products, since the bispectral LUTs used in the MODIS Collection 6 cloud products include theoretical re solutions as low as 4 µm. However, for retrievals with multiple LUT solutions the MODIS product only reports the larger re value, leading to a systematic bias if the observed cloud really includes a population of small droplets. As a consequence, for thin clouds with small droplet sizes one can expect the comparison of polarimetric and bispectral retrievals to disagree. This strong high bias for small re retrievals provides a plausible explanation for the large discrepancies observed in the small-droplet-size regime in the intercomparison of MODIS and POLDER retrievals (Bréon and Doutriaux-Boucher, 2005). Absent a solution to this issue, future intercomparisons or combined climatological data sets should be limited to retrievals of re(pol) exceeding 5–7 µm (depending on the respective bispectral LUT multiple-solution-space properties).

At the coarse spatial resolutions of most satellite instruments, cloud inhomogeneity can significantly impact retrievals. In the context of this study we find that the influence of unresolved spatial inhomogeneity is a dominant source of bias between the polarimetric and bispectral re retrievals. In this study we found that even for 100 % cloudy pixels (at a coarse 800 m horizontal resolution) the influence of the PPH bias is significant, with the average re bias exceeding 1 µm in the most inhomogeneous LES scene (ATEX clean). Based on these results we still expect that the overall systematic bias observed in the MODIS and POLDER intercomparison of moderate droplet size regimes is in large part attributable to the influence of this PPH bias (Bréon and Doutriaux-Boucher, 2005). Recently, great effort has been made to account for the influence of the PPH bias on bispectral MODIS retrievals. The 2-D Taylor expansion technique implemented by Zhang et al. (2016) offers the possibility of quantifying (and potentially correcting for) the impact of PPH bias on bispectral retrievals. This approach requires high-spatial-resolution measurements in at least one spectral band to obtain the sub-pixel reflectance variability, which is used to determine corrections for the bias of re and τ. In addition to PPH bias, 3-D radiative effects are also influenced by spatial resolution. The focus on 1-D radiative transfer in this study leaves questions for future studies regarding the influence of these 3-D radiative effects. Future work will need to identify the relative differences between 3-D radiative effects on total and polarized reflectances and retrievals.

Sufficient angular resolution is one of the more important requirements of the polarimetric retrieval technique. We find that resolving the multi-angular polarized reflectance at a resolution coarser than the Nyquist angular resolution of the supernumerary bow results in greater uncertainty (re(pol) and ve(pol)) and biased (ve(pol)) polarimetric retrievals. The required angular resolution is dependent both on droplet size and wavelength. Future cloud polarimetric instrumentation should consider these angular-resolution requirements. While we have not explicitly tested the so-called “super-pixel” approach implemented for POLDER retrievals, these coarse-spatial- and coarse-angular-resolution studies lead to some anticipated biases induced by this technique. We would expect such an approach to further bias ve(pol) retrievals low, due to the lack of sensitivity to unresolved high-ve populations. In addition, this current study indicates that re(pol) retrieval variance might increase, but the mean bias might not increase significantly. However, if there is significant correlation between the unresolved re and ve populations within an observation footprint, the mean re bias would be expected to suffer.

Ultimately, the utility of any optical property data set depends on the science questions for which the data set will be used. These questions may focus on the determination of domain-averaged water mass, radiative flux calculations, or microphysical process studies on a range of scales. The appropriate retrieval may differ for each of these science questions, and as a consequence the comparison of the bispectral and polarimetric retrievals discussed here ought to be viewed through the lens of a particular application.

Appendix A

We often treat the droplet size distribution observed by in situ instruments (on the order of meters) as relatable to the inferred size distribution properties obtained by remote-sensing retrievals (on the order of kilometers). This mathematical analysis addresses how resolution and scale influence the inferred cloud microphysical distribution. The modified gamma distribution not only suits observations of in situ cloud droplet size distributions, but it also exhibits several useful mathematical relationships:

$\begin{array}{ll}& 〈{r}^{\mathrm{2}}〉={r}_{\mathrm{e}}^{\mathrm{2}}\left({v}_{\mathrm{e}}-\mathrm{1}\right)\left(\mathrm{2}{v}_{\mathrm{e}}-\mathrm{1}\right)\\ & 〈{r}^{\mathrm{3}}〉={r}_{\mathrm{e}}^{\mathrm{3}}\left({v}_{\mathrm{e}}-\mathrm{1}\right)\left(\mathrm{2}{v}_{\mathrm{e}}-\mathrm{1}\right)\\ \text{(A1)}& & 〈{r}^{\mathrm{4}}〉={r}_{\mathrm{e}}^{\mathrm{4}}\left({v}_{\mathrm{e}}-\mathrm{1}\right)\left(\mathrm{2}{v}_{\mathrm{e}}-\mathrm{1}\right)\left({v}_{\mathrm{e}}+\mathrm{1}\right).\end{array}$

From a retrieval perspective all droplet size distributions are treated as gamma-distributed. There is a potential disconnect here, from the perspective of scale analysis, when retrievals at a 50 m spatial resolution (our LES resolution) and retrievals at 1 km (MODIS retrieval resolution), or even 150 km (POLDER retrieval resolution), each are being treated as gamma-distributed. However, not all droplet microphysics information is created equal; the droplet size distributions at higher resolution (subscript i) influence the low-resolution (subscript lr) droplet size distributions. With high-resolution information the different moments of the coarser-resolution droplet size distribution should be able to be constructed from the high-resolution microphysics. For a distribution made up of the summation of gamma size distributions the moments of the low-resolution distribution can be expressed by the following relationship, because summation and integration are each linear operators:

$\begin{array}{ll}& {〈{r}^{n}〉}_{\mathrm{lr}}={\int }_{r}{r}^{n}\left[\sum _{i}^{k}{N}_{i}\left(r,{r}_{e,i},{v}_{e,i}\right)\right]\mathrm{d}r\\ \text{(A2)}& & {〈{r}^{n}〉}_{\mathrm{lr}}=\sum _{i}^{k}\left[{\int }_{r}{r}^{n}{N}_{i}\left(r,{r}_{e,i},{v}_{e,i}\right)\mathrm{d}r\right]=\sum _{i}^{k}\left[{〈{r}^{n}〉}_{i}\right].\end{array}$

With this mathematical rule in mind, the values of re and ve for the low-resolution droplet size distribution can be obtained by substitution into Eqs. (2) and (3):

$\begin{array}{}\text{(A3)}& & {r}_{\mathrm{e}}^{\prime }\equiv \frac{{〈{r}^{\mathrm{3}}〉}_{\mathrm{lr}}}{{〈{r}^{\mathrm{2}}〉}_{\mathrm{lr}}}=\frac{\sum _{i}^{k}{〈{r}^{\mathrm{3}}〉}_{i}}{\sum _{i}^{k}{〈{r}^{\mathrm{2}}〉}_{i}},\text{(A4)}& & {v}_{e}^{\prime }\equiv \frac{{〈{r}^{\mathrm{4}}〉}_{\mathrm{lr}}{〈{r}^{\mathrm{2}}〉}_{\mathrm{lr}}}{{\left({〈{r}^{\mathrm{3}}〉}_{\mathrm{lr}}\right)}^{\mathrm{2}}}-\mathrm{1}=\frac{\sum _{i}^{k}{〈{r}^{\mathrm{4}}〉}_{i}{〈{r}^{\mathrm{2}}〉}_{i}}{{\left(\sum _{i}^{k}{〈{r}^{\mathrm{3}}〉}_{i}\right)}^{\mathrm{2}x}}-\mathrm{1}.\end{array}$

Henceforth, we will refer to the ${r}_{\mathrm{e}}^{\prime }$ and ${v}_{\mathrm{e}}^{\prime }$ relationships in Eqs. (7) and (8) as microphysical “aggregation rules.” It should be noted that these rules fundamentally treat the DSD as gamma-distributed at all scales.

The microphysical aggregation rules allow for the explanation of some features of the coarse polarimetric retrieval experiments displayed in Shang et al. (2015). Referring to the inhomogeneous polarimetric retrieval experiments in Table 2 and Fig. 4 of their paper, we reproduced their results and calculated the corresponding ${r}_{\mathrm{e}}^{\prime }$ and ${v}_{\mathrm{e}}^{\prime }$ in our Table 2, which contains the same retrieval examples and corresponding ${r}_{\mathrm{e}}^{\prime }$ and ${v}_{\mathrm{e}}^{\prime }$ results for the cases examined in their study. There is a clear difference between the mean re or ve and the polarimetric retrieval results. Using the microphysical aggregation rules defined above, we derived that the appropriate distribution properties, ${r}_{\mathrm{e}}^{\prime }$ and ${v}_{\mathrm{e}}^{\prime }$, are generally in closer agreement with the polarimetric retrievals of re(pol). These results offer a possible explanation as to why the polarimetric retrieval does not agree with the average of the sub-scale microphysics in the Shang et al. study. A couple of things should be noted here: (1) when there is little variability in the unresolved re (e.g., re= [15, 20] µm), the mean, retrieval, and estimated mixture are generally all in agreement (e.g., re〉= 17.5, re(pol) = 18, and ${r}_{\mathrm{e}}^{\prime }$= 18.2 µm). (2) When large variability in the unresolved re (e.g., re= [5, 20]) is present, both the retrieved and estimated mixture strongly favor the larger droplet effective radius (e.g., re(pol) = 19 and ${r}_{\mathrm{e}}^{\prime }=$ 19.12 µm). (3) Large variability in unresolved re sometimes results in large differences between ve(pol) and ${v}_{\mathrm{e}}^{\prime }$. The last two points are likely a consequence of the resulting coarse-resolution (multi-modal) distribution differing significantly from the gamma-distribution assumption stated previously.

Applying this analysis to the aggregation of LES scene microphysics will allow for the determination of how accurate a spatial mean aggregation reflects the true coarse-resolution microphysical parameters. We first assumed that all of the highest-resolution vertically weighted size distributions can be assumed to be appropriately characterized by a gamma distribution with re=re(VW) and ve=ve(VW). We then aggregated these LES microphysical properties at the 50 m native resolution to increasingly coarser resolutions (100, 200, 400, and 800 m), using both the mean and the aggregation rules. We found that the differences between the two techniques are negligible (Δre 0.01 µm and Δve 0.001) and do not significantly vary with final resolution. Apparently, the importance of the aggregation rules in the LES are far less important than what we had found in the multiple-moment cases tested in Shang et al. (2015). One clear difference between the these multiple moment cases and the LES was that the toy models are reductive bimodal distributions, exhibiting very large sub-scale microphysical inhomogeneity in re. This non-physical variability is something that is not commonly observed in the LES or in observational studies. To address this, we performed a theoretical examination of how important the aggregation rules are for calculating the bias between simple average aggregation and mathematical rule aggregation. In this experiment we established various distributions of unresolved DSDs with varying re and ve populations. These joint distributions of re and ve were used to test how the variance (i.e., the unresolved variability) would influence the average and mathematical-rule-aggregated results. This test confirmed that large differences between the simple average and mathematical aggregation rules requires spatial inhomogeneity of microphysics much larger than those observed in the LES or typical observational studies. Based on these results we recommend that future studies focusing on the effect of unresolved microphysical inhomogeneity on polarized retrievals should consider more realistic inhomogeneity conditions on both re and ve.

Data availability
Data availability.

The data set produced for this study is built upon A. S. Ackerman's DHARMA LES with derived fields, radiative transfer, and retrieval products developed by D. J. Miller. The derived LES microphysical/optical properties and the radiatively modeled scenes and retrievals used in this study are all available here: https://www.dropbox.com/sh/xgqzsk8m017q76r/AACUX9TZaDLeioevCfb9xyqIa?dl=0 (last access: 12 June 2018).

Competing interests
Competing interests.

The authors declare that they have no conflict of interest.

Acknowledgements
Acknowledgements.

The hardware used in the computational studies is part of the UMBC High Performance Computing Facility (HPCF). The facility is supported by the US National Science Foundation through the MRI program (grant nos. CNS-0821258 and CNS-1228778) and the SCREMS program (grant no. DMS-0821311), with additional substantial support from the University of Maryland, Baltimore County (UMBC). See https://hpcf.umbc.edu/ (last access: 12 June 2018) for more information on HPCF and the projects using its resources.

Edited by: Manfred Wendisch
Reviewed by: two anonymous referees

References

Ackerman, A. S., Hobbs, P. V., and Toon, O. B.: A model for particle microphysics, turbulent mixing, and radiative transfer in the stratocumulus-topped marine boundary layer and comparisons with measurements, J. Atmos. Sci., 52, 1204–1236, 1995.

Ackerman, A. S., Kirkpatrick, M. P., Stevens, D. E., and Toon, O. B.: The impact of humidity above stratiform clouds on indirect aerosol climate forcing, Nature, 432, 1014–1017, 2004.

Ackerman, A. S., vanZanten, M. C., Stevens, B., Savic-Jovcic, V., Bretherton, C. S., Chlond, A., Golaz, J.-C., Jiang, H., Khairoutdinov, M., Krueger, S. K., Lewellen, D. C., Lock, A., Moeng, C.-H., Nakamura, K., Petters, M. D., Snider, J. R., Weinbrecht, S., and Zulauf, M.: Large-Eddy Simulations of a Drizzling, Stratocumulus-Topped Marine Boundary Layer, Mon. Weather Rev., 137, 1083–1110, https://doi.org/10.1175/2008MWR2582.1, 2009.

Alexandrov, M. D., Cairns, B., and Mishchenko, M. I.: Rainbow Fourier transform, J. Quant. Spectrosc. Ra., 113, 2521–2535, https://doi.org/10.1016/j.jqsrt.2012.03.025, 2012a.

Alexandrov, M. D., Cairns, B., Emde, C., Ackerman, A. S., and van Diedenhoven, B.: Accuracy assessments of cloud droplet size retrievals from polarized reflectance measurements by the research scanning polarimeter, Remote Sens. Environ., 125, 92–111, https://doi.org/10.1016/j.rse.2012.07.012, 2012b.

Alexandrov, M. D., Cairns, B., Wasilewski, A. P., Ackerman, A. S., McGill, M. J., Yorks, J. E., Hlavka, D. L., Platnick, S. E., Thomas Arnold, G., van Diedenhoven, B., Chowdhary, J., Ottaviani, M., and Knobelspiesse, K. D.: Liquid water cloud properties during the Polarimeter Definition Experiment (PODEX), Remote Sens. Environ., 169, 20–36, https://doi.org/10.1016/j.rse.2015.07.029, 2015.

Bréon, F. M. and Doutriaux-Boucher, M.: A comparison of cloud droplet radii measured from space, IEEE T. Geosci. Remote, 43, 1796–1805, https://doi.org/10.1109/TGRS.2005.852838, 2005.

Bréon, F. M. and Goloub, P.: Cloud droplet effective radius from spaceborne polarization measurements, Geophys. Res. Lett., 25, 1879–1882, 1998.

Cairns, B., Russell, E. E., and Travis, L. D.: Research Scanning Polarimeter: calibration and ground-based measurements, SPIE's Conference on Polarization: Measurement, Analysis, and Remote Sensing II, 186–196, https://doi.org/10.1117/12.366329, 1999.

Cho, H. M., Zhang, Z., Meyer, K., Lebsock, M., Platnick, S., Ackerman, A. S., Di Girolamo, L., C Labonnote, L., Cornet, C., Riedi, J., and Holz, R. E.: Frequency and causes of failed MODIS cloud property retrievals for liquid phase clouds over global oceans, J. Geophys. Res., 120, 4132–4154, https://doi.org/10.1002/2015JD023161, 2015.

De Haan, J. F., Bosma, P. B., and Hovenier, J. W.: The adding method for multiple scattering calculations of polarized light, Astron. Astrophys., 183, 371–391, 1987.

Deirmendjian, D.: Scattering and polarization properties of water clouds and hazes in the visible and infrared, Appl. Opt., 3, 187–196, 1964.

Deschamps, P. Y., Breon, F. M., Leroy, M., Podaire, A., Bricaud, A., Buriez, J. C., and Seze, G.: The POLDER mission: instrument characteristics and scientific objectives, IEEE T. Geosci. Remote, 32, 598–615, https://doi.org/10.1109/36.297978, 1994.

Diner, D. J., Xu, F., Garay, M. J., Martonchik, J. V., Rheingans, B. E., Geier, S., Davis, A., Hancock, B. R., Jovanovic, V. M., Bull, M. A., Capraro, K., Chipman, R. A., and McClain, S. C.: The Airborne Multiangle SpectroPolarimetric Imager (AirMSPI): a new tool for aerosol and cloud remote sensing, Atmos. Meas. Tech., 6, 2007–2025, https://doi.org/10.5194/amt-6-2007-2013, 2013.

Fridlind, A. M. and Ackerman, A. S.: Estimating the Sensitivity of Radiative Impacts of Shallow, Broken Marine Clouds to Boundary Layer Aerosol Size Distribution Parameter Uncertainties for Evaluation of Satellite Retrieval Requirements, J. Atmos. Ocean. Tech., 28, 530–538, https://doi.org/10.1175/2010JTECHA1520.1, 2011.

Hansen, J. E.: Circular Polarization of Sunlight Reflected by Clouds, J. Atmos. Sci., https://doi.org/10.1175/1520-0469(1971)028<1515:CPOSRB>2.0.CO;2, 1971.

Hansen, J. E. and Travis, L. D.: Light scattering in planetary atmospheres, Space Sci. Rev., 16, 527–610, 1974.

King, M. D., Menzel, W. P., Kaufman, Y. J., Tanré, D., Bo-Cai Gao, Platnick, S., Ackerman, S. A., Remer, L. A., Pincus, R., and Hubanks, P. A.: Cloud and aerosol properties, precipitable water, and profiles of temperature and water vapor from MODIS, IEEE T. Geosci. Remote, 41, 442–458, https://doi.org/10.1109/TGRS.2002.808226, 2003.

Knobelspiesse, K., Cairns, B., Mishchenko, M., Chowdhary, J., Tsigaridis, K., van Diedenhoven, B., Martin, W., Ottaviani, M., and Alexandrov, M.: Analysis of fine-mode aerosol retrieval capabilities by different passive remote sensing instrument designs, Opt. Express, 20, 21457–21484, https://doi.org/10.1364/OE.20.021457, 2012.

Knobelspiesse, K., Segal-Rosenhaimer, M., Redemann, J., Cairns, B., and Alexandrov, M. D.: Multi-angle, polarimetric cloud observations using a radiative transfer model trained neural network, College Park, MD, 2017.

Liu, Y. and Diner, D. J.: Multi-Angle Imager for Aerosols, Public Health Rep., 132, 14–17, https://doi.org/10.1177/0033354916679983, 2017.

Lohmann, U., Stier, P., Hoose, C., Ferrachat, S., Kloster, S., Roeckner, E., and Zhang, J.: Cloud microphysics and aerosol indirect effects in the global climate model ECHAM5-HAM, Atmos. Chem. Phys., 7, 3425–3446, https://doi.org/10.5194/acp-7-3425-2007, 2007.

Marbach, T., Phillips, P., Lacan, A., and Schlussel, P.: The 3MI Mission: Multi-Viewing-Channel-Polarisation Imager of the EUMETSAT Polar System – Second Generation (EPS-SG) dedicated to aerosol and cloud monitoring, in Sensors, Systems, and Next-Generation Satellites XVII, 8889, 88890I, International Society for Optics and Photonics, 2013.

Marshak, A., Platnick, S., Várnai, T., Wen, G., and Cahalan, R. F.: Impact of three-dimensional radiative effects on satellite retrievals of cloud droplet sizes, J. Geophys. Res., 111, D09207, https://doi.org/10.1029/2005JD006686, 2006.

Martin, G. M., Johnson, D. W., and Spice, A.: The measurement and parameterization of effective radius of droplets in warm stratocumulus clouds, J. Atmos. Sci., 51, 1823–1842, 1994.

Martins, J. V., Fernandez-Borda, R., McBride, B., Espinosa, R., and Remer, L.: Combination between in-situ and remote sensing of tropospheric aerosols, College Park, MD, 2017.

Miles, N. L., Verlinde, J., and Clothiaux, E. E.: Cloud Droplet Size Distributions in Low-Level Stratiform Clouds, J. Atmos. Sci., 57, 295–311, https://doi.org/10.1175/1520-0469(2000)057<0295:CDSDIL>2.0.CO;2, 2000.

Miller, D. J., Zhang, Z., Ackerman, A. S., Platnick, S., and Baum, B. A.: The impact of cloud vertical profile on liquid water path retrieval based on the bispectral method: A theoretical study based on large-eddy simulations of shallow marine boundary layer clouds, J. Geophys. Res., 121, 4122–4141, https://doi.org/10.1002/2015JD024322, 2016.

Mishchenko, M. I., Cairns, B., Travis, L. D., Kopp, G., Schueler, C. F., Fafaul, B. A., Hooker, R. J., Maring, H. B., Itchkawich, T., Hansen, J. E., Kopp, G., Schueler, C. F., Fafaul, B. A., Hooker, R. J., Maring, H. B., and Itchkawich, T.: Accurate Monitoring of Terrestrial Aerosols and Total Solar Irradiance: Introducing the Glory Mission, B. Am. Meteorol. Soc., 677–691, https://doi.org/10.1175/BAMS-88-5-677, 2007.

Nakajima, T. and King, M. D.: Determination of the Optical Thickness and Effective Particle Radius of Clouds from Reflected Solar Radiation Measurements. Part I: Theory, J. Atmos. Sci., 47, 1878–1893, https://doi.org/10.1175/1520-0469(1990)047<1878:dotota>2.0.co;2, 1990a.

Nakajima, T. and King, M. D.: Determination of the optical thickness and effective particle radius of clouds from reflected solar radiation measurements. Part I: Theory, J. Atmos. Sci., 47, 1878–1893, 1990b.

Planck, M.: The theory of heat radiation, 2nd Edn., P. Blakiston's Son & Co., Philadelphia, PA, available at: http://gutenberg.org/ebooks/40030 (last access: 12 June 2018), 1914.

Platnick, S.: Vertical photon transport in cloud remote sensing problems, J. Geophys. Res., 105, 22919–22935, 2000.

Platnick, S., King, M. D., Ackerman, S. A., Menzel, W. P., Baum, B. A., Riedi, J. C., and Frey, R. A.: The MODIS cloud products: algorithms and examples from terra, IEEE T. Geosci. Remote, 41, 459–473, https://doi.org/10.1109/TGRS.2002.808301, 2003.

Platnick, S., Meyer, K. G., King, M. D., Wind, G., Amarasinghe, N., Marchant, B., Arnold, G. T., Zhang, Z., Hubanks, P. A., Holz, R. E., Yang, P., Ridgway, W. L., and Riedi, J.: The MODIS Cloud Optical and Microphysical Products: Collection 6 Updates and Examples From Terra and Aqua, IEEE T. Geosci. Remote, 55, 502–525, https://doi.org/10.1109/TGRS.2016.2610522, 2017.

Pruppacher, H. R. and Klett, J. D.: Diffusion Growth and Evaporation of Water Drops and Ice Crystals, in Microphysics of Clouds and Precipitation, Springer, Netherlands, Dordrecht, 412–463, 1978.

Roebeling, R. A., Feijt, A. J., and Stammes, P.: Cloud property retrievals for climate monitoring: Implications of differences between Spinning Enhanced Visible and Infrared Imager (SEVIRI) on METEOSAT-8 and Advanced Very High Resolution Radiometer (AVHRR) on NOAA-17, J. Geophys. Res., 111, D20210, https://doi.org/10.1029/2005JD006990, 2006.

Rosenfeld, D., Liu, G., Yu, X., Zhu, Y., Dai, J., Xu, X., and Yue, Z.: High-resolution (375 m) cloud microstructure as seen from the NPP/VIIRS satellite imager, Atmos. Chem. Phys., 14, 2479–2496, https://doi.org/10.5194/acp-14-2479-2014, 2014.

Shang, H., Chen, L., Bréon, F. M., Letu, H., Li, S., Wang, Z., and Su, L.: Impact of cloud horizontal inhomogeneity and directional sampling on the retrieval of cloud droplet size by the POLDER instrument, Atmos. Meas. Tech., 8, 4931–4945, https://doi.org/10.5194/amt-8-4931-2015, 2015.

Stevens, B., Ackerman, A. S., and Albrecht, B. A.: Simulations of trade wind cumuli under a strong inversion, J. Atmos. Sci., 58, 1870–1891, https://doi.org/10.1175/1520-0469(2001)058<1870:sotwcu>2.0.co;2, 2001.

Stevens, B., Lenschow, D. H., Vali, G., Gerber, H., Bandy, A., Blomquist, B., Brenguier, J. L., Bretherton, C. S., Burnet, F., Campos, T., Chai, S., Faloona, I., Friesen, D., Haimov, S., Laursen, K., Lilly, D. K., Loehrer, S. M., Malinowski, S. P., Morley, B., Petters, M. D., Rogers, D. C., Russell, L., Savic-Jovcic, V., Snider, J. R., Straub, D., Szumowski, M. J., Takagi, H., Thornton, D. C., Tschudi, M., Twohy, C., Wetzel, M., and van Zanten, M. C.: Dynamics and chemistry of marine stratocumulus–DYCOMS-II, B. Am. Meteorol. Soc., 84, 579–593, https://doi.org/10.1175/BAMS-84-5-579, 2003.

Stevens, B., Moeng, C.-H., Ackerman, A. S., Bretherton, C. S., Chlond, A., de Roode, S., Edwards, J., Golaz, J.-C., Jiang, H., Khairoutdinov, M., Kirkpatrick, M. P., Lewellen, D. C., Lock, A., Müller, F., Stevens, D. E., Whelan, E., and Zhu, P.: Evaluation of Large-Eddy Simulations via Observations of Nocturnal Marine Stratocumulus, Mon. Weather Rev., 133, 1443–1462, https://doi.org/10.1175/MWR2930.1, 2005.

Tampieri, F. and Tomasi, C.: Size distribution models of fog and cloud droplets in terms of the modified gamma function, Tellus, 28, 333–347, https://doi.org/10.1111/j.2153-3490.1976.tb00682.x, 1976.

Twomey, S.: The Influence of Pollution on the Shortwave Albedo of Clouds, J. Atmos. Sci., 34, 1149–1152, https://doi.org/10.1175/1520-0469(1977)034<1149:TIOPOT>2.0.CO;2, 1977.

Werner, F., Siebert, H., Pilewskie, P., Schmeissner, T., Shaw, R. A., and Wendisch, M.: New airborne retrieval approach for trade wind cumulus properties under overlying cirrus, J. Geophys. Res., 118, 3634–3649, https://doi.org/10.1002/jgrd.50334, 2013.

Wiscombe, W. J.: Mie scattering calculations: Advances in technique and fast, vector-speed computer codes, NCAR Tech, National Center for Atmospheric Research, Boulder, Colorado, 1979.

Zhang, Z. and Platnick, S.: An assessment of differences between cloud effective particle radius retrievals for marine water clouds from three MODIS spectral bands, J. Geophys. Res., 116, D20215, https://doi.org/10.1029/2011JD016216, 2011.

Zhang, Z., Platnick, S., Yang, P., Heidinger, A. K., and Comstock, J. M.: Effects of ice particle size vertical inhomogeneity on the passive remote sensing of ice clouds, J. Geophys. Res., 115, D17203, https://doi.org/10.1029/2010JD013835, 2010.

Zhang, Z., Ackerman, A. S., Feingold, G., Platnick, S., Pincus, R., and Xue, H.: Effects of cloud horizontal inhomogeneity and drizzle on remote sensing of cloud droplet effective radius: Case studies based on large-eddy simulations, J. Geophys. Res., 117, D19208, https://doi.org/10.1029/2012JD017655, 2012.

Zhang, Z., Werner, F., Cho, H.-M., Wind, G., Platnick, S., Ackerman, A. S., Di Girolamo, L., Marshak, A., and Meyer, K.: A framework based on 2-D Taylor expansion for quantifying the impacts of subpixel reflectance variance and covariance on cloud optical thickness and effective radius retrievals based on the bispectral method, J. Geophys. Res.-Atmos., 121, 7007–7025, https://doi.org/10.1002/2016JD024837, 2016.

Zhang, Z., Dong, X., Xi, B., Song, H., Ma, P. L., Ghan, S. J., Platnick, S., and Minnis, P.: Intercomparisons of marine boundary layer cloud properties from the ARM CAP-MBL campaign and two MODIS cloud products, J. Geophys. Res., 122, 2351–2365, https://doi.org/10.1002/2016JD025763, 2017.

Zinner, T., Wind, G., Platnick, S., and Ackerman, A. S.: Testing remote sensing on artificial observations: impact of drizzle and 3-D cloud structure on effective radius retrievals, Atmos. Chem. Phys., 10, 9535–9549, https://doi.org/10.5194/acp-10-9535-2010, 2010.

Note that throughout this paper we will refer to “linearly polarized reflectances” simply as “polarized reflectances” in recognition of the negligible contribution of circularly polarized light in the atmosphere (Hansen, 1971).

The radiatively derived vertical weighting of Platnick (2000) implicitly depends on the re(z) profile, whereas a fixed parameter vertical weighting described here does not. However, the importance of this difference should be less than the vertical variability of optical depth or extinction cross section.

Note that  1 % of pixels in the LES retrieval data correspond to a “failed” bispectral retrieval due to falling outside of the LUT space. These pixels are omitted from the intercomparison. Different reasons for bispectral retrieval failure are discussed in Cho et al. (2015).

Additionally,  1 % of the cloudy pixels in this scene exhibit values below 4 µm.

Note that the MODIS LUT extends its range down to 4 µm, and in situations with multiple solutions the larger retrieval value is selected.