Estimates of top-of-the-atmosphere (TOA) radiative flux are essential for the
understanding of Earth's energy budget and climate system. Clouds, aerosols,
water vapor, and ozone (O
The Earth's energy budget constrains the general circulation of the atmosphere and determines the climate of the Earth–atmosphere system; it is therefore also an indicator of possible climate changes (Hatzianastassiou, et al., 2004). There is a long history of attempts to estimate Earth's albedo and energy budget (Dines, 1917; Hartmann et al., 1986). With the advent of the satellite remote sensing era, it became possible to directly measure the albedo of the Earth. Subsequently, the shortwave (SW) energy balance at the top of the atmosphere (TOA) and the role of clouds, aerosols, and trace gases has been studied using satellite measurements (Ramanathan et al., 1989; Yu et al., 2006; Bellouin et al., 2005; Loeb et al., 2005; Patadia et al., 2008; Joiner et al., 2009).
The Earth Radiation Budget Experiment (ERBE) was launched in October 1984 by the space shuttle Challenger and provided long- and shortwave radiation parameter measurements. TOA SW radiative parameter estimates from ERBE (Barkstrom, 1984; Barkstrom and Smith, 1986) showed that clouds approximately double the albedo of Earth to an all-sky value of 0.3 from an estimated clear-sky value of 0.15 (Ramanathan et al., 1989; Harrison et al., 1990). The next generation of broadband instruments, the Cloud and the Earth's Radiant Energy System (CERES), draws heavily on ERBE heritage. Since its first launch in 1997 on board the NASA Tropical Rainfall Measurement Mission (TRMM), CERES has provided continuous observations that can be used to understand the role of clouds and the energy cycle in global climate change (Wielicki et al., 1995; Loeb et al., 2012).
Continuous and coincident measurements of radiative fluxes and atmospheric components facilitate research studies to estimate and understand the role of different atmospheric components on the planetary energy balance. Although CERES provides state-of-the-art estimates of TOA radiative fluxes, it was not designed to make measurements of individual atmospheric components that impact those fluxes. Several studies have utilized aerosol and cloud information from high spatial resolution MODerate resolution Imaging Spectroradiometer (MODIS) measurements to quantify their impact on TOA fluxes (Yu et al., 2006; Patadia et al., 2008; Zhang et al., 2005b; Loeb et al., 2005; Oreopoulos et al., 2009). Several attempts have also been made to convert narrowband radiances into broadband fluxes using regression or more sophisticated statistical approaches (Chevallier et al., 1998; Hu et al., 2002; Domenech and Wehr, 2011; Vázquez-Navarro et al., 2013).
The Ozone Monitoring Instrument (OMI), flying on NASA's Aura satellite since 2004, provides information about components important for the Earth's SW radiation budget, including the effective cloud/aerosol fraction (Stammes et al., 2008; Joiner and Vasilkov, 2006) and total column ozone (TCO) (Veefkind et al., 2006; McPeters et al., 2008; Kroon et al., 2008). OMI-retrieved parameters can be utilized to understand their role in the Earth's SW energy budget.
Modeling the spatial and temporal distribution of the TOA shortwave flux (SWF) requires a description of the components that control the transfer of solar radiation within the Earth–atmosphere system. When required parameters are missing or incomplete, a statistical approach is an alternative for estimation of TOA SWF. Here, we develop an artificial neural network model (NNM) to estimate total reflected TOA outgoing SWF. Artificial NNs are algorithms that simulate biological NNs by learning and pattern recognition (Bishop, 1995). NNs have been used by many scientific disciplines, including Earth science, to identify patterns and extract trends in imprecise and complicated nonlinear data (e.g., Lee et al., 1990; Gupta and Christopher, 2009). In radiation studies, NNs have been used to estimate TOA and surface SWF based on radiative transfer calculations with or without data from satellites (e.g., Krasnopolsky et al., 2008, 2010; Takenaka et al., 2011; Vázquez-Navarro et al., 2012; Jiang et al., 2014). CERES TOA flux algorithms have also used NNs to generate angular distribution models (ADMs) in the absence of sufficient high-resolution imager information for reliable scene identification (Loukachine and Loeb, 2003, 2004).
In this study, we utilize OMI cloud and ozone products along with other ancillary data to estimate TOA SWF. We develop NNs that take OMI-derived quantities as inputs and provide CERES-equivalent TOA SWF as the output. In essence, the trained NNs have learned or incorporated all of the complexity that is essential to the CERES algorithms (ADMs, scene identification, etc.) and are then able to predict TOA SWF directly and efficiently based on a limited number of retrieved products from OMI or sensors that provide similar data. The trained NN models are optimized to run with data sets from OMI or similar sensors and can be applied generally to different seasons and years. For example, the NN-based models we develop here can be applied to similar measurements from the Total Ozone Mapping Spectrometer (TOMS) instruments. One objective of this study is to assess how well TOA SWF can be estimated using OMI-derived cloud and ozone products with NNs when nearly coincident CERES data are used for training. The developed NNs can then be applied to other data sets with similar products and accuracy. Alternatively, the general training approach could be applied with similar data sets such as with MODIS cloud and ozone products.
The strength of a NN approach is that it is highly efficient and, if well-trained, should be precise and accurate for this type of problem. NNs may be used to examine the sensitivity of TOA SWF to various input data sets but do not themselves provide specific insight into the physical mechanisms behind those sensitivities. NNs will of course only be as good as the data that are used for training. In addition, they may not perform well for unusual conditions that are not present in the training data set. Therefore, NNs cannot replace dedicated TOA SWF estimates from instruments like CERES.
The paper is organized as follows: Sect. 2 describes the various satellite data sets utilized in the study. Section 3 discusses the development of NN models including the selection of input parameters. Section 4 evaluates our NN estimation of TOA SWF using independent CERES data over ocean and land. Section 5 summarizes the results and discusses future work.
Under clear-sky conditions, TOA SWF is affected by the Earth's surface properties, atmospheric absorbers such as water vapor, ozone, and aerosols, and scattering by air molecules and particulates. Over ocean, surface properties can be characterized by ocean color and roughness of the ocean surface. Under cloudy sky conditions, cloud optical properties such as the cloud optical thickness, geometrical cloud fraction, effective radius, and phase function affect TOA SWF. In clear and cloudy skies, the solar zenith angle (SZA) and Sun–Earth distance (SED) impact the TOA SWF.
In this work, we make use of data sets mainly from two passive sensors in
A-train constellation of satellites that fly within 15 min of each
other: (1) Aura OMI with an equatorial crossing time of
The first CERES instrument flew on the TRMM satellite, launched in November
1997, and provided data until 2000. Five CERES instruments are currently
operating: two on NASA's Terra satellite (FM1 and FM2), two on NASA's Aqua
satellite (FM3 and FM4), and one on the Suomi National Polar-orbiting
Partnership (NPP) satellite (FM5). These CERES instruments provide
radiometric measurements of the Earth's atmosphere from three broadband
channels: (1) a shortwave channel to measure reflected sunlight (0.3–5
CERES radiances are converted to TOA fluxes using ADMs. The CERES science team has an extensive database of ADMs for clear- and cloudy-sky over both land and ocean (Loeb et al., 2005). The ADMs heavily depend upon the observed scene type and are sensitive to surface characteristics as well as cloud and aerosol optical properties (Loeb et al., 2003; Zhang et al., 2005a; Patadia et al., 2011). The ADMs over ocean are dependent upon wind speed and aerosol optical thickness along with sun-satellite geometry (Zhang et al., 2005a).
The Aqua spacecraft carries two identical CERES instruments: one operates in a cross-track scan mode (FM3) and the other in a biaxial scan mode (FM4). Measurements from the biaxial scan mode were used to develop the ADMs; this provided considerable improvement over the previous generation of instruments, including the ERBE (Loeb et al., 2003, 2007).
This study uses the Single Scanner Footprint (SSF, Edition 3A) TOA SWF
obtained from the Aqua CERES FM3. The SSF product is an instantaneous merge
of CERES parameters with coincident cloud and aerosol parameters derived
from the Aqua MODIS (Loeb et al., 2003) at the footprint level (i.e., not
daily averages). The high-resolution (1
OMI provides hyper-spectral measurements of Earth-backscattered sunlight
from UV to visible wavelengths (
Cloud–aerosol OCP, also known as effective cloud pressure, is a measure of
the reflectance-weighted pressure reached by incoming solar photons (Joiner
et al., 2012). It is distinct from the cloud-top pressure (CTP). While CTP
is the more important parameter needed for TOA long-wave flux, OCP is more
related to atmospheric absorption in the shortwave. OCP is derived from OMI
observations using two different methods (Stammes et al., 2008): (1) filling-in of solar Fraunhofer lines from rotational Raman scattering
in the UV (the OMCLDRR product) (Joiner and Bhartia, 1995; Joiner et al.,
2004) and (2) collision-induced oxygen absorption (O
OMI cloud and trace-gas algorithms use a simplified mixed Lambertian cloud
model to estimate observed radiances
Formally, the ECF is wavelength dependent because it is
defined by spectral quantities (Stammes et al., 2008). We conducted a
simulation experiment to evaluate the wavelength dependence of
If
The spectral dependence of the effective cloud fraction
(
We have used the following modified cloud fraction parameter,
Two-dimensional histogram of effective cloud fraction (ECF or
In addition to OMI data, a SeaWiFs-derived chlorophyll (Chl) concentration
climatology is used as an input predictor when
Because the sizes of the OMI (13 km
The NN inputs (predominantly retrievals from OMI measurements) are used to train the NN to match the output (CERES-derived TOA SWF). Once the network is trained, input data sets can be used to calculate TOA SWF with characteristics similar to the CERES product. Therefore, the NN-produced TOA SWF will be referred as OMI estimated SWF throughout the paper.
The general neural network architecture has three layers of neurons: an
input layer, a hidden layer, and an output layer with standard multi-layer
network architecture. We use the same number of neurons in the hidden layer
as in the input layer as this produced generally good results. The input
layer has an identity activation function; all other layers are connected by
sigmoid activation functions (Eq. 4).
A schematic of the neural network model used for estimation of TOA SW flux with OMI UV retrieved parameters. The table in the bottom lists all the input parameters corresponding to two NN models used.
NNMs require optimized training to produce accurate outputs. Here we use a standard back propagation training algorithm (Hertz et al., 1991), where inputs are iteratively sent to the neural network. In back propagation, the hidden layer weights associated with each input parameter are modified through the training process that minimizes errors between the targets and outputs (Bishop, 1995; Gardner and Dorling, 1998). After each iteration, the error is propagated backward through the network and weights are modified to bring the actual response of the network closer to the desired output in a statistical sense. The function minimized during the training is a sum of squared errors of each output for each training pattern. Once the network is trained, it can be evaluated using independent data (i.e., not used in the training data set).
Statistical analysis of the input parameter selection exercise. The
correlation coefficient
Here we examine the impact of using various input parameters on the derived
neural networks. This exercise is performed using data with
In model a, we have combined the effects of SZA, SED, and
Monthly mean (January 2007) maps of OMI minus CERES TOA SW flux (percent) for eight different NN models. The letters on the map corresponds to model number in Table 1.
The TOA SWF is estimated from a measured radiance and therefore the
observational geometry factors in. The addition of satellite-viewing
geometry parameters (VZA, RAA) to model a provides improvements in areas
of high biases and reduces the standard deviation from 37.1 to
31.4 Wm
The inclusion of TCO (model d) as an input parameter positively impacts
TOA SWF estimation as shown in Fig. 4d; the high positive biases in the
tropical Pacific and Indian oceans and in the region near 60
Model e adds cloud OCP to the input parameters included in model b.
OCP also improves SWF estimates; the regions where improvement occurs are
different from those improved by using TCO. Model f shows that when TCO
and OCP are used together as input parameters, there is further improvement
in SWF estimation. Although the global statistics in Table 1 do not clearly
reflect this improvement, Fig. 4f shows that inclusion of OCP and TCO
reduces biases in many regions, most prominently in the tropics. The
percentage of total OMI samples (monthly mean) within
The impact of surface winds and total column water vapor (model g in Fig. 4g) is
more prominent in the tropics than in other regions. Inclusion
of Chl and LERs in model h removes some of the notable low
biases in TOA SWF near the coast of northern China, the Caspian Sea, and the Black
Sea. Furthermore, model h corrects for negative biases in areas with
high TOA SWFs, most likely due to the inclusion of LER. The model “h”
produces 89 % (99 %) of OMI-estimated monthly mean TOA SWFs within
We next examine the performance of the NN model h with respect to
different input samples. We first examine the robustness of the NN for
detection of interannual variability. In this exercise, we trained the NN
with data from the first 15 days of January 2007 as above and applied it to
input data from the entire months of January 2007 (Fig. 5a) and January 2006 (Fig. 5b). Figure 5a–b
present 2-D histograms similar to that in Fig. 2 but here compare the TOA SWF from the NN with that from CERES for
January of the 2 different years over ocean. The colors represent the 2-D
histogram (or density) of coincident pairs using a bin size of 10 Wm
Two-dimensional histograms (similar to Fig. 2) of daily mean CERES- and
OMI-derived (NN) TOA SW flux for January 2007:
A NN trained on 1 particular month of data is not guaranteed to perform well for a different month. In the next test, we used the NN model trained on January 2007 data with input data from July 2007. Results for July (using CERES as the benchmark) were degraded as compared with application to January data. This is due presumably to changes in observing conditions between the 2 different months (changes in solar angles).
We then trained the same NN (identical input parameters) using a subset of data from July 2007 and applied it to data from the entire month of July 2007. Results (again using CERES as the benchmark) were of similar quality to those where the NN was trained and applied to January. This exercise suggests that we may need to use different models for different months or expand our training data set for application to different months.
We next use data from the 1st day of each month of 2007 for training and
data from the 16th day of each month of 2007 for evaluation. The
comparisons with CERES using the training and validation data are consistent
as shown in Fig. 6. The almost identical values of statistical parameters
for training and validation data demonstrate that the neural network has
been well trained. For example, there is a high degree of linear correlation
(
Two-dimensional histogram similar to Fig. 5 but showing training (top) and
validation (bottom) results from the combined all-sky NN models (input
parameters listed in Fig. 3, model “h” for
Root mean square errors (RMSE), normalized RMSE (NRMSE in percent), data samples (percent), and bias (percent) in training and validation data sets (same model and data as used in Fig. 6) as a function of effective cloud fraction for the data presented in Fig. 4. Model “h” is used to produce these results.
Further evaluation of the entire year reveals that this NN (model h) is appropriate for all months. Therefore this model will be used for subsequent analyses in this study. Creating more networks as a function of scene type or for different latitude belts or even for different months/seasons may improve results in certain regions. However, based on our results, we simplified the approach by minimizing the number of networks.
About 1–2 % of total coincident data correspond to
Two-dimensional histograms (similar to Fig. 5) of the daily OMI and CERES TOA
SW flux averaged over different spatial grid sizes for July 2007:
Similar to Fig. 8 but for monthly mean data (July 2007) OMI and
CERES TOA SW flux averaged over different spatial grid sizes:
Here onward, all the results presented in Figs. 7–12 and discussed are
produced using model h, which is the most optimized model. Figure 7
presents the RMSE, RMSE normalized by CERES flux
(NRMSE in percent), data sample (percent), and bias (percent) for 5 % ECF bins.
This analysis includes both training and validation data as presented in Fig. 6. The RMSE varies between about 24 and 35 Wm
Similar to Fig. 10c and d but with a NN trained using data from
the OMI cloud O
Similar to Fig. 10a, c, and d except over land for July 2007.
In order to evaluate the NN performance at different spatial and temporal
scales similar to those used by the climate community, we use data from July 2007. Figure 8 presents a comparison of daily CERES and OMI TOA SWF over
ocean for six spatial scales: the OMI native pixel (13
Statistical parameters corresponding daily and monthly intercomparisons of pixel and gridded TOA SW flux data from CERES and OMI.
Note:
For the daily data, as the spatial averaging scales increase from 0.5
to 10
Figure 9a–d show 2-D histograms of monthly mean gridded data over ocean at
0.5, 1, 2, and 5
Figure 10 presents the spatial distribution of 1
Figure 11 similarly shows differences between CERES and OMI TOA SWF over
ocean derived using
Figure 12a shows a time series of daily global mean values of TOA SWF over
ocean from OMI and CERES for 2007. Both instruments show almost identical
daily variations with differences within
We developed a similar land-only NN model that utilizes most of the input
parameters from our ocean NN (e.g., OMI RRS cloud parameters). The only
change is that for surface characterization we use a monthly climatology of
surface broadband albedo in place of the Chl concentration and
surface wind speed. The albedo product is derived using a combination of
CERES and MODIS observations at 1
Figure 13 shows results from the OMI-derived CERES-trained NN that produces TOA SWF over land. Statistical comparison with CERES over land provides results similar to those over ocean. The NN performs well over Asia and parts of Europe and the Americas. The OMI-based NN tends to underestimate TOA SWF over the high albedo desert areas of Northern Africa, Australia, and also over some regions of South America. Note that the large differences that occur in coastal regions may be due to imperfect collocations.
We have developed a neural network approach to estimate TOA SWF based primarily on UV parameters retrieved with the Aura OMI and Aqua CERES-derived TOA SWF used for training. One year of data from OMI and CERES has been used to train/validate/analyze several separate neural networks for different conditions, which together provide estimation of TOA SWF under all-sky conditions. The most important input parameters are ECF and sun-satellite geometry. TCO and cloud optical centroid pressure from OMI, as well as surface-related parameters, provide secondary positive impacts.
Independent validation at different spatial and temporal scales shows that
the OMI NN-based approach reproduces CERES-derived TOA SWF with high
fidelity. Correlation coefficients for all comparison are > 0.95,
and slopes are close to unity. A high percentage of OMI-estimated monthly
mean TOA SWF at 0.5
We plan to apply our derived neural networks to long-term, well-calibrated UV measurements from TOMS. The TOMS series provides a long-term data record dating back to late 1978 (about half a decade before the first ERBE launch) with a few small gaps between that time and the first CERES launch. We should be able to apply NN models derived with CERES/OMI to TOMS, provided that the input parameters are either available and compatible or can be estimated independently. For example, in place of actual cloud OCPs that are available from OMI, but not from TOMS, we could use a cloud OCP climatology that was developed from OMI data for use in the TOMS total ozone algorithm. The lower spatial resolution of TOMS is not expected to present any difficulties. This approach can also be extended to the future geostationary missions that provide the relevant input data, such as the NASA Earth Ventures Tropospheric Emissions: Monitoring of Pollution (TEMPO), the Korean Geostationary Environmental Monitoring Spectrometer (GEMS), and the European Space Agency (ESA) Sentinel 4 (Al-Saadi et al., 2015). Finally, we may apply the NN training and evaluation approach to data from CERES and the nadir mapper on the Ozone Mapping Profiling Suite (OMPS) that provides information similar to OMI. Both instruments fly on the Suomi NPP satellite. This may reduce collocation noise and small biases that result from the time difference between OMI and CERES measurements. The final NN models developed in this study (e.g., NNM-1 and NNM-2 in Fig. 3) along with the instructions on how to use them have been provided in the Supplement.
This material is based upon work supported by the National Aeronautics and Space Administration (NASA) issued through the Science Mission Directorate (SMD) for the Aura Science Team managed by Kenneth Jucks and Richard Eckman. We thank the CERES, OMI, MODIS, and GEOS-DAS data processing teams for providing the data used for this study. We would also like to thank Norman Loeb and Arlindo da Silva for useful discussion and comments during the preparation of the paper. Edited by: V. Sofieva