Journal cover Journal topic
Atmospheric Measurement Techniques An interactive open-access journal of the European Geosciences Union
Journal topic
Atmos. Meas. Tech., 11, 6833-6859, 2018
https://doi.org/10.5194/amt-11-6833-2018
Atmos. Meas. Tech., 11, 6833-6859, 2018
https://doi.org/10.5194/amt-11-6833-2018

Research article 21 Dec 2018

Research article | 21 Dec 2018

BOREAS – a new MAX-DOAS profile retrieval algorithm for aerosols and trace gases

The MAX-DOAS profile retrieval algorithm BOREAS
Tim Bösch1, Vladimir Rozanov1, Andreas Richter1, Enno Peters1,a, Alexei Rozanov1, Folkard Wittrock1, Alexis Merlaud4, Johannes Lampel3,9, Stefan Schmitt3, Marijn de Haij2, Stijn Berkhout6, Bas Henzing5, Arnoud Apituley2, Mirjam den Hoed2, Jan Vonk6, Martin Tiefengraber7,8, Moritz Müller7,8, and John Philip Burrows1 Tim Bösch et al.
• 1Institute of Environmental Physics, University of Bremen, Bremen, Germany
• 2Royal Netherlands Meteorological Institute (KNMI), De Bilt, the Netherlands
• 3Institute of Environmental Physics, University of Heidelberg, Heidelberg, Germany
• 4Royal Belgian Institute for Space Aeronomy (BIRA-IASB), Brussels, Belgium
• 5Netherlands Organization for Applied Scientific Research (TNO), Utrecht, the Netherlands
• 6National Institute for Public Health and the Environment (RIVM), Bilthoven, the Netherlands
• 7LuftBlick, Kreith, Austria
• 8Institute of Atmospheric and Cryospheric Sciences, University of Innsbruck, Innsbruck, Austria
• 9Airyx GmbH, Eppelheim, Germany
• anow at: Institute for Protection of Maritime Infrastructures, Bremerhaven, Germany
Abstract

We present a new MAX-DOAS profiling algorithm for aerosols and trace gases, BOREAS, which utilizes an iterative solution method including Tikhonov regularization and the optimal estimation technique. The aerosol profile retrieval is based on a novel approach in which the absorption depth of O4 is directly used in order to retrieve extinction coefficient profiles instead of the commonly used perturbation theory method. The retrieval of trace gases is done with the frequently used optimal estimation method but significant improvements are presented on how to deal with wrongly weighted a priori constraints and for scenarios in which the a priori profile is inaccurate.

Performance tests are separated into two parts. First, we address the general sensitivity of the retrieval to the example of synthetic data calculated with the radiative transfer model SCIATRAN. In the second part of the study, we demonstrate BOREAS profiling accuracy by validating the results with the help of ancillary measurements carried out during the CINDI-2 campaign in Cabauw, the Netherlands, in 2016.

The synthetic sensitivity tests indicate that the regularization between measurement and a priori constraints is insufficient when knowledge of the true state of the atmosphere is poor. We demonstrate a priori pre-scaling and extensive regularization tests as a tool for the optimization of retrieved profiles. The comparison of retrieval results with in situ, ceilometer, NO2 lidar, sonde and long-path DOAS measurements during the CINDI-2 campaign always shows high correlations with coefficients greater than 0.75. The largest differences can be found in the morning hours, when the planetary boundary layer is not yet fully developed and the concentration of trace gases and aerosol, as a result of a low night-time boundary layer having formed, is focused in a shallow, near-surface layer.

1 Introduction

Aerosols and trace gases play an important role for life on Earth as high concentrations have adverse impacts on human health and climate. Furthermore, aerosols impact the Earth's energy budget by radiative forcing. They interact with solar radiation by scattering and absorption processes. Additionally, they have an impact on the formation of clouds when acting as cloud condensation nuclei (CCN). Despite considerable efforts by the scientific community, uncertainties in aerosol radiative forcing are still high (Stocker2014).

In addition to aerosols, atmospheric trace gases are of importance in an increasingly urbanized world as they impact human health, agriculture, acid deposition and climate (e.g. ). Trace gases in the troposphere such as nitrogen dioxide and sulfur dioxide (NO2, SO2), nitrous acid (HONO), formaldehyde (HCHO) and glyoxal (CHOCHO) need to be measured to assess their impact on air quality and because their spatial and temporal inhomogeneous distributions provide information on different emission sources, e.g. the combustion of fossil fuels, biomass burning and agriculture . The separation and identification of anthropogenic from natural sources, as well as measurements of their temporal variability, improve the understanding of their effects on climate and life on Earth. The current lack of knowledge leads to uncertainties in temporal and spatial emission patterns of trace gases and aerosols and limits our ability to fully understand atmospheric processes. This results in a need for comprehensive measurements to fill these gaps.

For more than 15 years, Multi-Axis Differential Optical Absorption Spectroscopy (MAX-DOAS) measurements have been used to investigate the chemical composition of the troposphere . This passive remote-sensing method is based on absorption spectroscopy applied to measurements of scattered sunlight. The advantages of ground-based MAX-DOAS when compared to satellite observations are the high sensitivity for the lowermost layers of the troposphere, the high temporal and spatial resolution of measurements and the lower cost.

The strength of the absorption signal detected in scattered sunlight depends on the absorber amounts and their vertical distribution but also on the length of the light path. In general, this length in a certain altitude layer is a function of the measurement geometry. Therefore, a set of MAX-DOAS radiance measurements taken at different elevation angles (lines of sight, LOSs) contains information about the vertical distribution of trace gases, which can be retrieved. However, the retrieval of absorber profiles from MAX-DOAS measurements is an ill-posed problem that needs additional constraints (see Sect. 3.2). Thus, the inversion, which is applied to retrieve vertical concentration profiles, is done by elaborated mathematical methods such as optimal estimation (OE). In addition to the trace gas profile, the aerosol extinction profile needs to be retrieved as well as it has a non-linear effect on the trace gas retrieval and it is too variable to be approximated by climatologies. In contrast to the trace gas retrievals, aerosol retrievals are strongly non-linear, necessitating iterative inversion schemes such as the Gauss–Newton algorithm or the Levenberg–Marquardt algorithm (Rodgers2004).

Profile retrieval algorithms used in the scientific community are either inversion algorithms or parameterized approaches. Inversion algorithms directly link the measurement quantity to the vertical profile of the target absorber with the help of a forward model . This model is usually calculated with a radiative transfer model (RTM), assuming a selected set of atmospheric conditions. For trace gases, the model also includes a sensitivity matrix which can be considered derivative of the measurement quantity with respect to the trace gas concentration for each altitude level. For aerosols, this sensitivity matrix is normally calculated via perturbation theory. This perturbation approach is shortly summarized as follows: by consideration of a trace gas with a well-known vertical distribution such as the oxygen dimer O2−O2 (or short O4), the aerosol extinction for each layer is changed gradually and the resulting modelled observations for different LOSs are compared with measurements until the iteration converges (e.g. the difference between the measurement and modelled quantity is small enough).

Parameterization, on the other hand, means that a forward model, based on a limited set of parameters, is used to describe the measurement quantity. Frequently used forward model parameters are integrated values and profile information such as shape and height of certain absorber layers . The forward model results are calculated with an RTM and are least-squares fitted to the measurements. Generally, look-up tables (LUTs) are pre-calculated for a set of different scenarios, parameters and geometries covering all relevant atmospheric conditions, avoiding high computational efforts during near-real-time calculations.

Inversion algorithms have the advantage that they are not limited to the scenarios used when creating the LUT but unrealistic profiles are possible when the measurement, inversion or regularization (weighting between the information from measurements and a priori information) is poor. On the other hand, parameterized approaches evaluate profiles much faster, as the slow forward model computation was already done when creating the LUT.

Although efforts to derive concentration profiles from MAX-DOAS measurements have been made for more than one decade, profiling is still a difficult task and the results of different algorithms can differ strongly when the absorber of interest is highly variable and inhomogeneous on spatial and temporal scales . Comparison studies of different profiling algorithms for synthetic as well as for real data, summarizing the current state of the art and including results from the algorithm introduced here, will be reported by and .

The purpose of this study is the introduction of IUP Bremen's new MAX-DOAS profile retrieval algorithm for aerosols and trace gases BOREAS (Bremen Optimal estimation REtrieval for Aerosols and trace gaseS), which has been developed to improve the earlier profile retrieval algorithm (Wittrock2006). BOREAS uses a novel approach for the retrieval of aerosols but a similar optimal estimation technique for the retrieval of trace gases. In contrast to perturbation-based inversion algorithms, BOREAS uses the change (from an a priori state) of depth in an absorption band to get information on the aerosol content which caused this change. To the authors' knowledge, this approach has never been used within an operational profiling algorithm and it complements the variety of methods with another promising technique. In order to improve the standard OE method for trace gases, a simple way of changing the weighting between a priori and measurement constraints is introduced by varying a regularization factor so that oscillations are minimized. Additionally, the usage of a priori pre-scaling is highlighted as one way of improving the results when the a priori profile is far away from the real atmosphere.

The development of BOREAS is aimed at several key properties:

1. Flexibility: the algorithm should retrieve aerosol and trace gas profiles from any MAX-DOAS measurement with pre-filtering options for the data.

2. Accuracy and stability: the algorithm should be stable in terms of varying atmospheric conditions when retrieving profiles for several years of data. The profiling results (modelled observation) should fit the measured observations with a high accuracy.

3. Automation: the retrieval should respond to problematic data/settings (e.g. low information content, wrong regularization) automatically with an included problem solution scheme.

4. Fast: the algorithm should be fast enough to allow near-real-time profile retrievals.

The paper is organized as follows: in the first section, a typical MAX-DOAS measurement and its information content are shortly described. The next section focuses on the theoretical background of the retrieval algorithm. This section is followed by a detailed sensitivity study. The fourth section is divided into an analysis of synthetic data, a discussion of error sources and an example of application to real measurements, which shows that BOREAS is able to retrieve profiles with high reliability and accuracy. The final section (Sect. 5) concludes and summarizes the results presented in this study.

2 MAX-DOAS measurements and the DOAS retrieval

Modern ground-based MAX-DOAS instruments are capable of measuring scattered sunlight in the ultraviolet (UV) and visible (VIS) spectral range with a full azimuthal and elevation angle coverage of the hemisphere (see Fig. 1).

Figure 1Schematic representation of typical MAX-DOAS measurement geometries.

In general, the spectral least-squares fit of absorption cross sections and a polynomial to the slant optical thickness (logarithm of a Sun-normalized radiance at a certain LOS, also called slant optical thickness, SOT) $\mathit{\tau }=-\mathrm{ln}\left(I/{I}_{\mathrm{0}}\right)$ (DOAS fit) provides integrated concentrations ρ along the light path L, the slant column densities S (or SCD) . The length of the light path L depends on the Sun's position, the scattering properties of the atmosphere and on the viewing geometry. For ground-based measurements, the extraterrestrial solar spectrum I0 is usually replaced by an intensity spectrum measured in the zenith direction Iref (reference spectrum) to compensate for Fraunhofer lines and instrumental issues (e.g. temperature dependent wavelength shifts). As this zenith spectrum itself is usually contaminated by the absorber of interest to some extent, the difference between the two SCDs, ΔS is retrieved and referred to as differential SCD. The differential optical thickness Δτj of a certain absorber j only, is defined as

$\begin{array}{}\text{(1)}& \mathrm{\Delta }{\mathit{\tau }}_{j}\left(\mathit{\lambda },\mathbf{\Omega }\right)={\mathit{\sigma }}_{j}\left(\mathit{\lambda }\right)\mathrm{\Delta }{S}_{j}\left(\mathbf{\Omega }\right),\end{array}$

where σj(λ) is the spectral absorption cross section and $\mathbf{\Omega }=\mathit{\left\{}\mathit{\alpha },\mathit{\phi },\mathit{\vartheta }\mathit{\right\}}$ represents the angular variables. Here, α is the elevation angle (or LOS), φ is the relative azimuth angle (RAA) between the Sun and viewing azimuth, and ϑ is the solar zenith angle (SZA).

When using a single zenith spectrum measured around noon as Iref, the tropospheric signal in Iref is minimized, but for absorbers, which are also present in the stratosphere, stratospheric absorption contributes significantly to ΔS in the morning and evening. This is due to diurnal variations in the stratospheric amounts of some absorbers such as NO2 and the change in light paths through the stratosphere because of varying SZA (see e.g. ). To minimize the stratospheric signal, a reference spectrum measured close in time to the intensity I for a certain geometry is applied. Usually, a scan of measurements in different elevation angles is followed by a measurement in the zenith direction so that the whole scan can be fitted with the same reference spectrum.

The ratio of slant column density S and the vertically integrated absorber concentration $V={\int }_{\mathrm{0}}^{H}\mathit{\rho }\left(z\right)\text{d}z$ (vertical column density or VCD, V) is called the air mass factor, M (AMF).

$\begin{array}{}\text{(2)}& M\left(\mathbf{\Omega }\right)\equiv \frac{S\left(\mathbf{\Omega }\right)}{V}\end{array}$

Generally, this value depends on the radiative transfer through the atmosphere and on parameters such as SZA, surface albedo, wavelength and the profiles of aerosols and gaseous absorbers. For trace gases in the boundary layer, it can, in a first approximation, be described by the geometric air mass factor

$\begin{array}{}\text{(3)}& {M}_{\text{geom}}\left(\mathit{\alpha }\right)=\frac{S\left(\mathit{\alpha }\right)}{V}=\frac{\mathrm{1}}{\mathrm{sin}\left(\mathit{\alpha }\right)},\end{array}$

assuming that the trace gas layer is located below the last scattering point . The differential AMF (dAMF), assuming a zenith reference spectrum, is then $\mathrm{\Delta }M=S/V-{S}_{\text{ref}}/V=\left(\mathrm{1}-\mathrm{sin}\left(\mathit{\alpha }\right)\right)/\mathrm{sin}\left(\mathit{\alpha }\right)$. The difference between geometric and true AMF depends strongly on the wavelength, elevation angle, relative azimuth angle and aerosols.

Because of the altitude-dependent concentration of absorbers, there is a need for a similar quantity describing the ratio of S and V in a certain layer. The concept of the box air mass factor z (BAMF) introduces this height dependence by defining the ratio for each altitude layer z:

$\begin{array}{}\text{(4)}& {\mathcal{M}}_{z}\left(\mathbf{\Omega }\right)=\frac{{S}_{z}\left(\mathbf{\Omega }\right)}{{V}_{z}}.\end{array}$

Here, z depends on the altitude in contrast to the total AMF M. Box air mass factors are calculated by radiative transfer models and can be understood as sensitivity of the partial slant column to the partial vertical column at a certain altitude. On the other hand, multiplication of a profile of partial vertical columns with z leads to the corresponding slant column density.

3 Retrieval algorithm description

The profile retrieval algorithm BOREAS was developed in order to retrieve aerosol and trace gas vertical profiles from MAX-DOAS measurements. The aerosol retrieval is fully implemented within the RTM SCIATRAN to decrease computation time for the iterative minimization scheme. BOREAS is a Python-written analysis script, which calls SCIATRAN for the aerosol retrieval and for calculations of BAMF matrices, which are then used within an optimal-estimation-based trace gas retrieval. A flow chart depicting BOREAS is shown in Fig. 2. In the next subsections, we give an overview of the individual steps of the algorithm.

Figure 2BOREAS analysis flow chart. The algorithm is separated into two steps. Step 1: aerosol retrieval within a Tikhonov regularization. Step 2: optimal-estimation-based trace gas retrieval.

3.1 Retrieval of aerosol profiles

The standard DOAS fit does not provide direct information about aerosols present in the atmosphere. However, the scattering and absorption properties of aerosols have an impact on the measured differential slant column density ΔS(Ω) because scattering processes can significantly modify the light path. Since the single-scattering albedo ω (SSA) stays more or less constant when the aerosol type does not change over time and altitude, it is not a quantity to be retrieved in BOREAS and is kept constant with typical values for the prevalent aerosol type (e.g. urban pollution: SSA =0.92). The angular scattering distribution of aerosols is fully quantified by the scattering phase function. Within BOREAS, we use the Henyey–Greenstein approximation with constant values for the asymmetry factor g, which quantifies the amount of forward and backward scattering (e.g. urban pollution: g=0.68). In this parameterization, the optical properties of aerosols are fully defined with g, SSA and an extinction coefficient profile σe(z). The latter is the retrieval parameter for BOREAS aerosol retrievals. Note that the usage of measured phase functions will be implemented in BOREAS in the future, since Henyey–Greenstein is in some situations not an accurate representation of the atmospheric aerosol scattering distribution.

If σe(z) is known, then the aerosol optical thickness (AOT) can be determined as ${\mathit{\tau }}_{\text{aer}}={\int }_{\mathrm{0}}^{H}{\mathit{\sigma }}_{\text{e}}\left(z\right)\text{d}z$, where the integration is performed over the entire atmosphere. If the vertical profile of an absorber number density is known, differences between modelled and measured S for this absorber are the result of differences between the assumed and real aerosol profiles. As observations at different LOSs have varying sensitivity to the presence of aerosols at different altitudes, this can be used to retrieve an aerosol profile.

Generally, the vertical profiles of species in the troposphere are unknown because of the temporal and spatial variability of emission sources, transport and conversion processes. However, the oxygen monomer O2 is in this respect an exceptional species because it only depends on pressure and temperature. Furthermore, as the oxygen dimer O4 concentration is proportional to the squared monomer concentration, its profile exponentially decreases with altitude as well. The O4 slant column density can easily be determined because O4 has spectral absorption features in the wavelength regions of most DOAS fits. Detailed sensitivity studies of O4 ΔS(Ω) measurements regarding changes in atmospheric aerosol properties can be found in and .

In the BOREAS aerosol retrieval algorithm, the difference between modelled and measured O4 differential slant optical thicknesses Δτ(λ,Ω) (DSOT) is used to retrieve aerosol extinction profiles in an iterative process. The measured Δτ(λ,Ω) is calculated using Eq. (1), where ΔSi(Ω) is the DSCD retrieved in the framework of the standard DOAS fit, and the simulated O4 DSOT ($\mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega }\right)$) is calculated as follows:

$\begin{array}{}\text{(5)}& \mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)=\mathrm{ln}\frac{I\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)}{{I}_{\text{ref}}\left(\mathit{\lambda },{N}_{\text{a}}\left(z\right)\right)}\phantom{\rule{0.25em}{0ex}},\end{array}$

where $I\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)$ and Iref(λ,Na(z)) are the intensities calculated using SCIATRAN under the assumption that O4 is the only absorber. The dependency on the reference geometry Ωref is summarized with the index “ref” and will be neglected from now on. The vertical profiles of pressure and temperature are used according to the US standard atmosphere model (NASA1976) but can be replaced by measured atmospheric conditions when available. The aerosol loading is described using an a priori concentration number density vertical profile Na(z).

The inverse problem with respect to the aerosol optical depth is then formulated as

$\begin{array}{}\text{(6)}& {∥\phantom{\rule{0.125em}{0ex}}\mathrm{\Delta }\mathit{\tau }\left(\mathit{\lambda },\mathbf{\Omega }\right)-\mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)-P\left(\mathit{\lambda },\mathbf{\Omega }\right)∥}^{\mathrm{2}}⟶min\phantom{\rule{0.25em}{0ex}},\end{array}$

where P(λ,Ω) is a polynomial of lower order and its argument Ω emphasizes that polynomial coefficients depend on the LOS angle. The assumptions made in this formulation are that (1) the O4 absorption derived from the measurements does not depend on the concentration of other trace gases and (2) that the optical depth in an atmosphere without other absorbers can be described as the sum of the O4 optical depth and a low-order polynomial.

Since $\mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)$ is a non-linear and complicated functional of Na(z), an analytical solution of this minimization problem does not exist. To simplify the solution of the minimization problem given by Eq. (6), let us consider the variation of DSOT caused by the variation of the aerosol number density. The variation of DSOT in a linear approximation can be represented in the form of the following functional Tailor series :

$\begin{array}{ll}\text{(7)}& & \mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)=& \mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },\stackrel{\mathrm{‾}}{{N}_{\text{a}}}\left(z\right)\right)+\underset{\mathrm{0}}{\overset{H}{\int }}W\left(\mathit{\lambda },\mathbf{\Omega },z\right)\phantom{\rule{0.125em}{0ex}}\mathit{\delta }{N}_{\text{a}}\left(z\right)\text{d}z+{\mathit{\epsilon }}_{\text{lin}}\left(\mathit{\lambda }\right)\phantom{\rule{0.25em}{0ex}},\end{array}$

where

$W\left(\mathit{\lambda },\mathbf{\Omega },z\right)={\frac{\mathit{\delta }\phantom{\rule{0.125em}{0ex}}\mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },{N}_{\text{a}}\left(z\right)\right)}{\mathit{\delta }\phantom{\rule{0.125em}{0ex}}{N}_{\text{a}}\left(z\right)}|}_{\stackrel{\mathrm{‾}}{{N}_{\text{a}}}}$

is a functional derivative of DSOT with respect to the aerosol number density profile Na(z) around the initial guess $\stackrel{\mathrm{‾}}{{N}_{\text{a}}}\left(z\right)$. $W\left(\mathit{\lambda },\mathbf{\Omega },z\right)$ is usually referred to as the weighting function and εlin(λ) is the linearization error.

It follows that the weighting function provides a linear relationship between the variation of DSOT and the variation of an aerosol number density Na around the initial guess $\stackrel{\mathrm{‾}}{{N}_{\text{a}}}$. Now inserting Eq. (7) into Eq. (6) and approximating the integral with a finite sum, we have

$\begin{array}{ll}\text{(8)}& & ‖\mathrm{\Delta }\mathit{\tau }\left(\mathit{\lambda },\mathbf{\Omega }\right)-\mathrm{\Delta }\stackrel{\mathrm{̃}}{\mathit{\tau }}\left(\mathit{\lambda },\mathbf{\Omega },\stackrel{\mathrm{‾}}{{N}_{\text{a}}}\left(z\right)\right)& {-\sum _{k=\mathrm{1}}^{L}\mathcal{W}\left(\mathit{\lambda },\mathbf{\Omega },{z}_{k}\right)\left({x}_{k}-\stackrel{\mathrm{‾}}{{x}_{k}}\right)-P\left(\mathit{\lambda },\mathbf{\Omega }\right)∥}^{\mathrm{2}}⟶min\phantom{\rule{0.25em}{0ex}}.\end{array}$

Here, $\mathrm{\Delta }x={x}_{k}-\stackrel{\mathrm{‾}}{{x}_{k}}={N}_{\text{a}}\left({z}_{k}\right)-\stackrel{\mathrm{‾}}{{N}_{\text{a}}}\left({z}_{k}\right)$, L is the number of altitude levels, and $\mathcal{W}\left(\mathit{\lambda },\mathbf{\Omega },{z}_{k}\right)$ is defined to satisfy the trapezoidal integration rule.

The solution of the minimization problem given by Eq. (8) is performed on a discrete wavelength grid. In fact, only five wavelengths over the O4 absorption band centred, for example, at 477 nm are used.

The minimization problem can be reformulated in the following vector–matrix form:

$\begin{array}{}\text{(9)}& {∥\mathrm{\Delta }\mathbit{y}-\mathbf{K}\mathrm{\Delta }\mathbit{x}∥}^{\mathrm{2}}⟶min.\end{array}$

Here, the vector $\mathrm{\Delta }\mathbit{y}=\mathbit{y}-\stackrel{\mathrm{‾}}{\mathbit{y}}=|\mathrm{\Delta }{\mathbit{y}}_{\mathrm{1}},\mathrm{\Delta }{\mathbit{y}}_{\mathrm{2}},\mathrm{\dots },\mathrm{\Delta }{\mathbit{y}}_{N}{|}^{T}$ has the dimension NM and describes the difference between measured and simulated DSOT at N LOS angles and M wavelengths. The mth element of vectors yn and ${\stackrel{\mathrm{‾}}{\mathbit{y}}}_{n}$ is the differential slant optical depth of O4 at wavelength m, given by

$\begin{array}{}\text{(10)}& & \mathit{\left\{}{\mathbit{y}}_{n}{\mathit{\right\}}}_{m}=\mathrm{\Delta }{\mathit{\tau }}^{-}\left({\mathit{\lambda }}_{m},{\mathbf{\Omega }}_{n}\right)\phantom{\rule{0.25em}{0ex}},\phantom{\rule{0.25em}{0ex}}m=\mathrm{1},\mathrm{\dots },M,\text{(11)}& & \mathit{\left\{}\stackrel{\mathrm{‾}}{{\mathbit{y}}_{\mathbit{n}}}{\mathit{\right\}}}_{m}=\mathrm{\Delta }{\stackrel{\mathrm{̃}}{\mathit{\tau }}}^{-}\left({\mathit{\lambda }}_{m},{\mathbf{\Omega }}_{n},\stackrel{\mathrm{‾}}{{N}_{\text{a}}}\left(z\right)\right)\phantom{\rule{0.25em}{0ex}},\end{array}$

where the superscript “” denotes that a polynomial is subtracted. The state vector Δx has the dimension L and matrix K has the dimension $\left(N\cdot M\right)×L$; i.e. the matrix consists of NM rows and L columns. The rth row of matrix K is then given by

$\begin{array}{ll}\text{(12)}& & \mathit{\left\{}\mathbf{K}{\mathit{\right\}}}_{r}=& \left\{{\mathcal{W}}^{-}\left({\mathit{\lambda }}_{m},{\mathbf{\Omega }}_{n},{z}_{\mathrm{1}}\right),{\mathcal{W}}^{-}\left({\mathit{\lambda }}_{m},{\mathbf{\Omega }}_{n},{z}_{\mathrm{2}}\right),\mathrm{\dots },{\mathcal{W}}^{-}\left({\mathit{\lambda }}_{m},{\mathbf{\Omega }}_{n},{z}_{L}\right)\right\}\\ & \text{with}\phantom{\rule{0.25em}{0ex}}r=\left(n-\mathrm{1}\right)M+m\end{array}$

and contains weighting functions for the mth wavelength, nth LOS in L layers.

The minimization problem given by Eq. (9) is solved by employing an iterative Tikhonov regularization technique (Rodgers2004):

$\begin{array}{ll}\text{(13)}& {\mathbit{x}}_{i+\mathrm{1}}=& \phantom{\rule{0.25em}{0ex}}{\mathbit{x}}_{\mathrm{0}}+\left({\mathbf{K}}_{i}^{T}{\mathbf{S}}_{y}^{-\mathrm{1}}{\mathbf{K}}_{i}+{\mathbf{S}}_{\mathrm{0}}^{-\mathrm{1}}+\mathit{\gamma }{\mathbf{S}}_{t}^{T}{\mathbf{S}}_{t}{\right)}^{-\mathrm{1}}& {\mathbf{K}}_{i}^{T}{\mathbf{S}}_{y}^{-\mathrm{1}}\left(\mathbit{y}-{\mathbit{y}}_{i}+{\mathbf{K}}_{i}\left({\mathbit{x}}_{i}-{\mathbit{x}}_{\mathrm{0}}\right)\right).\end{array}$

Here, i is the iteration number, S0 and Sy are the a priori and measurement covariance matrices, Ki is the weighting function matrix calculated using the estimated aerosol number density profile xi, St is the first-order derivative matrix with Tikhonov parameter γ and ${\mathbit{x}}_{\mathrm{0}}=\stackrel{\mathrm{‾}}{\mathbit{x}}$ and ${\mathbit{y}}_{\mathrm{0}}=\stackrel{\mathrm{‾}}{\mathbit{y}}$ are the a priori profile and the representing a priori measurement vector respectively. Equation (13) is solved in the logarithmic space in order to avoid negative values.

There are three criteria to stop the iteration process:

• Convergence in parameter space, i.e. the maximum difference between the components of the state vector at two subsequent iterative steps does not exceed the selected criterion (e.g. 0.0001 km−1);

• The root mean square difference between measured and simulated O4 DSOT is less than selected (e.g. 0.001);

• The maximum number of iterations is reached.

In addition to the equations above, here we introduce other quantities which are useful for describing the retrieval. The gain matrix

$\begin{array}{}\text{(14)}& {\mathbf{G}}_{i}=\left({\mathbf{K}}_{i}^{T}{\mathbf{S}}_{y}^{-\mathrm{1}}{\mathbf{K}}_{i}+{\mathbf{S}}_{\mathrm{0}}^{-\mathrm{1}}+\mathit{\gamma }{\mathbf{S}}_{t}^{T}{\mathbf{S}}_{t}{\right)}^{-\mathrm{1}}{\mathbf{K}}_{i}^{T}{\mathbf{S}}_{y}^{-\mathrm{1}}\end{array}$

describes the sensitivity of the solution to the measurement. Note that this formulation also includes the Tikhonov term, leading to differences for γ≠0, with the definition given in . The averaging kernel Ai=GiKi (AK) characterizes the sensitivity of the solution to the true state. The trace of AK quantifies the degrees of freedom of the signal ds=tr(A) (DOFs). This quantity is commonly understood as the number of individual pieces of information which can be retrieved.

Table 1SCIATRAN settings for the calculation of differential slant column densities.

3.2 Retrieval of trace gas concentration profiles

Compared to aerosols, the inverse problem for trace gases is easier to solve because, under the assumption of an optically thin atmosphere, the relationship between trace gas concentration and measured differential slant column density is linear. Then, the forward model F(x,b) is equal to a set of measurements y,

$\begin{array}{}\text{(15)}& \mathbit{y}=\mathbf{F}\left(\mathbit{x},\mathbit{b}\right)+\mathit{ϵ},\end{array}$

where ϵ includes the error of measurement and forward model. F(x,b) depends on the retrieval quantity vector x (trace gas concentration profile) and on an additional parameter vector b. The latter includes quantities which have an impact on the measurement and are known with some accuracy (e.g. a priori information).

From the perspective of MAX-DOAS, this relationship is strongly ill posed as the number of retrieval parameters is usually much higher than the number of measurements. Furthermore, as the light paths of two geometrically close elevation angles traverse similar vertical layers near the surface, measurements cannot be considered independent. Consequently, the retrieval parameter vector x is not fully constrained by the input vector y, which introduces the need for additional a priori knowledge which constrains the solution. With the assumption of a Gaussian error distribution, the OE method is often used to prevent unstable solutions when dealing with ill-posed problems.  (Rodgers2004).

$\begin{array}{ll}{\mathbit{x}}_{n}& ={\mathbit{x}}_{\mathrm{0}}+\left({\mathbf{K}}^{T}{\mathbf{S}}_{y}^{-\mathrm{1}}\mathbf{K}+{g}^{-\mathrm{1}}{\mathbf{S}}_{\mathrm{0}}^{-\mathrm{1}}{\right)}^{-\mathrm{1}}{\mathbf{K}}^{T}{\mathbf{S}}_{y}^{-\mathrm{1}}\left(\mathbit{y}-\mathbf{K}{\mathbit{x}}_{\mathrm{0}}\right)\\ \text{(16)}& & \text{or}& ={\mathbit{x}}_{\mathrm{0}}+{\mathbf{S}}_{\mathrm{0}}{\mathbf{K}}^{T}\left({\mathbf{KS}}_{\mathrm{0}}{\mathbf{K}}^{T}+g{\mathbf{S}}_{y}{\right)}^{-\mathrm{1}}\left(\mathbit{y}-\mathbf{K}{\mathbit{x}}_{\mathrm{0}}\right)\end{array}$

Again, S0 and Sy are the covariance matrices, y is the measurement and x0 is the a priori profile. Equation (16) is solved in linear space. Negative values are possible but can be avoided by applying appropriate a priori constraints. Note that we introduced a scaling factor g which gives the possibility of regulating the weighting between a priori and measurement information. Here, the matrix K consists of BAMF values for every layer and measuring geometry (Eq. 4) instead of weighting functions. The covariance matrix of measurements has only diagonal elements, which are the absolute errors of the spectral DOAS fit of this absorber. S0 has constant variances on its diagonal elements but consists of additional non-diagonal elements based on a Gaussian distribution, which accounts for a possible correlation of different profile layers .

4 Results and discussion

In this section, we first present and discuss the results of synthetic sensitivity studies. For this purpose, noise-free differential slant columns densities of different aerosol and NO2 scenarios were simulated with SCIATRAN. Subsequently, these data sets were used as input for BOREAS, and the results are compared with the true profiles. In the next subsection, we discuss the error sources of the retrieval before; in the last section, the results of data measured during the CINDI-2 campaign (2nd Cabauw Intercomparison of Nitrogen Dioxide measuring Instruments, 2016) are shown. These results are validated by comparison with different ancillary measurements (sondes, lidar, long-path DOAS, ceilometer, AERONET, Pandora and in situ).

Table 2Parameters for aerosol and NO2 profiles used in the synthetic sensitivity study.

Figure 3Exponential aerosol profiles for the synthetic sensitivity study. Also shown is the a priori profile for the aerosol retrieval (dashed line).

4.1 Sensitivity study with synthetic data

A synthetic data set of differential slant column densities of NO2 and O4 was created using SCIATRAN's spherical mode including multiple scattering. The settings in Table 1 were chosen to describe conditions as they can be found in urban areas like the city of Bremen. Pressure and temperature profiles were taken from a U.S. Standard Atmosphere (NASA1976). The vertical grid was set to 25 m steps from the surface up to 6 km, 250 m steps up to 10 km and 1 km steps up to 60 km. The LOS angles were chosen as a representation of a typical MAX-DOAS measurement with more angles for the lower elevations, where the sensitivity is highest. Both the solar zenith angle and relative azimuth angle were kept constant.

Figure 4Retrieval results with a fixed (a) and a pre-scaled a priori profile (b) with varying Tikhonov parameters γ for SNR = 3000. Small γ values mean less smoothing of the resulting profiles.

The trace gas and aerosol profiles in Table 2 (see also Figs. 3, 6, A1 and A8) were chosen to assess the BOREAS retrieval capability under simple (exponential profiles with scale height SH =1 km) and non-ideal conditions (box profiles, which are not similar to the exponential a priori profiles). The NO2 scenarios did not include aerosols to avoid the mixture of uncertainties from the trace gas retrieval and inaccuracies within the aerosol retrieval. The box profiles are shown and shortly discussed in Appendix A.

4.1.1 Sensitivity of the aerosol retrieval

Figure 3 shows the true exponential extinction coefficient profiles (see Table 1) with aerosol optical thicknesses of 0.2, 0.6 and 1.0. In addition, an a priori profile is depicted with the same scale height (SH = 1 km) but with AOT = 0.18. In general, an a priori profile should comprise known information of the true atmosphere. Ancillary measurements are frequently used to gather information for a mean profile. Here, we try to evaluate the performance of the algorithm by using an a priori profile close to the true profile (E1) but also for atmospheres with highly varying aerosol loads (E2, E3).

The retrieval uses vertical grid steps of 50 m for synthetic data because smaller step widths introduce retrieval noise, whereas larger steps result in vertical structure being overly smoothed. Since the retrieval tends to overestimate the upper layers due to missing sensitivity (see Fig. 9), a height-dependent a priori variance was used for the preparation of the a priori covariance matrix Sa. The measurement covariance matrix Se consists of a constant signal-to-noise (SNR) value, which is multiplied with the measurement for each LOS to get absolute covariances. The Tikhonov parameter is changed in the first examples to demonstrate the retrieval response to this parameter. The iterations stop when convergence is reached or after 60 steps. Convergence in parameter space was set to 0.0001 km−1. In addition, convergence of the rms of measured and simulated DSOT differences was set to 0.001.

On the left side of Fig. 4 the retrieval results for different Tikhonov parameters γ are shown. E1, which is close to the a priori profile (dashed grey line) is retrieved well for all γ values. In general, smaller Tikhonov parameters give too much weight to the measurement by reducing smoothing and introducing retrieval noise, which leads to oscillations. Large values smooth the solution in the direction of the a priori profile. When the true atmosphere is far away from the a priori profile, the algorithm does not perform satisfyingly. For E2, strong smoothing (large γ values) leads to more or less straight lines in which insufficient curvature can be found, whereas weak (or no) smoothing means the beginning of oscillations indicated by the reduction of the bottom extinction. No parameter value is able to retrieve the proper bottom extinction. This can be found in the same way for E3, where all results show a clear lack of accuracy. This is not surprising as the a priori profile cannot be understood as the best guess or median of the atmosphere any more.

Table 3True and retrieved AOT for SNR = 3000 and γ=1.0 with and without a priori pre-scaling, including relative differences to the true profile in brackets.

Figure 5Variation of the relative difference in AOT and bottom extinction (BOT) compared to the true value as a function of SNR and Tikhonov parameter, colour coded for scenario E3 with a priori pre-scaling.

Several authors have already highlighted retrieval problems from MAX-DOAS measurements under highly variable (in time and space) atmospheric conditions (Frieß et al.2018; Vlemmix et al.2011, 2015; Wagner et al.2011). Nevertheless, studies that focus on improvements of the regularization or a priori information are rare with respect to MAX-DOAS retrievals. Here, we use a priori pre-scaling to cope with strong deviations of the true atmosphere from the a priori estimate. For this purpose, the BOREAS aerosol retrieval is started on a reduced vertical and spectral grid from a zero profile (all a priori extinction values were set to be zero) as a priori information. In Eq. (15), the a priori profile x0 is set to 0 but the remaining terms stay unchanged in order to retrieve a reliable AOT value without limiting the solution by a certain profile shape. Since the retrieval works directly on O4 absorption, aerosol optical thickness is in general the most reliable retrieval quantity. The AOT from this pre-calculation is used to scale the a priori profile for the main run by calculating a scaling factor from the initial a priori AOT and the new value, which is applied to all initial extinction coefficients before the main inversion. Note that the only a priori information used both within the pre-run and within the main run is the Tikhonov parameter. This method performs well when the a priori shape is similar to the true atmospheric profile shape (right side of Fig. 4). However, no improvement is found when the shape of the a priori profile deviates strongly from the true atmospheric condition but no deterioration is found either (see box profiles in Appendix A). On the right-hand side of Fig. 4, clear improvements can be seen in both the overall shape and bottom extinction. Nevertheless, the profiles tend to deviate more strongly at the surface, especially when the aerosol load is higher, which was also found by . The relative differences in retrieved AOT are high for the unscaled retrieval but can be significantly reduced with a priori pre-scaling (see Table 3). Further improvements could be made by increasing the allowed number of iterations together with stronger convergence criteria (not shown), but the average retrieval time would increase as well.

A comparison of all three profiles and their optimal γ ranges with and without pre-scaling indicates that each combination of a priori and true profile has a specific optimal Tikhonov parameter which might differ strongly between each scenario. In addition, the SNR is another quantity which has a large impact on the retrieval. In general, SNR values might differ strongly throughout a day, because dynamic ranges of exposure and integration times are frequently used to operate the CCD in the ideal saturation range when the number of photons might differ strongly (e.g. due to different viewing geometries). In Fig. 5 we varied SNR (as a representation of the measurement error) and the Tikhonov parameter γ from 500 to 5000 and from 0.25 to 30 respectively to find the best parameter range for the optimization of AOT and the bottom extinction (BOT). The figure shows results for the strong aerosol scenario E3 (E1, E2 and the box scenarios are shown in the appendix). The relative differences to the true value are colour coded with bright spots, indicating better retrievals.

For scenario E3, high γ (or low SNR) values apply too much smoothing (not enough weight on the measurement), which ends in strong AOT  BOT underestimations of the retrieval results. Furthermore, there is an ideal range of SNR values for γ≤2 where the AOT relative difference is ≤10 %. In this specific SNR γ range, the bottom values do not show the best results. On the other hand, BOT values are optimized for $\mathrm{3}\le \mathit{\gamma }\le \mathrm{10}$ for SNR =2500 and SNR =3000 where the AOT does not have the lowest difference. As a consequence, due to unequal vertical sensitivity, the choice of a priori constraints and parameterization, in some cases either AOT or the shape of the profile can be optimized. Stable solutions for the optimization of both profile characteristics could not be identified in the range of parameterization, regularization and convergence. As a result, a reasonable compromise for optimum values with respect to both AOT and BOT would be in the range SNR = 2000–3000 with low Tikhonov values $\mathrm{2}\le \mathit{\gamma }\le \mathrm{5}$. Note that the SNR range between 3500 and 4000 has larger relative deviations than the surrounding settings which could not be found for other scenarios. This is assumed to be either a unique solution, where the range of settings leads to poor retrieval results, or a numerical instability.

In contrast to this pragmatic approach, the so-called L-curve approach was found to be insufficient as it already assumes good regularization; i.e. the data residual and the parameter norm (xxa) can be described well by a specific regularization parameter , which was not found for the application of MAX-DOAS profiling.

4.1.2 Sensitivity of the trace gas retrieval

The optimal-estimation-based trace gas profile retrieval faces similar problems to the regularization difficulties discussed for the aerosol retrieval in the previous section. In many studies, an a priori variance of a fixed percentage of the a priori profile is used in combination with the differential slant column density errors from the DOAS fit to constrain the solution (Frieß et al.2018). Here, we show that this is an insufficient approach, since the fitting errors might be too small, giving too much weight to the measurement when the NO2 concentration is high and the a priori profile is not a good estimate of the true atmosphere.

Again, a priori pre-scaling is used to achieve a better estimate of the true atmospheric conditions. In contrast to the aerosol retrieval, which needed a pre-run for a better first guess, the trace gas a priori pre-scaling is achieved using the 30 elevation angle measurement. In the geometric approximation (see Eq. 3), the differential AMF is 1, which leads to ΔS30=V. This vertical column density V is used as a pre-scaling value for the a priori profile.

The different NO2 exponential profiles, based on the parameters in Table 2, are shown in Fig. 6 together with the a priori profile (box profiles can be found in Appendix A). Note that we used an a priori profile close to E2. These profiles are a good representation of the daily variability in an urban region, where high NO2 concentrations are found during phases of commuter traffic (rush hours) in the morning and late afternoon and low NO2 loads are found in between.

Figure 6Exponential NO2 profiles for the synthetic sensitivity study. Also shown is the a priori profile for the aerosol retrieval (dashed line).

Figure 7 depicts the profiling results without a priori pre-scaling on the left side and with pre-scaling on the right side. Different regularization ratios are achieved by varying parameter g in Eq. (16), which varies the measurement error by $\sqrt{g}$. The retrieval is done on the same vertical grid as in the previous section with similar variances for the a priori covariance matrix. In contrast to a constant SNR, we use variances in the range of typical DOAS fit errors of NO2 for the measurement covariance matrix, which can be understood as the standard approach of the MAX-DOAS profiling community. For small elevation angles and high NO2 concentrations, the relative ΔS error is often lower than 1 % and sometimes even lower than 0.3 %, indicating a high reliability in the measurement but neglecting errors due to pointing inaccuracies of the telescope or horizontal inhomogeneities which affect the profiling result but not the measurement.

The results again show the importance of pre-scaling. Whereas the E2 scenario is well retrieved for all g ranges without pre-scaling, the other scenarios tend to oscillate or underestimate the profile. Note that the standard regularization ratio (g=1) does not lead to satisfying results for E3. Applying pre-scaling leads to more accurate profiles, especially for E3.

Figure 7Retrieval results with a fixed (a) and a pre-scaled a priori profile (b) with g factors. Small g values mean less measurement weighting for the resulting profiles.

Figure 8Retrieval errors for E1–E3 of the synthetic aerosol (a) and NO2 study (b). Aerosol results are shown for SNR = 3000 and Tikhonov parameter γ=2. Trace gas errors are depicted without additional regularization (g=1). Pre-scaling was used.

The specific optimal g range for each scenario depends not only on the a priori profile and the true atmosphere but also on the variance values, their height dependencies and the definition of off-diagonal elements. However, the quantification of this optimal range is a difficult task for real data due to the lack of knowledge about the true atmosphere. Manual variations of g as well as iterative approaches are conceivable. Possible criteria for finding the best g values are, for example, the rms between measurement and retrieval quantity or ancillary measurements. Here, we used fixed values which were chosen to decrease oscillations while maintaining the best surface concentration.

4.2 Discussion of error sources and information content

The total error of both aerosol and trace gas retrieval can be separated into three different error sources (Rodgers2004).

$\begin{array}{}\text{(17)}& {\mathbf{S}}_{\text{tot}}={\mathbf{S}}_{\text{sm}}+{\mathbf{S}}_{\text{fw}}+{\mathbf{S}}_{\text{ns}}\end{array}$

The first error term describes the smoothing error ${\mathbf{S}}_{\text{sm}}=\left(\mathbf{A}-\mathbf{I}\right){\mathbf{S}}_{\text{e}}\left(\mathbf{A}-\mathbf{I}{\right)}^{T}$ which can be understood as error for the averaging kernel smoothed estimate of the true state of the atmosphere. A and I are the averaging kernel and identity matrix respectively. The second term of Stot is the forward model error Sfw, which is hard to quantify in real retrievals, because it needs the true state of the atmosphere. For near-linear problems, this error source can be neglected, when the forward model parameters are well estimated. The last error is called the retrieval noise Sns=GSeGT (G: gain matrix; see e.g. Eq. 14) and denotes the uncertainty of the retrieved profile due to measurement errors.

Figure 8 shows smoothing, retrieval and total error for the above-mentioned scenarios of the aerosol retrieval (left) and the trace gas retrieval (right).

The total errors for both aerosol and trace gas retrieval are dominated by the smoothing error with negligibly small retrieval noise. This shows that the measurement itself can be considered a small error source in contrast to the generally limited information content of MAX-DOAS measurements. Note that the error components of the NO2 profile show small oscillations, which also indicates that larger constraints are needed. For the aerosol retrieval, the largest errors can be found at an altitude at which the sensitivity is low, with an additional increase near the surface. The errors are nearly zero for higher altitudes because of the height-dependent variances, which were chosen in order to reduce possible retrieval instabilities in altitude regimes where the sensitivity is lowest. In general, NO2 also shows larger errors for higher altitudes but the surface error is dominant here, indicating a reduced sensitivity or poor parameterization. The errors for z≥3 km are close to zero because the smallest a priori covariance is used as a boundary condition for the highest altitudes, where the sensitivity is low.

Figure 9Averaging kernels of the aerosol retrievals for scenarios E1–E3 (first three figures on the left side). Also shown are the area and FWHM of the AK on the right side. FWHM values are depicted on their nominal heights (coloured) and on the height of their individual peaks (black).

For the quantification of the vertical sensitivity of the retrieval, Fig. 9 shows the averaging kernels AK (see Eq. 14 and text) for the aerosol scenarios E1–E3 calculated for a Tikhonov parameter γ=0.25 and SNR = 3000. We used a small γ to demonstrate the general vertical sensitivity rather than the smoothed one. Larger γ values would lead to peaks at lower altitudes with slightly reduced peak sizes. Within an ideal retrieval, the AK matrix should be equal to the identity matrix (Rodgers2004). Then, delta peaks in every layer would indicate a perfect sensitivity of the retrieval to the true state of the atmosphere. For MAX-DOAS retrievals, AK are much broader, with peaks close to the surface. Here, the sensitivity is highest and the kernels for a certain layer show correlations with neighbouring layers due to the finite width of each kernel. The comparison of AK for E1–E3 reveals that, besides the surface kernel, all peaks are reduced in size and width for higher aerosol loads. This demonstrates the smaller sensitivity due to both an insufficient a priori estimate and the additional aerosol load, which reduces the general number of photons scattered into the light path (for specific geometries, e.g. RAA =90). The area of the AK should ideally be 1 over a large range of altitudes. Here, we find that only within the first kilometre is the overall sensitivity high. Note that, especially for E2 and E3, the surface area value is much smaller than 1 due to the negative area of the lowermost kernels for higher altitudes. Negative kernel values indicate that an additional aerosol load in this altitude would decrease the retrieved solution close to the surface. The full width at half maximum (FWHM) of averaging kernels are a measure of the vertical retrieval resolution with the lowest values close to the surface. In the right-hand-side plot of Fig. 9 we show FWHM at the nominal altitudes (coloured lines) and the altitudes of their individual maxima (black lines). The calculation was done by using the individual AK instead of the FWHM of fitted Gaussian distributions. The limited vertical extent of the black curves indicate that a theoretical vertical resolution with values between 500 m and 1 km is available for all kernel levels, but the specific sensitivity for all layers lies within the lowest kilometre.

The trace of the averaging kernel matrix (degrees of freedom, DOFs) is often discussed as if it is the number of independent pieces of information for a retrieval. already pointed out that this number is not necessarily a strict limitation of the retrieval and might differ strongly depending on the regularization.

In Fig. 10, we depicted the rms between measured and retrieved O4 ΔS and the DOFs for scenario E3. Giving the retrieval more freedom will increase the degrees of freedom and decrease the rms, which might be understood as more information is used for a better measurement assimilation. However, in Sect. 4.3.1, we already demonstrated that profiles with high DOFs and low rms might oscillate. DOFs only state the maximum number of independent pieces of information for the current regularization parameters without considering if the retrieved profiles are optimal. Therefore, we do not use this quantity for the description of retrieval results.

Figure 10Rms between measured and simulated O4 ΔS and the matching degrees of freedom (DOFs) for the specific retrievals for different SNR and γ values.

In Fig. 11, we show the averaging kernel, area and FWHM of the NO2 retrieval for scenarios E1–E3. Similar conclusions can be drawn as for the aerosol retrieval for the shape and altitude variation of AK. The area is closer to 1 for all altitudes, which is due to the missing smoothing of the Tikhonov term. The FWHM again shows a lower vertical sensitivity for higher layers. On the other hand, the FWHM curves at their specific peak heights (black) show a larger vertical spread in comparison to the aerosol curves, which indicates a larger sensitivity for higher altitudes in the NO2 retrieval. The DOFs for the trace gas retrieval lie in the range between 2.3 and 3.2 but decrease significantly when applying values of g≠1.

Figure 11Averaging kernels of the NO2 retrievals for scenarios E1–E3 (first three figures on the left side). Also shown are the area and FWHM of the AK on the right side. FWHM values are depicted at their nominal heights (coloured) and at the heights of their individual peaks (black).

In addition to the above mentioned errors and resolution properties, further error sources and problems should be noted. For real measurements, the true meteorological conditions are mostly unknown. pointed out that wrong O4 profiles might lead to strong deviations between retrieved and true aerosol profiles. Furthermore, we did not implement the instruments field of view (FOV) in the retrieval because it will increase run time. However, larger differences can be expected for concentrations and extinctions in the lowermost layers, when parts of the FOV might gather photons reflected on the Earth's surface. Additionally, we assume a linearity between measurement and concentration, which might be inaccurate for high aerosol loads and NO2 concentrations when the telescope points to the horizon.

4.3 Retrieval of profiles during the CINDI-2 campaign

The 2nd Cabauw Intercomparison of Nitrogen Dioxide Measuring Instruments campaign (CINDI-2) took place in Cabauw (the Netherlands) from 25 August to 7 October 2016 and was funded by the European Space Agency (ESA) and the participating research groups. The campaign goals were the characterization of differences between NO2 measurements by various instruments and approaches, and the evaluation of the resulting data sets for the validation of the Copernicus satellite, Sentinel 5 Precursor (S5P). Over 40 different instruments operated by 30 groups from all over the world provided an outstanding ensemble of data sets for this task. The campaign was a successor of CINDI, which was held in Cabauw from 16 to 24 June 2009 (Pinardi et al.2013; Piters et al.2012; Zieger et al.2011).

The measurement site Cabauw is located in a rural region dominated by agriculture but is surrounded by four of the largest cities in the Netherlands (Rotterdam, Amsterdam, Den Haag and Utrecht). Thus, depending on the wind direction, long-range transport from highly industrialized areas is likely, which results in high-pollution events. Here, 3 of the 5 investigated days show a more or less steady wind from south-easterly directions, with a change in wind direction in the evening of 15 September (see Fig. 12). On 23 September, the wind came from the west, whereas 1 day later the wind came mostly from southerly directions.

Figure 12Wind speed and direction for the investigated days (CESAR2018) resampled to 5 min values from 06:00 to 17:00 UTC.

In this study, the instrumental set-up for the validation of aerosol and trace gas profiles consists of in situ and remote-sensing instruments. Near-surface concentration and extinction values are provided by a ceilometer, NO2 lidar, long-path DOAS (LP DOAS) and in situ samplers (ICAD, CAPS, NAQMN). Integrated values are provided by Pandora and AERONET direct Sun instruments. Besides the MAX-DOAS measurements, the only routinely retrieved profiling information comes from ceilometer and NO2 lidar data. In addition, three NO2 sondes were launched on one of the days investigated here (15 September 2016).

The profile validation is realized on 2 cloud-free days (13–14 September 2016) and 3 days with broken clouds (15 September and 23–24 September 2016). The operators of MAX-DOAS instruments were asked to perform elevation scans at the beginning of each hour and at 11:15 and 11:45 UTC every day. The ancillary measurements introduced in the following subsections were resampled or averaged in this time interval to prevent discrepancies due to time lags between two observations. Each scan took around 8 min and consisted of the following elevation angles: 1, 2, 3, 4, 5, 6, 8, 15 and 30. The telescope pointed towards the westerly direction with a viewing azimuth angle of 107 from the south.

In this section, BOREAS retrievals were performed on data from the IUPB MAX-DOAS instrument on a 100 m step width vertical grid from the surface up to 4 km. The aerosol retrieval uses the a priori pre-scaling option with an exponential a priori profile described by a 1 km scale height and an aerosol optical thickness of 0.18. The asymmetry factor and single-scattering albedo were chosen to be 0.92 and 0.68 respectively. The SNR was set to 2500 with a Tikhonov parameter of 2. The a priori variance decreased with altitude from 1.5 at the surface to 0.01 at 4 km. This definition was chosen to improve the profiling results for lower altitudes, where the main aerosol load/concentration can be expected, and in order to suppress instabilities at the upper grid boundary, where the sensitivity is lowest. The trace gas retrieval again uses a priori pre-scaling with an exponentially decreasing profile with a scale height of 1 km and a vertical column density of 9×1015 molecules cm−2. The a priori covariance matrix uses the same variances as the aerosol retrieval but includes Gaussian distributed side diagonal elements to account for correlations between individual layers . The measurement covariance matrix for NO2 consists of the total differential slant column error on the diagonal elements only. Vertical profiles of pressure and temperature were created by taking the mean of 16 different sonde measurements taken during the years 2013–2015 in De Bilt (the Netherlands).

Figure 13(a) Time series of AERONET (blue) and BOREAS AOT (red). Small triangles show the original AERONET measurement, small dots with connecting lines depict the resampled data. Grey areas indicate clouds. (b) Scatter plot of both data sets including parameters of the orthogonal regression and Pearson's correlation coefficient.

Figure 14(a) Time series of BOREAS (red), ceilometer (green) and NAQMN in situ (blue) and near-surface aerosol parameters. Small triangles show the original hourly NAQMN measurements; small dots with connecting lines depict the resampled data. Green squares show ceilometer near-surface extinction values evaluated by averaging the 10–50 m coefficients. Grey areas indicate clouds. (b) Scatter plot including parameters of the orthogonal regression and Pearson's correlation coefficients.

Instrumentation for the aerosol validation

Aerosol profiles were validated using three different instruments. The retrieved AOT was compared with values from an AERONET station (AErosol RObotic NETwork, ). The level 2 AOT at 440 nm was scaled with the Angström exponent from the ratio 440 nm/675 nm to calculate the AOT at 477 nm (see Fig. 13). Due to measurement intervals varying between 4 and 30 min, we decided to resample the AERONET signal. The errors from AERONET instruments are usually in the range of 0.01 (VIS, IR) to 0.02 (UV) . Here, we used a constant error of 0.01.

The bottom extinction coefficient of the retrieved aerosol profiles was compared with in situ PM10 concentrations from the National Air Quality Monitoring Network (NAQMN) operated by the Dutch National Institute for Public Health and the Environment (RIVM). The NAQMN measurement site Wielsekade is located at a distance of around 900 m from the MAX-DOAS station. The site is listed as a regional background station and the PM10 measurement principle is beta attenuation (Thermo Fisher Scientific FH62I-R). The instrument provides data on 1 min intervals but only the hourly values are validated by RIVM. These hourly values were resampled on the MAX-DOAS times and the minute values were used to calculate the errors as standard deviation for the data points shown (see Fig. 14).

Furthermore, we used AERONET-scaled ceilometer near-surface extinction as a further validation data set. The ceilometer (CHM15k Nimbus) was operated by the Royal Netherlands Meteorological Institute (KNMI) and sampled backscattering signals every hour at a wavelength of 1064 nm. The integrated backscattering signal was divided by the 1020 nm AERONET AOD to get a conversion factor which was applied to the backscattering signal for the conversion into extinction coefficients. This new ceilometer profile was again scaled with AERONET AOT at 477 nm. The error is expressed as the standard deviation calculated from the temporal and vertical averaging.

Comparison of aerosol retrieval parameters

Figure 13 shows the time series and the scatter plot of AERONET- and BOREAS-derived AOT. The BOREAS data were filtered for profiles which reached the iteration limit during the retrieval. Grey areas indicate clouds within the specified time period.

In the scatter plot, BOREAS error bars were calculated by integrating the total error (Eq. 17) vertically. On the first 2 cloud-free days, the temporal variability shows a similar pattern with a slight offset between the instruments, most likely introduced by an underestimation of the AOT by BOREAS. Due to the limited sensitivity of MAX-DOAS profiling for higher altitudes (see Fig. 9), elevated aerosols do not contribute to BOREAS AOT but can be measured by direct Sun instruments like AERONET Sun photometers.

In the evening, the results of both instruments vary more. Different reasons for this finding are possible. First, a developing planetary boundary layer (PBL) might exceed the vertical extension where BOREAS has sufficient sensitivity. Second, when pointing towards the Sun, saturation of the CCD becomes a problem, leading to low integration times, which decrease SNR and fitting quality. In addition, RTM calculations might introduce uncertainties at high SZA, when the aerosol load is high and the light path is strongly increased. Furthermore, the aerosol phase function used leads to uncertainties, as the forward scattering peak might be underestimated by the Henyey–Greenstein parameterization. The other days show a higher variability due to an increase in cloudy scenes. Note that, especially in the morning and around noon, clouds may influence the instruments in various ways because of their different azimuthal viewing directions. The BOREAS profile retrieval cannot be considered reliable when one or several elevation angles point at clouds during the measurements, as intensity and light path vary in unpredictable ways. This might also lead to large iteration numbers or to no convergence at all. On 15 September, the temporal patterns of the two instruments differ strongly. Although the extinction values around noon agree well, AERONET's AOT decrease before and the increase afterwards cannot be found this strongly by BOREAS. Especially in the evening, both curves deviate, indicating highly variable atmospheric conditions, which might also be introduced by a change in wind direction (see Fig. 12). The aerosol load on the last 2 days is much lower because of rainfall in the time between 16 and 22 September. In the morning of 23 September, several profiles were discarded due to exceeded iteration limits.

The correlation coefficient of 0.83 is high and shows the general good agreement on 3 of 5 days. The regression line indicates the above-mentioned small underestimation of the AOT by BOREAS.

Figure 14 shows the comparison of near-surface aerosol values. Extinction coefficients from BOREAS and the ceilometer are depicted together with PM10 concentrations of the NAQMN in situ instrument. Note that the y axis for PM10 concentrations was chosen to match the results of BOREAS on 14 September. The NAQMN in situ sampler and BOREAS show a good temporal agreement within the first 3 days. Only in the morning hours of the 13 and 15 September, do data from the two instruments show a larger spread. This deviation is most likely due to the insufficient vertical resolution of MAX-DOAS profile retrievals when the PBL has not yet evolved and near-surface aerosol loads are dominant (see Figs. 9 and 15 for further details). On 15 September, early morning clouds might also have an impact on the deviation. Again, a larger variability for BOREAS results can be found for the last 2 days, when the retrieval was influenced by broken clouds. The in situ instruments show a smoother daily variation here. The ceilometer bottom extinction shows a good agreement with BOREAS and NAQMN on the first day and an underestimation on the second day. On 15 September, bottom values are influenced by thick clouds which might interfere with the backscattering signal of altitude ranges below the clouds (see Fig. 16). The day 23 September shows more variability than the first 2 days but this can be found for the other instruments as well.

The correlation of BOREAS bottom extinctions with data from both validation instruments is high, indicating good agreement on cloud-free days. The correlation with ceilometer near-surface values is high as well but the regression line shows a general underestimation of ceilometer near-surface values in comparison to BOREAS.

Figure 15 shows the aerosol layer height found by the ceilometer for all days. Especially during the first 3 days, an upward-extending PBL can be identified from noon to the late afternoon. Only on 15 September, does a more or less stable residual PBL seems to exist in the morning hours. The other days show high variability in the morning with layer heights from 200 to 600 m with more or less individual high signals, which might be produced by smaller clouds at higher altitudes: 24 September shows a stable layer around 200 m and a diffuse developing PBL in the afternoon.

Figure 15Ceilometer aerosol layer height within the planetary boundary layer.

The underlying extinction coefficient profiles for the above-discussed bottom extinction and AOT values can be seen in Fig. 16. In the top row, temporal and vertically averaged ceilometer profiles are depicted. In the middle row, these data were smoothed to the MAX-DOAS vertical resolution by the application of BOREAS averaging kernels (${\mathbit{x}}_{\text{new}}={\mathbit{x}}_{\text{apri}}+\mathbf{A}\left({\mathbit{x}}_{\text{ceilo}}-{\mathbit{x}}_{\text{apri}}$); see ). The first day shows a good agreement between BOREAS and ceilometer near-surface extinction with a small offset between the curves. In the morning, both instruments find the aerosol load mainly located in the lowermost layers. Beginning around noon, the PBL starts developing with a maximum height of 2 km in the afternoon found by the ceilometer. BOREAS cannot resolve this increasing PBL as well as the averaged ceilometer but the AK smoothed data indicate that the reason might be limited sensitivity to the top boundary of the PBL.

On 15 September, the averaged ceilometer data show high and thick clouds in the morning and a rising PBL in the afternoon. In the AK-smoothed data, these clouds cannot be identified any more. BOREAS introduces elevated aerosol layers which can be understood as a retrieval artefact due to these cloudy scenes. In the afternoon, BOREAS finds an upward expansion in the PBL similar to the ceilometer, with the exception of the last profile, which was influenced by the telescope pointing towards the Sun. BOREAS extinction values are smaller in the PBL, which is a consequence of the already explained underestimation of BOREAS AOT to AERONETs AOT, which was used for the backscatter signal scaling.

Figure 16Comparison of aerosol extinction coefficient profiles from BOREAS and ceilometer for 13 September 2016 (a) and 15 September 2016 (b).

Instrumentation for the NO2 validation

Several instruments were used for the validation of BOREAS nitrogen dioxide profiling results. The VCD results were validated with the help of a Pandora instrument (no. 128) operated by LuftBlick and a NO2 lidar from RIVM. In addition, three NO2 sondes launched on 15 September by KNMI were also used.

The Pandora instrument was used with its direct Sun capability to retrieve total nitrogen dioxide columns on hourly time steps (see Fig. 17). Since the measurement time did not match to BOREAS scans, Pandora values were resampled. The stratospheric column was subtracted based on the approach of by using OMI stratospheric columns during the Cabauw overpass (L2 OVP, ). The data were quality filtered and the errors were provided by LuftBlick and are based on .

The NO2 lidar did not measure as frequently as Pandora and MAX-DOAS instruments but performed several measurements on 15 September. One lidar scan took approximately 30 min with the temporal midpoints of the scans on a non-regular temporal grid which necessitated resampling. The scan was done on different elevation angles which enabled the retrieval of vertical profiles with altitude points up to 2.5 km. For the VCD calculation, these profiles were vertically integrated. Errors were provided by the RIVM team and Gaussian error propagation was applied for the calculation of VCD errors. In addition to the remote-sensing instrument, NO2 sondes were launched on 15 September around 05:15, 08:04 and 10:25 UTC. The sondes provide NO2 VMR as well as pressure, temperature and altitude data for the conversion to concentrations, which were vertically integrated up to 4 km.

Near-surface concentration validation was done with the help of in situ (NAQMN, ICAD, CAPS) and remote-sensing instruments (lidar and LP DOAS). The in situ samplers NAQMN, CAPS and ICAD were operated by RIVM, the Royal Belgian Institute for Space Aeronomy (BIRA-IASB) and the Institute of Environmental Physics in Heidelberg (IUPH) respectively. In addition to the aerosol bottom concentration, NAQMN also provided NO2 concentrations (Teledyne API 200E; see Sect. 4.3.1 for details on data handling). IUPH and BIRA-IASB created a common NO2 in situ data set from IUPHs ICAD (Iterative CAvity DOAS, ) and BIRAs CAPS (Environment SA, AS32M-CAPS, Cavity Attenuated Phase Shift, ). This joint data set filled gaps in the individual measurement series and fixed a scaling issue for CAPS. BIRA-IASB operated two different CAPS instruments, which were installed at the 27 and 200 m levels of the Cabauw measurement tower. The joint ICAD/CAPS data set as well as the 200 m CAPS data were averaged over 15 min time steps. For both instruments (ICAD/CAPS) an error of 1 ppb was assumed.

Figure 17(a) Time series of Pandora, lidar and BOREAS NO2 VCD. Blue diamonds show the original Pandora measurements and green squares show the lidar data points. Small dots with connecting lines depict the resampled data sets of Pandora and lidar. BOREAS is shown as red circles. Grey areas indicate clouds. The plot includes orange triangles as a representation of the integrated NO2 sondes measurements with the ascent and descent separated into different triangles with the edge at the top or bottom respectively. (b) Scatter plot of the data sets including parameters of the orthogonal regression and Pearson's correlation coefficients.

Figure 18(a) Time series of in situ and remote-sensing NO2 near-surface concentrations. Triangles show in situ instruments for NAQMN (bright blue, triangles pointing upwards) and for ICAD/CAPS (dark blue, triangles pointing downwards). In addition, on 13 September 2016, CAPS at the 200 m level is depicted as green triangles with the edge up and the mean values of both CAPS with the edge to the side (yellow). The NO2 lidar is plotted as green squares and the LP DOAS as different-sized circles in shades of magenta (lowest altitude as smallest circle). BOREAS is shown as large red circles. (b) Scatter plot of the data sets including parameters of the orthogonal regression and Pearson's correlation coefficients.

In addition to the in situ instruments, a long-path DOAS (LP DOAS) provided by IUPH was used for the validation. Four reflectors attached at different altitude levels of the measurement tower (12.7, 47, 107 and 207 m) provided unique and well-defined light paths which enabled the calculation of concentration on several altitudes. The instruments' elevation measurements led to profiles every 30 min. Errors were calculated by the IUPH team.

Nitrogen dioxide profiles were provided only by the NO2 lidar and sonde launches. The LP DOAS did not cover enough altitude levels for a comparison. The conversion from trace gas VMR to concentrations for all data sets was done with the meteorological data from the CESAR observatory (CESAR2018).

Comparison of NO2 retrieval parameters

Figure 17 depicts the comparison of VCD for different instruments. The agreement with Pandora is good on all days with larger deviations in the morning and on 15 September in general with the exception around noon. Deviations in the morning might be (similarly to the AERONET comparison) due to spatial differences in the NO2 distribution, when Pandora points to the east and MAX-DOAS to the west. This argument is supported by the lidar measurement on 15 September that agreed well with BOREAS and shared the same azimuthal viewing direction. In the afternoon of that day, BOREAS VCD increased faster than Pandora and lidar, with a strong overestimation for the last profile at 16:00 when the Sun was low. Here, we can exclude horizontal gradients as a reason for the differences, because of the same viewing direction of lidar and MAX-DOAS. On the same day, NO2 sondes proved that the BOREAS VCD was correct only during the ascent (triangle with edge at the top) of the second launch. The matching descent (triangle with edge at the bottom) for the first flight was approximately 38 km in the north-westerly direction between Amsterdam and Rotterdam and agrees more with Pandora columns, indicating differences due to azimuthal viewing directions. The strong differences in VCD for ascent and descent of the first and second sonde launches show high temporal and spatial variability of the NO2 concentration, which is supported by all remote-sensing instruments.

The correlation with Pandora is slightly better than with lidar but the regression parameter shows that Pandora VCDs are higher than those from BOREAS, whereas the lidar regression is closer to 1.

In Fig. 18, near-surface NO2 concentrations are depicted. Both in situ data sets (NAQMN and ICAD/CAPS) agree well with each other and show larger differences only in the morning of 14 September. The LP DOAS profiles at 12.7 and 47 m show similar concentrations to the in situ instruments. Differences in the morning of the first 2 days between the lower and the higher LP DOAS values indicate a strong vertical inhomogeneity, which was also found for the aerosols (see Sect. 4.3.1).

BOREAS surface concentrations agree very well with all data sets except during the morning hours. On 13 September, in addition to the ICAD/CAPS data at the 27 m level (blue), the 200 m points are shown (green). The anti-correlated behaviour in the morning hours confirms the strong inhomogeneity and proves that nitrogen dioxide was mainly concentrated close to the surface at that time. The mean values of both CAPS instruments (yellow line) show a better agreement with BOREAS than the 27 and 200 m data.

It seems that BOREAS cannot fully resolve this thin near-surface layer and retrieves rather smooth profiles instead of sharp concentration peaks (see also the discussion about vertical sensitivity in Sect. 4.2). This is again in agreement with the LP DOAS results on 24 September, where the 200 m level LP DOAS finds similar concentrations to BOREAS. Note that the NO2 lidar has a finer resolution within the lowest 100 m, which might be the reason it depicts similar concentration numbers to the other instruments in the morning hours of 15 September. As a rough estimation of the thickness of the individual thin layers in the morning, we divided BOREAS VCD by the ICAD/CAPS concentration and found that the layer height lies around 200 m (06:00–08:00), 300 m (09:00) and 400 m (10:00) on 13 September. The mean AK FWHM for the surface layer on that day is 214 m, indicating that the lowermost layer can be resolved only from around 09:00 UTC where the curves approach each other. A good agreement is found at 10:00 UTC when the concentration is focused in a layer twice as thick as BOREAS surface resolution.

The correlations are high for all instruments with the highest value for NAQMN and the NO2 lidar. The in situ instruments show similar correlations, but a slight underestimation of BOREAS can be found with slopes between 1.22 and 1.44.

Figure 19 depicts NO2 profiles for BOREAS, lidar and sondes on 15 September. Similarly to the aerosol profile comparison, averaging kernels were applied to lidar and sondes measurements. The vertical profiles of the unsmoothed lidar show the previously discussed high vertical inhomogeneity. We can clearly identify a high concentration close to the surface and another elevated NO2 layer with altitudes varying between 100 and 500 m. The unsmoothed sondes agree well with the lidar measurements only at the ascent of the second launch. After the application of averaging kernels, the two distinct layers with high concentrations are smoothed to one layer with NO2 concentrated at the surface, again indicating a lower vertical sensitivity of MAX-DOAS profiling. In addition, smoothed and unsmoothed lidar find larger concentrations for altitudes over 500 m before 08:00 UTC, which was not found by BOREAS and the sonde profiles.

Figure 19Comparison of NO2 concentration profiles from BOREAS, lidar and sonde measurements for 15 September 2016.

5 Conclusions

In this study, we introduced BOREAS, a new profiling algorithm for MAX-DOAS measurements. BOREAS retrieves aerosol extinction coefficient profiles around O4 absorption bands from the oxygen dimers differential slant column densities. In contrast to existing inversion algorithms, which are based on perturbation theory, BOREAS directly minimizes the difference between modelled and measured optical depth by varying aerosol extinction coefficient profiles in an iterative Tikhonov regularization scheme. In addition, trace gas concentration profiles are retrieved via optimal estimation using the aerosol properties determined in the first step. For both retrieval parts, we introduced a priori pre-scaling as a simple method for improving results under variable atmospheric conditions.

The retrieval performance was demonstrated on the example of synthetic data calculated with SCIATRAN and with real measurements taken during the CINDI-2 campaign in Cabauw 2016. The synthetic scenarios were chosen to work as a stress test for BOREAS. One fixed a priori profile was used for the retrieval of different large aerosol loads and NO2 concentrations. The results are good when the scenario is relatively close to the a priori profile but fails for strong loads and concentrations. In a comprehensive variation of parameters we showed that, in non-ideal situations, either the shape/near-surface value or the integrated quantity can be optimized. The information content of a MAX-DOAS scan with degrees of freedom between ≈1 and ≈3.5 is not enough for a perfect retrieval. Furthermore, the frequently utilized assumption of using the trace gas errors from DOAS fits only as measurement variance was found to be insufficient since the regularization ratio might be wrongly weighted. We suggest a manual change in measurement errors or an iterative approach for finding the best ratio.

In the second part of this study, BOREAS retrieval results from measurements of the IUPB MAX-DOAS instrument during the CINDI-2 campaign were validated with ancillary measurements. The agreement for all parameters was good with correlations equal or higher than 0.75. Systematic offsets or deviations depending on geometry, atmospheric condition and investigated air mass indicate limitations due to the specific measurement characteristics, which were also found in earlier MAX-DOAS profiling studies. The correlation with AERONET AOT is 0.83 with an underestimation by BOREAS, leading to a regression slope of 1.30. The bottom value correlation is 0.75 for the ceilometer near-surface extinction and 0.81 for NAQMN in situ measurements. The total column of NO2 shows a good agreement with Pandora and lidar with correlations of 0.86 and 0.79 respectively. In contrast to the underestimation of BOREAS integrated aerosol, Pandora nitrogen dioxide columns are overestimated, especially in the morning hours. The correlations for near-surface NO2 concentrations are high with values larger than 0.84. Stronger deviations between BOREAS and in situ instruments were found in the morning. This was explained by the limited vertical resolution of BOREAS, while in the morning the highest concentrations are found close to the surface.

As a conclusion, BOREAS profiling capabilities are strong, which was proven by high correlations with all validating instruments. Discrepancies were found due to different azimuthal viewing directions of the instruments and the limited vertical resolution of MAX-DOAS profiling when aerosol load or concentration is close to the surface in layers thinner than the BOREAS vertical resolution. A comparison of BOREAS results to the performance of other MAX-DOAS profiling algorithms will be discussed in two upcoming papers .

Data availability
Data availability.

The profiling data are available upon request (contact persons are Tim Bösch and Andreas Richter). The individual data sets of ancillary instruments are available from the individual PIs upon request.

Appendix A: Additional plots and retrieval of box profiles

A1 Additional aerosol scenarios and plots

Figure A1Box aerosol profiles for the synthetic sensitivity study. Also shown is the a priori profile for the aerosol retrieval (dashed line).

The retrieval results for the box scenarios in Fig. A1 are shown in Fig. A2 for different Tikhonov parameters and SNR = 3000. A priori pre-scaling leads to a clear improvement only for scenario B3 since the bottom extinction is reached. For B2 and B3, the resulting profile shapes change slightly, which indicates the above-mentioned lack of improvements through pre-scaling, when the a priori profile shape differs strongly from the true atmosphere. Nevertheless, since exponential profiles are improved a lot and box profiles show no degradation, we highly recommend the usage of better and more flexible a priori profiles.

Figure A2Retrieval results with a fixed (a) and a pre-scaled a priori profile (b) with varying Tikhonov parameters γ for SNR = 3000. Small γ values mean less smoothing of the resulting profiles.

Figure A3Variation of SNR and Tikhonov parameter with the relative differences in AOT and bottom extinction compared to the true value, colour coded for scenario B1 with a priori pre-scaling.

Figure A4Same as Fig. A3 but for scenario B2.

Figure A5Same as Fig. A3 but for scenario B3.

Figures A3A5 show the SNR and Tikhonov variation for the three aerosol box profiles. One can see that for B2 the smallest γ values lead to the best bottom value, whereas for scenario B1 these values would result in an overestimation. Also the optimal SNR γ range for the AOT differs strongly for all box profiles, which again shows that regularization should be considered a flexible choice of parameters, depending on the specific atmospheric conditions.

Figures A6 and A7 show the SNR and Tikhonov variation for the exponential profiles E1 and E2. Note that for E1 all parameter combinations lead to good profiling results because the a priori profile is close to the true atmosphere. In comparison to E3, E2 shows a better adaptation of AOT with a coincidently similar performance for the bottom extinction.

Figure A6Same as Fig. 5 but for scenario E1.

Figure A7Same as Fig. 5 but for scenario E2.

A2 Additional NO2 scenarios and plots

Figure A8 shows the box NO2 scenarios, which are retrieved for different g values in Fig. A9. The standard regularization ratio for g=1 is again found to be oscillating without reaching the proper bottom concentrations for E2 and E3. Higher g factors show an improvement for both the fixed a priori and pre-scaled initial guess.

Figure A8Box NO2 profiles for the synthetic sensitivity study. Also shown is the a priori profile for the aerosol retrieval (dashed line).

Figure A9Retrieval results with a fixed (a) and a pre-scaled a priori profile (b) with g factors. Small g values mean less measurement weighting for the resulting profiles.

Competing interests
Competing interests.

The authors declare that they have no conflict of interest.

Acknowledgements
Acknowledgements.

This study was supported by the University of Bremen and by the FP7 Project Quality Assurance for Essential Climate Variables (QA4ECV), no. 607405. Participation of the University of Bremen in the CINDI-2 campaign received funding from ESA via RFQ/3-14594/16/I-SBo. The authors would like to thank the CINDI-2 team for excellent support on site. Furthermore, we thank François Hendrick and Marc Allaart for the provision of mean pressure and temperature profiles used within the BOREAS retrieval of CINDI2 data.

The article processing charges for this open-access
publication were covered by the University of Bremen.

Edited by: Michel Van Roozendael
Reviewed by: three anonymous referees

References

Barret, B., De Mazière, M., and Demoulin, P.: Retrieval and Characterization of Ozone Profiles from Solar Infrared Spectra at the Jungfraujoch, J. Geophys. Res.-Atmos., 107, 4788, https://doi.org/10.1029/2001JD001298, 2002. a, b

Bobrowski, N., Hönninger, G., Galle, B., and Platt, U.: Detection of Bromine Monoxide in a Volcanic Plume, Nature, 423, 273–276, https://doi.org/10.1038/nature01625, 2003. a

Brinksma, E. J., Pinardi, G., Volten, H., Braak, R., Richter, A., Schönhardt, A., van Roozendael, M., Fayt, C., Hermans, C., Dirksen, R. J., Vlemmix, T., Berkhout, A. J. C., Swart, D. P. J., Oetjen, H., Wittrock, F., Wagner, T., Ibrahim, O. W., de Leeuw, G., Moerman, M., Curier, R. L., Celarier, E. A., Cede, A., Knap, W. H., Veefkind, J. P., Eskes, H. J., Allaart, M., Rothe, R., Piters, A. J. M., and Levelt, P. F.: The 2005 and 2006 DANDELIONS NO2 and Aerosol Intercomparison Campaigns, J. Geophys. Res., 113, D16S46, https://doi.org/10.1029/2007JD008808, 2008. a

CESAR: Cesar Observatory, available at: http://www.cesar-observatory.nl/index.php, last access: 11 December 2018. a, b

Clémer, K., Van Roozendael, M., Fayt, C., Hendrick, F., Hermans, C., Pinardi, G., Spurr, R., Wang, P., and De Mazière, M.: Multiple wavelength retrieval of tropospheric aerosol optical properties from MAXDOAS measurements in Beijing, Atmos. Meas. Tech., 3, 863–878, https://doi.org/10.5194/amt-3-863-2010, 2010. a, b

Dubovik, O., Smirnov, A., Holben, B. N., King, M. D., Kaufman, Y. J., Eck, T. F., and Slutsker, I.: Accuracy Assessments of Aerosol Optical Properties Retrieved from Aerosol Robotic Network (AERONET) Sun and Sky Radiance Measurements, J. Geophys. Res.-Atmos., 105, 9791–9806, https://doi.org/10.1029/2000JD900040, 2000. a

Engl, H. W. and Grever, W.: Using the L–Curve for Determining Optimal Regularization Parameters, Numer. Math., 69, 25–31, https://doi.org/10.1007/s002110050078, 1994. a

Eriksson, P.: Analysis and Comparison of Two Linear Regularization Methods for Passive Atmospheric Observations, J. Geophys. Res.-Atmos., 105, 18157–18167, https://doi.org/10.1029/2000JD900172, 2000. a

Frieß, U., Monks, P. S., Remedios, J. J., Rozanov, A., Sinreich, R., Wagner, T., and Platt, U.: MAX-DOAS O4 Measurements: A New Technique to Derive Information on Atmospheric Aerosols: 2. Modeling Studies, J. Geophys. Res., 111, D14203m https://doi.org/10.1029/2005JD006618, 2006. a, b, c, d

Frieß, U., Klein Baltink, H., Beirle, S., Clémer, K., Hendrick, F., Henzing, B., Irie, H., de Leeuw, G., Li, A., Moerman, M. M., van Roozendael, M., Shaiganfar, R., Wagner, T., Wang, Y., Xie, P., Yilmaz, S., and Zieger, P.: Intercomparison of aerosol extinction profiles retrieved from MAX-DOAS measurements, Atmos. Meas. Tech., 9, 3205–3222, https://doi.org/10.5194/amt-9-3205-2016, 2016. a, b, c

Frieß, U., Beirle, S., Alvarado Bonilla, L., Bösch, T., Friedrich, M. M., Hendrick, F., Piters, A., Richter, A., van Roozendael, M., Rozanov, V. V., Spinei, E., Tirpitz, J.-L., Vlemmix, T., Wagner, T., and Wang, Y.: Intercomparison of MAX-DOAS Vertical Profile Retrieval Algorithms: Studies using Synthetic Data, Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2018-423, in review, 2018. a, b

Hansen, P.: Analysis of Discrete Ill-Posed Problems by Means of the L-Curve, SIAM Review, 34, 561–580, https://doi.org/10.1137/1034115, 1992. a

Hartl, A. and Wenig, M. O.: Regularisation model study for the least-squares retrieval of aerosol extinction time series from UV/VIS MAX-DOAS observations for a ground layer profile parameterisation, Atmos. Meas. Tech., 6, 1959–1980, https://doi.org/10.5194/amt-6-1959-2013, 2013. a, b, c

Hendrick, F., Barret, B., Van Roozendael, M., Boesch, H., Butz, A., De Mazière, M., Goutail, F., Hermans, C., Lambert, J.-C., Pfeilsticker, K., and Pommereau, J.-P.: Retrieval of nitrogen dioxide stratospheric profiles from ground-based zenith-sky UV-visible observations: validation of the technique through correlative comparisons, Atmos. Chem. Phys., 4, 2091–2106, https://doi.org/10.5194/acp-4-2091-2004, 2004. a

Hendrick, F., Müller, J.-F., Clémer, K., Wang, P., De Mazière, M., Fayt, C., Gielen, C., Hermans, C., Ma, J. Z., Pinardi, G., Stavrakou, T., Vlemmix, T., and Van Roozendael, M.: Four years of ground-based MAX-DOAS observations of HONO and NO2 in the Beijing area, Atmos. Chem. Phys., 14, 765–781, https://doi.org/10.5194/acp-14-765-2014, 2014. a, b

Henyey, L. G. and Greenstein, J. L.: Diffuse Radiation in the Galaxy, Astrophys. J., 93, 70–83, https://doi.org/10.1086/144246, 1941. a

Herman, J., Cede, A., Spinei, E., Mount, G., Tzortziou, M., and Abuhassan, N.: NO2 Column Amounts from Ground-Based Pandora and MFDOAS Spectrometers Using the Direct-Sun DOAS Technique: Intercomparisons and Application to OMI Validation, J. Geophys. Res.-Atmos., 114, D13307, https://doi.org/10.1029/2009JD011848, 2009. a, b

Holben, B. N., Eck, T. F., Slutsker, I., Tanré, D., Buis, J. P., Setzer, A., Vermote, E., Reagan, J. A., Kaufman, Y. J., Nakajima, T., Lavenu, F., Jankowiak, I., and Smirnov, A.: AERONET – A Federated Instrument Network and Data Archive for Aerosol Characterization, Remote Sens. Environ., 66, 1–16, https://doi.org/10.1016/S0034-4257(98)00031-5, 1998. a

Hönninger, G. and Platt, U.: Observations of BrO and Its Vertical Distribution during Surface Ozone Depletion at Alert, Atmos. Environ., 36, 2481–2489, https://doi.org/10.1016/S1352-2310(02)00104-8, 2002. a

Hönninger, G., von Friedeburg, C., and Platt, U.: Multi axis differential optical absorption spectroscopy (MAX-DOAS), Atmos. Chem. Phys., 4, 231–254, https://doi.org/10.5194/acp-4-231-2004, 2004. a

Irie, H., Takashima, H., Kanaya, Y., Boersma, K. F., Gast, L., Wittrock, F., Brunner, D., Zhou, Y., and Van Roozendael, M.: Eight-component retrievals from ground-based MAX-DOAS observations, Atmos. Meas. Tech., 4, 1027–1044, https://doi.org/10.5194/amt-4-1027-2011, 2011. a

Kebabian, P. L., Wood, E. C., Herndon, S. C., and Freedman, A.: A Practical Alternative to Chemiluminescence-Based Detection of Nitrogen Dioxide: Cavity Attenuated Phase Shift Spectroscopy, Environ. Sci. Technol., 42, 6040–6045, https://doi.org/10.1021/es703204j, 2008. a

Knepp, T., Pippin, M., Crawford, J., Chen, G., Szykman, J., Long, R., Cowen, L., Cede, A., Abuhassan, N., Herman, J., Delgado, R., Compton, J., Berkoff, T., Fishman, J., Martins, D., Stauffer, R., Thompson, A. M., Weinheimer, A., Knapp, D., Montzka, D., Lenschow, D., and Neil, D.: Estimating Surface NO2 and SO2 Mixing Ratios from Fast-Response Total Column Observations and Potential Application to Geostationary Missions, J. Atmos. Chem., 72, 261–286, https://doi.org/10.1007/s10874-013-9257-6, 2015. a

Lee, H., Irie, H., Kim, Y. J., Noh, Y., Lee, C., Kim, Y., and Chun, K. J.: Retrieval of Aerosol Extinction in the Lower Troposphere Based on UV MAX-DOAS Measurements, Aerosol Sci. Technol., 43, 502–509, https://doi.org/10.1080/02786820902769691, 2009. a

Lelieveld, J., Berresheim, H., Borrmann, S., Crutzen, P. J., Dentener, F. J., Fischer, H., Feichter, J., Flatau, P. J., Heland, J., Holzinger, R., Korrmann, R., Lawrence, M. G., Levin, Z., Markowicz, K. M., Mihalopoulos, N., Minikin, A., Ramanathan, V., de Reus, M., Roelofs, G. J., Scheeren, H. A., Sciare, J., Schlager, H., Schultz, M., Siegmund, P., Steil, B., Stephanou, E. G., Stier, P., Traub, M., Warneke, C., Williams, J., and Ziereis, H.: Global Air Pollution Crossroads over the Mediterranean, Science, 298, 794–799, https://doi.org/10.1126/science.1075457, 2002. a

Leser H., Hönninger G., and Platt U.: MAX-DOAS Measurements of BrO and NO2 in the Marine Boundary Layer, Geophys. Res. Lett., 30, 1537, https://doi.org/10.1029/2002GL015811, 2003. a, b

Li, X., Brauers, T., Shao, M., Garland, R. M., Wagner, T., Deutschmann, T., and Wahner, A.: MAX-DOAS measurements in southern China: retrieval of aerosol extinctions and validation using ground-based in-situ data, Atmos. Chem. Phys., 10, 2079–2089, https://doi.org/10.5194/acp-10-2079-2010, 2010. a

NASA: US Standard Atmosphere, Office, U.S. Government Printing, Washington, DC, 1976. a, b

NASA: L2 OVP, Aerosol Optical Thickness & Single Scattering Albedo, Station overpass data, available at: https://avdc.gsfc.nasa.gov/index.php?site=2045907950, last access: 30 January 2018. a

Pinardi, G., Van Roozendael, M., Abuhassan, N., Adams, C., Cede, A., Clémer, K., Fayt, C., Frieß, U., Gil, M., Herman, J., Hermans, C., Hendrick, F., Irie, H., Merlaud, A., Navarro Comas, M., Peters, E., Piters, A. J. M., Puentedura, O., Richter, A., Schönhardt, A., Shaiganfar, R., Spinei, E., Strong, K., Takashima, H., Vrekoussis, M., Wagner, T., Wittrock, F., and Yilmaz, S.: MAX-DOAS formaldehyde slant column measurements during CINDI: intercomparison and analysis improvement, Atmos. Meas. Tech., 6, 167–185, https://doi.org/10.5194/amt-6-167-2013, 2013. a, b

Piters, A. J. M., Boersma, K. F., Kroon, M., Hains, J. C., Van Roozendael, M., Wittrock, F., Abuhassan, N., Adams, C., Akrami, M., Allaart, M. A. F., Apituley, A., Beirle, S., Bergwerff, J. B., Berkhout, A. J. C., Brunner, D., Cede, A., Chong, J., Clémer, K., Fayt, C., Frieß, U., Gast, L. F. L., Gil-Ojeda, M., Goutail, F., Graves, R., Griesfeller, A., Großmann, K., Hemerijckx, G., Hendrick, F., Henzing, B., Herman, J., Hermans, C., Hoexum, M., van der Hoff, G. R., Irie, H., Johnston, P. V., Kanaya, Y., Kim, Y. J., Klein Baltink, H., Kreher, K., de Leeuw, G., Leigh, R., Merlaud, A., Moerman, M. M., Monks, P. S., Mount, G. H., Navarro-Comas, M., Oetjen, H., Pazmino, A., Perez-Camacho, M., Peters, E., du Piesanie, A., Pinardi, G., Puentedura, O., Richter, A., Roscoe, H. K., Schönhardt, A., Schwarzenbach, B., Shaiganfar, R., Sluis, W., Spinei, E., Stolk, A. P., Strong, K., Swart, D. P. J., Takashima, H., Vlemmix, T., Vrekoussis, M., Wagner, T., Whyte, C., Wilson, K. M., Yela, M., Yilmaz, S., Zieger, P., and Zhou, Y.: The Cabauw Intercomparison campaign for Nitrogen Dioxide measuring Instruments (CINDI): design, execution, and early results, Atmos. Meas. Tech., 5, 457–485, https://doi.org/10.5194/amt-5-457-2012, 2012. a

Platt, U. and Stutz, J.: Differential Optical Absorption Spectroscopy: Principles and Applications, Physics of Earth and space environments, Springer, Berlin, 2008. a

Pöhler, D., Vogel, L., Frieß, U., and Platt, U.: Observation of Halogen Species in the Amundsen Gulf, Arctic, by Active Long-Path Differential Optical Absorption Spectroscopy, P. Natl. Acad. Sci USA, 107, 6582–6587, https://doi.org/10.1073/pnas.0912231107, 2010. a

Pöhler, D., Lutz, E., Horbanski, M., Lampel, J., and Platt, U.: High Precision NO2 and NO Measurements with the ICAD Instrument during S-b-s Campaign Hohenpeißenberg 2016, vol. 19, p. 14336, 2017. a

Rodgers, C. D.: Inverse Methods for Atmospheric Sounding: Theory and Practice, in: Series on atmospheric oceanic and planetary physics, Vol. 2, World Scientific, Singapore, reprinted edn., oCLC: 254137862, 2004. a, b, c, d, e, f

Rodgers, C. D. and Connor, B. J.: Intercomparison of Remote Sounding Instruments, J. Geophys. Res.-Atmos., 108, 4116, https://doi.org/10.1029/2002JD002299, 2003. a

Rozanov, V. V. and Rozanov, A. V.: Differential optical absorption spectroscopy (DOAS) and air mass factor concept for a multiply scattering vertically inhomogeneous medium: theoretical consideration, Atmos. Meas. Tech., 3, 751–780, https://doi.org/10.5194/amt-3-751-2010, 2010. a

Rozanov, V. V., Rozanov, A. V., Kokhanovsky, A. A., and Burrows, J. P.: Radiative Transfer through Terrestrial Atmosphere and Ocean: Software Package SCIATRAN, J. Quant. Spectrosc. Ra., 133, 13–71, https://doi.org/10.1016/j.jqsrt.2013.07.004, 2014. a

Sayer, A. M., Hsu, N. C., Bettenhausen, C., and Jeong, M.-J.: Validation and Uncertainty Estimates for MODIS Collection 6 “Deep Blue” Aerosol Data, J. Geophys. Res.-Atmos., 118, 7864–7872, https://doi.org/10.1002/jgrd.50600, 2013. a

Sinreich, R., Frieß, U., Wagner, T., and Platt, U.: Multi Axis Differential Optical Absorption Spectroscopy (MAX-DOAS) of Gas and Aerosol Distributions, Faraday Discuss., 130, 153–164, https://doi.org/10.1039/b419274p, 2005. a

Sinreich, R., Merten, A., Molina, L., and Volkamer, R.: Parameterizing radiative transfer to convert MAX-DOAS dSCDs into near-surface box-averaged mixing ratios, Atmos. Meas. Tech., 6, 1521–1532, https://doi.org/10.5194/amt-6-1521-2013, 2013. a

Stocker, T. (Ed.): Climate Change 2013: The Physical Science Basis: Working Group I Contribution to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, Cambridge University Press, New York, 2014. a

Tirpitz, J.-L., Frieß, U., Hendrick, F., et al.: Intercomparison of MAX-DOAS Vertical Profile Retrieval Algorithms: Studies on Field Data from the CINDI-2 Campaign, in preparation, 2019. a, b

van Roozendael, M., Fayt, C., Post, P., Hermans, C., and Lambert, J. C.: Retrieval of Tropospheric BrO and NO2 from UV-Visible Observations, in: Sounding the Troposphere from Space, Springer, Berlin, Heidelberg, 155–165, https://doi.org/10.1007/978-3-642-18875-6_19, 2004. a

Vlemmix, T., Piters, A. J. M., Berkhout, A. J. C., Gast, L. F. L., Wang, P., and Levelt, P. F.: Ability of the MAX-DOAS method to derive profile information for NO2: can the boundary layer and free troposphere be separated?, Atmos. Meas. Tech., 4, 2659–2684, https://doi.org/10.5194/amt-4-2659-2011, 2011. a, b

Vlemmix, T., Hendrick, F., Pinardi, G., De Smedt, I., Fayt, C., Hermans, C., Piters, A., Wang, P., Levelt, P., and Van Roozendael, M.: MAX-DOAS observations of aerosols, formaldehyde and nitrogen dioxide in the Beijing area: comparison of two profile retrieval approaches, Atmos. Meas. Tech., 8, 941–963, https://doi.org/10.5194/amt-8-941-2015, 2015. a, b

Wagner, T., Dix, B., v. Friedeburg, C., Frieß, U., Sanghavi, S., Sinreich, R., and Platt, U.: MAX-DOAS O4 measurements: A new technique to derive information on atmospheric aerosols – Principles and information content, J. Geophys. Res.-Atmos., 109, https://doi.org/10.1029/2004JD004904, 2004.  a, b, c

Wagner, T., Beirle, S., Brauers, T., Deutschmann, T., Frieß, U., Hak, C., Halla, J. D., Heue, K. P., Junkermann, W., Li, X., Platt, U., and Pundt-Gruber, I.: Inversion of tropospheric profiles of aerosol extinction and HCHO and NO2 mixing ratios from MAX-DOAS observations in Milano during the summer of 2003 and comparison with independent data sets, Atmos. Meas. Tech., 4, 2685–2715, https://doi.org/10.5194/amt-4-2685-2011, 2011. a, b

Wang, Y., Lampel, J., Xie, P., Beirle, S., Li, A., Wu, D., and Wagner, T.: Ground-based MAX-DOAS observations of tropospheric aerosols, NO2, SO2 and HCHO in Wuxi, China, from 2011 to 2014, Atmos. Chem. Phys., 17, 2189–2215, https://doi.org/10.5194/acp-17-2189-2017, 2017. a, b

Wittrock, F.: The Retrieval of Oxygenated Volatile Organic Compounds by Remote Sensing Techniques, Dissertation, University of Bremen, Bremen, 2006. a, b

Wittrock, F., Oetjen, H., Richter, A., Fietkau, S., Medeke, T., Rozanov, A., and Burrows, J. P.: MAX-DOAS measurements of atmospheric trace gases in Ny-Ålesund –Radiative transfer studies and their application, Atmos. Chem. Phys., 4, 955–966, https://doi.org/10.5194/acp-4-955-2004, 2004. a, b, c

Zieger, P., Weingartner, E., Henzing, J., Moerman, M., de Leeuw, G., Mikkilä, J., Ehn, M., Petäjä, T., Clémer, K., van Roozendael, M., Yilmaz, S., Frieß, U., Irie, H., Wagner, T., Shaiganfar, R., Beirle, S., Apituley, A., Wilson, K., and Baltensperger, U.: Comparison of ambient aerosol extinction coefficients obtained from in-situ, MAX-DOAS and LIDAR measurements at Cabauw, Atmos. Chem. Phys., 11, 2603–2624, https://doi.org/10.5194/acp-11-2603-2011, 2011. a, b