2.1 TROPOMI aboard Sentinel-5 Precursor

2.2 OMI aboard EOS-Aura

2.3 Latitudinal range for uncertainty studies

To investigate the stability and uncertainties of the NO₂ SCD retrieval the “tropical latitude” (TL hereafter) range is defined as all scanlines that have their sub-satellite latitude point – corresponding approximately to the nadir-viewing detector rows – within a 30^∘ range that moves along with the seasons, in an attempt to filter out seasonality in the NO₂ columns: on 1 January the TL range covers [−30 ^∘ , 0 ^∘] for the sub-satellite latitude points, while half a year later it covers [0 , +30 ^∘]. The TL range is also used for the across-track “de-striping” of the SCDs discussed in Sect. 4.3. For TROPOMI (OMI) data the TL range contains about 475 (250) scanlines; after the along-track pixel size reduction in TROPOMI there are about 610 scanlines in the TL range.

Table 1Specifics for the NO₂ slant column retrieval of TROPOMI and OMI–QA4ECV. The reference spectra (second group of entries) have all been convolved with the row-dependent instrument spectral response function (ISRF or slit function). Last access dates for all websites mentioned in the table is 17 March 2020.

^a Specifics of the OMI–OMNO2A retrieval are mentioned in Sect. 3.4. ^b Offline (re)processing uses E₀ measured nearest in time to I, except for the period mid-October 2018 to mid-March 2019, when the most recent E₀ with regard to I was used due to an issue with the processor; the version-2 reprocessing will use the nearest E₀ for all orbits.

Download Print Version | Download XLSX

3.1 DOAS technique

The NO₂ SCD retrieval is performed using a DOAS technique (Platt, 1994; Platt and Stutz, 2008), which provides the amount of NO₂ along the effective light path, from sun through atmosphere to satellite. This technique attempts to model the reflectance spectrum R_meas(λ) observed by the satellite instrument:

with I(λ) the radiance at the top of the atmosphere, E₀(λ) the extraterrestrial solar irradiance measured by the same instrument and μ₀=cos (θ₀) the cosine of the solar zenith angle; given that the processing is limited to ground pixels measured at ${\mathit{\theta }}_{\mathrm{0}}\le {\mathrm{88}}^{\circ }$, the division by μ₀ in Eq. (1) will not cause problems. Note that both I and E₀ also depend on viewing geometry, but those arguments are left out for brevity.

The modelled reflectance, R_mod(λ), is determined from reference spectra of a number of species known to absorb in the wavelength window used for the SCD retrieval, as well as a correction for scattering and absorption by rotational Raman scattering (RRS), the so-called “Ring effect” (see Grainger and Ring, 1962; Chance and Spurr, 1997), while a polynomial $P\left(\mathit{\lambda }\right)=\sum a_m{\mathit{\lambda }}^m$ ($m=\mathrm{0},\mathrm{1},\mathrm{\dots },n_p$) is used to account for spectrally smooth structures resulting from molecular (single and multiple) scattering and absorption, aerosol scattering and absorption, and surface albedo effects.

The precise formulation of R_mod(λ) and the method used to minimise the difference between the modelled and measured reflectance differs slightly between the TROPOMI and OMI retrievals. Details of these DOAS approaches are listed in Table 1. (The difference in the degree of the DOAS polynomial is not relevant: n_p=4 and n_p=5 give practically the same results; for TROPOMI n_p=5 is chosen following the traditional setting in the OMNO2A processing (cf. Sect. 3.4) of OMI data.)

3.2 TROPOMI intensity fit retrieval

In the TROPOMI NO₂ processor (van Geffen et al., 2019) R_mod(λ) is formulated in an intensity fit (IF hereafter) approach:

with σ_k(λ) the absolute cross section and N_s, k the slant column amount of molecule $k=\mathrm{1},\mathrm{\dots },n_k$ taken into account in the fit: NO₂, ozone, water vapour, liquid water and the O₂−O₂ collision complex. The physical model accounts for inelastic Raman scattering of incoming sunlight by N₂ and O₂ molecules that leads to the filling-in of the Fraunhofer lines in the radiance spectrum, i.e. the Ring effect. In Eq. (2), C_ring is the Ring fit coefficient and I_ring(λ)∕E₀(λ) the sun-normalised synthetic Ring spectrum, with E₀(λ) is the measured irradiance. The term between parentheses in Eq. (2) describes both the contribution of the direct differential absorption (i.e. the 1), and the modification of these differential structures by inelastic scattering (the $+C_{\mathrm{ring}}{I}_{\mathrm{ring}}\left(\mathit{\lambda }\right)/E_{\mathrm{0}}\left(\mathit{\lambda }\right)$ term) to the reflectance spectrum.

The IF minimises the chi-squared merit function:

with n_λ the number of wavelengths (spectral pixels) in the fit window (405–465 nm) and ΔR_meas(λ_i) the uncertainty in the measured reflectance, which depends on the precision of the radiance and irradiance measurements as given in the level-1b product, i.e. on the signal-to-noise ratio (SNR) of the measurements. Radiance spectral pixels flagged in the level-1b data as bad or as suffering from saturation (Sect. 2.1.1) are filtered out before any further processing step.

In the final data product ground pixels are flagged when the slant column retrieval uncertainty ΔN_s>33 µmol m⁻² (2×10¹⁵ molec. cm⁻²). SCD error values this large occur rarely: usually <0.1 % of the pixels per orbit with original ground pixel sizes; for the smaller-size pixel orbits there are about 50 % more pixels with high SCD error values (based on one test day of data), taking into account that the SCD error itself increases with reduced pixel size. Note, however, that the ground pixel size reduction leads to about 28 % more ground pixels per orbit and thus a significant increase in the number of successfully retrieved ground pixels.

The magnitude of χ² is a measure of how good the fit is. Another measure of the goodness of the fit is the so-called root-mean-square (rms) error:

where the difference $R_{\mathrm{res}}\left(\mathit{\lambda }\right)=R_{\mathrm{meas}}\left(\mathit{\lambda }\right)-R_{\mathrm{mod}}\left(\mathit{\lambda }\right)$ is usually referred to as the residual of the fit.

In the TROPOMI processor χ² is minimised using an optimal estimation (OE; based on Rodgers, 2000) routine, with suitable a priori values of the fit parameters and a priori errors set very large, so as not to limit the solution of the fit (for example, the NO₂ SCD a priori error is set at $\mathrm{1.0}\times {\mathrm{10}}^{-\mathrm{2}}$ mol m${}^{-\mathrm{2}}=\mathrm{6}\times {\mathrm{10}}^{\mathrm{17}}$ molec. cm⁻²), while for numerical stability reasons a pre-whitening of the data is performed. Estimated slant column and fitting coefficient uncertainties are obtained from the diagonal of the covariance matrix of the standard errors, while the off-diagonal elements represent the correlation between the fit parameters.¹ The SCD error estimates are scaled with the square root of the normalised χ², where χ² is normalised by (n_λ−D), with D the degrees of freedom of the fit, which is almost equal to the number of fit parameters: $\mathrm{\Delta }{N}_{\mathrm{s}}=\mathrm{\Delta }{N}_{\mathrm{s}}^{\mathrm{OE}}\cdot \sqrt{{\mathit{\chi }}^{\mathrm{2}}/(n_{\mathit{\lambda }}-D)}$, with $\mathrm{\Delta }{N}_{\mathrm{s}}^{\mathrm{OE}}$ the SCD error reported by the OE routine. The NO₂ output data product provides $\mathrm{\Delta }{N}_{\mathrm{s}},{\mathit{\chi }}^{\mathrm{2}},n_{\mathit{\lambda }},D$ and rms error.

3.2.1 TROPOMI wavelength calibration

Before forming the reflectance of Eq. (1) both I(λ) and E₀(λ) are calibrated, after which the calibrated E₀(λ_cal) is interpolated, using information from a high-resolution reference spectrum (E_ref; see Table 1), to the calibrated I(λ_cal), which serves as the common grid for the reflectance. In the TROPOMI processor these steps are performed prior to the DOAS fit (van Geffen et al., 2019).

A wavelength calibration essentially replaces the nominal wavelength λ_nom that comes along with the level-1b spectra (Ludewig et al., 2020) by a calibrated version:

$$\begin{array}{}\text{(5)}& {\mathit{\lambda }}_{\mathrm{cal}}={\mathit{\lambda }}_{\mathrm{nom}}+w_{\mathrm{s}}+w_q({\mathit{\lambda }}_{\mathrm{nom}}-{\mathit{\lambda }}_{\mathrm{0}}),\end{array}$$

where w_s represents a wavelength shift and w_q a wavelength stretch (w_q>0) or squeeze (w_q<0), with w_q defined with regard to the central wavelength of the fit window λ₀. Each radiance ground pixel and each irradiance row has its own wavelength grid and calibration results. In the TROPOMI processor fitting w_q is turned off; see below for a short discussion of this.

The wavelength calibration is performed over the full NO₂ fit window (405–465 nm), using a high-resolution solar reference spectrum (E_ref, pre-convolved with the TROPOMI instrument spectral response function (ISRF); see Table 1) and the OE routine also in use for solving the DOAS equation. For the I(λ) calibration a second-order polynomial as well as a term representing the Ring effect are included: the model function used for the radiance wavelength calibration is a modified version of Eq. (2); including the Ring effect allows for a wavelength calibration to be performed across the full fit window. For the E₀(λ) calibration the Ring term is obviously excluded. The a priori error of the wavelength shift is set to 0.07 nm, one-third of the spectral sampling in the NO₂ wavelength range, so as to ensure that w_s will not exceed the spectral sampling distance.

Figure 2Wavelength calibration shifts w_s for the NO₂ fit window (405–465 nm) of the TROPOMI irradiance (red) and radiance (blue), where the latter is an average over the tropical latitude (TL) range. (a) Shifts for 1 July 2018 (radiance orbit 03711, with irradiance from orbit 03718) as a function of the across-track ground pixel index; the dashed horizontal lines are the across-track averages, with the exception of the outer rows. (b) Time evolution of the across-track average shifts.

Download

Figure 2a shows the wavelength shifts w_s for an orbit on 1 July 2018 of the irradiance (red) and radiance (blue) as a function of across-track ground pixel (row), where the radiance shift of each row is an along-track average over the TL range defined in Sect. 2.3. When taking a different latitude range the across-track shape of the radiance wavelength shift shown in Fig. 2a does not noticeably change, while the absolute value of the average shifts increases by about 5 % going south to north – it is not known what causes this small increase, but it is well within instrument specifications. Due to only partial instrument slit illumination at the outer two rows, 0 and 449, w_s shows markedly different values for these rows. To avoid these peaks from overshadowing the effects discussed below, the outer two rows are skipped from the following analysis.

The broad across-track shape and the average value of w_s visible in Fig. 2a are not important, as they result from the choice of the nominal grid of the level-1b data. The change in time of the average $\stackrel{\mathrm{‾}}{w_{\mathrm{s}}}$ and of the row-to-row variation in w_s, however, give an idea of the stability of the level-1b data and hence of the instrument. Figure 2b shows the temporal change in $\stackrel{\mathrm{‾}}{w_{\mathrm{s}}}$. There seems to be a small long-term oscillation in this, with an amplitude of about 0.0016 and 0.0020 nm for radiance and irradiance, respectively, which looks likely to be a seasonal effect. A similar seasonal variation of similar amplitude is seen in the wavelength calibration data of OMI's visible channel (Schenkeveld et al., 2017, Fig. 34). Both for TROPOMI and OMI this amplitude does not exceed scatter levels and is thus well within instrument requirements.

For a given field of view (ground pixel), the dominant term in the overall magnitude of the radiance is the inhomogeneous illumination of the instrument slit as a result of the presence of clouds. Variation in the presence of clouds may therefore show up as differences in the w_s of ground pixels (e.g. along a row) and from day to day. The magnitude of the day-to-day variation in the average is much smaller than the long-term oscillation visible in Fig. 2b. The row-to-row variation in the shift, visible in Fig. 2a, is small and the evolution of that across-track variation shows a slow increase over time (not shown), probably related to degradation of the instrument (Erwin Loots, personal communication, 2019).

With the forthcoming update of the level-1b data to v2.0.0 the nominal UV–visible wavelength grids of both irradiance and radiance are adjusted by 0.027 nm, for all rows and all days (Ludewig et al., 2020). As a result of this the average $\stackrel{\mathrm{‾}}{w_{\mathrm{s}}}$ will be reduced by that amount, but the across-track and in-time variations will remain the same. Level-1b v2.0.0 will contain an improved degradation correction (Rozemeijer and Kleipool, 2019; Ludewig et al., 2020), probably reducing the slow increase over time of the across-track variation mentioned above. All in all, the wavelength calibration results show that TROPOMI is a rather stable instrument, but further monitoring of the wavelength shifts seems worthwhile.

Turning on the stretch fit parameter in the radiance calibration for orbit 03711 leads to a small stretch of 0.2–$\mathrm{5}\times {\mathrm{10}}^{-\mathrm{4}}$, depending on latitude, with an associated error estimate of 3–$\mathrm{6}\times {\mathrm{10}}^{-\mathrm{4}}$ (averaging over 30^∘ latitude ranges with varying central latitudes): the stretch found is smaller than its error for most latitudes. At the same time the radiance wavelength shift, the NO₂ SCD and SCD error, and the rms error of the DOAS fit change on average by less than 1 %, with a standard deviation comparable to that change or larger. In other words: including the stretch fit parameter in the radiance calibration does not significantly alter the retrieval results, and hence the w_q fit parameter will remain turned off.

3.3 OMI–QA4ECV optical density fit retrieval

The OMI data are processed in the QA4ECV framework with the QDOAS software (Danckaert et al., 2017), wherein R_mod(λ) is formulated in an optical density fit (ODF hereafter) approach:

with σ_ring(λ) the differential (pseudo-absorption) reference spectrum of the Ring effect and C_ring its fitting coefficient, where σ_ring(λ) equals I_ring(λ)∕E_ref(λ) minus a second-order polynomial, with E_ref a (constant) solar reference spectrum (which is different from the measured solar spectrum E₀(λ) used in Eq. 2). Note that except for the way the Ring effect is treated, the IF and ODF modelled reflectances are the same to first order; see Appendix A for a discussion of this difference.

The ODF minimises the merit function (cf. Eq. 3):

without weighting with the level-1b uncertainty estimate ΔR_meas, though QDOAS has the option to include the weighting. To minimise ${\mathit{\chi }}_{\mathrm{ODF}}^{\mathrm{2}}$, QDOAS uses a Levenberg–Marquardt non-linear least-squares fitting procedure (Press et al., 1997), which also provides an estimate of the uncertainties in the fit parameters.

In the ODF formulation the rms error is defined as

which is different from the R_rms of the intensity fit as given in Eq. (4); see Appendix B for a relationship between the two.

Like many other DOAS applications, the OMI–QA4ECV processing includes a correction for an intensity offset in the radiance:

with P_off(λ) a low-order polynomial (in OMI–QA4ECV a constant) and S_off a suitable scaling factor (QDOAS computes this dynamically from an average of the measured solar spectrum E₀(λ) in the DOAS fit window). Sect. 5.1 discusses the possible origin and implication of this correction term.

QDOAS also has the option to be run in intensity fit mode, in which case the modelled reflectance includes the Ring effect as a pseudo-absorber like it does in the optical density fit mode Eq. (6) rather than as the non-linear term like in Eq. (2).

3.4 OMI–OMNO2A intensity fit retrieval

The official OMI NO₂ SCD data processing, running at NASA, is called OMNO2A. OMNO2A v1.2.x delivers the SCD data for the DOMINO v2 NO₂ vertical column density (VCD) processing (results of which are released via http://www.temis.nl/airpollution/no2.html, last access: 17 March 2020). A number of improvements intended for OMNO2A v2.0, which have not yet been implemented, were investigated by van Geffen et al. (2015), but the SCD retrieval of OMNO2A v2.0 can be run locally at the Royal Netherlands Meteorological Institute (KNMI) for testing and comparisons. The OMNO2A processor does not include an intensity offset correction term.

OMNO2A v2.0 uses the intensity fit approach with the modelled reflectance formulated in the same manner as TROPOMI, viz. Eq. (2) and the settings listed for TROPOMI in Table 1, with the exception that χ² is minimised using a Levenberg–Marquardt (LM) solver and wavelength calibration is performed over part of the NO₂ fit window (409–428 nm), the 2005 average irradiance spectrum as reference and an older ozone reference spectrum (van Geffen et al., 2015). Tests have shown that the LM and OE solvers essentially give the same fit results when used with the same settings. Furthermore, KNMI has a local tool to convert the OMI level-1b data into the TROPOMI level-1b format, enabling direct comparisons between the two processors.

Figure 3NO₂ geometric column density (GCD, defined in Sect. 4; a, b, c) and slant column density (SCD) error estimate from the DOAS fit (d, e, f) averaged over the TL range as function of the across-track viewing zenith angle (θ) of Pacific Ocean orbits of TROPOMI and OMI on 1 July 2018 and of OMI on 1 July 2005. (a, d) Regular TROPOMI processing of TROPOMI compared with OMI–QA4ECV processing. (b, e) Regular TROPOMI processing of TROPOMI compared with QDOAS processing with TROPOMI settings and with QA4ECV settings. (c, f) Regular TROPOMI processing of OMI compared with OMI–QA4ECV and OMNO2A (v2) results.

Download

4.1 GCD and SCD error comparison for one orbit

Figure 3 provides comparisons of the GCD (left column) and SCD error estimate from the DOAS fit (right column), averaged over the TL range for the Pacific Ocean orbits of TROPOMI and OMI on 1 July 2018. In view of the OMI row anomaly, the corresponding OMI orbit of 1 July 2005 is shown as well, noting that the NO₂ concentrations in 2005 are likely to be different from those in 2018.

The TROPOMI orbit used here is representative of all Pacific Ocean orbits in across-track shape and variability, as is shown in subsequent sections by the stability of stripe amplitude (Sect. 4.3) and slant column uncertainties (Sect. 4.6).

4.1.1 Geometric column density

In Fig. 3a the GCD results of the regular TROPOMI processing are compared with the OMI–QA4ECV processing. The TROPOMI and OMI GCD of 1 July 2018 compare well in magnitude, in as far as such a comparison is possible in view of the large row-to-row variation in the OMI data and the row anomaly: averaged over the viewing zenith angle range $\mathit{\theta }=[-{\mathrm{55}}^{\circ },-{\mathrm{10}}^{\circ }]$ TROPOMI's GCD is about 3 % higher than OMI's. Near the western (left) edge of the swath, TROPOMI seems to report lower NO₂ values than OMI, which might be related to the fact that nadir of the OMI orbit lies 9^∘ east of TROPOMI nadir. The OMI GCD of 1 July 2005 clearly shows less row-to-row variation than the OMI 2018 data but more than the TROPOMI data (cf. Sect. 4.3).

In Fig. 3b the regular TROPOMI results are compared with a processing of the TROPOMI level-1b data with QDOAS, using settings as close as possible to those of the TROPOMI processor and settings used for QA4ECV (viz. Table 1). When using TROPOMI settings the QDOAS results match those of the regular TROPOMI processing very closely: averaged over the central 150 (of the 450) detector rows the difference is about 0.2 %. The QDOAS QA4ECV settings are different from the TROPOMI settings at three points (type of DOAS fit, use of level-1b uncertainly in χ² minimisation and intensity offset correction), as a result of which the GCDs (and thus the SCDs) are lower by about 6.1 % for this orbit. Sect. 4.2 discusses the effect of the QDOAS settings somewhat further.

In Fig. 3c the OMI results of the regular QA4ECV processing are compared with a processing of the OMI level-1b data with the OMNO2A and TROPOMI SCD processors for the OMI orbit of 2005 in Fig. 3a, in order to investigate the impact of retrieval method details. Differences in the results of the OMNO2A and TROPOMI processor are likely mainly due to differences in the wavelength calibration: TROPOMI's radiance wavelength calibration includes a correction for the Ring effect, which allows the use of a larger calibration window (in this case the NO₂ fit window; viz. Sect. 3.2.1), while OMNO2A's calibration window is necessarily limited (viz. Sect. 3.4).

As with the TROPOMI data in Fig. 3b, the QA4ECV settings clearly give the lowest GCD results: averaged over the central 20 (of the 60) detector rows, the QA4ECV GCD is lower than the OMNO2A processor GCD by about 3.7 % and lower than the TROPOMI processor GCD by about 7.0 %. Note that the across-track striping in the OMI results differs markedly between the different processor results, which is related to a combination of processor differences and the response to instrumental issues (OMI striping data quoted in Sect. 4.3 is taken from OMI–QA4ECV).

Table 2NO₂ geometric column density (GCD), slant column density (SCD) error and rms error from the DOAS fit averaged over the TL range and the central 150 detector rows of TROPOMI Pacific orbit 03711 of 1 July 2018 retrieved with QDOAS using different settings. For comparison, the regular v1.2.2 TROPOMI results (used in this study) and a local reprocessing using the forthcoming v2.1.0 are also listed. Given the difference in rms error definitions, their values from QDOAS and TROPOMI retrievals cannot be compared directly (Sect. 3.3).

^a With respect to v1.2.2, v2.1.0 entails two small bug fixes and spike removal (Sect. 4.1.3); all QDOAS runs include spike removal.

Download Print Version | Download XLSX

4.2 TROPOMI NO₂ SCD: different QDOAS options

As mentioned in the previous section (and visible in Fig. 3), the retrieval results depend on the details of the DOAS NO₂ SCD retrieval: the type of the DOAS fit (IF or ODF) and the retrieval settings used (in particular whether the intensity offset correction is included or not).

Table 2 presents the GCD, SCD error and rms error of the DOAS fit for four combinations of QDOAS settings when processing TROPOMI orbit 03711, with other configuration settings as much as possible matching those of the TROPOMI processor (if included, the intensity offset correction polynomial P_off(λ) is a constant), as well as the results from the TROPOMI NO₂ processor. Conclusions from these results are as follows:

• Turning on the intensity offset correction in QDOAS has quite a large impact on the results: the GCD goes down by ∼5 %, while the SCD error goes down by ∼8%.

• That turning on the intensity offset correction in QDOAS leads to a lower rms error is logical, since an extra fit parameter is introduced; it cannot be determined which part of the reduction in the rms error (by ∼9 %) is due to this extra fit parameter and which part is due to a physically better fit.

• In IF mode QDOAS retrieves slightly larger GCDs (∼1 %) and slightly lower SCD errors (∼1 %), showing that the precise fit method itself does not affect the fit results much.

• The rms error calculation of the TROPOMI IF mode and the QDOAS ODF mode, given in Eqs. (4) and (8), respectively, lead to different results; a relation between these two is given in Appendix B.

• Given that the rms error in the QDOAS IF mode is ∼9 % higher than in the QDOAS ODF mode the rms definitions of these two QDOAS modes may be slightly different for the two modes and the definition of the QDOAS IF mode is different from the TROPOMI IF mode.

As a reference, Table 2 also includes the results of the regular TROPOMI retrieval of the currently officially available processor version v1.2.2, as well as the results from a local reprocessing with the forthcoming v2.1.0 processor (Sect. 4.1.3). That processor has an experimental option to also include an intensity offset correction, implemented in the form of an extra term on the right-hand side of Eq. (2):

with P_off(λ) a low-order polynomial and S_off a suitable scaling factor with the same unit as E₀(λ). Table 2 shows that including a constant P_off in the TROPOMI retrieval has a similar effect as in the case of QDOAS: the GCD and the SCD error decrease by a few percent.

Another small difference in the retrieval methods is that the TROPOMI NO₂ processor uses the level-1b uncertainty in χ² minimisation (cf. Eq. 3) whereas OMI–QA4ECV does not (cf. Eq. 7). QDOAS has the option to turn the χ² weighting on in its ODF mode, the impact of which on the fit results (not shown) is minimal for the GCD and rms, while the SCD error seems to be unrealistically much reduced, indicating that perhaps the error propagation in the ODF mode is not done entirely correctly.

All in all, the retrieval method itself (IF or ODF) does not seem to have a significant impact, while the intensity offset correction has quite a large impact on the GCD (and thus on the SCD) values. The intensity offset term is further discussed in Sect. 5.1.

Figure 4Evaluation of the NO₂ SCD stripe amplitude. (a) SCD stripe amplitude $N_{\mathrm{s}}^{\mathrm{str}}$ (blue) and $N_{\mathrm{s}}^{\mathrm{str}}/M_{\mathrm{geo}}$, i.e. the GCD stripe amplitude (red), for orbit 03711 of 1 July 2018. (b) The measured (blue) and corrected (red) GCD for the same orbit, averaged over the TL range. (c) Time evolution of the rms of the SCD stripe amplitude.

Download

4.3 De-striping: correcting across-track features

Since the beginning of the OMI mission, non-physical across-track variations in the NO₂ SCDs have been observed, which shows up as small row-to-row jumps or “stripes” (Boersma et al., 2011; Veihelmann and Kleipool, 2006). Given that the geophysical variation in NO₂ in the across-track direction (east–west) is smooth rather than stripe-like over non-contaminated areas (Boersma et al., 2007), a procedure to “de-stripe” the SCDs is implemented in the CTM/DA processing system used for DOMINO and QA4ECV. Even though in TROPOMI the row-to-row variation is much smaller than in OMI (cf. Fig. 3a), as of v1.2.0 it was decided to turn on de-striping to remove small but systematic across-track features and improve the data product quality.

The operational TROPOMI de-striping is determined from the TL range of orbits over the Pacific Ocean, and a slant column stripe amplitude is determined for each viewing angle. The SCD stripe amplitude ($N_{\mathrm{s}}^{\mathrm{str}}$) is defined as the difference between the measured total SCD (N_s) and the total SCD ($N_{\mathrm{s}}^{\mathrm{corr}}=N_{\mathrm{s}}-N_{\mathrm{s}}^{\mathrm{str}}$) derived from the CTM/DA profiles using the averaging kernel and air-mass factor from the retrieval. In order to retain only features which are slowly varying over time, and in order to reduce the sensitivity to features observed during a single overpass, the SCD stripe amplitudes are averaged over a time period of 7 d, or about seven Pacific orbits, before subtracting them from the SCDs. The NO₂ data product file contains N_s and $N_{\mathrm{s}}^{\mathrm{str}}$, so that a user of the slant column data can or must apply the stripe correction.

As an example, Fig. 4a shows $N_{\mathrm{s}}^{\mathrm{str}}$ for the Pacific Ocean orbit of 1 July 2018 (blue) and $N_{\mathrm{s}}^{\mathrm{str}}/M_{\mathrm{geo}}$ (red) for the stripe amplitude in GCD space. For the same orbit Fig. 4b shows the GCD (blue) averaged over the TL range and the corrected GCD, i.e. $N_{\mathrm{s}}^{\mathrm{corr}}/M_{\mathrm{geo}}$ (red). The across-track structure and the magnitude of the $N_{\mathrm{s}}^{\mathrm{str}}$ vary in time, but the overall behaviour is fairly constant.

Figure 5Comparison of the time evolution of the rms of the NO₂ SCD stripe amplitude over the first year of TROPOMI data (red; cf. Fig. 4c) and over selected OMI–QA4ECV years (blue); the main increases in the OMI rms occur during 2006, 2010–2011 and 2014–2015. Dashed lines indicate averages over the year periods.

Download

A measure of the stability of the SCD stripe amplitude is the rms of the across-track stripe amplitude, i.e. of the blue line in Fig. 4a: $\sqrt{\mathit{\left\{}\stackrel{\mathrm{‾}}{{\sum }_i(N_{\mathrm{s},i}^{\mathrm{str}}{)}^{\mathrm{2}}}\mathit{\right\}}}$, with summation over rows $i=\mathrm{0},\mathrm{1},\mathrm{\dots },\mathrm{449}$. Fig. 4c shows this rms as function of time: there is quite some variation, but on average the rms seems constant at 2.15±0.13 µmol m⁻² ($\mathrm{0.13}\pm \mathrm{0.08}\times {\mathrm{10}}^{\mathrm{15}}$ molec. cm⁻²); nothing special is seen at 6 August 2019, when the pixel size changes. Further monitoring will have to show whether the stripe amplitude remains stable.

Figure 5 shows the same quantity for the first year of TROPOMI data (average: 2.10 µmol m⁻²) and for selected years of OMI–QA4ECV data: 2005 (3.96 µmol m⁻² or 1.9 times the TROPOMI average), 2012 (6.83 µmol m⁻² or 3.3 times) and 2018 (10.63 µmol m⁻² or 5.1 times). The increase in the stripe amplitude of OMI NO₂ data is not uniform over time and is also present in the case daily solar irradiance spectra being used for the retrieval (Sergey Marchenko, personal communication, 2019); hence the increase is not (or at least not solely) caused by the use of a fixed irradiance in the OMI–QA4ECV data processing (viz. Table 1),

4.4 Quantitative TROPOMI-OMI GCD comparison

The comparison of TROPOMI and OMI–QA4ECV Pacific Ocean orbits of 1 July 2018 in Fig. 3a is merely qualitative because (a) of the row anomaly in the OMI data, (b) of the stripiness of the OMI data and (c) the orbits do not exactly overlap. For a more quantitative comparison, TROPOMI and OMI data are gridded to a common longitude–latitude grid of ${\mathrm{0.8}}^{\circ }\times {\mathrm{0.4}}^{\circ }$, after applying the respective de-striping of the SCDs described in the previous subsection on both datasets.

Figure 6Comparison of TROPOMI and OMI–QA4ECV NO₂ GCD for clear-sky ground pixels for July 2018 after conversion to a common longitude–latitude grid of ${\mathrm{0.8}}^{\circ }\times {\mathrm{0.4}}^{\circ }$ for (a) the Pacific Ocean and (b) the India-to-China area. The area covered, the difference between TROPOMI and OMI–QA4ECV, the linear fit coefficients, and the correlation coefficient are listed in the panels.

Download

Figure 7Same as Fig. 6 but for cloudy ground pixels.

Download

Figure 6 shows the scatter plot of the TROPOMI and OMI/Q4ERCV GCDs of (almost) clear-sky ground pixels (i.e. cloud radiance fraction <0.5) for July 2018 for two regions: the remote Pacific Ocean and the polluted area covering India and China in the Northern Hemisphere; the definition of these two areas is included in the figure panel legends. Both regions show a very good correlation with R²≈0.99. Over the Pacific Ocean area (Fig. 6a) the clear-sky TROPOMI GCD is on average 2.20±1.65 µmol m⁻² ($\mathrm{1.33}\pm \mathrm{0.99}\times {\mathrm{10}}^{\mathrm{14}}$ molec. cm⁻²) or 5.23±3.93 % larger than the OMI–QA4ECV GCD. For January 2019 the result (not shown) is quite similar: the clear-sky TROPOMI GCD over the Pacific Ocean is on average 2.19±1.56 µmol m⁻² or 5.78±4.61 % larger than OMI–QA4ECV. Over the polluted India-to-China area (Fig. 6b) the clear-sky TROPOMI GCD is on average 2.02±2.08 µmol m⁻² or 3.79±4.06 % larger than OMI–QA4ECV; i.e. the relative difference is a little smaller than from the Pacific Ocean.

For cloudy pixels (i.e. cloud radiance fraction >0.5) the difference between the TROPOMI and OMI–QA4ECV GCD is smaller, both in absolute and in relative terms, and the scatter is less, as can be seen from Fig. 7. Over the Pacific Ocean area (Fig. 7a) the cloudy TROPOMI GCD is on average 1.27±0.93 µmol m⁻² ($\mathrm{0.76}\pm \mathrm{0.56}\times {\mathrm{10}}^{\mathrm{14}}$ molec. cm⁻²) or 3.04±2.39 % larger than the OMI–QA4ECV GCD. Over the polluted India-to-China area (Fig. 7b) the clear-sky TROPOMI GCD is on average 1.38±1.26 µmol m⁻² or 2.74±2.37 % larger than OMI–QA4ECV.

These differences between the TROPOMI and the OMI–QA4ECV GCDs (and thus between the SCDs) is comparable to the difference found in Sect. 4.2 due to turning on the intensity offset correction (discussed further in Sect. 5.1) and may therefore be related mainly to the specific settings of the retrieval methods.

4.5 Impact of time difference between radiance and irradiance measurements

In the offline TROPOMI NO₂ (re-)processing of a certain radiance orbit, the processor is configured to use the irradiance spectrum measured nearest in time to the radiance orbit. Given that TROPOMI takes irradiance measurements once every 15 orbits (once every ∼25 h and 22 min) and that currently the offline processing is running at least a week after the radiance measurements, the difference in time between the radiance and irradiance measurements will usually be not larger than eight orbits. In this sense, the TROPOMI processing is very different from the OMI processing (whether QA4ECV, OMNO2A or other): for OMI the 2005 average irradiance is used for the full dataset (2004–present) (van Geffen et al., 2015; Zara et al., 2018).

If for the TROPOMI processor one was to use a fixed irradiance, the errors on the retrieval results become larger. Figure 8a illustrates this by showing the across-track TL range average SCD error for radiance orbit 07513 using the irradiance measurement of the same orbit and of orbit 05428 (2085 orbits, 147 d earlier) and of orbit 03058 (4455 orbits, 314 d earlier): the larger the difference in measurement time between radiance and irradiance, the larger the SCD error and the larger the row-to-row variation in the SCD error.

Figure 8Effect of a difference between the radiance and irradiance orbit numbers on the NO₂ GCD and the SCD error, averaged over the TL range. (a) SCD error of radiance orbit 07513 (26 March 2019; red) using irradiance measurements from orbits 03058 (16 May 2018; blue), 05428 (30 October 2018; grey) and 07513. (b) SCD error averaged over detector rows 25–424 (solid) and the corresponding standard deviation (dashed) of two radiance orbits (red and blue colours) using a series of irradiance measurements, normalised to 1 for matching orbits, as function of the number of days between radiance and irradiance measurement. (c) Idem for the GCD (solid) and corresponding standard deviation (dashed); note that the two solid GCD curves almost exactly overlap at normalisation value 1.0.

Download

Figure 8b shows the SCD error averaged over detector rows 25–424 (so as to avoid including the higher uncertainties of the outer rows related to the lower on-board pixel binning) and the corresponding standard deviation (SD) for two radiance orbits using selected irradiance measurements from between these two; in the case of radiance orbit 03058 (07513) future (past) irradiances are used. The average SCD error itself increases gradually with increasing time difference, while the SD – a measure of the stripiness of the SCD error – increases more than linearly with time.

For the same series Fig. 8c shows that the average GCD value itself is not affected by the time difference between radiance and irradiance: for radiance orbit 03058 (07513) the average GCD is 41.11±0.18 µmol m⁻² (32.79±0.18 µmol m⁻²). The SD of this averaging – the stripiness of the GCD – increases steeply, levelling off to a factor of around 3. If the TROPOMI processing were to use a fixed irradiance, the de-striping (Sect. 4.3) would show an ever increasing stripe amplitude in Fig. 4c.

It is unclear why the time difference between radiance and irradiance measurements has such a big impact on the TROPOMI NO₂ retrieval errors. The solar output varies somewhat over time, but it seems unlikely that this variation is large enough to cause the increase in the retrieval errors. TROPOMI suffers from a small degradation (Rozemeijer and Kleipool, 2019) of 1 %–2 % in the absolute irradiance but with little to no wavelength dependency; hence this degradation is not expected to significantly affect the reflectance and the NO₂ SCD retrieval results.

The increased stripiness observed in the OMI NO₂ results depicted in Fig. 5, and shown by Boersma et al. (2011) and discussed in detail by Zara et al. (2018), is at least in part the result of the increasing difference in time between radiance and irradiance measurement, but acting over a longer timescale than the effect seen in Fig. 8b and c for TROPOMI. The fact that the GCD value itself (Fig. 8c) is not appreciably affected by the time difference is very reassuring, both for the TROPOMI and the OMI–QA4ECV retrieval results.

Figure 9NO₂ SCD statistical uncertainties (red) and SCD error estimates from the DOAS fit (blue) as function of time. (a) All pixels with successful retrieval. (b) Pixels with cloud radiance fraction <0.5. (c) Pixels with cloud radiance fraction >0.5. The vertical dotted line marks 6 August 2019, when the along-track ground pixel size was reduced. Averages, marked by dashed lines, are listed in Table 3.

Download

4.6 Time dependence of the slant column uncertainty

The spatial variability of the SCDs over a remote Pacific Ocean sector can be used as an independent statistical estimate of the random component of the SCD uncertainty. This approach was used in the QA4ECV project by Zara et al. (2018) to compare OMI and GOME-2A NO₂ and formaldehyde SCD values retrieved by different retrieval groups, as well as to compare the SCD error estimates following from the different DOAS fits.

Figure 9 shows the NO₂ SCD statistical uncertainties (red) and SCD error estimates from the DOAS fit (blue) as function of time for all ground pixels for which the retrieval was successful (i.e. with quality assurance value 𝚚𝚊_𝚟𝚊𝚕𝚞𝚎>0.50; panel a), for clear-sky pixels (𝚚𝚊_𝚟𝚊𝚕𝚞𝚎>0.75, corresponding to cloud radiance fraction <0.5; panel b), and for cloudy pixels ($\mathrm{0.50}<\mathtt{\text{qa\_value}}<\mathrm{0.75}$; panel c). For this exercise the Pacific Ocean orbits (Sect. 2.1.2) were evaluated over the latitude range $[-{\mathrm{60}}^{\circ },+{\mathrm{60}}^{\circ }]$. Averages over the data period shown in Fig. 9 are listed in Table 3, along with the OMI–QA4ECV results from Zara et al. (2018), who also showed that the OMI–QA4ECV SCD statistical uncertainties and SCD error estimates have increased over the years by about 1 % and 2 % per year, respectively.

Table 3NO₂ SCD statistical and SCD DOAS fit uncertainties, averaged over the listed period, given in two units; cf. Fig. 9.

^a OMI–QA4ECV results taken from Zara et al. (2018), Table 4; additional data
provided by the author.

Download Print Version | Download XLSX

The reduction in the along-track ground-pixels size from 7.2 km to 5.6 km on 6 August 2019 effectively entails a reduction in the integration time from 1.08 to 0.84 s, as a result of which the per-pixel noise on the level-1b radiances data increased by a factor of $\sqrt{\mathrm{1.08}/\mathrm{0.84}}=\mathrm{1.134}$, which in turn caused an increase in the NO₂ SCD error by somewhat less than 13 % (because the SCD error is not solely determined by the noise on the radiance spectra). This increase in the SCD error is visible in Fig. 9 as a jump at that date (marked by a vertical dotted line) and is reflected in the averages given in Table 3: the DOAS uncertainty increases by 7 –8%, depending on the pixel type. The pixel size change does not impact the average magnitude of the NO₂ GCD (except for polluted regions where due to the smaller pixels size larger peak values may be expected), but it does have an effect on the inter-pixel variation of the GCD: the statistical uncertainty increases by 7%–8%.

All in all, the TROPOMI statistical uncertainties are clearly much lower than those of OMI–QA4ECV, even after the ground pixel size reduction. The SCD error estimates from the DOAS fit routine are on average larger than the statistical uncertainties (for TROPOMI about 10 % and for OMI–QA4ECV about 20 %). From the TROPOMI data it appears that the SCD uncertainty is only about 5 % larger than the statistical uncertainty in the case of cloudy pixels but about 12 % in the case of clear-sky pixels. The main reason for the difference between the DOAS and statistical uncertainties is that, unlike the statistical uncertainties, the SCD error estimates also include systematic retrieval issues, and these appear to play a larger role for clear-sky pixels, i.e. pixels for which the radiance signal is lowest.

From Fig. 9 and Table 3 it is furthermore clear that the statistical and the DOAS uncertainties of TROPOMI appear to be stable over the currently available data period: the standard deviation of the quantities given in Table 3 are small and Fig. 9 shows no systematic change over time. The jumps in the quantities on 6 August 2019 are caused by the along-track pixel size change, not by an instrumental issue, and this change has not affected the stability: the standard deviations of the quantities given in Table 3 are not markedly different between the two measurement modes.

5.1 Intensity offset correction

Many DOAS applications, including the OMI–QA4ECV processing, include a correction for an intensity offset in the radiance, e.g. in the form given in Eq. (9). The precise physical origin of such an intensity offset is not specified in the literature, but it is thought to be related to instrumental issues (e.g. incomplete removal of stray light or dark current in level-1b spectra) and/or atmospheric issues (e.g. incomplete removal of Ring spectrum structures, vibrational Raman scattering (VRS) in clear ocean waters); see, for example, Platt and Stutz (2008), Richter et al. (2011), Peters et al. (2014) and Lampel et al. (2015).

From OMI–QA4ECV evaluations (Müller et al., 2016; Boersma et al., 2018) and a preliminary study using TROPOMI data (Oldeman, 2018), it appears that the largest impact of the intensity offset correction occurs over clear-sky clear ocean water (i.e. with little to no chlorophyll). If indeed absorption by VRS is the key aspect here, it would on physical grounds be more appropriate to include a VRS absorption spectrum (σ_VRS) in the DOAS fit because the intensity offset corrections are proportional to the irradiance, while σ_VRS has a different spectral structure; i.e. an intensity offset correction will not fully compensate for VRS absorption. Investigating this matter further falls outside the scope of the present paper.

Turning on the intensity offset correction (IOC) in QDOAS for the TROPOMI and OMI orbits shown in Fig. 3 reduces the GCD values on average by ∼5 %, with the relative impact largest for the lower GCDs. Since this decrease in the GCDs is comparable for both TROPOMI and OMI data, when using the same SCD processor, it seems unlikely that the IOC is correcting for instrumental effects. The quantitative comparison discussed in Sect. 4.4 revealed that for clear-sky cases (Fig. 6) the differences are a little larger than for the cloudy cases (Fig. 7), and for clear-sky cases the difference is larger for the remote Pacific Ocean area (almost completely water) than for the polluted India-to-China area (mainly land surface), while for the cloudy cases the differences are comparable for the two areas. These differences thus seem to indicate that the IOC may be correcting for some absorption effects in ocean waters, but not only for such absorption effects given that the reduction in GCD is also seen over land and over clouds.

It must be noted that the effect of the IOC in QDOAS (viz. Eq. 9) on the GCDs is nearly twice as large as the effect of the experimental IOC in the TROPOMI processor (viz. Eq. 10); apparently these two implementations of the IOC do not behave exactly the same.

All in all an intensity offset correction will not be included in the regular TROPOMI NO₂ processing, also because instrumental effects such as stray light and dark current are corrected for in the spectral calibration in the level 0 to 1b processor (Kleipool et al., 2018; Ludewig et al., 2020)

5.2 Validation of stratospheric NO₂

Routine validation of TROPOMI data products is being carried out by the Validation Data Analysis Facility (VDAF; http://mpc-vdaf.tropomi.eu/, last access: 17 March 2020), with support from the S5P Validation Team (S5PVT), which issues quarterly validation reports, such as Lambert et al. (2019). Since NO₂ over the Pacific Ocean, i.e. away from anthropogenic sources of NO₂, is primarily located in the stratosphere, validation of stratospheric NO₂ can also be seen as validation of Pacific Ocean NO₂ SCDs.

Stratospheric NO₂ column data are compared to reference measurements from zenith-sky light (ZSL) DOAS instruments, which are operated in the context of the Network for the Detection of Atmospheric Composition Change (NDACC). ZSL-DOAS measurements, obtained twice daily at twilight, are adjusted to the TROPOMI overpass time in order to account for the diurnal cycle of NO₂. Quoting the fifth quarterly report (Lambert et al., 2019), the TROPOMI stratospheric NO₂ columns are “generally lower by approximately 0.15×10¹⁵ molec cm⁻² [2.5 µmol m⁻²] than the NDACC ZSL-DOAS ground-based measurements, deployed at 19 stations from pole to pole. The bias of roughly −10 % is within the S5P mission requirements, which is equivalent to 0.2–0.4×10¹⁵ molec cm⁻², depending on latitude and season”. The −10 % bias mentioned is the average bias; the median bias is about −7 %. Note that the ZSL-DOAS measurements have their own uncertainties (a bias of at most 10 % and a random uncertainty better than 1 %; Lambert et al., 2019) and that the interpolation to the TROPOMI overpass time introduces uncertainties in the ground-based data of the order of 10 % (Lambert et al., 2019; see also Dirksen et al., 2011).

In other words: the agreement between stratospheric NO₂ of TROPOMI and ground-based instruments is rather good, where TROPOMI seems to give SCD column values that are slightly too low. Including an intensity offset correction in the DOAS fit (Sect. 5.1) would lead to a reduction in the Pacific Ocean NO₂ SCD by a few percent (Sect. 4.2), which in turn would imply worsening of the validation results.

5.3 NO₂ retrieval over strongly polluted areas

In the case NO₂ concentrations being no longer optically thin, assumptions lying at the basis of the DOAS retrieval approach may no longer be valid (Richter et al., 2014; Andreas Richter, personal communication, 2019): the relationship between SCD and VCD may become non-linear for single wavelengths, the AMF of boundary layer NO₂ may become strongly wavelength dependent and decrease with increasing NO₂ columns, and the temperature dependence of the NO₂ reference spectrum (usually corrected for a posteriori in the AMF application) may be wavelength dependent. During a dramatic pollution episode in China in January 2013, with NO₂ up to 1×10¹⁷ molec. cm⁻² (1660 µmol m⁻²), these effects seemed to become significant, as shown by Richter et al. (2014).

When measuring NO₂ over strongly polluted areas with high spatial resolution, such as provided by TROPOMI, the chance of detecting very large NO₂ concentrations for individual ground pixels increases. The area with the largest NO₂ columns is probably China, but since the reductions in air pollution in China over the past years, it is currently unlikely to encounter NO₂ concentrations that are not optically thin in the TROPOMI data, except in a few individual pixels.

NO₂ concentrations over China are highest in winter. In January 2019, for example, the highest GCD found over China is 701±16 µmol m⁻² in orbit 06637 (24 January), which has 577 pixels (0.05 % of the 1 204 367 pixels with a successful retrieval) with a GCD exceeding 300 µmol m⁻²; 73 pixels have a GCD values exceeding 400 µmol m⁻². Orbit 06580 (20 January) has in that month the largest number of pixels with a GCD exceeding 300 µmol m⁻², namely 1609, with a peak value of 512±14 µmol m⁻²; 256 pixels have GCD values exceeding 400 µmol m⁻².