Understanding uncertainty is essential for utilizing atmospheric volatile organic compound
(VOC) measurements in robust ways to develop atmospheric science. This study
describes an inter-comparison of the VOC data, and the derived uncertainty
estimates, measured with three independent techniques (PTR-MS,
proton-transfer-reaction mass spectrometry; GC-FID-MS,
gas chromatography with flame-ionization and mass spectrometric detection;
and DNPH–HPLC, 2,4-dinitrophenylhydrazine derivatization followed by analysis by high-performance
liquid chromatography)
during routine monitoring as part of the Sydney Particle Study (SPS) campaign in 2012. Benzene, toluene,
Some sources of uncertainty that are poorly quantified by the bottom-up uncertainty analysis method were identified, including: contributions of non-target compounds to the measurement of the target compound for benzene, toluene and isoprene by PTR-MS as well as the under-reporting of formaldehyde, acetaldehyde and acetone by the DNPH technique. As well as these, this study has identified a specific interference of liquid water with acetone measurements by the DNPH technique.
These relationships reported for Sydney 2012 were incorporated into a larger analysis with 61 similar published inter-comparison
studies for the same compounds. Overall, for the light aromatics, isoprene and the
Volatile organic compounds (VOCs) in the atmosphere have important roles in processes leading to formation of ozone and secondary organic aerosol, and quantitative measurements of VOCs are important for source reconciliation, verification of atmospheric models and exposure assessment. While atmospheric VOC measurements commenced around 60 years ago, measurement techniques are still rapidly evolving and the uncertainties associated with these measurements are often poorly understood. Assessment of uncertainty for VOC measurement techniques by standard methods (Harris, 2003; JCGM, 2008) often underestimates what happens in practice because of the presence of poorly understood or neglected processes that affect the measurement and its uncertainty. However comparison of independent techniques for measuring individual VOCs provides a more critical test of uncertainties. Inter-comparison of independent techniques and their quantification of measurement uncertainty can collectively contribute significantly to the tasks of validation of a wider range of new knowledge, particularly where atmospheric VOC observations are used to validate VOC emissions inventories, air chemistry models and human exposure to air toxins.
Uncertainty in measurements of atmospheric constituents, including VOCs, can arise from four components of the measurement process:
the pretreatment of the sample (e.g. in the inlet or adsorption, storage and desorption on
a cartridge), the matrix in which the sample (and calibration standards) are presented to
the detector (e.g. in nitrogen, helium, air or some complex mixture), the presence of interfering compounds in the sample (e.g. co-eluting in
chromatography or isobaric compounds in mass spectrometry) and the instrument calibration (e.g. calibration standards used, linearity of
detector response).
There are three distinct methods of determining these uncertainties in VOC measurements. In the first approach, one can examine the
individual components of a single measurement technique and assess the uncertainty of each and combine these to get a total uncertainty
for that method as described in the Guide to Expression of Uncertainty in Measurement (JCGM, 2008). With this approach, one question always remains:
were any sources of uncertainty overlooked? The second method is to make multiple paired measurements with different measurement
techniques, of either synthetic VOC mixtures in cylinders or from air in chambers, and determine the uncertainty from the resulting
paired and replicate measurements. This again only captures a partial contribution to the uncertainty, but it is particularly effective
in identifying the presence of unknown sources of uncertainty and complements the first approach. The third approach, used here, is to
undertake multiple paired measurements of ambient air. This approach does not allow replicate analyses but has the advantage of
including the influence of environmental and operational factors on the measurement uncertainty.
Three independent VOC measurement systems were employed in the study presented here: continuous measurements by
proton-transfer-reaction mass spectrometry (PTR-MS), integrated 5–10
The Sydney Particle Study (SPS) was an intensive field experiment designed to provide a detailed characterization of the chemical and
aerosol composition of the urban atmosphere in Sydney, Australia, in summer 2011 and autumn 2012 (Cope et al., 2014). Sydney is
Australia's largest city (population
The second SPS campaign, SPS 2, occurred in autumn from 15 April to 13 May 2012. The measurement site was approximately 1000
The compounds selected for discussion in the proceeding analysis are a subset of the species measured by the PTR-MS, AT-VOC (adsorbent tube VOC sampling) and DNPH techniques in SPS 2. For the full results of the PTR-MS, AT-VOC and DNPH analysis from SPS 2, the reader is referred to Keywood et al. (2016).
We present quantitative comparisons of concentrations of VOCs including (a)
The results from this study are compared with other inter-comparison data from the scientific literature and some conclusions about the uncertainty in current VOC measurements presented.
The sampling site (33.802
Ambient air was drawn from the main VOC sample inlet via
Three samples per day (05:00–10:00, 11:00–19:00 and 19:00–05:00; all times are in local time)
were collected by the automated sampler which actively drew air
through DNPH coated solid silica adsorbent cartridges (Supelco LpDNPH S10, Supelco, Pennsylvania, USA), using a constant flow air
sampling pump at a set flow rate of 1
There is a known deterioration, over 1 or more days, of derivatized DNPH-carbonyl samples at room temperature. Because of this, the
compartment housing the DNPH cartridges in the automated sampler was maintained at
The method of DNPH–HPLC sampling employed in this study is compatible with USEPA Method TO-11A (USEPA, 1999b). Following sampling, the
derivatives were eluted from the cartridge in 2.5
In SPS 2, three samples per day (05:00–10:00, 11:00–19:00 and 19:00–05:00) were collected by the automated sampler which actively
drew air through two multi-adsorbent tubes in series (Markes Carbograph plus Carbopack X)
using a constant flow air sampling pump at a set flow
rate of 20
The adsorbent tubes were analysed by a PerkinElmer TurboMatrix
The method of adsorbent tube VOC sampling and analysis employed in this study was compatible with ISO16017-1:2000 and in accordance with USEPA Compendium Method TO-17 (USEPA, 1999a).
A series of certified gas standards including a BTEX standard (benzene, toluene, ethylbenzene and xylenes) (manufacturer stated
accuracy
A flow of 1.5
In SPS 2 a commercially built PTR-MS (Ionicon Analytik, GmbH, Innsbruck, Austria) was utilized for continuous VOC measurements. For a detailed description of PTR-MS the reader is referred to Ellis and Mayhew (2014), de Gouw and Warneke (2007), and Lindinger et al. (1998). Briefly, the instrument consists of a hollow cathode ion source where reagent ions were generated, a drift tube where the reagent ions and the sample were mixed and chemical ionization reactions occurred between the reagent and the analytes, and a quadrupole mass spectrometer (Balzers QMG422) with a secondary electron multiplier operating in pulse counting mode, for sorting and detecting reagent and product ions.
The drift tube was operated at 60
Ambient sampling times from SPS 2 for the PTR-MS, AT-VOC and DNPH–HPLC methods; zero and calibration times for the PTR-MS.
The PTR-MS operated with the aid of custom-built auxiliary equipment that regulated the flow of air in the sample inlet and controlled
whether the PTR-MS was sampling ambient or zero air, or calibration gas. The timing and duration of zero, calibration and ambient
measurements for SPS 2 are detailed in Table 1. Zero readings were made by diverting ambient air through a zero furnace
(350
All PTR-MS ion signals from calibration and ambient measurements referred to in this study were background corrected.
The PTR-MS calibration factors for each of the VOCs included in this work, normalized to 10
The minimum detectable limit (MDL) for each
The PTR-MS was calibrated with three certified gas standards containing in total 20 VOC species. These certified gas standards were
supplied by Apel-Riemer Environmental Inc. (Broomfield, CO, USA) and Air Liquide–Scott Specialty Gases (Plumsteadville, PA, USA). The
stated accuracy for each component in the standards was
The gravimetrically prepared Apel-Riemer standard used to calibrate the PTR-MS contained benzene, toluene and
Interference in the identification and quantification of a target compound in PTR-MS measurements of ambient air can and frequently does occur due to the presence of products from other reaction pathways such as isobaric compounds, fragment ions from other compounds, isotopologues and products of secondary reactions (Warneke et al., 2003; Rogers et al., 2006; Inomata et al., 2008; Dunne et al., 2012; Kaser et al., 2013). When comparing PTR-MS measurements to more selective VOC measurement techniques such as chromatographic methods, the presence of interference in the target ion signal often results in an apparent positive bias in the PTR-MS reported values. The uncertainty related to mass interference is not incorporated in the bottom-up uncertainty analysis and is investigated here to determine its role where there were significant differences at the 95 % confidence limit between the mean values measured by each instrument.
If the identity of the interferents are known, and their concentration and PTR-MS response (fragmentation patterns and instrument
sensitivity) is also known or can be estimated, their contribution to the target
The MDL and summary statistics (ppb) for the PTR-MS, AT-VOC and DNPH data for each of the seven compounds selected for this study. Note that the DNPH MDL differs between morning afternoon and night samples due to different sampling times result. For the purposes of this table the DNPH MDLs and median / MDLs are quoted as a range.
While a number of compounds were measured by both the PTR-MS and AT-VOC or DNPH techniques, only compounds whose data met the following
criteria were retained for the analysis:
Each PTR-MS sample had an ambient data acquisition period that was Each compound known to substantially contribute to a given An empirical calibration from measurements of a certified standard containing the compound(s) of interest was available for both
techniques being compared. The ratio of the median / MDL was
The averaging periods used to merge the PTR-MS, AT-VOC and DNPH data from SPS 2 are listed in Table 1. Three DNPH cartridges and three
pairs of VOC adsorbent tubes were collected daily: a 5
Of the range of compounds measured by each of the three VOC measurement systems (PTR-MS, AT-VOC, DNPH), the data for seven
compounds or compound groups satisfied criteria 1, 2, 3 and 4 for inclusion in the inter-comparison presented here; they were benzene,
toluene, the
There were two methods of determining uncertainties in VOC measurements assessed in this study, bottom-up and top-down. The first
approach, the bottom-up method, examined the individual components of a single measurement technique, assessed the uncertainty of each
and combined these to get a total uncertainty for that method (Harris, 2003; JCGM, 2008). The uncertainty analysis proceeded via the
mathematical model, here called the measurement equation, for the measurement as described in the Guide to Expression of Uncertainty in
Measurement (JCGM, 2008). Details of the uncertainty analysis procedure for each of the selected compounds and for the measurement
technique are described in the Supplement 1. All uncertainties in this analysis are expanded uncertainties with a coverage factor
The bottom-up uncertainty analysis for the AT-VOC method included uncertainty due to
the accuracy of the certified calibration standards; the variance in the response factors of the GC-FID in measurements of certified calibration gas
standards; the uncertainty in the loop volume, temperature and pressure; and the variance in a series of replicate ambient measurements of the target VOCs by the AT-VOC
method. the accuracy of the certified calibration standards, the variance in the response factors of the HPLC in measurements of a series of replicate DNPH cartridges spiked with a certified
liquid standard mixture and the variance in a series of replicate ambient measurements of the target VOCs by the DNPH
method. the accuracy of the certified calibration standards, the variance in the performance of the mass flow controllers which were used to control the flows of the dilution and calibration
gas standards and The variance in the response factors of the PTR-MS in measurements of certified calibration gas
standards.
The bottom-up uncertainty analysis for the DNPH method included uncertainty due to
The bottom-up uncertainty analysis for the PTR-MS method included uncertainty due to
In the second approach to assessing uncertainty, the top-down method, we evaluated the systematic difference between two methods by
evaluating the slope and intercept of a linear regression between two sets of paired simultaneous measurements. We evaluate random
deviations of individual measurements as the root mean square of the orthogonal distance between the location of the pair of
observations (
When comparing two observational datasets, reduced major axis (RMA) regression, also called geometric mean regression, is preferable to
simple least squares linear regression because the analysis is not between an independent and dependent variable, and RMA accounts for
random measurement error on both the
Contributions to the uncertainty of these measurements that are not included in the bottom-up analyses but are apparent from the top-down analyses are discussed. These contributions are described as poorly understood and poorly quantified processes that do not occur in the measurement equation. Some examples of these for PTR-MS and DNPH are identified. None were immediately apparent for AT-VOC.
The results of this inter-comparison are compared with similar published studies from the scientific literature and some conclusions about the uncertainty in current VOC measurements are presented. The other studies examined were published in the peer-reviewed literature, in which all employed PTR-MS as one of the instruments being compared; only results of ambient air studies were included (direct measurements of VOC emission sources such as biomass burning plumes were excluded) and in all comparisons both instruments were calibrated for the species of interest.
Seven sets of inter-comparisons matched the criteria presented in Sect. 2.6. These were
benzene, toluene, the sum of the formaldehyde, acetaldehyde and acetone measured by both the PTR-MS and the
DNPH techniques in SPS 2.
For simplicity, the subsequent text is organized around the names of the most common compound(s) occurring in the instrument response,
while the discussion recognizes that other interfering or co-eluting compounds can be contributing to the instrument response.
The MDL, summary statistics (25th percentile, median, 75th percentile) and the median / MDL for each compound are presented in Table 3.
The means and SDs of the atmospheric data;
the estimated measurement uncertainties of the means (
The uncertainty associated with measurement of these VOCs is evaluated via the methods in the Guide to Expression of Uncertainty in Measurement (JCGM, 2008) and presented in the Supplement. While there is some overlap between the observed uncertainty and the calculated measurement uncertainty, they also include distinct components. The observed uncertainty of a set of atmospheric VOC measurements includes a component due to atmospheric variability that is not included in the calculated uncertainty. The calculated measurement uncertainty can include a component due to uncertainty in the calibration standards, which does not occur in the observed variability of atmospheric measurements which are measured against one reference standard.
Here we analyse whether the sets of simultaneous measurements of VOCs by two different methods have uncertainties such that their mean values plus or minus the measurement uncertainties overlap within the 95 % confidence limit or not. Table 4 shows that for benzene, isoprene, acetaldehyde and acetone, the mean values do not overlap within the 95 % confidence limits. In contrast, for toluene, xylenes and formaldehyde, the mean values do overlap within the 95 % confidence limits.
The inter-comparisons for benzene, toluene, the sum of the
Intercomparisons of PTR-MS vs. AT-VOC and DNPH measurements of selected VOCs in SPS 2 (2012). RMA correlation coefficients
(
Time series of
In PTR-MS, protonated benzene is detected at
Reduced major axis regression analysis between the PTR-MS data for
It is possible the slope of
Ethylbenzene was measured in the AT-VOC samples; however, propyl- and isopropyl benzene were not, and their contribution to the PTR-MS
ion signal at
The slope of the RMA regression between the corrected PTR-MS data and the AT-VOC data improved slightly to
The quantitative agreement between the measurement of benzene by PTR-MS and AT-VOC in this study was poorer than those reported in similar real-world inter-comparisons, most of which have reported slopes between 0.8 and 1.2 shown graphically in Fig. 2 (Warneke et al., 2001; de Gouw et al., 2003; Kato et al., 2004; Kuster et al., 2004; Jobson et al., 2010; Rogers et al., 2006; de Gouw and Warneke, 2007; Kaser et al., 2013; Wang et al., 2014; Kajos et al., 2015; Cui et al., 2016).
The degree of interference will vary with the relative concentrations of higher aromatics to benzene in the atmosphere being
studied. As the higher aromatics have shorter atmospheric lifetimes than benzene, the interference will vary with ageing of an air
mass. Thus, when measuring aged air masses, PTR-MS reported values should show better agreement with more selective GC techniques. In
this study, within a large city, fresh emissions would be present, containing on average a greater fraction of higher aromatics. Thus, we
would expect a larger contribution to the
In summary, a comparison between the measurements of benzene by PTR-MS and the AT-VOC technique indicates a significant difference in the measured concentrations which is unresolved but is likely to vary according to the relative contribution of higher aromatics in different atmospheres.
In PTR-MS, toluene undergoes non-dissociative proton transfer from
RMA regression analysis between the PTR-MS data at
The
These potential interferent compounds, with the exception of
This correction had a minor impact on the slope of the RMA regression (
With the exception of two of studies (Kato et al., 2004; Kajos et al., 2015) previous inter-comparisons between toluene measurements by
PTR-MS and GC techniques have reported slopes of 0.8–1.2 and generally good correlations (
In summary, a comparison between the measurements of toluene by PTR-MS and the AT-VOC technique indicates that there was not a significant difference in the measured concentrations at the 95 % confidence limit. There may be some residual unquantified interference with the PTR-MS toluene measurement which may vary due to contributions from the many additional monoterpene species commonly present in the atmosphere but not accounted for here (Geron et al., 2000; Maleknia et al., 2007).
In PTR-MS, the signal at
The slope (
RMA regression analysis between the PTR-MS signal at
The concentration of
This correction resulted in a minor increase in the slope to
A minor contribution to the PTR-MS signal at
The results reported here are similar to many previous intercomparison studies that have reported good quantitative agreement, within
In summary, a comparison between the measurements of
In measurements of the atmosphere the PTR-MS signal at
Isoprene emissions are dominated by biogenic sources and are strongly light and temperature dependent with maxima in the afternoon. For
SPS 2, when only the afternoon data were considered, closer agreement was observed between the PTR-MS and AT-VOC data for isoprene
attributed to a 0.2
Park et al. (2013) observed three peaks at
The results reported here are consistent with previous inter-comparisons studies between PTR-MS and GC techniques which have reported
slopes of 0.79–2.15 often with significant (up to 0.39
In summary, a comparison between the measurements of isoprene by PTR-MS and the AT-VOC technique indicates a significant difference in
the measured concentrations which may vary according to the relative contribution of other species that contribute to the PTR-MS signal
at
In the following section, the inter-comparisons for formaldehyde, acetaldehyde and acetone measured by both the PTR-MS and the DNPH–HPLC techniques in SPS 2 will be discussed in turn. The MDL, summary statistics (25th percentile, median, 75th percentile) and the median / MDL values for the PTR-MS and DNPH data for each compound are presented in Table 3. The results of the analysis of measurement uncertainty are presented in Table 4.
The results of the RMA regression analysis and the RMSD for each compound are summarized in Table 5. Scatterplots of the comparisons for the three carbonyl compounds are presented in Fig. 1f–h.
As part of this analysis, we have identified a loss process in the DNPH method due to condensation of
The inter-comparisons for formaldehyde, acetaldehyde and acetone measured by both the PTR-MS and the DNPH techniques are presented in
Table 5 as the slope and intercept of the RMA regression analysis, correlation (
In PTR-MS, protonated formaldehyde is detected at
The comparisons presented in Table 4 indicate that the mean values reported by PTR-MS and DNPH agree within 95 % confidence
limits. RMA regression analysis between the PTR-MS signal at
To examine any possible effect of liquid water, the analysis was repeated excluding the data of 16–24 April. The results yielded
a slope of
The slope of 1.25 may be a result of contributions to the PTR-MS signal at
The dominant ion signal in the PTR-MS spectra of glyoxal is detected at
Using the PTR-MS and DNPH data for methanol and glyoxal respectively, along with laboratory measurements and literature values of the
PTR-MS response variables for formaldehyde, methanol and glyoxal – branching ratios (BR
Previous studies have reported PTR-MS values for formaldehyde that were systematically higher than DNPH–HPLC measurements (Wisthaler et al., 2008; Cui et al., 2016) and higher than differential optical absorption spectroscopy (DOAS) and Hantzsch techniques (Wisthaler et al., 2008; Warneke et al., 2011). Other studies report DNPH–HPLC values for formaldehyde that were systematically lower than those reported by other analytical methods (DOAS, FTIR, Hantzsch, TDLAS) (Kleindienst et al., 1988; Lawson et al., 1990; Gilpin et al., 1997; Hak et al., 2005; Wisthaler et al., 2006).
In summary, a comparison between the measurements of formaldehyde by PTR-MS and the DNPH technique indicates there was not a significant difference in the measured concentrations although some discrepancy between the two instruments remains unresolved.
In measurements of the atmosphere the signal at
To examine any possible effect of liquid water, the analysis was repeated excluding the data of 16–24 April (see Sect. 3.2.1).
The results were a slope of
A positive bias in PTR-MS measurements of acetaldehyde may result from contributions to the
Two studies using high-resolution PTR-ToF-MS have observed a single peak at
In an atmospheric simulation chamber study three PTR-MS instruments reported acetaldehyde values close to the known injected
value, whereas a DNPH method significantly underestimated (
Herrington et al. (2007) reported the collection efficiency of acetaldehyde on DNPH cartridges declined from
As shown in Fig. 2, other real-world inter-comparison studies have reported variable agreement between measurements of acetaldehyde by
PTR-MS and GC methods (slopes of 0.87–1.7, intercepts of
In summary, a comparison between the measurements of acetaldehyde by PTR-MS at
In PTR-MS measurements, the ion signal at
The RMA regression analysis between the PTR-MS signal at
As discussed previously the compartment housing the DNPH cartridges in the automated sampler was maintained at
Datapoints that coincided with average dew point temperatures
Ho et al. (2014) also identified a significant negative bias in the collection efficiency of acetone on DNPH cartridges that was
related to humidity, sample flow rate and sample duration. While Ho et al. (2014) used a similar DNPH cartridge type, these authors
reported 35–80 % of acetone was lost under similar conditions as those experienced in this study (RH
In PTR-MS, the ion signal at
Using the DNPH data for propanal and glyoxal, along with literature values of the PTR-MS response variables – branching ratios (Spanel
et al., 1997; Stonner et al., 2016) and reaction rates (Cappellin et al., 2012) – a correction was applied to the PTR-MS
Applying the correction for propanal and glyoxal interference to the reduced
Previous published atmospheric and chamber study measurements reported PTR-MS values for acetone that were
Overall, the PTR-MS signal at
Comparisons have been made between measurements of benzene, toluene,
The slope and intercept as determined by reduced major axis regression gives a different story. The slopes vary considerably with
a median of 1.25 and a range of 1.16–2.01. The intercepts vary with a median of 0.04 and a range of
The reasons for the variations in slope include the contributions of non-target compounds to the measurement of the target compound for benzene, toluene and isoprene by PTR-MS and the under-reporting of formaldehyde, acetaldehyde and acetone by the DNPH technique. This study has identified specific issues with (a) the use of PTR-MS in urban areas at night when interferences from other compounds in isoprene measurements are significant and (b) an interference of liquid water in the sample trap with acetone measurements by the DNPH technique. Despite attempting to correct for these issues, significant discrepancies between the PTR-MS and the AT-VOC and DNPH–HPLC methods remain unresolved. The PTR-MS always has a larger response than the AT-VOC and DNPH–HPLC method and the slopes reported here were often at the higher end when compared with other published inter-comparison studies for the same compounds (Fig. 3). Additional, unquantified uncertainty due to mass interference in PTR-MS and interference in the collection of efficiency of aldehydes and ketones on DNPH may be responsible for the unresolved discrepancies reported here.
Other sources of uncertainty that may arise when comparing two observational datasets that are not included in the bottom-up
uncertainty analyses and were not assessed here include the following:
These are generic issues that should be addressed in future VOC inter-comparison studies.
The relationships reported for Sydney 2012 were incorporated into a larger analysis with 61 other inter-comparison studies for the same
compounds (found in the recent scientific literature; see Fig. 3). For the whole available set of inter-comparisons, the
There are two qualifications concerning this overall uncertainty analysis. This analysis in no way indicates what the uncertainty is in measurements of other VOC compounds. A smaller uncertainty has been reported for alkanes (Hoerger et al., 2015). Similarly, if the emissions and concentrations of a VOC are measured with the same technique or with techniques that are compared, then the uncertainties associated with an atmospheric mass balance compiled using these measurements may be smaller than the case where different VOC measurement techniques that have not been compared are used.
The uncertainties in VOC measurements identified here should be considered when assessing the reliability of VOC measurements from individual instruments, when utilizing VOC data to constrain and inform air quality and climate models, when using VOC observations for human exposure studies and when comparing ambient VOC data with satellite retrievals.
The Sydney Particle Study (SPS) involved a comprehensive suite of measurements of atmospheric gases and aerosols in order to characterize the sources, size distribution and composition of aerosols in Sydney and better understand the characteristics of gas-phase secondary aerosol precursors. The dataset was published by Keywood et al. (2016) and the final report was provided by Cope et al. (2014).
The supplement related to this article is available online at:
The authors declare that they have no conflict of interest.
The NSW Environmental Trust has provided support for this study through the “Atmospheric Particles in Sydney: model-observation verification study”. The NSW Office of Environment and Heritage supported the Sydney Particle Study. We also thank the staff and management of Westmead Hospital for assistance during the field campaigns. This work was initiated when E. Dunne was undertaking her PhD studies at Monash University. E. Dunne thanks Monash University for a postgraduate scholarship. Edited by: Yoshiteru Iinuma Reviewed by: three anonymous referees