2.1 Swisens Poleno

We used the first unit of the commercially available Poleno device developed by Swisens AG (Switzerland). The device provides in-flight measurement of particle shape, size, and fluorescence using various light sources and detectors. The schematic structure of the device is presented in Fig. 1. Laser light scattering triggers the measurement, together with providing a first estimation of particle size, velocity, and alignment by combining the information of two trigger lasers. Following the trigger, two focused images at 90 ^∘ from each other are reconstructed using digital holography as in Berg and Videen (2011), and UV-induced fluorescence produces information regarding the particle composition. UV-induced fluorescence lifetime and spectra are measured at three different excitation wavelengths (280, 365, and 405 nm) using five measurement emission windows between 320 and 720 nm. Finally, a measurement of the time-resolved optical polarization characteristics of the particle is acquired before it exits the device.

Figure 1Measurement principle of the Swisens Poleno (courtesy Swisens AG).

Since atmospheric pollen concentrations are typically low compared to other aerosols (on the order of 10–10 000 pollen grains per cubic metre), a high sampling rate is necessary to sample pollen effectively. This is all the more relevant since the threshold for allergic response is typically even lower, varying depending on the taxa, from just a few grains to a few tens of grains per cubic metre. In the Swisens Poleno, this level of sampling is achieved using a concentrator based on the principle of a virtual impactor that enables an effective flow rate of 40 L min⁻¹. An hourly time resolution providing concentrations relevant for pollen exposure thresholds can thus easily be achieved with this sampling rate. The drawback, however, is that the saturation level occurs at coarse particle (>10 µm) concentrations above 30 000 particles per cubic metre. Note that size-dependent particle loss occurs in the concentrator; corrected concentration factors were determined from the controlled chamber experiments presented at the end of the paper. A Sigma-2 inlet was chosen to protect the device from precipitation as its sampling, in particular the role of wind speed and particle size, has been documented in detail (Verein Deutscher Ingenieure, 2013).

2.2 Calibration dataset

A large (>750 particles per pollen taxon) calibration dataset was collected for eight different pollen taxa using online measurements from the Swisens Poleno device. The taxa were chosen to present a good range of particle size – from small nettle pollen grains through to large pine pollen grains – and morphology. Note that the list includes taxa relevant for pollen allergies such as two different Betulaceae, Dactylis glomerata as a proxy for grasses, and ash. These samples were used to train a machine learning algorithm applied to identify the different pollen taxa. Only one particle type was calibrated at a time, allowing the data points to be labelled directly, although dirt, debris, and agglomerates needed to be eliminated from the dataset manually through visual inspection of the holographic images. To generate a large number of events without saturating the detector, pollen samples were continuously aerosolized using sound waves in a closed chamber around the detector inlet. Figure 2 shows examples of the reconstructed images generated for the calibration dataset. The pollen identification presented here is based just on these reconstructed images since they are expected to contain enough relevant morphological information to permit sufficient identification of the taxa of interest. Fluorescence and lifetime measurements are expected to be pertinent for extending the scope of the device to characterize other bioaerosols (e.g. spores) and pollutants. The dataset obtained includes 12 234 pollen grains (two images per grain) and is summarized in Table 1; 80 % of this dataset was used for algorithm calibration and 20 % for validation purposes. The images are greyscale and have a resolution of 200 pixels × 200 pixels. Each pixel represents a 0.56 µm by 0.56 µm physical domain.

Figure 2Reconstructed holographic images from the Swisens Poleno for different pollen taxa: 1. Ambrosia artemisiifolia, 2. Corylus avellana, 3. Dactylis glomerata, 4. Fagus sylvatica, 5. Fraxinus excelsior, 6. Pinus sylvestris, 7. Quercus robur, and 8. Urtica dioica.

Download

Table 1List of pollen taxa used to train the classification algorithm, including the number of events used for training and validation of the machine learning algorithm for each taxa. Note that all pollen were in a dry state.

Download Print Version | Download XLSX

2.3 Shape analysis for pollen detection

A large range of coarse aerosol particles were seen in the events recorded by the Swisens Poleno. To provide a clean dataset to the pollen classification algorithm, the pollen particles needed to be discriminated from all others. In principle this can be done by applying thresholds to the confidence estimates provided by deep-learning pollen-classification algorithms; however, this simple method did not yield the required level of accuracy. An additional step was therefore implemented in the algorithm (thus becoming a two-step classifier), using shape analysis to discriminate between pollen and non-pollen particles prior to applying the full pollen classification.

In general, unbroken biological particles tend to have a smooth, convex shape, while dust, debris, or other nonbiological particles have rougher, more chaotic shapes (see, for example, Fig. 3). Two deterministic image analysis routines were developed and evaluated to select the best available method for distinguishing pollen from other detected particles. Both use the contour of each particle, which is extracted from the reconstructed holographic images in three steps: (1) pixels are separated into two classes using the Otsu binarization algorithm (Otsu, 1979), which is based on a dynamic intensity threshold; (2) the largest cluster corresponding to the particle of interest is then identified; and (3) a convolution operation extracts the contour of the particle.

The first routine for biological particle identification uses the OpenCV2 library (Bradski, 2000) to fit (in a least-squares sense) an ellipse to each contour (Fitzgibbon et al., 1999). As a feature for biological particle identification the fraction f_c of the contour located further than a certain distance from the fitted ellipse is considered (red pixels in Fig. 3). For pollen grains, this value is typically low, while for more fragmented particles this fraction can reach up to 60 % of the contour (0 % and 46 % respectively for the examples shown in Fig. 3).

Figure 3Illustration of the image analysis routines applied to (a–c) a pollen grain (Quercus robur) and (d–f) coarse particulate matter.

Download

The second method is based on the fractal dimension, which characterizes the state of self-similarity or roughness, and is also estimated from the particle contour. Such analysis of natural objects was first introduced by Mandelbrot (1983) and is now widely used in a variety of applications, such as plant leaf or sediment identification (Backes et al., 2009; Orford and Whalley, 1983). We make use of the so-called box-counting algorithm to estimate the fractal dimension of the holographic images because of its computational simplicity (Theiler, 1990). This method consists of splitting each image into grid boxes and counting the number of boxes N(s) that contain a fragment of the perimeter of the particle, where s is the box size. By repeating the procedure for decreasing values of s, the fractal dimension is then estimated by computing the slope of log(N(s)) over $\text{log}\left(\frac{\mathrm{1}}{s}\right)$ (see Fig. 3c).

The performance of these two methods (ellipse fitting and fractal dimension) is compared using the reference dataset, which contains images from all calibrated pollen taxa (1640 particles) and manually selected coarse aerosols (1554 non-pollen particles), measured during late spring and summer 2018 in Payerne, Switzerland. Smaller particles were filtered out prior to this comparison to keep only particles roughly in the pollen size range (10 to 200 µm). Note that only a reduced subset of pollen calibrations was used to keep a balanced dataset with respect to the non-pollen category.

Using manually labelled data to verify the output of the classification algorithm, we performed a grid optimization to find the set of parameters that best discriminates between pollen and other airborne particles; that is, a filter that simultaneously provides sufficient recall (ensuring that most pollen grains are classified as pollen) and precision (ensuring that only pollen grains are classified as pollen).

2.4 Pollen classification using deep learning

Developments in computer hardware have made it possible to perform efficient training of complex artificial, or “deep”, neural networks; their use in image recognition problems is the iconic application of deep learning. Mimicking the visual cortex, a series of so-called convolutional layers identify relevant patterns and concentrate the information diluted over a large image. Extracted features are then used as input for fully connected layers of artificial neurons, which combine the features to determine associated labels for each image. This technique is part of the family of supervised-learning algorithms; networks need to be trained using images for which the label is known. We used the open-source software library Keras (Chollet, 2015) with TensorFlow (Abadi et al., 2015) as computational back end to implement the pollen identification algorithm.

Figure 4Vision model based on the VGG16 architecture (Simonyan and Zisserman, 2014) as used here for pollen classification.

Download

The model used to classify pollen grains is based on the VGG16 architecture (Simonyan and Zisserman, 2014), which has successfully been applied to a wide range of different image classification problems (Russakovsky et al., 2015). The basic model is shown in Fig. 4, with the vision model being applied separately to the two orthogonal images and the output then being processed with two fully connected layers. This ensures that the model is able to use the information from both images. For the final layer, softmax activation is used to map the network output to a probability distribution (Chollet, 2015). The predicted pollen label is determined by taking the most probable class. Note that probability information is also useful since the plausibility of the final classifications can be easily verified (Crouzy et al., 2016; Šauliene et al., 2019). Furthermore, although not carried out here, a threshold could be applied to the classification when performing operational measurements to retain only the pollen grains classified above a sufficient confidence level.

3.1 Pollen identification

The discrimination between pollen and other coarse aerosols is evaluated in Fig. 5 in the form of a normalized confusion matrix for each of the two image analysis algorithms. Each line in Fig. 5 is normalized to 1 and the values along the diagonal provide the recall for each category.

Figure 5Normalized confusion matrix summarizing the performance of (a) the ellipse-based classifier and (b) fractal-based classifier. “Pollen” refers to a mix of different pollen taxa, while “Non-pollen” encompasses all other coarse aerosols.

Download

The convexity hypothesis for unbroken biological particles seems to hold particularly well for pollen grains. Indeed, in nearly all cases one of the two images has an almost perfect elliptical fit for pollen particles, which translates into only a very small fraction of contour pixels that strongly deviates from the optimal ellipse, i.e. f_c≈0. Other particles most often present values of f_c≥20 % for both images. Good precision can be obtained by using a low f_c threshold, albeit at the expense of recall. The best results (achieving both good precision and recall) were obtained by imposing different thresholds on the two images: at least one of the two images needs to satisfy a hard condition on the value of f_c, while the other should not present excessive deviation from its fitted ellipse. When using optimal values of the two f_c thresholds, an overall accuracy of 96 % was achieved for the discrimination of pollen from other particles.

Visually, the contours of pollen grains clearly exhibit smoother shapes than non-pollen particles. However, it can be noted from Fig. 5 that the fractal dimension method did not function as well as the ellipse fitting one. While an overall accuracy of 77 % is reached, the number of non-pollen particles classified as pollen (false positives) is still too high to ensure a satisfactory classification in the second step of the deep-learning algorithm. Note that the ellipse fitting algorithm alone performs better than any combination of the two methods (not shown here).

Estimating the fractal dimension of an object from a holographic image is sensitive to the image resolution, which is thought to have a significant influence on the determination of the fractal dimension (see Baveye et al., 1998). Indeed, more detailed images tend to improve the estimation of the fractal dimension of an object as more details of the contour become apparent. Furthermore, the binarization process (i.e. reducing the greyscale holographic image to black and white) may also affect the box-counting calculation. Should higher-resolution images be available in future versions of the Poleno, the fractal dimension method may be worth implementing. At this point, given the better accuracy of the ellipse-fitting technique, we utilize this method in the final algorithm.

3.2 Pollen classification

Once a particle has been identified as a pollen grain it needs to be classified into the right taxa. Using the convolutional neural network (CNN) described in Sect. 2.4, each airborne particle is assigned a taxa with a corresponding confidence level of prediction. Results from the classification model are presented as a normalized confusion matrix in Fig. 6. The sum of each line is normalized to 1, and the diagonal values indicate the recall for the different pollen taxa.

Overall the classification algorithm performs very well, with six of the eight pollen taxa being classified with an accuracy of over 90 %. The exceptions are Corylus, which is confused in 10 % of cases with Fraxinus, and Dactylis, which is confused 22 % of the time with Corylus. Note that in this regard the problem presented to the algorithm is somewhat artificial; Corylus and grass pollen are not likely to be simultaneously present in the atmosphere in concentrations relevant for pollen allergies. Nevertheless, a larger mix of pollen taxa is likely to be observed in reality, highlighting the need for further developments to the classification algorithm using a larger number of species and including fresh pollen. In this line, it will be essential to include birch in the identification algorithm. This may, however, prove to be challenging given the morphological similarities of the members of the Betulaceae family.

Figure 6Normalized confusion matrix for the pollen taxa identification, the second step of the classification algorithm.

Download

To better understand the functioning of the neural network, Fig. 7 presents activation heat maps (Kotikalapudi and contributors, 2017) of pollen particles. These show which parts of the image the network focuses on to make the taxa prediction; in our case, strongly on the particle shape. This is apparent in the heat maps (Fig. 7), as the highest activation regions follow the outline of the pollen grains. This may appear to be an obvious result but confirms the validity of the CNN step of the classification algorithm and indicates that the predictions are based on a physical feature of the particle and not on some other information embedded in the images. This verification is essential, as differences in light intensity or the presence of dust on lenses could lead to discrimination between calibrations not based on pollen morphology but on artefacts.

Figure 7Visualization of the areas on which the convolutional neural network for pollen classification focuses.

Download

Figure 8(a, b) Concentrations (5 and 10 µm PSL spheres) scaled to the METAS reference measurements in UTC. (c, d) Comparison of fluorescence measurements. Solid lines are the reference fluorescence intensities measured by the Max Planck Institute of Chemistry presented in Könemann et al. (2018). Median measurements from the Poleno are shown with error bars (interquartile range). Each excitation wavelength is scaled individually (see text for details).

Download

Although there are some limitations to the use of dry pollen for model training purposes, the performance obtained suggests that holography alone is sufficient to distinguish between different pollen taxon. Combined with the results of the previous section on pollen identification, we propose a two-step approach for operational pollen monitoring using digital holography, first applying classical image analysis to identify pollen and subsequently using deep learning to classify these particles into individual pollen taxa. As mentioned in Sect. 2.4, the identification algorithm provides a measure of confidence in addition to the predicted label. Note that raw results are presented in the confusion matrix (Fig. 6); in an operational setup confidence thresholds could be used to increase precision further. Due to the large sampling of such an automatic system, a certain loss of particles from introducing confidence thresholds can be accepted without losing statistical significance of the sampling.

3.3 Reference particle counts and fluorescence observations

The focus of this study was to assess the performance of the Swisens Poleno in terms of pollen identification. While this is key, it is equally as important to accurately quantify airborne pollen concentrations. At present, this remains a difficult task since no method, standardized or other, exists to aerosolize a known quantity of a known pollen taxa. Pollen grains are both considerably larger than other, nonbiological aerosol particles and relatively fragile, so producing homogenized airborne concentrations is currently not possible with conventional techniques.

To assess the accuracy of the particle concentrations obtained with the Swisens Poleno, a measurement campaign was carried out at the Swiss Federal Institute of Metrology (METAS). The custom-made facility at METAS has been described in detail in Horender et al. (2019). The aim was to compare the Poleno device with reference particle concentrations and fluorescence observations in a controlled calibration chamber using polystyrene latex (PSL) spheres. Different sizes, ranging from 0.5–20 µm, were tested along with three types of fluorescent PSL (blue 2.07 µm, plum purple 2.07 µm, and red 2.07 µm) to provide a first insight into the quality of the fluorescence measurements. For each size, the concentrations measured by the Poleno were compared to the reference concentrations for approximately 20 min. The fluorescent PSL used here have been fully characterized by the Max Planck Institute for Chemistry (MPIC) (Könemann et al., 2018) for a large range of excitation wavelengths. Those corresponding to the Poleno excitation wavelengths have been reproduced in Fig. 8 and serve as a reference for the fluorescence measurements. Since fluorescence intensity is measured in arbitrary units ($\mathrm{a}.\mathrm{u}.$), the fluorescence measured by the Poleno (filled dots) is scaled to the MPIC reference values (solid lines) using the maximum for each of the five emission windows located between 335 and 700 nm.

The results presented here are encouraging, both in terms of particle concentration and fluorescence measurements. The Poleno seems to follow the fluctuations in terms of particle concentration very well, with Pearson correlation values of 0.905 and 0.916 for the 5 and 10 µm sizes respectively (see Fig. 8). Similar results are observed for the other particle sizes tested (not shown), indicating that the Poleno measures the size of the certified PSL particles correctly. It is important to note, however, that the Poleno values have been scaled to the METAS values since the particle concentrator is size selective, with larger particles being better sampled. Once a size-scaling curve has been established it can be effectively applied to all future measurements, which is a significant improvement compared to the current practice of deriving scaling factors for automatic pollen monitors from Hirst-type measurements (Crouzy et al., 2016; Šauliene et al., 2019). The systematic analysis of the efficiency of the concentrator goes beyond the scope of this paper but will be described in future work. The reproducibility of the scaling factors obtained was verified by repeating the experiments with the 2 µm particles three times.

Despite the fact that the Poleno does not measure a continuous fluorescence emission spectrum, Fig. 8 confirms that it already provides an insight into the shape of the spectra for the different excitation wavelengths. The Poleno fluorescence signals agree well with the offline reference measurements performed at MPIC (Könemann et al., 2018) for all five emission windows and combined with the holographic images, potentially provide the opportunity to extend the number of particle types that can be recognized (e.g. further pollen taxa, spores, or pollutants).