Journal cover Journal topic
Atmospheric Measurement Techniques An interactive open-access journal of the European Geosciences Union
Journal topic
Atmos. Meas. Tech., 11, 1417-1436, 2018
https://doi.org/10.5194/amt-11-1417-2018
Atmos. Meas. Tech., 11, 1417-1436, 2018
https://doi.org/10.5194/amt-11-1417-2018

Research article 12 Mar 2018

Research article | 12 Mar 2018

# A simple biota removal algorithm for 35 GHz cloud radar measurements

Non-hydrometeor echo filtering technique
Madhu Chandra R. Kalapureddy1, Patra Sukanya1,3, Subrata K. Das1, Sachin M. Deshpande1, Govindan Pandithurai1, Andrew L. Pazamany2, Jha Ambuj K.1, Kaustav Chakravarty1, Prasad Kalekar1, Hari Krishna Devisetty1, and Sreenivas Annam4 Madhu Chandra R. Kalapureddy et al.
• 1Indian Institute of Tropical Meteorology (IITM), Dr Homi Bhabha road, Pashan, Pune 411008, Maharashtra, India
• 2ProSensing Inc., 107 Sunderland road, Amherst, MA 01002, USA
• 3Savitribai Phule Pune University, Pune 411007, India
• 4VFSTR University, Vadlamudi 522213, India
Abstract

Cloud radar reflectivity profiles can be an important measurement for the investigation of cloud vertical structure (CVS). However, extracting intended meteorological cloud content from the measurement often demands an effective technique or algorithm that can reduce error and observational uncertainties in the recorded data. In this work, a technique is proposed to identify and separate cloud and non-hydrometeor echoes using the radar Doppler spectral moments profile measurements. The point and volume target-based theoretical radar sensitivity curves are used for removing the receiver noise floor and identified radar echoes are scrutinized according to the signal decorrelation period. Here, it is hypothesized that cloud echoes are observed to be temporally more coherent and homogenous and have a longer correlation period than biota. That can be checked statistically using  4 s sliding mean and standard deviation value of reflectivity profiles. The above step helps in screen out clouds critically by filtering out the biota. The final important step strives for the retrieval of cloud height. The proposed algorithm potentially identifies cloud height solely through the systematic characterization of Z variability using the local atmospheric vertical structure knowledge besides to the theoretical, statistical and echo tracing tools. Thus, characterization of high-resolution cloud radar reflectivity profile measurements has been done with the theoretical echo sensitivity curves and observed echo statistics for the true cloud height tracking (TEST). TEST showed superior performance in screening out clouds and filtering out isolated insects. TEST constrained with polarimetric measurements was found to be more promising under high-density biota whereas TEST combined with linear depolarization ratio and spectral width perform potentially to filter out biota within the highly turbulent shallow cumulus clouds in the convective boundary layer (CBL). This TEST technique is promisingly simple in realization but powerful in performance due to the flexibility in constraining, identifying and filtering out the biota and screening out the true cloud content, especially the CBL clouds. Therefore, the TEST algorithm is superior for screening out the low-level clouds that are strongly linked to the rainmaking mechanism associated with the Indian Summer Monsoon region's CVS.

1 Introduction

Table 1KaSPR specifications.

2 System, data and methodology

This investigation employs vertically oriented Doppler spectral moment profile observations of IITM's Ka-band scanning polarimetric radar (KaSPR) for the study of vertical cloud structure. Specifically, KaSPR employs an improved variation of the well-known linear frequency modulated (LFM) pulse compression technique. The KaSPR pulse compression technique is amplitude taper (window) (using a Tukey taper with 0.7 taper coefficient) on the transmitted LFM pulse and the compression is implemented in the digital signal processor system using a least mean squares filter (Mudukutore et al., 1998) to achieve much improved (lower) range side lobes compared to untapered LFM pulse compressed with a matched filter. Thus, KaSPR uses the 3.3 µs pulse length with 10X LFM chirp compression with effective range resolution of 50 m (i.e. compressed to 0.33 µs) and sampling in range (range gate spacing) at every 25 m with pulse reception frequency of 5 kHz. So, the radar data set used for this work has the range samples at every 25 m with first range gate altitude at 942 m a.g.l. KaSPR has been providing high-resolution (25 m and 1 s) resourceful measurements of cloud and precipitation at a tropical site (Mandhardev: 18.0429 N, 73.8689 E; 1.3 km a.m.s.l.) on a mobile platform since June 2013. Its other main technical features are given in Table 1. KaSPR possesses a sensitivity of $\sim -$60 (45) dBZ at 1 (5) km and is therefore sensitive to the cloud droplet. According to T-matrix Rayleigh computations, a single 0.1 mm drop size of target at  35 GHz may have the reflectivity of $\sim -\mathrm{60}$ dBZ, whereas it needs near 63 drops each of 0.05 (0.01) mm size (or near 1 000 000 drops of 0.01 mm size) to give the same reflectivity. Furthermore, if there are 5000 (pulses per second) hits on the target in the radar scattering volume in 1 s, the mean of those 5000 samples at a range bin (height) will be affected by the mean characteristics of the target, such as composition, orientation, number density and kinematics associated with it. Therefore, it is safer to assume that the atmospheric or meteorological targets (in this case cloud particle) are distributive in nature and passive in the sense that their motion and/or orientation are in resonance with the kinematics of the background atmosphere. By comparison, birds and insects are point targets in nature and active in the sense that they can change their motion, direction and orientation within a few seconds. This leads to the irregular nature of intermittent or spurious radar returns that are characteristic of atmospheric biota due to the much smaller decorrelation time associated with them. This study utilizes the high-resolution profile of cloud radar reflectivity factor (Z) to construct the cloud vertical structures by filtering out the returns from the noise and biota.

Figure 1(a) Ten vertical-looking cloud-radar-measured sample reflectivity height profiles on 27 April 2014 during 23:03–23:08 UT. S0 to S5 are the theoretical noise-equivalent reflectivity curves with their respective threshold values in bracket. HTI plot of (b) the same reflectivity profile for the duration of 306 s (c) screened out reflectivity profile for the receiver noise floor and the biota (insects) using running average constrained with standard deviation (d) constrained with NER (S5).

Figure 1a represents the height profiles of zeroth moment (radar echo peak power) based Z on 27 April 2014 at 23:03 UT with various theoretical radar sensitivity (noise-equivalent reflectivity, NER) curves (S0–S5; the range profile correction with the start range sensitivity value of reflectivity, i.e. r2×Zstart range, where r is range or height and Z is reflectivity; e.g. for S1, Z is 60 dBZ). These different NER or sensitivity curves are utilized to qualify the observed radar returns that are indeed above the NER, the inherent radar receiver noise level. The receiver noise level is the inherent thermal noise associated with electronic components in the receiver chain, as well as other sources that are taken into account through the noise figure (Table 1), and it remains approximately constant over the length of the pulse returns. However, range correction is intuitive in the radar equation due to the decrease in echo signal strength with increasing height (for vertical orientation). In order to determine the noise range in every range bin, S0 to S5 are computed and overlaid on Z. This allows for identification and characterization of the signal that overlays the background system noise level. As discussed earlier, the signal at any level may have contributions due to either volumetric meteorological cloud particulates or strong non-meteorological/non-hydrometeor point targets (e.g. biota). In Fig. 1a the echoes at  3.7 km and below 2 km can be marked as cloud and biota, respectively, as they exceed the profile S5. The noise variations around 15 dB are mostly confined between S0 and S2 with S1 as mean NER. Contrasting echo texture associated with the cloud and atmospheric biota is evident from the height–time–intensity (HTI) plot of Z in Fig. 1b. This is a weak cloud case with reflectivity 38 dBZ at  3.7 km altitude with the presence of intermittent, non-homogeneous echo texture from the biota below 2.7 km altitude. A similarly weak cloud case of $-\mathrm{38}±\mathrm{2}$ dBZ at 5.4 km altitude is confirmed as cloud with the sharp increase in relative humidity of  80 % at that altitude by collocated radiosonde (GPS–RS) measurements but is not shown here (see Fig. A2). Biota echoes are observed to be confined most densely below 1.7 km and fall in the reflectivity range of 50 to 20 dBZ. The observed standard deviation (SD) is always more than 2 dBZ and can indirectly infer the decorrelation period of  4–5 s (returns due to biota are observed to vanish at an interval of  3–8 s; see the lower part of the HTI plot). In the decorrelation period, it is hypothesized here that the running mean and SD of  4 s sliding window reflectivity profiles work in identifying all non-hydrometeor returns. Furthermore, the time coherence of radar returns at every range sample can be checked every 4 s as a window period to infer the echo power decorrelation time or degree of coherence period associated with biota returns based on the SD of Z value. Two sensitivity (S1 and S5) tests have been performed on Z profile to quantify noise floor, biota and the meteorological cloud returns. All the tests have been affected by the presence of non-meteorological echo due to biota even though these are mostly present in the ABL. Reflectivity values associated with the cloud boundaries are very faint and fall within or close to system noise floor by 2–5 dBZ. The profile S5 seems to be better at screening out the cloud echoes 10 dBZ above the radar receiver mean noise floor, but this can eliminate a significant portion of the weakest reflectivity area at the cloud edge (Fig. 1d). Apart from clouds, biota also show higher reflectivity values than S5. Figure 1d is similar to Fig. 1b except it is completely screened out for cloud by applying the typical threshold of radar system sensitivity profiles S1 and S5. In addition to this, in the case of Fig. 1c, a contiguous set of four reflectivity profiles has been considered for computing running mean and SD. The method used to generate Fig. 1c is the main objective of this paper and is outlined by the flowchart in Fig. 6. This method will be explained below; Sect. 3 contains thorough information. In this case, insect reflectivity values are similar to those of the cloud but their altitude levels are significantly different. The contribution due to biota can therefore be removed by S5 curve thresholding, leaving the contribution due to clouds untouched (Fig. 1d). Thus, for the simultaneous presence of cloud and biota echoes at around same altitude this NER method fails to identify the contributions separately. This NER method also fails whenever sharp reflectivity changes occur, usually seen with cloud boundaries and edges. This issue therefore demands the development of a robust algorithm that explores the fundamental difference between cloud and biota returns so that it could be identified and separated out automatically.

In order to make the algorithm more robust for running it automatically, a close re-inspection of Fig. 1b shows that cloud returns are much more regular and near homogeneous when compared to biota returns, which appear to be spurious or intermittent in occurrence. Therefore, the NER criterion works reasonably well for the case of homogeneous, isolated stable cloud layers but its robustness is questionable whenever there are vigorous and quick changes associated with cloud edge and/or structure (explained in the discussion of clouds 1–2 in Fig. 5). An additional criterion makes the current algorithm robust for complete revival of cloud information from the Z observations by utilizing the decorrelation periods of biota (close to 3–5 s). During this time interval significant changes are not seen within the cloud. To explore this fact, in the next section the same weak low-level cloud case has been chosen to understand the coherence period associated with cloud and biota.

Figure 2(a) Same as Fig. 1a but for 220 profiles. Extra NER curves, here in grey (S04, S14 and S54), are computed on the basis of the point target radar equation (i.e. r4×Zstart range, where r is range and Z is reflectivity; e.g. for S04, Z is 68 dBZ). (b) HTI plot of Z profiles. The smoothly varying homogeneous cloud layer is at altitudes of 3.5–3.8 km and sharp, rounded and spurious kinds of echoes below 2.7 km are due to biota.

3 Results and discussions

Figure 3(a) Same as Fig. 1a but for 105 profiles. (b) Mean and (c) standard deviation of 15 profiles of Z pertinent to cloud height region (3.5–3.9 km) and (d) and (e) same as (b) and (c) but pertinent to biota height region (0.9–1.5 km).

Figure 4Same as Fig. 3 but for total duration 35 s; the mean and standard deviation profiles are for every 5 s interval.

Figure 5Same as Fig. 3 but for total duration 10 s; the mean and standard deviation profiles are for 4-point-moving average.

To investigate the similarities and contrasting features associated with various contributions to the cloud reflectivity profile, it is important to explore further the case of Fig. 1. Statistical echo coherence periods associated with three types (cloud, biota and noise) have been computed for their identification and separation. Both the cloud in the  3.7 km narrow region and biota returns below  1.5 km in Fig. 3 are evident above the maximum noise level. Both cloud and biota parts of the Z profiles are expanded to allow for review of the mean (Fig. 3b and d) and SD (σ; Fig. 3c and e) of Z for every set of consecutive 15 profiles. Figure 3b shows that the patterns of the seven mean cloud reflectivity profiles are organized and more consistent or correlated to one another during 105 s; this is in comparison to less organized reflectivity profiles due to biota that are much less consistent or correlated with one another in Fig. 3d. Moreover, the corresponding seven σ profiles show differences for cloud that are less than 1.5σ (Fig. 3c). By comparison, differences in profiles due to biota are more than 4.0σ most of the time (Fig. 3e). It is seen that the mean cloud reflectivity peak values gradually extend from 3.7 to 3.8 km, where the corresponding SD values are less than 1σ. In order to further test the minimum decorrelation time associated with cloud and biota, the averaging time is reduced to a set of five profiles (5 s) with the same data (see Fig. 4). In this case, Fig. 4c also depicts σ for all seven mean cloud reflectivity profiles below 1.5 dBZ with peak < 1σ. This shows that the volumetric distribution nature of cloud particles is statistically more homogeneous or show less dispersion. However, Z values associated with biota show random behaviour with significant dispersion > 1.5σ dBZ (Fig. 4e). This high dispersion in the Z values shows that the echo due to biota decorrelates quickly within  5 s time interval (see Fig. 4d–e). It is seen in Fig. 3 that for vertical levels from 0.9 km to 1.5, the sharp peaks in reflectivity profiles and strong dispersion of > 3σ dBZ are associated with the return from biota. This is attributed mostly to the observed intermittent point target nature of biota echoes plausibly due to the rambling or meandering motion of biota within the radar sampling volume. Moreover, the inherent radar system noise (random in nature) dispersion is observed between the cloud and biota (1.5–3.0σ dBZ). It is evident from the top panels of Fig. 3–4 that cloud reflectivity profiles show a relatively more consistent trend and correlation among the contiguous mean profiles computed from the set of 15 Z profiles than computed from the five profiles. This may be mainly due to the homogeneities or inhomogeneities associated with the manner in which the sample points are chosen from the data set. Therefore, in order to preserve the real time sequence of observations for the study of cloud evolution as well as to recover underlying smooth trends pertinent to natural clouds, a four-point moving or running average is applied on the time series of Z data instead of deriving a simple average. The 4 s moving average time is optimal for yielding the best cloud results (Fig. 5) by characterizing the cloud to biota echoes as coherent to incoherent property during the moving average period. By this four-point running average, biota echoes become incoherent due to their short decorrelation period ( 4 s), whereas those echoes decorrelating over longer periods indicate the presence of clouds. To understand the degree of dispersion, the absolute deviations in mean and median values also must be analysed along with σ. The mean absolute deviation is slightly smaller than σ, σ∕1.253, whereas the median absolute deviation is smallest, σ∕1.483. This work makes use of the statistical mean and σ but, by using the above relation, one can relate the present results with other central statistical tendencies of data distribution. Next, the filtering of noise and biota from the presence of cloud using the cloud radar reflectivity profile will be explored. The segregation has been carried out using theoretical radar echo sensitivity curves and statistically computed echo decorrelation periods and finally tracking the cloud echo peak to its adjacent sides until it is close to the S1 profile for the cloud height. The above set of tasks, theoretical echo sensitivity and observed echo-based statistics for cloud height tracking (TEST), is repetitively performed on the cloud radar Z measurements under an algorithm whose flowchart is in Fig. 6. The algorithm used in this work is named as TEST and can be summarized below:

1. Wherever the moving mean Z values in the profile are equal to or above the S5, this can be qualified as cloud or biota echo. This step ensures removal of the system noise floor.

2. Those altitude regions of the qualified echo are then further scrutinized to identify clouds using the minimum thickness of greater than 100 m (to strictly avoid biota that are found to extend less than two range gates, each of size 50 m) and mean standard deviation below 1.5σ dBZ.

3. In order to keep the identified cloud's structure intact, the identified cloud peak(s) are tracked back on either side (towards upper and bottom heights) up to around (preferably 1–2 dBZ) the mean noise profile S1.

4. Finally, the remnant isolated echo floor is not related to cloud but occurs due to the abrupt disconsolation at the subsequent running average by the restrictions of step 2. In order to remove those, the frequency count of Z profile has been constrained at those height levels where the Z frequency count falls below 5 % of total measurement.

Figure 6TEST algorithm flow chart that identifies and filters out the biota and noise echoes for screening out the cloud contributions with the Z measurements.

The first two steps ensure the identification and removal of non-hydrometeor contributions from the cloud radar reflectivity profile, which can then be used for inferring unbiased vertical cloud structure. However, these two steps are insufficient for recovering the weakly echoing cloud boundaries associated with the sharp reduction in cloud droplet size and concentrations. To have intact cloud height information (step 3), identified cloud echo peaks need to be backtracked along the either sides on the reflectivity profile until its value falls close to the mean noise floor for radar receiver. It is interesting to note that the cloud echo regions are always stronger and above the mean noise fluctuations, i.e. S1. Therefore on the left side of the curve, S0 to S1 always appear as a void region in the two-dimensional reflectivity plot wherever there are clouds present, no matter how weak or strong (just below 4 km in the left panel of Figs. 1 and 3). This causes sharp boundary gradients between cloud and noise in the vertical profiles of Z and hence the corresponding σ. This can be used as a visual criterion for detection of cloud.

Figure 7 is similar to Fig. 1 but it represents a multilayer pre-monsoon cloud system for the period 12:00–12:05 UT, 29 May 2014. Various labelled altitude regions (biota, noise and cloud) of the vertical reflectivity structure show typical mean features that can broadly classify the returns into cloud and non-cloud (biota and noise) portion. Furthermore, Fig. 7 shows the typical variety of cloud layers existing within the vertical structure of tropical cloud as well as morphological features pertinent to pre-monsoon thunderstorm activity. The cirrus layer at 12–14 km shows gradual structural change with peak reflectivity values of  5 dBZ. Here, the high reflectivity values contribute to form deep, single convective clouds by merging with the cloud layer that exists at lower heights.

Figure 7(a–c) Same as Fig. 1a–c but on 29 May 2014 during 12:00–12:05 UT for the duration of 306 s. Statistics correspond to the labels on the Z profile can be seen in Table 2.

Figure 8Time series of the mean and standard deviation (SD) of Z (a–b) for two biota and four noise floor regions as per Table 2. Bold solid lines are the five-point-running mean over the actual time series data (lines with symbol). (c–d) Same as (a–b) but for the cloud regions as per Table 2.

Figure 9(a) Uncorrected mean reflectivity profile on 29 May 2014 during 12:00–12:05 UT, superimposed with curves S1 (dashed red line) and S5 (solid green line). (b) Histogram of whole uncorrected Z profile. (c) Three sub-panels of low (< 3.6 km), middle (3.6 km > =ht < 8.6 km) and high (> = 8.6 km) altitude regions. Each histogram peak in the three top right sub-panels is mapped onto the corresponding three peaks, indicated by dotted circles in panel (b). This shows that the noise clearly suppresses the meteorological information. (d) Same as Fig. 9a but corrected by filtering out noise and biota. (e) The correction applied to Z profile pops up the true meteorological cloud reflectivity distribution, which can be seen in panel (f) (same as c but for corrected Z).

Figure 8a and b reveal the reflectivity time series associated with the labelled non-cloud and cloud portion of Table 2, respectively. Noise and biota show maximum 2 dBZ fluctuations around the four-point-running mean reflectivity, whereas for biota the max fluctuation is 3–5 dBZ (bold solid line). It can be understood that noise values increase gradually with altitude with σ values  2.3 whereas sharp boundary gradients associated with biota and ragged shallow cloud regions (clouds 1 and 2 in Fig. 7) also show higher σ values > 3 dBZ. Stable or layered cloud regions (clouds 4 and 5 in Fig. 7) show significantly standard deviations below 2σ (dBZ). Further, it is interesting to examine the time series plots for the contrasting variations between the biota and noise and cloud regions with Fig. 8a and b. The range of dBZ variability is 4–10 dBZ for biota and 2–4 dBZ for noise and less than 1 dBZ for cloud within an interval of 5–10 s. The corresponding variability in SD is observed to be 4–10σ for biota, 1.5–3.5σ for noise and  1σ for cloud (< 1σ for cloud peak) except for weaker cloud regions. These statistical characteristics of all types of observed cloud echoes have been tabulated in Table 2.

Figure 9 demonstrates the application of the work presented here and illustrates the significant differences between the uncorrected (Fig. 9a–c) and corrected (Fig. 9d–f) reflectivity profiles. The peaks in frequency distribution of uncorrected cloud reflectivity profiles at just below 50 dBZ, between 50 and 40 and just above 40 dB are the predominant contributions from noise (Fig. 9b). These noise regions severely bias the corresponding histogram frequency distribution at three different altitude levels associated with Johnson's trimodal cloud distribution (Fig. 9c). In order to infer the distribution of cloud reflectivity values in the various altitude regions pertinent to trimodal CVS (Johnson et al., 1999), the observed vertical structure is subdivided into warm or low (< 3.6 km), mixed or middle (3.6 km  altitude  8.6 km) and ice or high (> 8.6 km) phase and/or level clouds. The plots of uncorrected reflectivity distribution clearly show skewness towards lowest values of reflectivity (below 50, 40 and 30 dB for low, middle and high levels, respectively, seen in Fig. 9c). This is mainly due to the predominance of noise contribution, except for the low cloud regions, where the contribution of biota is also included. After applying the TEST algorithm the corrected reflectivity distribution peaks at 42, 35 and 22 dB for low, middle and high levels, respectively (Fig. 9f), reflecting the actual scenario of the cloud system. This method is simple and has the potential to bring out the statistically significant micro- and macro-physical characteristics from meteorological information (i.e. cloud) and hence help better characterize the CVS over a region.

Table 2Statistical mean and standard deviation of cloud radar reflectivity corresponds to the selected height regions, which are labelled, on the Fig. 7.

Figure 10Same as Fig. 7 but for vertical-looking KaSPR measurements at 04:00 UT on 28 August 2014 using (a) FFT processing and (b) PP processing on 15 min prior observations. PP case will be used to evaluate the polarimetric algorithm performance.

Figure 11HTI plots of (a) LDR, Φdp and KDP parameters pertinent to PP processed data of Fig. 10 and (b) biota-filtered reflectivity after applying corresponding polarimetric thresholds of the respective top panels.

In order to test the merit of the current algorithm for filtering out the non-hydrometeor contribution with Z profile, the parametric thresholds on pulse-pair (PP) processed Z and the polarimetric variable profiles of the cloud radar measurements have also been considered in place of the usual fast Fourier transformation (FFT) process. The FFT process is able to provide only the polarimetric parameter, i.e. LDR. Figure 10 is similar to Fig. 1, illustrating FFT (top) and PP (bottom) processed Z profiles on 28 August 2014; however, they are 15 min apart from one another (04:15 and 04:00 UT, respectively), which causes some dissimilarities in the observed three-layer cloud structure between the two plots (upper and lower panel). The minimum range of the noise floor in the Z profiles (2-D plot in the first panel) is seen to be greater for PP than FFT processing. The TEST algorithm performs in a similar way for both the FFT and PP processed Z profiles and is able to isolate the cloud structure as well as possible. Figure 11 further explores the polarimetric capability of the KaSPR to separate out the meteorological–hydrometeor contribution of Z by using critical thresholds on the PP-polarimetric measurements that correspond to Fig. 10b. Figure 11a stand for HTI plots of, three polarimetric parameters namely, LDR, Φdp and KDP. Computation of LDR is inherently limited to the cross-polar isolation of the radar system that is 27 dB for KaSPR. Hence, high LDR values above 17 dB are mostly seen with biota and low LDR values below 17 dB are seen with cloud. Low to lower LDR values (i.e. <−17 to 25 dB) are strictly confined within the peak values of co-polar reflectivity (>−10 dB) of cloud altitude regions,  8–10 km. Except for the inherent limitations associated with LDR, these results are in agreement with earlier reported results (e.g. Bauer-Pfundstein and Görsdorf, 2007, and Khandwalla et al., 2003). The LDR, Φdp and KDP threshold values are set below 17 dB, 56 and 15 km−1, respectively, and can be used to filter out biota from the corresponding Z profiles that are shown in Fig. 11b. The thresholds used for Φdp and KDP are subjective depending on the observed case for better filtering of biota. These polarimetric threshold methods are successful in filtering out the non-hydrometeor contributions, but they are bound to sacrifice the weaker portion of the cloud where polarimetric computations are not perfect. Thus, the polarimetric method is unable to preserve the weaker portions of the whole cloud regions, where the TEST method performs better (bottom right panel of Fig. 10). This further proves the efficiency of the proposed TEST method. TEST is implemented in the post-processing of high-resolution reflectivity measurements. The method developed here is far simpler and provides a superior solution to filtering out signal due to noise and biota and preserving cloud data in the form of pure hydrometeor reflectivity measurements which can be used to infer the true characteristics of clouds.

Figure 12(a) Same as Fig. 7b (uncorrected) and (b) same as Fig. 7c (corrected) but integrated for duration of 00:00–06:30 UT taken at an interval of  15 min on 21 May 2013. (c) Merged form of Fig. 9a and d (line profiles) and Fig. 9c and f (histograms) using uncorrected and corrected Z data of (a) and (b) can be seen in the grey and black bars, respectively.

Figure 13Cloud radar measurements of reflectivity (Z), LDR, spectral width (SW) with noise (a–c) and filtered out for noise using S5 curve (d–f), TEST algorithm screened output Z for clouds (g), (g) + biota filtering using LDR > 14 dB (h) and (h) + SW filter for biota using SW < 0.5 m2 s−2 (i).

Figure 12a demonstrates further application of the current work on filtered cloud reflectivity profiles (bottom plot) by considering the 6 h evolution of a variety of tropical cloud systems. On 21 May 2013, a typical convective cloud system present during the pre-monsoon season was observed. This event is composed of three systems: the first 3 h (00:00–03:12 UT) show stratiform cloud confirmed by bright band occurrence at an altitude of 4 km a.g.l.; convective system around 05:00 UT, which is a cumulus congestus initially; and above it cirrus (ice) cloud in the altitude range of 13–14 km. The screened-out reflectivity profile can therefore be utilized to fully characterize the trimodal cloud episode, as shown in Fig. 12b. The mean reflectivity profile with SD bars reveals the nature of important phase change regions associated with CVS. The change in cloud processes in the CVS is closely associated with the phase of cloud water that is strongly linked with the predominant change of temperature.

Figure 14Same as Fig. 13 but for typical high-density biota noticeed at 07:38 UT on 10 September 2013.

Figure 15Simple ACF-inferred decorrelation periods associated with base (green line), middle (blue line), and top (magenta line) height levels of shallow cumulus cloud and biota height levels.

The merit of TEST algorithm lies mainly in its ability to recover shallow cumulus clouds present with biota. Figs. 13 and 14 are the typical examples to demonstrate the performance of TEST on screening out the shallow low-level clouds. These are the concluding figures, in which TEST (first column panels), LDR (second column panels) and SW (last column panels) measurements are all considered. The top and middle row panels are unfiltered and filtered, respectively, for noise using the S5 curve. In the noise-filtered data, both cloud and biota echoes at the lower level can be seen separately with middle panels of Figs. 13–14. The higher-level biota are more organized just above 2.5 km. Shallow ABL cloud regions show LDR values <−20 dB whereas insects show varied LDR values in the range of 25 to 5 dB. Thus, LDR alone is not sufficient to remove all insects (Fig. 13e). Smaller echo coherence periods associated with biota are further confirmed with smaller spectral width values (< 0.3 m2 s−2; Fig. 13f). Higher spectral width values, on the order of  1 m2 s−2 of the cloud, indicate that the random motion of the smaller particles of cloud within the radar scattering volume is affected by the ABL turbulence. The discussed TEST algorithm (Fig. 13g) is able to screen out the cloud and filter out the biota significantly. Further, TEST fails to isolate relatively stronger biota returns existing within the cloud due to the lack of strong reflectivity gradient (both in short intervals of height and timescale), which fails to give the required high SD values to filter out biota. In order to ensure those echoes as biota and then to isolate those returns, the LDR values larger than 14 dB and SW values much smaller than 0.5 m2 s−2 have been chosen here. Isolated biota returns outside the cloud identified by TEST and the above critical thresholds with LDR and SW are found to be similar, except in a few places. It infers that the threshold value fails to filter out all biota returns due to either persistently low LDR or high SW values associated with those biota. However, it can be seen in Fig. 14 (similar to Fig. 13 but a typical case of high number density of biota on 10 September 2013 during 07:38–07:42 UT) that TEST alone is unable to remove biota (Fig. 14g), but by using LDR it becomes more promising (Fig. 14f). Furthermore, in the case of weakly turbulent cloud portions, they possess nearly comparable lower SW values as those of biota; under such conditions it is difficult to screen out clouds using SW alone (see Fig. 14i). Similarly, LDR alone cannot filter all biota and screen out weak clouds well. However, these two diverse and independent radar parameters, Doppler spectral width- and power-based polarimetric LDR measurements of KaSPR, will be an additional measure to identify cloud and non-hydrometeor radar echoes.

4 Summary and conclusions

Millimetre-band radars are very sensitive to the detection of small targets such as cloud droplets, insects and other biological particulates (biota) present in great numbers in the lower atmosphere. Polarization measurement is an efficient means to discriminate cloud echoes from non-hydrometeor scatterers that share very low reflectivity. Unfortunately not all radars are equipped with polarization measurements. This paper proposes, for these standard radars, a simple technique to separate meteorological and non-meteorological echoes. It uses only successive vertical reflectivity profiles acquired by a 35 GHz radar operated at vertical incidence with a 50 m pulse length and 1 s temporal sampling. Because of the high spatial and temporal resolution, most of the time only one (or no) biota target is present in the pulse resolution volume. In contrast, cloud echo is due to millions of droplets that occupy the pulse volume. As a consequence signal variability at a given range between two vertical profiles is much more important for biota scatterers than for cloud echoes. Signal variability is given here by the SD of the reflectivity over the course of four profiles, which corresponds to the typical duration of the biota echoes crossing the antenna beam. The threshold value that distinctly separates biota from cloud is obtained from statistical analysis of a large radar observation set. Indeed this value should be adjusted for a radar with different characteristics. This study responds to a real issue for anybody who wants to extract physical quantities from radar signal. The methodology used is validated with polarization measurements provided by the same radar.

Appendix A

Figure A1Instantaneous height profiles of Z during 12:00–12:05 UT on 29 May 2014 with centred number profile notice to be the strong biota return identified with HTI plot of Fig. 4b. (b) Corresponds to standard deviation (SD) from four-point running average.

Figure A2(a, b, c) Same as Fig. 1a–c but on 8 July 2016 during 05:31 UT for the duration of 108 s. S0–S5 are NER curves. Collocated GPS–RS relative humidity (%) profile is shown as a white solid line in (b).

Figure A3Same as Fig. 13 but during 10:21 UT on 11 September 2015 for the duration of 449 s.

Data availability
Data availability.

Competing interests
Competing interests.

The authors declare that they have no conflict of interest.

Acknowledgements
Acknowledgements.

IITM is an autonomous organization that is fully funded by MOES, govt. of India. The authors are thankful to the director of IITM not only for his whole-hearted support for establishing the radar programme but also for monitoring and acting as a source of inspiration to promote this research to the next level. The authors are highly indebted to Ernest Raj, Devara and all those who were involved and helped in setting up the IITM's Cloud Radar Facility, KaSPR as well as KaSPR design and development, which was done at ProSensing, USA. Authors are grateful to three anonymous referees for the constructive comments that helped in improving the quality of the paper. Authors would also like to thank Gianfranco Vulpiani, the handling editor, and Thomas Wagner, the chief editor of AMT, for all the support rendered.

Edited by: Gianfranco Vulpiani
Reviewed by: three anonymous referees

References

Alberoni, P. P., Ducrocq, V., Gregoric, G., Haase, G., Holleman, I., Lindskog, M., Macpherson, B., Nuret, M., and Rossa, A.: Quality and Assimilation of Radar Data for NWP–A Review, COST 717 document, ISBN: 92-894-4842-3, 38, 2003.

Bauer-Pfundstein, M. R. and Görsdorf, U.: Target separation and classification using cloud radar Doppler-spectra, paper presented at the 33rd International Conference on Radar Meteorology, Am. Meteorol. Soc., Cairns, Australia, 6–10 August, 2007.

Bony, S., Colman, R., Kattsov, V. M., Allan, R. P., Bretherton, C. S., Dufresne, J., Hall, A., Hallegatte, S., Holland, M. M., Ingram, W., Randall, D. A., Soden, B. J., Tselioudis, G., and Webb, M. J.: How well do we understand and evaluate climate change feedback processes?, J. Climate, 19, 3445–3482, 2006.

Chandra, A. S., Kollias, P., and Albrecht, B. A.: Multiyear Summertime Observations of Daytime Fair–Weather Cumuli at the ARM Southern Great Plains Facility, J. Climate, 26, 10031–10050, 2013.

Chandra, A. S., Kollias, P., Giangrande, S. E., and Klein, S. A.: Long Term Observations of the Convective Boundary Layer Using Insect Radar Returns at the SGP ARM Climate Research Facility, J. Climate, 23, 5699–5714, 2010.

Chandrasekar, V., Keränen, R., Lim, S., and Moisseev, D.: Recent advances in classification of observations from dual polarization weather radars, Atmos. Res., 119, 97–111, 2013.

Cho, Y.-H., Lee, G. W., Kim, K.-E., and Zawadzki, I.: Identification and removal of ground echoes and anomalous propagation using the characteristics of radar echoes, J. Atmos. Ocean. Tech., 23, 1206–1222, 2006.

Clothiaux, E. E., Ackerman, T. P., Mace, G. G., Moran, K. P., Marchand, R. T., Miller, M. A., and Martner, B. E.: Objective determination of cloud heights and radar reflectivities using a combination of active remote sensors at the ARM CART sites, J. Appl. Meteor., 39, 645–665, 2000.

Doviak, R. J. and Zrnić, D. S.: Doppler Radar and Weather Observations, Academic press, London, UK, 1984.

Dufton, D. R. L. and Collier, C. G.: Fuzzy logic filtering of radar reflectivity to remove non-meteorological echoes using dual polarization radar moments, Atmos. Meas. Tech., 8, 3985–4000, https://doi.org/10.5194/amt-8-3985-2015, 2015.

Ecklund, W. L., Carter, D. A., and Balsley, B. B.: A UHF wind profiler for the boundary layer: Brief description and initial results, J. Atmos. Ocean. Tech., 5, 432–441, 1988.

Frisch, A. S., Fairall, C. W., and Snider, J. B.: Measurement of stratus cloud and drizzle parameters in ASTEX with a Ka-band Doppler radar and a microwave radiometer, J. Atmos. Sci., 52, 2788–2799, 1995.

Geerts, B. and Miao, Q.: The Use of Millimeter Doppler Radar Echoes to Estimate Vertical Air Velocities in the Fair-Weather Convective Boundary Layer, J. Atmos. Ocean. Tech., 11, 225–246, 2005.

Gossard, E. E.: Radar research on the atmospheric boundary layer, Radar in Meteorology, edited by: Atlas, D., Amer. Meteor. Soc., 477–527, 1990.

Gourley, J. J., Tabary, P., and Parent du Chatele, J.: A fuzzy logic algorithm for the separation of precipitating from non-precipitating echoes using polarimetric radar observations, J. Atmos. Ocean. Tech., 24, 1439–1451, 2007.

Hajovsky, R. G., Deam, A. P., and LaGrone, A. H.: Radar Reflections from insects in the lower atmosphere, IEEE T. Antenn. Propag., M-4 4(2), 224–227, 1966.

Hardy, K. R., Atlas, D., and Glover, K. M.: Multiwavelength backscatter from the clear atmosphere, J. Geophys. Res., 71, 1537–1552, 1966.

Harrison, D. L., Driscoll, S. J., and Kitchen, M.: Improving precipitation estimates from weather radar using quality control and correction techniques, Meteorol. Appl., 7, 135–144, 2000.

Harrison, D. L., Georgiou, S., Gaussiat, N., and Curtis, A.: Longterm diagnostics of precipitation estimates and the development of radar hardware monitoring within a radar product data quality management system, Hydrolog. Sci. J., 59, 1277–1292, 2014.

Hogan, R. J., Gaussiat, N., and Illingworth, A. J.: Stratocumulus liquid water content from dual-wavelength radar, J. Atmos. Ocean. Tech., 22, 1207–1218, 2005.

Hurtado, M. and Nehorai, A.: Polarimetric detection of targets in heavy inhomogeneous clutter, IEEE T. Signal Proces., 56, 1349–1361, 2008.

Johnson, R. H., Rickenbach, T. M., Rutledge, S. A., Ciesielski, P. E., and Schubert, W. H : Trimodal characteristics of tropical convection, J. Climate, 12, 2397–2418, 1999.

Khandwalla, A., Sekelsky, S., and Quante, M.: Algorithms for filtering insect echoes from cloud radar measurements, Thirteenth ARM Science Team Meeting Proceedings, Broomfield, Colorado, 2003.

Kollias, P. and Albrecht, B. A.: The turbulent structure in a continental stratocumulus cloud from millimeter wavelength radar observations, J. Atmos. Sci., 57, 2417–2434, 2000.

Lhermitte, R.: A 94-Ghz Doppler Radar for Cloud Observation, J. Atmos. Ocean. Tech., 4, 36–48, 1987.

Lhermitte, R. M.: Probing air motion by Doppler analysis of radar clear air returns, J. Atmos. Sci., 23, 575–591, 1966.

Luke, E. P., Kollias, P., and Johnson, K. L.: A Technique for the Automatic Detection of Insect Clutter in Cloud Radar Returns, J. Atmos. Ocean. Tech., 25, 1498–1513, 2008.

Martner, B. E. and Moran, K. P.: Using cloud radar polarization measurements to evaluate stratus cloud and insect echoes, J. Geophys. Res., 106, 4891–4897, 2001.

Meischner, P., Collier, C., Illingworth, A., Joss, J., and Randeu, W.: Advanced Weather Radar Systems in Europe: The COST 75 Action, B. Am. Meteor. Soc., 78, 1411–1430, 1997.

Mudukutore, A., Chandrasekar, V., and Keeler, R. J.: Pulse compression for weather radars, IEEE T. Geosci. Remote, 36, 125–142, https://doi.org/10.1109/36.655323, 1998.

Mueller, E. A.: Differential reflectivity of birds and insects. Preprints, 21st Conf. on Radar Meteorology, Edmonton, AB, Canada, Am. Meteor. Soc., 465–466, 1983.

Mueller, E. A. and Larkin, R. P.: Insects observed using dualpolarization radar, J. Atmos. Ocean. Tech., 2, 49–54, 1985.

Nguyen, C. M., Moisseev, D. N., and Chandrasekar, V.: A parametric time domain method for spectral moment estimation and clutter mitigation for weather radars, J. Atmos. Ocean. Tech., 25, 83–92, 2008.

Pazmany, A., McIntosh, R., Kelly, R., and Vali, G.: An airborne 95 GHz dual-polarized radar for cloud studies, IEEE T. Geosci. Remote Sens., 32, 731–739, 1994.

Russell, R. W. and Wilson, J. W.: Radar-observed “fine lines” in the optically clear boundary layer. Reflectivity contributions from aerial plankton and its predators, Bound.-Lay. Meteor., 82, 235–262, 1997.

Sassen, K., Mace, G. G., Wang, Z., Poellot, M. R., Sekelsky, S. M., and McIntosh, R. E.: Continental stratus clouds: A case study using coordinated remote sensing and aircraft measurements, J. Atmos. Sci., 56, 2345–2358, 1999.

Sekelsky, S. M., Li, L., Calloway, J., McIntosh, R. E., Miller, M. A., Clothiaux, E. E., Haimov, S., Mace, G. C., and Sassen, K.: Comparison of millimeter-wave cloud radar measurements for the fall 1997 cloud IOP, Proceedings of 8th ARM Science Team Meeting, 671–675, Dep. of Energy, Tucson, Ariz, 1998.

Steiner, M. and Smith, J.: Use of three-dimensional reflectivity structure for automated detection and removal of nonprecipitating echoes in radar data, J. Atmos. Ocean. Tech., 19, 673–686, 2002.

Teixeira, J., Stevens, B., Bretherton, C. S., Cederwall, R., Doyle, J. D., Golaz, J. C., Holtslag, A. A., Klein, S. A., Lundquist, J. K., Randall, D. A., and Siebesma, A. P.: Parameterization of the atmospheric boundary layer: A view from just above the inversion, B. Am. Meteor. Soc., 89, 453–458, 2008.

Teschke, G., Gorsdorf, U., Korner, P., and Trede, D.: A new approach for target classification of Ka-band radar data, Fourth European Conference on Radar in Meteorology and Hydrology (ERAD), Barcelona, 2006.

Tiedtke, M.: A comprehensive mass flux scheme for cumulus parameterization in large-scale models, Mon. Weather Rev., 117, 1799–1801, 1989.

Torres, S. M. and Zrnić, D. S: Ground clutter canceling with a regression filter, J. Atmos. Ocean. Tech., 16, 1364–1372, 1999.

Unal, C.: Spectral Polarimetric Radar Clutter Suppression to Enhance Atmospheric Echoes, J. Atmos. Ocean. Tech., 26, 1781–1797, 2009.

Warde, D. A. and Torres, S. M.: Automatic detection and removal of ground clutter contamination on weather radars, Proc. 34th Conference on Radar Meteorology, Williamsburg, VA, USA, AMS, P10.11, 2009.

Zhang, P., Ryzhkov, A. V., and Zrnic, D. S.: Observations of insects and birds with a polarimetric prototype of the WSR-88D radar, Preprints, 32d Int. Conf. on Radar Meteorology, Albuquerque, NM, Amer. Meteor. Soc., CD-ROM, P6.4, 2005.

Zrnić, D. S. and Ryzhkov, A. V.: Observations of insects and birds with a polarimetric radar, IEEE T. Geosci. Remote Sens., 36, 661–668, 1998.