Csu radartools tutorial

Overview

---

This tutorial will walk through the steps to read and convert a raw radar file, quality control the data (removing unwanted artifacts and echoes, unfold radial velocity and calculate Kdp), then grid the data, and finally process the gridded data to end up with hydrometeor identification, precipitation rates, and DSD information. This is generally focused on surface polarimetric X, C, or S-band radar. We will apply a series of corrections

• Unfold radial velocity
• Calculate Kdp
• Apply thresholds to remove non-meteorological data
• Calculate the attenuation and differential attenuation
• Despeckle and remove 2nd trip

Modules available from GitHub

---

• https://github.com/ARM-DOE/pyart
• https://github.com/CSU-Radarmet/CSU_RadarTools

Scientific Background

---

Kdp calculation

Hubbert, J., and V. N. Bringi. "An iterative filtering technique for the analysis of copolar differential phase and dual-frequency radar measurements." Journal of Atmospheric and Oceanic Technology 12.3 (1995): 643-648.

Step-by-step Instructions

---

The Example file is from the C-band polarimetric SEA-POL radar. The raw data are in sigmet native format.

Python Imports

 import numpy as np
 from copy import deepcopy
 import os
 import pyart
 from CSU_RadarTools.csu_radartools import csu_kdp
 from CSU_RadarTools.csu_radartools import csu_misc
 import scipy.ndimage as ndi

Convert to CFRADIAL

Convert the data to cfradial data to be more easily transferable to other programs.
We can do this using RadxConvert from a command line, but we can also do it from python with the os command.

 mydir = '/Users/bdolan/scratch/LROSE/SEA-POL/ppi/'
 os.system(f'RadxConvert -f {mydir}SEA20190917_071005 -outdir {mydir}cfrad')

This will put the output in a directory for the date in directory 'cfrad'.
Now we have a file called:
cfrad.20190917_071006.545_to_20190917_071353.149_SEAPOL_SUR.nc

We can read this into radar with pyart:

 radar = pyart.io.read(f'{mydir}cfrad/cfrad.20190917_071006.545_to_20190917_071353.149_SEAPOL_SUR.nc')

There are a lot of variables in this file, but the ones of interest are:

 radar.fields.keys()

DBZ: Reflectivity
ZDR: Differential Reflectivity
UNKNOWN_ID_82: Correlation Coefficient
VEL: Radial velocity
SQI: Signal Quality Index
PHIDP: Differential phase
SNR: Signal to noise ratio

QC with CSU_Radartools and PyART

Do some quality control using PyART and CSU_Radartools.

Here is the original reflectivity field:

Special handling for SEA-POL

These are PECULIARS to the SEA-POL data. YMMV

Blanked sector over the wheelhouse

We will look for large blocks of azimuthal jumps to mask them. Get the difference between each azimuth and look for jumps larger than 30º. If so, mask the data so it plots correctly.

az_diff = np.diff(radar.azimuth['data'])
jumps = np.where(np.abs(az_diff) >= 30.0)[0]
if len(jumps):
  for f in radar.fields.keys():
     for j in jumps:
        radar.fields[f]['data'][j].mask = True
        radar.fields[f]['data'][j+1].mask = True

The new reflectivity field is now masked over the wheelhouse:

The correlation coefficient is incorrect, but we can calculate it from UNKNONW_ID_82 field.

Now correct the UNKNOWN_ID_82 field to transform into the correlation coefficient (Again, this is only needed for this data).

 ccorr = deepcopy(radar.fields['UNKNOWN_ID_82']['data'])
 ccorr[ccorr<0] += 65535
 ccorr = (ccorr-1)/65533.0
 radar.add_field_like('RHOHV','CC',ccorr)

Calculate Kdp

This method uses the Hubbert and Bringi FIR filter method. We note that the window / gate spacing must be an even number or it will fail. it will also calculate the standard deviation of the phase which can be used to remove non-meteorological data.

 def extract_unmasked_data(radar, field, bad=-32768):
   """Simplify getting unmasked radar fields from Py-ART"""
   return radar.fields[field]['data'].filled(fill_value=bad)

 dz_um = extract_unmasked_data(radar,'DBZ')
 dp_um = extract_unmasked_data(radar, 'PHIDP')

Range needs to be supplied as a variable, and it needs to be the same shape as dzN, etc.

 rng2d, az2d = np.meshgrid(radar.range['data'], radar.azimuth['data'])
 kdN, fdN, sdN = csu_kdp.calc_kdp_bringi(dp=dp_um, dz=dz_um, rng=rng2d/1000.0, thsd=12, gs=250.0, window=7, nfilter=2)

kdN is the specific differential phase, fdN is the filtered differential phase, and sdN is the standard deviation of the phase
Add this back into the radar structure.

 field_dict = {'data': kdN,
               'units': 'deg/km',
               'long_name': 'specific_differential_phase',
               'standard_name': 'Kdp',
               '_FillValue': '-32768'}
 radar.add_field('Kdp', field_dict, replace_existing=True)

Removing Non-meteorological Echoes

Standard Deviation of Differential Phase

Sometimes the sdp goes nuts in high reflectivity regions, and pair this mask with relatively low dBZ regions. Here I am using a threshold of 10, but I usually do a sensitivity study to figure out a good value, typically ranging from 10 to 25. However, remember that things like hail can have high sdN.

 sdp_mask = csu_misc.differential_phase_filter(sdN, thresh_sdp=13)
 sdp_mask = np.logical_and(sdp_mask, radar.fields['DBZ']['data'] <= 10)

 old_var_mask = deepcopy(radar.fields['DBZ']['data'].mask)
 new_var_mask = np.logical_or(old_var_mask, sdp_mask)
 radar.fields['DBZ']['data'].mask = new_var_mask

Well that didn't get us vary far (this is not unexpected given the RF interference with this particular dataset). But we have other tools at our disposal, and this is usually paired with another filter like rho_hv or SQI.

SQI or Rho_HV

Here I'm using SQI because it's available in the Sigmet data. We have found that 0.35 is an ok threshold for now.

 sq_mask = radar.fields['SQI']['data'] < 0.35]

Much better, but now we need a despeckler to pick up those random points.

Despeckler

We found that 4 gates works pretty well for this data. (remember that dz_um is the unmasked reflectivity that has the above corrections already applied).

 mask_ds = csu_misc.despeckle(dz_um, ngates=4)

Looking pretty good!

Insect Filter

Occasionally you will need to remove insects. This typically has high Zdr and moderate reflectivity. There is an insect_filter built into CSU_Radartools.
Note: This image does not have any insects so there is no change when applying this field, but here is how one does that.

 insect_mask = csu_misc.insect_filter(dz_um, zdr_um,  
   dz_range=[[-100, 10], [10, 15], [15, 20], [20, 25], [25, 30], [30, 35]],
   dr_thresh=[1, 1.3, 1.7, 2.1, 2.5, 2.8])

Second Trip

Second trip can be hard to blanket-remove with any one variable, unless you have Ldr, in which case 2nd trip often shows up as positive values. The following methodology was developed by Dr. B. Fuchs and was found to work well on the SEA-POL data, using a smoothed spectrum width and DZ field. As always, be sure to use with caution on other datasets.

1. First, fill the QCed reflectivity field with a background value (-20 dBZ here)

   dz_qc_filled = deepcopy(dz_um)
   dz_qc_filled[dz_qc_filled < -999] = -20.0

1. Apply a smoothing to that field.

   sm_rad = 2
   dz_qc_smooth = ndi.gaussian_filter(dz_qc_filled, (sm_rad, sm_rad), mode='nearest')
   sm_g_rad = 5
   width_smooth = ndi.gaussian_filter(radar.fields[ycfg['orig_width_name']]['data'].filled(2.0), (sm_g_rad, sm_g_rad), mode='nearest')