Actions

Difference between revisions of "RadxPid"

From Lrose Wiki

 
(17 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
RadxPid produces three-dimensional fields of the NCAR particle identification (PID).
 
RadxPid produces three-dimensional fields of the NCAR particle identification (PID).
  
=== Prerequesites ===
+
=== '''Overview''' ===
 +
 
 +
RadxPid ingests polarimetric data to estimate the dominant hydrometeor type using NCAR’s Particle Identification algorithm. First, KDP is calculated from PHIDP using the same KDP-specific parameter file as RadxKdp. Next, the polarimetric radar data and calculated KDP are combined with sounding data to determine the most likely dominant hydrometeor. The user decides whether or not the algorithm will use the raw Zh and Zdr data from the radar or correct for attenuation. Finally, the resulting three-dimensional PID categories are sent to an output file.
 +
 
 +
 
 +
=== '''Scientific Background''' ===
 +
For more information on the fuzzy logic algorithm used to estimate the NCAR PID, please refer to [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic this page].
 +
 
 +
Information about how attenuation is estimated from <math>K_{DP}</math> for use in determining the PID can be found [http://wiki.lrose.net/index.php/Estimated_Attenuation here].
 +
 
 +
Information on how to convert soundings into the Spdb format can be found [http://wiki.lrose.net/index.php/Storing_soundings_in_SPDB here].
 +
 
 +
=== '''Prerequesites''' ===
  
 
The following items are required:
 
The following items are required:
Line 10: Line 22:
 
* Three parameter files to run RadxPid, calculate KDP, and PID.
 
* Three parameter files to run RadxPid, calculate KDP, and PID.
  
=== Overview ===
 
 
RadxPid ingests polarimetric data to estimate the dominant hydrometeor type using NCAR’s Particle Identification algorithm. First, KDP is calculated from PHIDP using the same KDP-specific parameter file as RadxKdp. Next, the polarimetric radar data and calculated KDP are combined with sounding data to determine the most likely dominant hydrometeor. The user decides whether or not the algorithm will use the raw Zh and Zdr data from the radar or correct for attenuation. Finally, the resulting three-dimensional PID categories are sent to an output file.
 
  
=== PID thresholds file ===
+
=== '''PID thresholds file''' ===
  
 
These thresholds describe the membership functions for each combination of the PID category and polarimetric field as well as the variable weights for each particle type. The file also contains a place to enter a sounding, if the radar files cover a narrow time period; otherwise, the sounding will be overwritten by data provided by the user, which is explained below. The particle types are most sensitive to temperature, Zh, Zdr, and the standard deviations of Zdr and PhiDP.
 
These thresholds describe the membership functions for each combination of the PID category and polarimetric field as well as the variable weights for each particle type. The file also contains a place to enter a sounding, if the radar files cover a narrow time period; otherwise, the sounding will be overwritten by data provided by the user, which is explained below. The particle types are most sensitive to temperature, Zh, Zdr, and the standard deviations of Zdr and PhiDP.
Line 44: Line 53:
 
|}
 
|}
  
=== Sounding data ===
+
=== '''Sounding data''' ===
  
If sounding data varies in time, sounding data in a non-gridded data format (SPDB) will need to be ingested. SPDB data can come from either observations or numerical simulations (e.g., RAP or HRRR). For additional information regarding this data format, please see the documentation here ('''insert link when written''').
+
If sounding data varies in time, sounding data in a non-gridded data format (SPDB) will need to be ingested. SPDB data can come from either observations or numerical simulations (e.g., RAP or HRRR). For help converting soundings to this data format, please see the documentation [http://wiki.lrose.net/index.php/spdb_description here].
  
=== Parameter files ===
+
=== '''Parameter files''' ===
  
 
RadxPid uses three separate parameter files. The first includes the basic parameters related to data location and field names, which is similar to the parameter files for the other applications. The second includes the parameters that set the filtering length, method, and relevant coefficients necessary to calculate KDP and estimate attenuation. The third specifies filter parameters and relevant information related to the soundings used by the PID.
 
RadxPid uses three separate parameter files. The first includes the basic parameters related to data location and field names, which is similar to the parameter files for the other applications. The second includes the parameters that set the filtering length, method, and relevant coefficients necessary to calculate KDP and estimate attenuation. The third specifies filter parameters and relevant information related to the soundings used by the PID.
  
==== 1) Main parameter file ====
+
==== '''1) Main parameter file''' ====
  
===== Ensure file is up to date =====
+
===== '''Ensure file is up to date''' =====
  
 
To obtain the default parameter file, use the following command:
 
To obtain the default parameter file, use the following command:
Line 64: Line 73:
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params orig_param_file_name -print_params &gt; new_param_file_name</code>
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params orig_param_file_name -print_params &gt; new_param_file_name</code>
  
===== Important parameters =====
+
===== '''Important parameters''' =====
  
 
Input params
 
Input params
Line 91: Line 100:
 
* output_format: select the preferred file type (usually CFRADIAL)
 
* output_format: select the preferred file type (usually CFRADIAL)
  
==== 2) KDP-specific parameter file ====
+
==== '''2) KDP-specific parameter file''' ====
  
===== Ensure file is up to date =====
+
===== '''Ensure file is up to date''' =====
  
 
To obtain the default parameter file, use the following command:
 
To obtain the default parameter file, use the following command:
Line 103: Line 112:
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params_kdp orig_param_file_name -print_params_kdp &gt; new_param_file_name</code>
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params_kdp orig_param_file_name -print_params_kdp &gt; new_param_file_name</code>
  
===== Important parameters =====
+
===== '''Important parameters''' =====
  
 
Unfolding and filtering
 
Unfolding and filtering
Line 113: Line 122:
 
* KDP_psob_method: Method to handle PSOB
 
* KDP_psob_method: Method to handle PSOB
  
==== 3) PID-specific parameter file ====
+
==== '''3) PID-specific parameter file''' ====
  
===== Ensure file is up to date =====
+
===== '''Ensure file is up to date''' =====
  
 
To obtain the default parameter file, use the following command:
 
To obtain the default parameter file, use the following command:
Line 125: Line 134:
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params_pid orig_param_file_name -print_params_pid &gt; new_param_file_name</code>
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params_pid orig_param_file_name -print_params_pid &gt; new_param_file_name</code>
  
===== Important parameters =====
+
===== '''Important parameters''' =====
  
 
NCAR PID Method
 
NCAR PID Method
Line 136: Line 145:
 
* PID_sounding_spdb_url: path to SPDB sounding data
 
* PID_sounding_spdb_url: path to SPDB sounding data
  
=== Running RadxPid ===
+
=== '''Running RadxPid''' ===
  
 
To check all command line options for RadxPid, including debugging options and file paths, type the following command into a terminal.
 
To check all command line options for RadxPid, including debugging options and file paths, type the following command into a terminal.
Line 142: Line 151:
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -h</code>
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -h</code>
  
Once your parameter file is complete, use a command similar to the following to run the application:
+
Once your parameter file is complete, use a command similar to the following to run the application. ''Note: this command will '''''not''''' work for the FILELIST mode.''
  
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params param_file_name</code>
 
<code lang="bash">/path/to/lrose/install/bin/RadxPid -params param_file_name</code>
  
The user can also specify the location of cfradial files in the following manner:
+
If using FILELIST mode, the user needs to specify the location of cfradial files using the <code lang="bash">-f</code> flag:
  
<code lang="bash">/path/to/lrose/install/bin/RadxPid -f /path/to/cfradial/files/ -params param_file_name</code>
+
<code lang="bash">/path/to/lrose/install/bin/RadxPid -f /path/to/cfradial/files/*.nc -params param_file_name</code>
  
=== Other relevant items ===
+
=== '''Other relevant items''' ===
  
==== Additional parameters to edit ====
+
==== '''Additional parameters to edit''' ====
  
 
Caution: this is not an exhaustive list. We urge each user to read through the entire parameter file carefully.  
 
Caution: this is not an exhaustive list. We urge each user to read through the entire parameter file carefully.  
  
===== Main parameter file =====
+
===== '''Main parameter file''' =====
  
 
Specifying copy-through fields
 
Specifying copy-through fields
Line 162: Line 171:
 
* copy_selected_input_fields_to_output: determines whether original fields are copied to the output file
 
* copy_selected_input_fields_to_output: determines whether original fields are copied to the output file
  
===== KDP-specific parameter file =====
+
===== '''KDP-specific parameter file''' =====
  
 
Precip-induced attenuation correction for DBZ and ZDR
 
Precip-induced attenuation correction for DBZ and ZDR
Line 168: Line 177:
 
* KDP_specify_coefficients_for_attenuation_correction: the user can either set the coefficients themselves or use the default coefficients based on the radar wavelength
 
* KDP_specify_coefficients_for_attenuation_correction: the user can either set the coefficients themselves or use the default coefficients based on the radar wavelength
  
===== PID-specific parameter file =====
+
===== '''PID-specific parameter file''' =====
  
 
Computing PID
 
Computing PID

Latest revision as of 22:02, 15 April 2024

RadxPid produces three-dimensional fields of the NCAR particle identification (PID).

Overview

RadxPid ingests polarimetric data to estimate the dominant hydrometeor type using NCAR’s Particle Identification algorithm. First, KDP is calculated from PHIDP using the same KDP-specific parameter file as RadxKdp. Next, the polarimetric radar data and calculated KDP are combined with sounding data to determine the most likely dominant hydrometeor. The user decides whether or not the algorithm will use the raw Zh and Zdr data from the radar or correct for attenuation. Finally, the resulting three-dimensional PID categories are sent to an output file.


Scientific Background

For more information on the fuzzy logic algorithm used to estimate the NCAR PID, please refer to this page.

Information about how attenuation is estimated from [math]\displaystyle{ K_{DP} }[/math] for use in determining the PID can be found here.

Information on how to convert soundings into the Spdb format can be found here.

Prerequesites

The following items are required:

  • Polarimetric radar data (any Radx supported format, e.g., cfradial)
  • PID fuzzy logic thresholds file
  • Sounding data (either from observations or a numerical model)
  • Three parameter files to run RadxPid, calculate KDP, and PID.


PID thresholds file

These thresholds describe the membership functions for each combination of the PID category and polarimetric field as well as the variable weights for each particle type. The file also contains a place to enter a sounding, if the radar files cover a narrow time period; otherwise, the sounding will be overwritten by data provided by the user, which is explained below. The particle types are most sensitive to temperature, Zh, Zdr, and the standard deviations of Zdr and PhiDP.


Set this to a file suitable for the radar transmit mode and wavelength:


Wavelength Transmit mode thresholds_file_example
S-band Simultaneous pid_thresholds.sband.shv
S-band Alternating pid_thresholds.sband.alt
C-band Simultaneous pid_thresholds.cband.shv
X-band Simultaneous pid_thresholds.xband.shv

Sounding data

If sounding data varies in time, sounding data in a non-gridded data format (SPDB) will need to be ingested. SPDB data can come from either observations or numerical simulations (e.g., RAP or HRRR). For help converting soundings to this data format, please see the documentation here.

Parameter files

RadxPid uses three separate parameter files. The first includes the basic parameters related to data location and field names, which is similar to the parameter files for the other applications. The second includes the parameters that set the filtering length, method, and relevant coefficients necessary to calculate KDP and estimate attenuation. The third specifies filter parameters and relevant information related to the soundings used by the PID.

1) Main parameter file

Ensure file is up to date

To obtain the default parameter file, use the following command:

/path/to/lrose/install/bin/RadxPid -print_params > param_file_name

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

/path/to/lrose/install/bin/RadxPid -params orig_param_file_name -print_params > new_param_file_name

Important parameters

Input params

  • input_dir: directory containing radar data (if not specified on the command line and if mode = REALTIME)
  • mode: determines if the program waits for new files (REALTIME), moves through start and end times specified on the command line (ARCHIVE), or moves through list of files specified on the command line (FILELIST)

Input field information

  • SNR_available: determines if SNR data is in the file or needs to be calculated from DBZ
  • VARIABLE_field_name: tells RadxPid the polarimetric variable names in the ingested cfradial files
  • LDR_available: determines if LDR data is in the file

Computing KDP

  • KDP_params_file_path: path to the KDP-specific parameter file

Computing PID

  • PID_params_file_path: file path for the PID-specific parameter file
  • PID_use_attenuation_corrected_fields: determines whether PID uses Z and ZDR fields that are corrected for attenuation

Writing the output files

  • output_dir: files will be written to this directory
  • output_format: select the preferred file type (usually CFRADIAL)

2) KDP-specific parameter file

Ensure file is up to date

To obtain the default parameter file, use the following command:

/path/to/lrose/install/bin/RadxPid -print_params_kdp > param_file_name

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

/path/to/lrose/install/bin/RadxPid -params_kdp orig_param_file_name -print_params_kdp > new_param_file_name

Important parameters

Unfolding and filtering

  • KDP_fir_filter_len: Filter length applied to PHIDP

Handling phase shift on backscatter (PSOB)

  • KDP_psob_method: Method to handle PSOB

3) PID-specific parameter file

Ensure file is up to date

To obtain the default parameter file, use the following command:

/path/to/lrose/install/bin/RadxPid -print_params_pid > param_file_name

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

/path/to/lrose/install/bin/RadxPid -params_pid orig_param_file_name -print_params_pid > new_param_file_name

Important parameters

NCAR PID Method

  • pid_thresholds_file_path: file path for the PID thresholds file

Sounding input for PID temperatures

  • use_soundings_from_spdb: tells RadxPid whether to override the sounding in the pid thresholds file with SPDB data
  • PID_sounding_spdb_url: path to SPDB sounding data

Running RadxPid

To check all command line options for RadxPid, including debugging options and file paths, type the following command into a terminal.

/path/to/lrose/install/bin/RadxPid -h

Once your parameter file is complete, use a command similar to the following to run the application. Note: this command will not work for the FILELIST mode.

/path/to/lrose/install/bin/RadxPid -params param_file_name

If using FILELIST mode, the user needs to specify the location of cfradial files using the -f flag:

/path/to/lrose/install/bin/RadxPid -f /path/to/cfradial/files/*.nc -params param_file_name

Other relevant items

Additional parameters to edit

Caution: this is not an exhaustive list. We urge each user to read through the entire parameter file carefully.

Main parameter file

Specifying copy-through fields

  • copy_selected_input_fields_to_output: determines whether original fields are copied to the output file
KDP-specific parameter file

Precip-induced attenuation correction for DBZ and ZDR

  • KDP_specify_coefficients_for_attenuation_correction: the user can either set the coefficients themselves or use the default coefficients based on the radar wavelength
PID-specific parameter file

Computing PID

  • PID_snr_threshold: mininmum SNR required for the PID to be calculated
  • PID_min_valid_interest: mininimum interest value required in order for a PID category to be accepted
  • PID_apply_median_filter_to_VARIABLE: determines whether RadxPid applies a filter to specific polarimetric variables before running the PID
  • PID_ngates_for_sdev: sets the number of gates used to calculate the standard deviations of ZDR and PHIDP

Sounding input for PID temperatures

  • sounding_search_time_margin_secs: the maximum allowable time difference between the radar files and sounding data
  • sounding_location_name: directs RadxPid to the appropriate sounding
  • sounding_required_pressure_range_hpa: sets the minimum pressure range required for a valid sounding
  • sounding_required_height_range_m: sets the minimum altitude range required for a valid sounding