Lrose Wiki - User contributions [en]

RadxBeamBlock

2022-08-08T16:07:40Z

Barbero: /* Prerequesites */

RadxBeamBlock estimates the amount of beam blockage due to terrain near a radar.

=== '''Overview''' ===

Overall, RadxBeamBlock takes elevation data, the radar location, and radar characteristics to estimate how much blockage occurs due to terrain.

=== '''Prerequesites''' ===

The following items are required:

* Digital elevation model data
* Radar antenna location
* Radar characteristics
* Parameter files for RadxBeamBlock

=== '''Digital elevation model data''' ===
Note: a NASA Earthdata account is needed to download topography data. Creating an account is free and quick to do, and can be done [https://www.earthdata.nasa.gov/ here].

The supported digital elevation models include Shuttle Radar Topography Mission (SRTM) and ESRI grid data (spheroid). SRTM data can be found online '''[https://lpdaac.usgs.gov/products/srtmgl3sv003/#tools here]'''. SRTM data for the entire globe is 22GB and can be a lot to download! If you only need a subsection for your region of interest, use a recursive script to download them. If you only need a few files, this is easy enough to do manually. Remember to unzip your files. Once completed, you should have a directory with .hgt files spanning your region.

Example:
1 #!/bin/bash
2
3 # download SRTM data for Taiwan region
4
5 for lat in `seq 19 25`
6 do
7 for lon in `seq 117 123`
8 do
9 # get files
10 wget --user=$USERNAME --password=$PASSWORD https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N${lat}E${lon}.SRTMGL1.hgt.zip
11 done
12 done

=== '''Radar location and characteristics''' ===

The radar location requires the latitude, longitude, and elevation of the antenna (ground level + radar height). In addition, characteristics including the wavelength, beamwidth, gate spacing, azimuths, and elevation angles are required. This information can be found using RadxPrint on a raw radar file or converted radar CfRadial file.
RadxPrint -f /path/to/radar/file

=== '''Parameter files''' ===

RadxBeamBlock uses one parameter file. Compared to other Radx applications, the RadxBeamBlock parameter file requires just a few edits in order to run. Most of the parameters that need to be changed relate to the radar characteristics.

==== '''1) Main parameter file''' ====

===== '''Ensure file is up to date''' =====

To obtain the default parameter file, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -print_params > param_file_name</code>

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params orig_param_file_name -print_params > new_param_file_name</code>

===== '''Important parameters''' =====

Input params

* input_dem_parth: directory containing the elevation data
* input_data_format: specifies the format of the elevation data (examples provided in the parameter file)

Radar parameters

* radar_location: latitude, longitude, and elevation (ground elevation
** radar height in km) of the radar antenna
* radar_wavelength_cm: radar wavelength (cm)
* horiz_beam_width_deg: horizontal beamwidth (degrees)
* vert_beam_width_deg: horizontal beamwidth (degrees)
* gates: starting distance, gate spacing (km), and number of gates
* azimuths: starting azimuth, azimuth spacing (degrees), and number of azimuths
* elevations: starting elevation angle, angle spacing (degrees), and number of elevation angles

Output directory and file name

* output_dir: files will be written to this directory
* output_format: select the preferred file type (usually CFRADIAL)

=== '''Running RadxBeamBlock''' ===

To check all command line options for RadxBeamBlock, including debugging options and file paths, type the following command into a terminal.

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -h</code>

Once your parameter file is complete, use a command similar to the following to run the application:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params param_file_name</code>

RadxBeamBlock

2022-08-08T16:06:47Z

Barbero: /* Digital elevation model data */

RadxBeamBlock estimates the amount of beam blockage due to terrain near a radar.

=== '''Overview''' ===

Overall, RadxBeamBlock takes elevation data, the radar location, and radar characteristics to estimate how much blockage occurs due to terrain.

=== '''Prerequesites''' ===

The following items are required:

* Digital elevation model data
* Radar antenna location
* Radar characteristics
* Parameter files for RadxBeamBlock.

=== '''Digital elevation model data''' ===
Note: a NASA Earthdata account is needed to download topography data. Creating an account is free and quick to do, and can be done [https://www.earthdata.nasa.gov/ here].

The supported digital elevation models include Shuttle Radar Topography Mission (SRTM) and ESRI grid data (spheroid). SRTM data can be found online '''[https://lpdaac.usgs.gov/products/srtmgl3sv003/#tools here]'''. SRTM data for the entire globe is 22GB and can be a lot to download! If you only need a subsection for your region of interest, use a recursive script to download them. If you only need a few files, this is easy enough to do manually. Remember to unzip your files. Once completed, you should have a directory with .hgt files spanning your region.

Example:
1 #!/bin/bash
2
3 # download SRTM data for Taiwan region
4
5 for lat in `seq 19 25`
6 do
7 for lon in `seq 117 123`
8 do
9 # get files
10 wget --user=$USERNAME --password=$PASSWORD https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N${lat}E${lon}.SRTMGL1.hgt.zip
11 done
12 done

=== '''Radar location and characteristics''' ===

The radar location requires the latitude, longitude, and elevation of the antenna (ground level + radar height). In addition, characteristics including the wavelength, beamwidth, gate spacing, azimuths, and elevation angles are required. This information can be found using RadxPrint on a raw radar file or converted radar CfRadial file.
RadxPrint -f /path/to/radar/file

=== '''Parameter files''' ===

RadxBeamBlock uses one parameter file. Compared to other Radx applications, the RadxBeamBlock parameter file requires just a few edits in order to run. Most of the parameters that need to be changed relate to the radar characteristics.

==== '''1) Main parameter file''' ====

===== '''Ensure file is up to date''' =====

To obtain the default parameter file, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -print_params > param_file_name</code>

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params orig_param_file_name -print_params > new_param_file_name</code>

===== '''Important parameters''' =====

Input params

* input_dem_parth: directory containing the elevation data
* input_data_format: specifies the format of the elevation data (examples provided in the parameter file)

Radar parameters

* radar_location: latitude, longitude, and elevation (ground elevation
** radar height in km) of the radar antenna
* radar_wavelength_cm: radar wavelength (cm)
* horiz_beam_width_deg: horizontal beamwidth (degrees)
* vert_beam_width_deg: horizontal beamwidth (degrees)
* gates: starting distance, gate spacing (km), and number of gates
* azimuths: starting azimuth, azimuth spacing (degrees), and number of azimuths
* elevations: starting elevation angle, angle spacing (degrees), and number of elevation angles

Output directory and file name

* output_dir: files will be written to this directory
* output_format: select the preferred file type (usually CFRADIAL)

=== '''Running RadxBeamBlock''' ===

To check all command line options for RadxBeamBlock, including debugging options and file paths, type the following command into a terminal.

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -h</code>

Once your parameter file is complete, use a command similar to the following to run the application:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params param_file_name</code>

Elle full

2022-08-05T20:19:47Z

Barbero: /* Obtain GFS and topographic data */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files during intermediate steps, and which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the [https://lpdaac.usgs.gov/products/srtmgl3sv003/#tools SRTM topography data] using wget, curl, or manually downloading the files. A full example of how to download the data using wget is shown in the [http://wiki.lrose.net/index.php/RadxBeamBlock#Digital_elevation_model_data RadxBeamBlock documentation]. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget --user=$USERNAME --password=$PASSWORD https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N${lat}E${lon}.SRTMGL1.hgt.zip

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. You will need to choose a grid that contains your area of interest as well as the radar(s) you are using.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv
Note: before moving on, check that your Spdb sounding files are populated with data by using the SpdbQuery command below. All parameters except w wind and divergence should be populated.
SpdbQuery -url /path/to/spdb/folder -start "YYYY MM DD HH MM SS" -end "YYYY MM DD HH MM SS"

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data. A tutorial on how to add and view retrieve/calculated variables in HawkEye such as the RATE variables, PID, etc., can be found [http://wiki.lrose.net/index.php/HawkEye#Adding_variables_to_HawkEye_parameter_file here].</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

RadxBeamBlock

2022-08-05T20:09:15Z

Barbero: /* Radar location and characteristics */

RadxBeamBlock estimates the amount of beam blockage due to terrain near a radar.

=== '''Overview''' ===

Overall, RadxBeamBlock takes elevation data, the radar location, and radar characteristics to estimate how much blockage occurs due to terrain.

=== '''Prerequesites''' ===

The following items are required:

* Digital elevation model data
* Radar antenna location
* Radar characteristics
* Parameter files for RadxBeamBlock.

=== '''Digital elevation model data''' ===
Note: a NASA Earthdata account is needed to download SRTM data. Creating an account is free and quick to do, and can be done [https://www.earthdata.nasa.gov/ here].

The supported digital elevation models include Shuttle Radar Topography Mission and ESRI grid data (spheroid). SRTM data can be found online '''[https://lpdaac.usgs.gov/products/srtmgl3sv003/#tools here]'''. SRTM data for the entire globe is 22GB and can be a lot to download! If you only need a subsection for your region of interest, use a recursive script to download them. If you only need a few files, this is easy enough to do manually. Remember to unzip your files. Once completed, you should have a directory with .hgt files spanning your region of interest.

Example:
1 #!/bin/bash
2
3 # download SRTM data for Taiwan region
4
5 for lat in `seq 19 25`
6 do
7 for lon in `seq 117 123`
8 do
9 # get files
10 wget --user=$USERNAME --password=$PASSWORD https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N${lat}E${lon}.SRTMGL1.hgt.zip
11 done
12 done

=== '''Radar location and characteristics''' ===

The radar location requires the latitude, longitude, and elevation of the antenna (ground level + radar height). In addition, characteristics including the wavelength, beamwidth, gate spacing, azimuths, and elevation angles are required. This information can be found using RadxPrint on a raw radar file or converted radar CfRadial file.
RadxPrint -f /path/to/radar/file

=== '''Parameter files''' ===

RadxBeamBlock uses one parameter file. Compared to other Radx applications, the RadxBeamBlock parameter file requires just a few edits in order to run. Most of the parameters that need to be changed relate to the radar characteristics.

==== '''1) Main parameter file''' ====

===== '''Ensure file is up to date''' =====

To obtain the default parameter file, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -print_params > param_file_name</code>

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params orig_param_file_name -print_params > new_param_file_name</code>

===== '''Important parameters''' =====

Input params

* input_dem_parth: directory containing the elevation data
* input_data_format: specifies the format of the elevation data (examples provided in the parameter file)

Radar parameters

* radar_location: latitude, longitude, and elevation (ground elevation
** radar height in km) of the radar antenna
* radar_wavelength_cm: radar wavelength (cm)
* horiz_beam_width_deg: horizontal beamwidth (degrees)
* vert_beam_width_deg: horizontal beamwidth (degrees)
* gates: starting distance, gate spacing (km), and number of gates
* azimuths: starting azimuth, azimuth spacing (degrees), and number of azimuths
* elevations: starting elevation angle, angle spacing (degrees), and number of elevation angles

Output directory and file name

* output_dir: files will be written to this directory
* output_format: select the preferred file type (usually CFRADIAL)

=== '''Running RadxBeamBlock''' ===

To check all command line options for RadxBeamBlock, including debugging options and file paths, type the following command into a terminal.

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -h</code>

Once your parameter file is complete, use a command similar to the following to run the application:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params param_file_name</code>

RadxBeamBlock

2022-08-05T19:59:16Z

Barbero: /* Digital elevation model data */

RadxBeamBlock estimates the amount of beam blockage due to terrain near a radar.

=== '''Overview''' ===

Overall, RadxBeamBlock takes elevation data, the radar location, and radar characteristics to estimate how much blockage occurs due to terrain.

=== '''Prerequesites''' ===

The following items are required:

* Digital elevation model data
* Radar antenna location
* Radar characteristics
* Parameter files for RadxBeamBlock.

=== '''Digital elevation model data''' ===
Note: a NASA Earthdata account is needed to download SRTM data. Creating an account is free and quick to do, and can be done [https://www.earthdata.nasa.gov/ here].

The supported digital elevation models include Shuttle Radar Topography Mission and ESRI grid data (spheroid). SRTM data can be found online '''[https://lpdaac.usgs.gov/products/srtmgl3sv003/#tools here]'''. SRTM data for the entire globe is 22GB and can be a lot to download! If you only need a subsection for your region of interest, use a recursive script to download them. If you only need a few files, this is easy enough to do manually. Remember to unzip your files. Once completed, you should have a directory with .hgt files spanning your region of interest.

Example:
1 #!/bin/bash
2
3 # download SRTM data for Taiwan region
4
5 for lat in `seq 19 25`
6 do
7 for lon in `seq 117 123`
8 do
9 # get files
10 wget --user=$USERNAME --password=$PASSWORD https://e4ftl01.cr.usgs.gov/MEASURES/SRTMGL1.003/2000.02.11/N${lat}E${lon}.SRTMGL1.hgt.zip
11 done
12 done

=== '''Radar location and characteristics''' ===

The radar location requires the latitude, longitude, and elevation of the antenna (ground level + radar height). In addition, characteristics including the wavelength, beamwidth, gate spacing, azimuths, and elevation angles are required.

=== '''Parameter files''' ===

RadxBeamBlock uses one parameter file. Compared to other Radx applications, the RadxBeamBlock parameter file requires just a few edits in order to run. Most of the parameters that need to be changed relate to the radar characteristics.

==== '''1) Main parameter file''' ====

===== '''Ensure file is up to date''' =====

To obtain the default parameter file, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -print_params > param_file_name</code>

If you already have a parameter file and simply want to check for (and add) updated parameters while retaining current parameters, use the following command:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params orig_param_file_name -print_params > new_param_file_name</code>

===== '''Important parameters''' =====

Input params

* input_dem_parth: directory containing the elevation data
* input_data_format: specifies the format of the elevation data (examples provided in the parameter file)

Radar parameters

* radar_location: latitude, longitude, and elevation (ground elevation
** radar height in km) of the radar antenna
* radar_wavelength_cm: radar wavelength (cm)
* horiz_beam_width_deg: horizontal beamwidth (degrees)
* vert_beam_width_deg: horizontal beamwidth (degrees)
* gates: starting distance, gate spacing (km), and number of gates
* azimuths: starting azimuth, azimuth spacing (degrees), and number of azimuths
* elevations: starting elevation angle, angle spacing (degrees), and number of elevation angles

Output directory and file name

* output_dir: files will be written to this directory
* output_format: select the preferred file type (usually CFRADIAL)

=== '''Running RadxBeamBlock''' ===

To check all command line options for RadxBeamBlock, including debugging options and file paths, type the following command into a terminal.

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -h</code>

Once your parameter file is complete, use a command similar to the following to run the application:

<code lang="bash">/path/to/lrose/install/bin/RadxBeamBlock -params param_file_name</code>

HawkEye

2022-07-08T20:31:10Z

Barbero: /* Colors */

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colormaps can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.

Example: Add PID and RATE_ZH variables from QPE file.

> View metadata of variable in CfRadial .nc file.

<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>

> Add variables of interest to HawkEye parameter file

...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };

Rerun HawkEdit

<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>
[[File:HawkEdit_Add_Variables.png|700px|center]]

==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user. A list of colormaps can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here].

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

HawkEye

2022-07-08T20:30:59Z

Barbero: /* Adding variables to HawkEye parameter file */

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colormaps can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.

Example: Add PID and RATE_ZH variables from QPE file.

> View metadata of variable in CfRadial .nc file.

<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>

> Add variables of interest to HawkEye parameter file

...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };

Rerun HawkEdit

<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>
[[File:HawkEdit_Add_Variables.png|700px|center]]

==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here].

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

HawkEye

2022-07-08T20:30:22Z

Barbero: /* Adding variables to HawkEye parameter file */

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.

Example: Add PID and RATE_ZH variables from QPE file.

> View metadata of variable in CfRadial .nc file.

<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>

> Add variables of interest to HawkEye parameter file

...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };

Rerun HawkEdit

<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>
[[File:HawkEdit_Add_Variables.png|700px|center]]

==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here].

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

Elle full

2022-07-08T19:47:29Z

Barbero: /* Finished! */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files during intermediate steps, and which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. You will need to choose a grid that contains your area of interest as well as the radar(s) you are using.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv
Note: before moving on, check that your Spdb sounding files are populated with data by using the SpdbQuery command below. All parameters except w wind and divergence should be populated.
SpdbQuery -url /path/to/spdb/folder -start "YYYY MM DD HH MM SS" -end "YYYY MM DD HH MM SS"

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data. A tutorial on how to add and view retrieve/calculated variables in HawkEye such as the RATE variables, PID, etc., can be found [http://wiki.lrose.net/index.php/HawkEye#Adding_variables_to_HawkEye_parameter_file here].</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

HawkEye

2022-07-08T19:43:07Z

Barbero: /* Colors */

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.
Example: Add PID and RATE_ZH variables from QPE file
> View metadata of variable in CfRadial .nc file
<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>
> Add variables of interest to HawkEye parameter file
...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };
> Rerun HawkEdit
<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>
[[File:HawkEdit_Add_Variables.png|700px|center]]

==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here].

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

HawkEye

2022-07-08T19:42:27Z

Barbero: /* Adding variables to HawkEye parameter file */

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.
Example: Add PID and RATE_ZH variables from QPE file
> View metadata of variable in CfRadial .nc file
<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>
> Add variables of interest to HawkEye parameter file
...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };
> Rerun HawkEdit
<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>
[[File:HawkEdit_Add_Variables.png|700px|center]]

==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user.

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

HawkEye

2022-07-08T19:41:43Z

Barbero:

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.
Example: Add PID and RATE_ZH variables from QPE file
View metadata of variable in CfRadial .nc file
<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>
Add variables of interest to HawkEye parameter file
...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };
Rerun HawkEdit
<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>
[[File:HawkEdit_Add_Variables.png|700px|center]]
==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user.

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

File:HawkEdit Add Variables.png

2022-07-08T19:33:51Z

Barbero: Showing example of added variable "PID" in this screenshot.

== Summary ==
Showing example of added variable "PID" in this screenshot.

HawkEye

2022-07-08T19:31:38Z

Barbero:

=== '''Overview''' ===
HawkEye is a next generation lidar and radar display tool. It can display real-time and archived CfRadial data either in BSCAN, PPI, or RHI geometry. Editing capability will be added in future releases.

=== '''Running HawkEye''' ===

While relatively new to the research community, HawkEye is a mature viewing tool that has been powering real-time displays for NCAR S-Pol, HIAPER Cloud Radar, and the CSWR Doppler on Wheels for several years. As part of the LROSE project the tool has been updated to work in a research mode with archived CfRadial files. Like the Radx tools, HawkEye can be run in the Virtual Toolbox or compiled as a native application. The look and feel of HawkEye will be slightly different depending on the window environment, but the functionality remains the same across platforms.

To run HawkEye as a basic viewer it is not necessary to create a parameter file, but like the LROSE tools there is more functionality provided via the parameter options. In its simplest invocation:

<code lang="bash">/path/to/HawkEye -f </path/to/CfRadial_files></code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -f </path/to/CfRadial_files></code>

Depending on how the CfRadial files are stored in the directory, it may be necessary to use </path/to/CfRadial_files/cfrad*> to point the program directly to the CfRadials.

If the data is organized under the YYYYMMHH date directory in the path, the user can specify specific time span to display:

<code lang="bash">/path/to/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -archive_url /scr/rain1/rsfdata/projects/pecan/CfRadial/kddc/moments -start_time "2015 06 26 00 00 00" -time_span 7200</code>

This command would start it up to look at the specified data location, from the specified start time, for a time span of 2 hours (argument taken in units of seconds). In this case, since we are searching by time, you are required to have a day directory in the path, so that actual data would be in the 20150626 subdirectory underneath the 'moments' directory.

To check all command line options for HawkEye, type the following command into a bash.

<code lang="bash">/path/to/HawkEye -h</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -h</code>

To obtain the default parameter file for more options, use the following command:

<code lang="bash">/path/to/HawkEye -print_params > $PWD/HawkEye.params</code>

or

<code lang="bash">/Applications/HawkEye.app/Contents/MacOS/HawkEye -print_params > $PWD/HawkEye.params</code>

While HawkEye will do its best to display data properly, the parameter file can force the viewer to use POLAR_DISPLAY for PPI and RHI display and BSCAN_DISPLAY for BSCAN mode. The full parameter file description can be found here: '''[http://wiki.lrose.net/index.php/HawkEye_parameter_file HawkEye parameter file]'''

==== '''Adding variables to HawkEye parameter file''' ====

The default parameter file contains the variables "DBZ", "REF", "VEL", "WIDTH", "SW", "ZDR", "PHIDP", "RHOHV". If you would like to view other retrieved/calculated variables (e.g., KDP, PID, RATE_ZH) in HawkEye, add another struct with the variable of interest in the HawkEye parameter file in lines 345-417. Make sure the raw_name matches the variable in the file which you are viewing and the units are correct. However, the label and color_map are up to the user's discretion. A list of colors can be found [https://github.com/NCAR/lrose-displays/tree/master/color_scales here]. You can check the variable name, units, longname etc., using the ncdump command.
Example: Add PID and RATE_ZH variables from QPE file
View metadata of variable in CfRadial .nc file
<code lang="bash">ncdump -c ./cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc </code>
Add variables of interest to HawkEye parameter file
...
417 ,
418 {
419 label = "PID",
420 raw_name = "PID",
421 filtered_name = "",
422 units = "unitless",
423 color_map = "pid.colors",
424 shortcut = "p"
425 }
426 ,
427 {
428 label = "RATE_ZH",
429 raw_name = "RATE_ZH",
430 filtered_name = "",
431 units = "mm/h",
432 color_map = "prate.colors",
433 shortcut = "r"
434 }
435 };
Rerun HawkEdit
<code lang="bash">HawkEye -params ./HawkEye.qpe.params -f cfrad.20170602_001311.000_to_20170602_001311.000_RCWF_SUR.nc</code>

==== '''Colors''' ====

HawkEye supports many different color scales for different fields that can be specified via external files. Default color palettes are applied to variable names with commonly used meanings (ie. ZDR), but can be overridden by the user.

While most of the GUI viewing options are intuitive, additional documentation on all the details of the HawkEye GUI is under development.

Elle full

2022-07-08T18:39:43Z

Barbero: /* Sounding */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files during intermediate steps, and which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. You will need to choose a grid that contains your area of interest as well as the radar(s) you are using.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv
Note: before moving on, check that your Spdb sounding files are populated with data by using the SpdbQuery command below. All parameters except w wind and divergence should be populated.
SpdbQuery -url /path/to/spdb/folder -start "YYYY MM DD HH MM SS" -end "YYYY MM DD HH MM SS"

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:51:17Z

Barbero: /* Full elle tutorial */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files during intermediate steps, and which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv
Note: before moving on, check that your Spdb sounding files are populated with data by using the SpdbQuery command below. All parameters except w wind and divergence should be populated.
SpdbQuery -url /path/to/spdb/folder -start "YYYY MM DD HH MM SS" -end "YYYY MM DD HH MM SS"

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:47:33Z

Barbero: /* Sounding */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv
Note: before moving on, check that your Spdb sounding files are populated with data by using the SpdbQuery command below. All parameters except w wind and divergence should be populated.
SpdbQuery -url /path/to/spdb/folder -start "YYYY MM DD HH MM SS" -end "YYYY MM DD HH MM SS"

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:36:27Z

Barbero: /* RadxConvert */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:29:49Z

Barbero: /* Sounding */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, the grid will span from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:29:11Z

Barbero: /* Sounding */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798.</li></ol>
minx: initial longitude [-180 to 180]
miny: initial latitude [-90 to 90]
nx: # of longitude points (# of dx)
ny: # of latitude points (# of dy)
dx: preferred longitude grid spacing
dy: preferred latitude grid spacing
Example: if minx=118, miny=20, nx=50, ny=50, dx=0.1, dy=0.1, The box will range from 118º to 123º E and 20º to 25º N.
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar's longitude, latitude, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:04:55Z

Barbero: Undo revision 943 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:04:38Z

Barbero: /* RadxBeamblock */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:03:58Z

Barbero: /* RadxBeamblock */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
Note: for steps 4, 5, and 6, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:01:51Z

Barbero: Undo revision 940 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:01:35Z

Barbero: Undo revision 939 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:01:19Z

Barbero: Undo revision 938 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:01:03Z

Barbero: Undo revision 937 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T17:00:43Z

Barbero: Undo revision 936 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T16:58:38Z

Barbero: /* RadxBeamblock */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T16:53:54Z

Barbero: Undo revision 934 by Barbero (talk)

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T16:53:22Z

Barbero: /* RadxBeamblock */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<li>Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.</li>
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-08T16:52:16Z

Barbero: /* RadxBeamblock */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the latitude, longitude, and altitude (terrain + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
Note: for steps 4 and 5, you can use RadxPrint on the raw or converted .nc radar file to view this information.
RadxPrint -printVars -f /path/to/radar/file
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-07T23:05:56Z

Barbero: /* RadxConvert */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run the command in a for loop or simply run each day one at a time replacing $days with each date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the lat, lon, and altitude (terrin + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-07T23:02:19Z

Barbero: /* Running Radx Applications */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the lat, lon, and altitude (terrin + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-07T23:01:16Z

Barbero:

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of this [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
export days=“date1 date2 date3”
export daysbb="date1"
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the lat, lon, and altitude (terrin + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>

Elle full

2022-07-07T22:58:31Z

Barbero: /* Full elle tutorial */

=== '''Full elle tutorial''' ===

This workflow will convert raw radar files to the CfRadial format and estimate the surface rain rate, emphasizing some parameters that need to be defined for each application. This workflow assumes that soundings need to be estimated from GFS analysis data. Note, the current parameter files are currently optimized to work on Unix systems. It is also helpful to have the programs [https://formulae.brew.sh/formula/ncview ncview] and [https://formulae.brew.sh/formula/nco nco] to debug by plotting and dumping the contents of .nc files, which can be downloaded using [https://brew.sh/ Homebrew].

For more information, refer to the following pages on how LROSE [http://wiki.lrose.net/index.php/KDP_estimation calculates KDP], estimates the [http://wiki.lrose.net/index.php/RadxPid_fuzzylogic dominant PID category], calculates [http://wiki.lrose.net/index.php/RadxRate_equations rain rates], and estimates the [http://wiki.lrose.net/index.php/RadxQpe_expanded surface precipitation rate].

==== '''Download parameter files''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the [https://drive.google.com/file/d/1lHZCGtI-0E20xlVvEJjPd7glO7DHCVoJ/view?usp=sharing elle_full.tar.gz] file</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Extract contents into the desired directory</li></ol>
tar -zxvf elle_full.tar.gz

The user can create their own parameter files for each function as shown in this [http://wiki.lrose.net/index.php/RadxKdp#Ensure_file_is_up_to_date example], but then directories in each parameter file will need to be set manually. The environment files included in the elle_full.tar.gz file will set the directory structure with modifications to only two lines of code. These lines will have to be rerun if opening a new terminal workspace.

==== '''Set up environment variables''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Either place the raw radar data in a subdirectory of elle_full called "raw" or make a note of the path to the radar data.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In /elle_full/env_dirs, edit the directories (note: there is purposely no slash at the beginning of PROJ). Together, HOMED/PROJ should direct you to the elle_full directory. RAW is the full path to the raw radar data.</li></ol>
export HOMED="/path/to/home/directory"
export PROJ=“short/path/to/elle_full”
export RAW="/path/to/elle_full/raw"
<ol start="3" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, edit radar name (e.g., KHGX for Houston). This will set future directories in other parameter files and reduce the number of edits that need to be made.</li></ol>
export RADAR_NAME="RADAR"
export radar_name="radar"
export days="date1 date2 date3"
export daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>In /elle_full/env_vars, set the coefficients for the rain rate equations. More information regarding the equations available in LROSE are described on [http://wiki.lrose.net/index.php/RadxRate_equations this page] and on page 4 of [https://ams.confex.com/ams/37RADAR/webprogram/Paper275705.html this AMS manuscript]. The default Z-R relationship included in env_vars is from the [https://archive.eol.ucar.edu/projects/dynamo/spol/parameters/rain_rate/rain_rates.html Mismo dataset].</li></ol>

==== '''Obtain GFS and topographic data''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Download the GFS analysis from [https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/global-forcast-system-gfs NCEI]. Under 'Data on NCEI Servers,' select the HAS access option for the 0.5º domain of GFS Analysis (GFS-ANL). Since the tutorial includes two different files, select all UTC cycles and enter the start and end dates. Select the batch option and enter your email. Download the data when your order is ready and put it in a folder named gfs within the elle_full directory.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Download the topography data using the following command. ''Note: the folder is approximately 22 GB.'' If you know which region of the globe you need, you can poke around the website to determine which subfolder is necessary for your dataset. Unzip the contents and make note of the resulting directory.</li></ol>
wget -r https://dds.cr.usgs.gov/srtm/version2_1/SRTM3/ *link has moved*

=== '''Running Radx Applications''' ===
<ol start="1" style="list-style-type: decimal;">
<li>Set the variables in the terminal and set the dates in the format YYYYMMDD. Print out the directory structure to make sure it's pointing to the right directory.</li></ol>
source env_dirs
source env_vars
export days=“date1 date2 date3”
export daysbb="date1"
echo $HOMED/$PROJ

==== '''RadxConvert''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Convert the raw file to a CfRadial file. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxConvert -params ./params/RadxConvert.* -f $RAW/$RADAR_NAME/$days/*.raw*

==== '''RadxBeamblock''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Replace "RCWF" in the filename for RadxBeamBlock.RCWF.new with the name of the radar (e.g., RadxBeamBlock.KHGX.new).</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In the params folder, modify line 53 of RadxBeamBlock.RCWF.new to indicate the directory of the unzipped topographic data.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In the params folder, modify line 170 of RadxBeamBlock.RCWF.new to indicate the radar name.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In the params folder, modify line 188-191 of RadxBeamBlock.RCWF.new to indicate the lat, lon, and altitude (terrin + antenna height) of the radar.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>In the params folder, insert the radar wavelength (cm), horizontal (degrees), and vertical (degrees) beamwidths in lines 215, 225, and 235 of RadxBeamBlock.RCWF.new, respectively.</li></ol>
<ol start="6" style="list-style-type: decimal;">
<li>In the params folder, insert the range gate, azimuth, and elevation geometries in lines 263-306 of RadxBeamBlock.RCWF.new, respectively. The range gate and azimuth fields should match the observed spacing. If your elevation angles are unevenly spaced, it might be easier to artificially set the spacing to 0.25 or 0.5 degrees. The programs will interpolate the estimated blockage to the actual elevation angle later. The max elevation angle will depend on the radar location. Near Houston, TX, for example, only 2-3 degrees is probably sufficient, but a 10-20 degrees might be necessary near Taiwan.</li></ol>
<ol start="7" style="list-style-type: decimal;">
<li>Run RadxBeamBlock for the RCWF radar location</li></ol>
/path/to/RadxBeamBlock -params ./params/RadxBeamBlock.$RADAR_NAME.*

==== '''Sounding''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the params directory, rename the Grib2toMdv.gfs.taiwan file to whatever you prefer.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the projection parameters in lines 764-770. PROJ_LATLON is a fine choice and does not require the other parameters to be set.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Grib2toMdv.gfs.taiwan, insert the desired output grid in lines 792-798. minx (lon) and miny (lat) set the lower left corner of the box around the radar, dx and dy indicate the spacing in degrees, and nx and ny indicate the number of points.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In Mdv2SoundingSpdb.gfs, insert the radar lat, lon, and elevation in lines 286-289.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>Convert the GFS Grib2 files to Mdv (Grib2toMdv) and then extract the sounding closest to the RCWF radar (Mdv2SoundingSpdb)</li></ol>
/path/to/Grib2toMdv -params ./params/Grib2toMdv.* -f ./gfs/*.grb2
/path/to/Mdv2SoundingSpdb -params ./params/Mdv2SoundingSpdb.* -f ./gfs/mdv/$i/*.mdv

==== '''RadxRate''' ====
<ol start="1" style="list-style-type: decimal;">
<li>Ensure that you have the proper [http://wiki.lrose.net/index.php/RadxPid#PID_thresholds_file pid_thresholds file] for your radar. Make any necessary modifications.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Make sure the filtering/processing parameters in [http://wiki.lrose.net/index.php/RadxKdp RadxKdp], [http://wiki.lrose.net/index.php/RadxPid RadxPid], and [http://wiki.lrose.net/index.php/RadxRate RadxRate] are appropriate for your needs.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>In RadxPid.new, insert the accurate path to your pid_thresholds file in line 27.</li></ol>
<ol start="4" style="list-style-type: decimal;">
<li>In RadxRate.new, make sure the paths to the RadxKdp, RadxPid, and rain rate parameter files are correct in lines 284, 303, and 350.</li></ol>
<ol start="5" style="list-style-type: decimal;">
<li>Run RadxRate to calculate Kdp, the PID, and various rain rates. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxRate -params ./params/RadxRate.* -f ./convert/$RADAR_NAME/$days/*.nc

==== '''RadxQpe''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In RadxQpe.new, set your preferred output azimuthal resolution in line 32.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>In RadxQpe.new, check that the path to the beamblock file matches the path in line 269.</li></ol>
<ol start="3" style="list-style-type: decimal;">
<li>Run RadxQpe to estimate the near-surface rainfall using the hybrid method. If you have data from more than one day, run a for loop or run each day one at a time replacing $days with the date (YYYYMMDD).</li></ol>
/path/to/RadxQpe -params ./params/RadxQpe.* -f ./rate/$RADAR_NAME/$days/*.nc

==== '''Finished!''' ====
<ol start="1" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 3 new directories: convert, rate, and qpe. You can look through the the files in those directories to inspect the new files and check the quality of the data.</li></ol>

=== '''Run programs from a sample script''' ===
<ol start="1" style="list-style-type: decimal;">
<li>If you'd like to run these applications through a script or on a cluster, you can use the script run_radx.sh (or make your own!). Follow all of the steps above, except for last item in each subsection where you run the Radx applications. Make sure you source the env_dirs and env_vars files.</li></ol>
<ol start="2" style="list-style-type: decimal;">
<li>Edit directories in run_radx.sh script (lines 3 and 12)</li></ol>
cd /PATH/TO/elle_full
rdir="/PATH/TO/lrose/bin"
<ol start="3" style="list-style-type: decimal;">
<li>Edit the days (YYYYMMDD) in run_radx.sh script (lines 10-11)</li></ol>
days="date1 date2 date3..."
daysbb="date1"
<ol start="4" style="list-style-type: decimal;">
<li>Make the file executable, if it isn't already</li></ol>
chmod u+x run_radx.sh
<ol start="5" style="list-style-type: decimal;">
<li>Run shell script</li></ol>
./run_radx.sh
<ol start="6" style="list-style-type: decimal;">
<li>In the elle_full directory, you should now see 4 new directories: beamblock, convert, rate, and qpe. </li></ol>