LROSE Wind Tutorial

This interactive tutorial takes you through the steps of how to run FRACTL and SAMURAI. FRACTL is a fast traditional solver with integrated interpolation using LROSE infrastructure, and it is able to perform both gridding and multi-Doppler synthesis for airborne radars and multiple ground-based radars. This is different than Radx2Grid which is only capable of gridding data for a single ground-based radar. FRACTL adopts techniques from older REORDER and CEDRIC software, but with some new improvements. In contrast to the older tools, in which interpolation is followed by synthesis, FRACTL does both steps together. FRACTL doesn’t require the CfRadial file with an aggregation of the sweeps, which is different than Radx2Grid. FRACTL performs a nearest neighbor interpolation of the reflectivity instead of bilinear interpolation in Radx2Grid. It combines the Doppler velocities together using the traditional ‘normal’ equations but with singular value decomposition of raw velocities in spherical space rather than Cramer’s rule in gridded space like CEDRIC.

SAMURAI is a variational analysis technique that is described in Bell et al. (2012), Foerster et al. (2014), Foerster and Bell (2017), Cha and Bell (2021), Cha and Bell (2023), and other publications. The SAMURAI analysis yields a maximum likelihood estimate of the atmospheric state for a given set of observations and error estimates by minimizing a variational cost function. SAMMURAI employs a finite element approach using a series of overlapping cubic B-spline basis functions. This approach offers several advantages over a conventional grid-point representation, including:

• Allowing for a scaled-controlled analysis that can incorporate multiple spatial filters in the background error covariance and analytic spatial derivatives in observational space.
• No need to interpolate irregular data distributions to a Cartesian grid, and the data can be used in the variational minimization in their native locations.
• Spatial derivative constraints are considered as pseudo-observations, which can be implicitely integrated during the cost function minimmization at any point in their native coordinate.

It has more features and more development than FRACTL, and is generally recommended over the former for publication-quality analysis. The two programs together provide a powerful combination to produce high quality multi-Doppler wind fields from ground-based, airborne, or mixed configurations.

Tutorial Overview¶

1. Setup¶

QC-ed input data, a parameter file, and a center file:¶

a. QC-ed radar data files are provided (courtesy Meteorological Service of Catalonia, an overview of the tornado case can be found in DD_Tornado_ERAD24.ipynb):

*The QC process is not included in this tutorial. An example of one type of QC can be found in the QC tutorial and HawkEdit/solo are another tool to use for QC-ing data.

b. A terrain file for SAMURAI is provided in order to activate wind retrievals over complex terrain

• terrain.hgt

*If a user wishes to retrieve wind flows over complex terrain, a terrain.hgt is required to be included in the data folder. More details about how to generate a terrain.hgt can be found in Section X.

2. Basic FRACTL and SAMURAI “how to”¶

Both FRACTL and SAMURAI are run on the command line, using simple commands like this:

• fractl -params /path/to/fractl.params
• samurai -params /path/to/samurai.params

All of the information about how FRACTL and SAMURAI should analyze data (e.g., paths to input data, variable names, grids, filters) is defined in parameter files for each application. Therefore, most of the work to run FRACTL and SAMURAI goes into setting up the parameter files and placing files in the correct location.

FRACTL requirements:

• radar data (CfRadial)
• parameter file

SAMURAI requirements:

• data (e.g., radar data, radiosondes, in-situ aircraft data)
• center file
• parameter file
• optional: terrain file
• optional: background file

How to set up FRACTL¶

a. Set up directory structure:

All data that FRACTL will use need to be in the same directory. This tutorial will set up input and output directories for the initial FRACTL exercises. For additional FRACTL runs, we recommend creating new input and output directories to keep all analyses separate.

b. Parameter files:

FRACTL’s parameter file contains information about data I/O, the cartesian grid, and variable names. In this tutorial, the user will create their own parameter file from scratch and fill in the important parameters.

How to set up SAMURAI¶

a. Set up directory structure:

All data and the center file defining the Cartesian grid that SAMURAI will use need to be in the same directory. This tutorial will set up input and output directories for the initial SAMURAI exercises. For additional SAMURAI runs, we recommend creating new input and output directories to keep all analyses separate.

b. Center file:

The .cen file will be generated by the user through the provided Perl script. SAMURAI uses the center file to define the Cartesian frame of reference for the analysis, which be either be static or moving. It can be used to time-space correct the position of radar data in a moving storm system. It also sets the temporal limits for data to be included in the analysis. The center files are named according to the following convention: YYYYMMDD.cen

• Provided: Generate_center.pl

c. Parameter files: SAMURAI’s parameter file contains information about data I/O, the cartesian grid, variable names, filters, and other scientific choices. In this tutorial, the user will create their own parameter file from scratch and fill in the important parameters.

Environment and packages¶¶

First, we import the required python packages to run this notebook. Most of the LROSE processing can be done with the os package and shell commands.

We will copy the required data to one directory as SAMURAI requires and organize the directory by performing the following commands:

2. Prepare data for analysis¶

In this tutorial, the provided data have already been QC-ed. For a standard procedure, users are recommended to perform the following procedure:

• Convert radar data from level 2 or other native format to CfRadial using RadxConvert

  • SAMURAI can also incorporate non-radar data, and can read many commonly used formats. All data files in the input directory will be included in the analysis. The read subroutine for each datatype is determined by the file suffix.

• Quality-controlled raw data by desired QC-tools. See the QC tutorial for an example of how QC can be done; HawkEdit and Solo can also be used to QC data.

3. Setup the FRACTL parameter file¶

3.1 Generate the parameter file¶

First, we need to generate the parameter file! LROSE applications use a standard -print_params flag to generate parameter files with the defaults. We just need to specify the location and name of the parameter file. We have already set up a directory where you can place the parameter files. Feel free to change the name of the file if you prefer.

Now that we have generated the parameter file, we can inspect the file. In JupyterHub, you can edit the file either in a terminal with vi or by opening the text file via the lefthand menu - whichever you prefer! The file should be located here: /home/jovyan/lrose-hub/params/wind_guided

3.2 Modify the parameter file¶

Now we will step through and edit the parameter file.

First, we’ll start with the parameters associated with the input/output data directories and filenames.

Next, we’ll define the cartesian grid.

Finally, we’ll define the variable names so that FRACTL knows which variables to use in the analysis.

Congrats! You’ve finished setting up the FRACTL parameter file!

4. Run FRACTL¶

After modifying the parameter file, direct to the parameter file by using the -params flags

5. Plot FRACTL results¶

Environment and packages¶

Import the packages to plot the FRACTL output. These are pre-installed for this tutorial, but if you run this notebook on your own computer you may need to install them yourself using pip or conda.

Load the netcdf file

If you can see the above plot, then congratulations! You have successfully created and plotted a dual-Doppler analysis. Now let’s try SAMURAI¶

6. Set the SAMURAI parameters¶

6.1 Set up the center file¶

One difference between FRACTL and SAMURAI is the requirement of a centerfile, which allows for a moving domain. We will modify and run the Generate_center.pl perl script to generate a time-series of center locations by providing an estimation of the moving speed of the target and the reference location.

6.2 Generate the center file¶

!chmod u+x ./Generate_center.pl

Generate the .cen file in the input directory using the following command: !Generate_center.pl

Generate_center.pl creates the file in the directory containing this notebook. We just need to move it to the input directory, which we can do with the following command. Using either a terminal or the nagivation at the left of the notebook, double check that the file is now in the input directory.

6.3 Generate the SAMURAI parameter file¶

Like FRACTL, SAMURAI uses the -print_params flag to generate parameter files with the defaults. We just need to specify the location and name of the parameter file. We have already set up a directory where you can place the parameter files. Feel free to change the name of the file if you prefer.

!${LROSE_DIR}/samurai -print_params > ./lrose_params/samurai_params

6.4 Modify the SAMURAI parameter file¶

Now we will step through and edit the parameter file.

First, we’ll start with the parameters associated with the input/output data directories and filenames.

Next, we’ll define the cartesian grid.

SAMURAI requires the user to set the background time.

Next, we’ll set some remaining parameters which define the radar variable names, radii of influence, and how much data is processed.

Finally, we’ll correct some default parameters..

For a list of additional SAMURAI parameters, please refer to the appendix at the end of this notebook.

6.5 Generate a terrain file¶

a. Data source and structure of input terrain file

Terrain data is chosen by the user and can be obtained from different sources such as model data, ASTER-DEM, SRTM, and others. Users are required to save the file as terrain.hgt. The subroutine for reading the file is determined by its suffix (hgt).

The terrain file data should be structured as follows:

X and Y should match the desired output grid which corresponds to the “GRID DEFINITION SECTION” in the parameter file.

In this tutorial, the terrain file is provided.

b. Modify the SAMURAI parameter file related to the terrain boundary conditions

Now, we can modify the weights associated with the terrain calculation. When a terrain file is included in the directory, users can determine the strength of the boundary condition by changing the magnitude of neumann_u_weight, neumann_v_weight, and dirichlet_w_weight. The neumann_u_weight, neumann_v_weight regulate the impermeability of the horizontal flow as it encounters the terrain, and the dirichlet_w_weight governs the vertical motion generated by topographic influences when horizontal flow interacts with the terrain.

The default value is 0.1 for all the boundary condition weights. However, the choice of settings must be carefully evaluated on a case-by-case basis and their interpretations should be made accordingly when drawing scientific conclusions.

Now let’s take a look at the provided terrain.hgt text file:

7. Run SAMURAI¶

After modifying the parameter file, direct to the parameter file by using the -params flags

8. Plot SAMURAI results¶

Environment and packages¶¶

The packages were already imported when we plotted the FRACTL results.

Load the netcdf file¶

9. Apply FRACTL Condition Number¶

The FRACTL ‘Condition Number’ represents how well-posed the geometry is for a particular multi-Doppler retrieval. It is similar to the USTD and VSTD field from CEDRIC, which represent the standard deviation of the expected error in the resolved wind field based on the geometry. Since SAMURAI solves for the wind field globally using spline basis functions, it is difficult to calculate this error at each gridpoint. If a FRACTL analysis is run on the same domain, then it can be used to remove regions with poor geometry. Here, we use a value of 10 to threshold the SAMURAI analysis.

Plot cross sections of Divergence and Absolute Vorticity Fields along 41.89 latitude

Congratulations! You have successfully completed the LROSE Wind tutorial. There are other options to set in both FRACTL and SAMURAI, but the steps and parameters in this tutorial can produce a good quality wind field in many cases. Be critical with your own analysis and feel free to reach out to the LROSE team for questions as you analyze your own data.¶

Appendix - additional SAMURAI parameters¶

The following are optional parameters:¶

• BACKGROUND SECTION
  • load_background [line 107]
    • TRUE: A first guess of the analysis stored in the samurai_Background.in will be loaded. This can come from a model, sounding, or other data source if available. Most of the time it is not available so it should be set to FALSE.
    • FALSE (default setting)
  • adjust_background [line 132]
    • TRUE: the background will be adjusted to satisfy the mass continuity and match the supplied data using a variational adjustment. Without this, it is just interpolated simply to the grid.
• OPERATION SECTION
  • mode [line 170]: The default mode is MODE_XYZ, which is Cartesian grid. You can run in cylindrical mode with MODE_RTZ. Stay tuned for more radar centric RTZ mode in the near future!
  • preprocess_obs [line 219]
    • TRUE: the raw data files will be preprocessed according to their file suffixes, and a samurai_Observations.in file will be generated.
    • FALSE: SAMURAI will load the observations from the samurai_Observations.in file located in the data_directory. This is useful for generating fake data, or for saving some time if you already ran the preprocessing of lots of files.
  • num_iterations [line 232]: SAMURAI has the ability to take the output analysis and use it as a first guess for another analysis. This can be useful to create a ‘coarse’ analysis, followed by a ‘fine’ analysis. It is very important to set the observation and background errors appropriately if you choose this option.
• BACKGROUND SECTION
  • ref_state [line 380]: a sounding file to define the hydrostatic reference state used in the analysis. The default is the Dunion (2011) moist tropical sounding file. The analysis is generally not sensitive to this sounding, as it is just used to provide a reference density field and fall speed relationships in most cases. If you are analyzing thermodynamic information or have a very different freezing level it may be useful to provide a more appropriate sounding. The file format is similar to that of WRF or CM1 idealized soundings.
  • i_background_roi [line 402], j_background_roi [line 412]: These set the background radius of influence when loading a background field.
• RADAR SECTION
  • qr_variable [line 452]: In the default ‘dbz’ mode, the reflectivity is just interpolated and not included in the cost function minimization. If this is set to ‘qr’, then reflectivity is converted to liquid water using Z-M relationships defined in Gamache et al. (1993) and used as an additional variable in the cost function minimization. This is useful if you have other liquid water measurements to assimilate. Since this is a relatively simple Z-M it should be used with caution.
  • dbz_pseudow_weight [line 516]: SAMURAI has the ability to set a ‘soft’ w=0 boundary condition at echo top, and setting this weight will determine how soft or hard that constraint is enforced. Usually a hard w=0 is enforced via the spline boundary conditions at the bottom and top of the domain, so this is optional. It can be useful if the vertical velocities are believed to be too strong at the top of the cloud.
  • melting_zone_width [line 532], mixed_phase_dbz [line 540], rain_dbz [line 548]: SAMURAI has some basic terminal fall speed corrections that use reflectivity and Z-VT relationships. These parameters control which relationships are used depending on dbz and the height of the zero C level (which is determined from the reference sounding).
• BOUNDARY CONDITIONS SECTION

Available options are R0, R1T0, R1T1, R1T2, R2T10, R2T20, R3, and PERIODIC following Ooyama (2002). The default “non”- boundary condition (R0) adds a buffer set of gridpoints that are used to calculate the solution but are discarded for output. Different boundary conditions can be set on the left (L) or right (R) side of the domain for each variable and dimension. The most common option other than R0 would be R1T0 for vertical velocity (rhow = 0) at the surface and/or domain top. Periodic domains are only valid for the i and j dimension, but are available in both the XYZ and RTZ mode. Improved boundary conditions are currently in development.

• OBSERVATION ERRORS SECTION

Specified error is given in terms of a standard deviation, and is fixed for all observations from a particular instrument except radar. In the radar case, the spectrum width and elevation angle (proportional to terminal fall speed contribution) are used to define the error for each radar gate. A minimum error (radar_min_error) is also enforced.

• ITERATION DEPENDENT SECTION
  • mc_weight [line 1450]: specify the weight given to the mass continuity constraint. Default is set to 1, and is generally not recommended to change unless you have a good reason.
  • i_filter_length [line 1546], j_filter_length [line 1558], k_filter_length [line 1570]
  • Gaussian recursive filter is a low-pass filter. Smaller filter lengths retain more detail, and larger filter lengths smooth more. The general recommendation is 4, 4, 2, which removes features less than approximately 4dx, 4dy, and 2dz from the analysis.
  • i_spline_cutoff [line 1582], j_spline_cutoff [line 1594], k_spline_cutoff [line 1606]
  • Spline cutoff is implemented as a third derivative constraint on the cubic B-spline basis during the spline transform. It is a sharper filter than the Gaussian, roughly equivalent to a sixth order filter.
  • i_max_wavenumber [line 1618], j_max_wavenumber [line 1630], k_max_wavenumber [line 1642]
  • The Fourier spectral filter is the sharpest filter. It can be used for desired effects, such as explicitly removing high-wavenumber features in the spectral domain (usually for RTZ analysis), or for producing a ‘mean’ field by restricting to wavenumber zero.