ARM Logo

Py-ART Gridding


Overview

Within this notebook, we will cover:

  1. What is gridding and why is it important?

  2. An overview of gridding with Py-ART

  3. Test out a different gridding routine

  4. Apply Gridding to a Selection of Files

Prerequisites

Concepts

Importance

Notes

Py-ART Basics

Helpful

Basic features

Intro to Cartopy

Helpful

Basic features

Matplotlib Basics

Helpful

Basic plotting

NumPy Basics

Helpful

Basic arrays

  • Time to learn: 15 minutes


Imports

import os
from pathlib import Path
import glob
import warnings

import cartopy.crs as ccrs
import matplotlib.pyplot as plt


import pyart
from pyart.testing import get_test_data
import xarray as xr

warnings.filterwarnings('ignore')

What is gridding and why is it important?

Antenna vs. Cartesian Coordinates

Radar data, by default, is stored in a polar (or antenna) coordinate system, with the data coordinates stored as an angle (ranging from 0 to 360 degrees with 0 == North), and a radius from the radar, and an elevation which is the angle between the ground and the ground.

This format can be challenging to plot, since it is scan/radar specific. Also, it can make comparing with model data, which is on a lat/lon grid, challenging since one would need to transform the model daa cartesian coordinates to polar/antenna coordiantes.

Fortunately, PyART has a variety of gridding routines, which can be used to grid your data to a Cartesian grid. Once it is in this new grid, one can easily slice/dice the dataset, and compare to other data sources.

Why is Gridding Important?

Gridding is essential to combining multiple data sources (ex. multiple radars), and comparing to other data sources (ex. model data). There are also decisions that are made during the gridding process that have a large impact on the regridded data - for example:

  • What resolution should my grid be?

  • Which interpolation routine should I use?

  • How smooth should my interpolated data be?

While there is not always a right or wrong answer, it is important to understand the options available, and document which routine you used with your data! Also - experiment with different options and choose the best for your use case!

An overview of gridding with Py-ART

Let’s dig into the regridding process with PyART!

Read in and Visualize a Test Dataset

Let’s start with the same file used in the previous notebook (PyART Basics), which is a radar file from Northern Oklahoma.

file = "../../data/uah-armor/RAW_NA_000_125_20080411182229"
radar = pyart.io.read(file)

Let’s plot up quick look of reflectivity, at the lowest elevation scan (closest to the ground)

fig = plt.figure(figsize=[12, 12])
display = pyart.graph.RadarDisplay(radar)
display.plot_ppi('reflectivity',
                 cmap='pyart_HomeyerRainbow')
../../_images/2daddf7741e6825b7e58cc46d882005f804f1e30f2a54260a972f62cded29bb5.png

As mentioned before, the dataset is currently in the antenna coordinate system measured as distance from the radar

Setup our Gridding Routine with pyart.map.grid_from_radars()

Py-ART has the Grid object which has characteristics similar to that of the Radar object, except that the data are stored in Cartesian coordinates instead of the radar’s antenna coordinates.

pyart.core.Grid?
Init signature:
pyart.core.Grid(
    time,
    fields,
    metadata,
    origin_latitude,
    origin_longitude,
    origin_altitude,
    x,
    y,
    z,
    projection=None,
    radar_latitude=None,
    radar_longitude=None,
    radar_altitude=None,
    radar_time=None,
    radar_name=None,
)
Docstring:     
A class for storing rectilinear gridded radar data in Cartesian coordinate.

Refer to the attribute section for information on the parameters.

To create a Grid object using legacy parameters present in Py-ART version
1.5 and before, use :py:func:`from_legacy_parameters`,
grid = Grid.from_legacy_parameters(fields, axes, metadata).

Attributes
----------
time : dict
    Time of the grid.
fields : dict of dicts
    Moments from radars or other variables.
metadata : dict
    Metadata describing the grid.
origin_longitude, origin_latitude, origin_altitude : dict
    Geographic coordinate of the origin of the grid.
x, y, z : dict, 1D
    Distance from the grid origin for each Cartesian coordinate axis in a
    one dimensional array. Defines the spacing along the three grid axes
    which is repeated throughout the grid, making a rectilinear grid.
nx, ny, nz : int
    Number of grid points along the given Cartesian dimension.
projection : dic or str
    Projection parameters defining the map projection used to transform
    from Cartesian to geographic coordinates. None will use the default
    dictionary with the 'proj' key set to 'pyart_aeqd' indicating that
    the native Py-ART azimuthal equidistant projection is used. Other
    values should specify a valid pyproj.Proj projparams dictionary or
    string. The special key '_include_lon_0_lat_0' is removed when
    interpreting this dictionary. If this key is present and set to True,
    which is required when proj='pyart_aeqd', then the radar longitude and
    latitude will be added to the dictionary as 'lon_0' and 'lat_0'.
    Use the :py:func:`get_projparams` method to retrieve a copy of this
    attribute dictionary with this special key evaluated.
radar_longitude, radar_latitude, radar_altitude : dict or None, optional
    Geographic location of the radars which make up the grid.
radar_time : dict or None, optional
    Start of collection for the radar which make up the grid.
radar_name : dict or None, optional
    Names of the radars which make up the grid.
nradar : int
    Number of radars whose data was used to make the grid.
projection_proj : Proj
    pyproj.Proj instance for the projection specified by the projection
    attribute. If the 'pyart_aeqd' projection is specified accessing this
    attribute will raise a ValueError.
point_x, point_y, point_z : LazyLoadDict
    The Cartesian locations of all grid points from the origin in the
    three Cartesian coordinates. The three dimensional data arrays
    contained these attributes are calculated from the x, y, and z
    attributes. If these attributes are changed use :py:func:
    `init_point_x_y_z` to reset the attributes.
point_longitude, point_latitude : LazyLoadDict
    Geographic location of each grid point. The projection parameter(s)
    defined in the `projection` attribute are used to perform an inverse
    map projection from the Cartesian grid point locations relative to
    the grid origin. If these attributes are changed use
    :py:func:`init_point_longitude_latitude` to reset the attributes.
point_altitude : LazyLoadDict
    The altitude of each grid point as calculated from the altitude of the
    grid origin and the Cartesian z location of each grid point.  If this
    attribute is changed use :py:func:`init_point_altitude` to reset the
    attribute.
Init docstring: Initalize object.
File:           /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/grid.py
Type:           type
Subclasses:     

We can transform our data into this grid object, from the radars, using pyart.map.grid_from_radars().

Beforing gridding our data, we need to make a decision about the desired grid resolution and extent. For example, one might imagine a grid configuration of:

  • Grid extent/limits

    • 20 km in the x-direction (north/south)

    • 20 km in the y-direction (west/east)

    • 15 km in the z-direction (vertical)

  • 500 m spatial resolution

The pyart.map.grid_from_radars() function takes the grid shape and grid limits as input, with the order (z, y, x).

Let’s setup our configuration, setting our grid extent first, with the distance measured in meters

z_grid_limits = (0.,15_000.)
y_grid_limits = (-150_000.,150_000.)
x_grid_limits = (-150_000.,150_000.)

Now that we have our grid limits, we can set our desired resolution (again, in meters)

grid_resolution = 500

Let’s compute our grid shape - using the extent and resolution to compute the number of grid points in each direction.

def compute_number_of_points(extent, resolution):
    return int((extent[1] - extent[0])/resolution)

Now that we have a helper function to compute this, let’s apply it to our vertical dimension

z_grid_points = compute_number_of_points(z_grid_limits, grid_resolution)
z_grid_points
30

We can apply this to the horizontal (x, y) dimensions as well.

x_grid_points = compute_number_of_points(x_grid_limits, grid_resolution)
y_grid_points = compute_number_of_points(y_grid_limits, grid_resolution)

print(z_grid_points,
      y_grid_points,
      x_grid_points)
30 600 600

Use our configuration to grid the data!

Now that we have the grid shape and grid limits, let’s grid up our radar!

grid = pyart.map.grid_from_radars(radar,
                                  grid_shape=(z_grid_points,
                                              y_grid_points,
                                              x_grid_points),
                                  grid_limits=(z_grid_limits,
                                               y_grid_limits,
                                               x_grid_limits),
                                  fields=['reflectivity'],
                                  gridding_algo='map_gates_to_grid',
                                  h_factor=0.,
                                  nb=0.6,
                                  bsp=1.,
                                  min_radius=200.
                                 )
grid
<pyart.core.grid.Grid at 0x7f1f460430a0>

We now have a pyart.core.Grid object!

Plot up the Grid Object

Plot a horizontal view of the data

We can use the GridMapDisplay from pyart.graph to visualize our regridded data, starting with a horizontal view (slice along a single vertical level)

display = pyart.graph.GridMapDisplay(grid)
display.plot_grid('reflectivity',
                  level=2,
                  vmin=-20,
                  vmax=60,
                  cmap='pyart_HomeyerRainbow')
../../_images/c9f4c6da1f594918500ec247bf8837d5245921b658635151a1e6accdeacf3fbc.png

Plot a Latitudinal Slice

We can also slice through a single latitude or longitude!

display.plot_latitude_slice('reflectivity',
                            lat=35.1,
                            vmin=-20,
                            vmax=60,
                            cmap='pyart_HomeyerRainbow')
plt.xlim([-20, 20]);
../../_images/c8a4a9925ceb361a900e20268f7533491160bc72b840d6041701976ed154f028.png

Plot with Xarray

Another neat feature of the Grid object is that we can transform it to an xarray.Dataset!

ds = grid.to_xarray()
ds
<xarray.Dataset>
Dimensions:       (time: 1, z: 30, y: 600, x: 600)
Coordinates:
  * time          (time) object 2008-04-11 18:22:42
  * z             (z) float64 0.0 517.2 1.034e+03 ... 1.448e+04 1.5e+04
    lat           (y, x) float64 33.29 33.29 33.29 33.29 ... 35.98 35.98 35.98
    lon           (y, x) float64 -88.39 -88.38 -88.37 ... -85.12 -85.11 -85.1
  * y             (y) float64 -1.5e+05 -1.495e+05 ... 1.495e+05 1.5e+05
  * x             (x) float64 -1.5e+05 -1.495e+05 ... 1.495e+05 1.5e+05
Data variables:
    reflectivity  (time, z, y, x) float32 nan nan nan nan ... nan nan nan nan
    ROI           (time, z, y, x) float32 2.222e+03 2.218e+03 ... 2.972e+03

Now, our plotting routine is a one-liner, starting with the horizontal slice:

ds.isel(z=2).reflectivity.plot(cmap='pyart_HomeyerRainbow',
                               vmin=-20,
                               vmax=60);
../../_images/0a7a3026f692fcde380786b21502dedff4d4dc386999e3e8e29ae1b43d19c450.png

And a vertical slice at a given y dimension (latitude)

ds.sel(y=-3000,
       method='nearest').reflectivity.plot(cmap='pyart_HomeyerRainbow',
                                           vmin=-20,
                                           vmax=60);
../../_images/eb3b5babb4cbac845cb66fa3492ba0a12278fe0175bcd352c25fa838b4adecf9.png

Try a Different Gridding Technique

nearest_grid = pyart.map.grid_from_radars(radar,
                                  grid_shape=(z_grid_points,
                                              y_grid_points,
                                              x_grid_points),
                                  grid_limits=(z_grid_limits,
                                               y_grid_limits,
                                               x_grid_limits),
                                  fields=['reflectivity'],
                                  weighting_function='Nearest',
                                  nb=0.6,
                                  bsp=1.,
                                  min_radius=200.
                                 )
display.plot_latitude_slice('reflectivity',
                            lat=35.1,
                            vmin=-20,
                            vmax=60,
                            cmap='pyart_HomeyerRainbow')
plt.xlim([-20, 20]);
../../_images/c8a4a9925ceb361a900e20268f7533491160bc72b840d6041701976ed154f028.png
display = pyart.graph.GridMapDisplay(nearest_grid)
display.plot_latitude_slice('reflectivity',
                            lat=35.1,
                            vmin=-20,
                            vmax=60,
                            cmap='pyart_HomeyerRainbow')
plt.xlim([-20, 20]);
../../_images/0158aa805bb28c30da0e19384fa09cc269d31ca7a2a4fec022f219ff72970e19.png

Apply Gridding to a Selection of Files

We would like to apply our gridding transformation to an entire list of files, that will enable tracking in future notebooks (ex. tobac). We need to:

  • Setup a function to grid our data (including saving it to disk)

  • Provide a list of files to grid

def grid_radar_data(infile, outdir, grid_resolution=500):
    """
    Parameters
    ==========
    infile: string, path to file to be gridded
    outdir: string, path to directory to save to
    grid_resolution: int, desired resolution of field in meters
    
    Returns
    =======
    print statement with saved file
    """
    
    # Read the data
    radar = pyart.io.read(infile)
    
    # Configure the bounds + compute points
    z_grid_limits = (0.,15_000.)
    y_grid_limits = (-150_000.,150_000.)
    x_grid_limits = (-150_000.,150_000.)
    x_grid_points = compute_number_of_points(x_grid_limits, grid_resolution)
    y_grid_points = compute_number_of_points(y_grid_limits, grid_resolution)
    z_grid_points = compute_number_of_points(z_grid_limits, grid_resolution)
    
    
    # Grid the radar data
    grid = pyart.map.grid_from_radars(radar,
                                  grid_shape=(z_grid_points,
                                              y_grid_points,
                                              x_grid_points),
                                  grid_limits=(z_grid_limits,
                                               y_grid_limits,
                                               x_grid_limits),
                                  fields=['reflectivity'],
                                  weighting_function='Nearest',
                                  nb=0.6,
                                  bsp=1.,
                                  min_radius=200.
                                 )
    
    infile_path = Path(infile)
    
    outfile = f"{outdir}/{infile_path.stem}.nc"
    
    # Save to disk
    grid.to_xarray().to_netcdf(outfile, mode='w')
    print(f"Done gridding + saving {outfile}")
    return outfile
import fsspec

fs = fsspec.filesystem("https")
files = ["RAW_NA_000_125_20080411181219",
         "RAW_NA_000_125_20080411181722",
         "RAW_NA_000_125_20080411182229",
         "RAW_NA_000_125_20080411182732",
         "RAW_NA_000_125_20080411183259",
         "RAW_NA_000_125_20080411183827",
         "RAW_NA_000_125_20080411184409",
         "RAW_NA_000_125_20080411184937",
         "RAW_NA_000_125_20080411185504",
         "RAW_NA_000_125_20080411190016",
         "RAW_NA_000_125_20080411190525",
         "RAW_NA_000_125_20080411191031",
         "RAW_NA_000_125_20080411191548",
         "RAW_NA_000_125_20080411192104",
         "RAW_NA_000_125_20080411192613",
         "RAW_NA_000_125_20080411193127",
         "RAW_NA_000_125_20080411193638",
         "RAW_NA_000_125_20080411194151",
         "RAW_NA_000_125_20080411194703",
         "RAW_NA_000_125_20080411195218",
         "RAW_NA_000_125_20080411195728",
         "RAW_NA_000_125_20080411200416",
         "RAW_NA_000_125_20080411200945",
         "RAW_NA_000_125_20080411201520",
         "RAW_NA_000_125_20080411202047"
        ]
import urllib.request
import glob
def download_file(file, outdir='../../data/uah-armor/'):
    urllib.request.urlretrieve(f"https://github.com/openradar/open-radar-data/raw/main/data/{file}", f"{outdir}{file}")
    return f"{outdir}{file}"
downloaded_files = [download_file(file) for file in files]
gridded_data = [grid_radar_data(file,
                                "../../data/uah-armor/gridded") for file in downloaded_files]
    
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411181219.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411181722.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411182229.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411182732.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411183259.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411183827.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411184409.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411184937.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411185504.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411190016.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411190525.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411191031.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411191548.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411192104.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411192613.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411193127.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411193638.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411194151.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411194703.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411195218.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411195728.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411200416.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411200945.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411201520.nc
Done gridding + saving ../../data/uah-armor/gridded/RAW_NA_000_125_20080411202047.nc

Read in the Gridded Data and Plot It

Now that we have our output, we can read all of it into the same xarray.Dataset, and visualize our data!

ds = xr.open_mfdataset(gridded_data).squeeze()
ds.reflectivity.isel(z=2).plot(x='x',
                               y='y',
                               col='time',
                               vmin=-20,
                               vmax=70,
                               cmap='pyart_ChaseSpectral',
                               col_wrap=5);
../../_images/78a87587dc4f5fb359def367fefae3d889ed79f022b3c6167ebfafad102331b2.png

Summary

Within this notebook, we covered the basics of gridding radar data using pyart, including:

  • What we mean by gridding and why is it matters

  • Configuring your gridding routine

  • Visualize gridded radar data

What’s Next

In the next few notebooks, we walk through applying data cleaning methods, and advanced visualization methods!

Resources and References

Py-ART essentials links:

  • Landing page

  • Examples

  • Source Code

  • Mailing list

  • Issue Tracker