Py-ART Gridding#
Overview#
Within this notebook, we will cover:
What is gridding and why is it important?
An overview of gridding with Py-ART
How to choose a gridding routine
Gridding multiple radars to the same grid
Prerequisites#
Concepts |
Importance |
Notes |
|---|---|---|
Helpful |
Basic features |
|
Helpful |
Basic features |
|
Helpful |
Basic plotting |
|
Helpful |
Basic arrays |
Time to learn: 45 minutes
Imports#
import os
import warnings
import cartopy.crs as ccrs
import matplotlib.pyplot as plt
import pyart
from pyart.testing import get_test_data
warnings.filterwarnings('ignore')
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel_launcher.py", line 18, in <module>
app.launch_new_instance()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 739, in start
self.io_loop.start()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 205, in start
self.asyncio_loop.run_forever()
File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
self._run_once()
File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
handle._run()
File "/srv/conda/envs/notebook/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
await self.process_one()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 534, in process_one
await dispatch(*args)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
await result
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
await super().execute_request(stream, ident, parent)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
reply_content = await reply_content
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
res = shell.run_cell(
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
return super().run_cell(*args, **kwargs)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3048, in run_cell
result = self._run_cell(
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3103, in _run_cell
result = runner(coro)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3308, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3490, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3550, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_320/2480179226.py", line 8, in <module>
import pyart
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/__init__.py", line 11, in <module>
from . import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/__init__.py", line 70, in <module>
from .pattern import read_pattern #noqa
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/pattern.py", line 23, in <module>
from ..core.radar import Radar
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/__init__.py", line 40, in <module>
from .grid import Grid #noqa
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/grid.py", line 26, in <module>
import xarray
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/__init__.py", line 3, in <module>
from xarray import groupers, testing, tutorial
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/groupers.py", line 15, in <module>
import pandas as pd
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/__init__.py", line 26, in <module>
from pandas.compat import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/compat/__init__.py", line 27, in <module>
from pandas.compat.pyarrow import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/compat/pyarrow.py", line 8, in <module>
import pyarrow as pa
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyarrow/__init__.py", line 65, in <module>
import pyarrow.lib as _lib
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
AttributeError: _ARRAY_API not found
A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.
If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.
Traceback (most recent call last): File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel_launcher.py", line 18, in <module>
app.launch_new_instance()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/traitlets/config/application.py", line 1075, in launch_instance
app.start()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 739, in start
self.io_loop.start()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 205, in start
self.asyncio_loop.run_forever()
File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
self._run_once()
File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
handle._run()
File "/srv/conda/envs/notebook/lib/python3.9/asyncio/events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
await self.process_one()
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 534, in process_one
await dispatch(*args)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
await result
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
await super().execute_request(stream, ident, parent)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
reply_content = await reply_content
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
res = shell.run_cell(
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
return super().run_cell(*args, **kwargs)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3048, in run_cell
result = self._run_cell(
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3103, in _run_cell
result = runner(coro)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
coro.send(None)
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3308, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3490, in run_ast_nodes
if await self.run_code(code, result, async_=asy):
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3550, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_320/2480179226.py", line 8, in <module>
import pyart
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/__init__.py", line 11, in <module>
from . import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/__init__.py", line 70, in <module>
from .pattern import read_pattern #noqa
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/pattern.py", line 23, in <module>
from ..core.radar import Radar
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/__init__.py", line 40, in <module>
from .grid import Grid #noqa
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/grid.py", line 26, in <module>
import xarray
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/__init__.py", line 3, in <module>
from xarray import groupers, testing, tutorial
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/groupers.py", line 15, in <module>
import pandas as pd
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/__init__.py", line 49, in <module>
from pandas.core.api import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/core/api.py", line 9, in <module>
from pandas.core.dtypes.dtypes import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/core/dtypes/dtypes.py", line 24, in <module>
from pandas._libs import (
File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyarrow/__init__.py", line 65, in <module>
import pyarrow.lib as _lib
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
AttributeError: _ARRAY_API not found
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[1], line 8
4 import cartopy.crs as ccrs
5 import matplotlib.pyplot as plt
----> 8 import pyart
9 from pyart.testing import get_test_data
11 warnings.filterwarnings('ignore')
File /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/__init__.py:11
7 from os import environ as _environ
9 # import subpackages
10 # print out helpful message if build fails or importing from source tree
---> 11 from . import (
12 __check_build, # noqa
13 aux_io, # noqa
14 bridge, # noqa
15 config, # noqa
16 core, # noqa
17 correct, # noqa
18 filters, # noqa
19 graph, # noqa
20 io, # noqa
21 map, # noqa
22 retrieve, # noqa
23 testing, # noqa
24 util, # noqa
25 )
26 from ._debug_info import _debug_info # noqa
28 # root level functions
File /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/__init__.py:76
74 from .arm_vpt import read_kazr #noqa
75 from .edge_netcdf import read_edge_netcdf #noqa
---> 76 from .odim_h5 import read_odim_h5, read_odim_grid_h5, read_odim_vp_h5 #noqa
77 from .odim_h5_writer import write_odim_h5, write_odim_grid_h5 #noqa
78 from .gamic_hdf5 import read_gamic #noqa
File /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/odim_h5.py:24
21 import numpy as np
23 try:
---> 24 import h5py
25 _H5PY_AVAILABLE = True
26 except ImportError:
File /srv/conda/envs/notebook/lib/python3.9/site-packages/h5py/__init__.py:25
19 # --- Library setup -----------------------------------------------------------
20
21 # When importing from the root of the unpacked tarball or git checkout,
22 # Python sees the "h5py" source directory and tries to load it, which fails.
23 # We tried working around this by using "package_dir" but that breaks Cython.
24 try:
---> 25 from . import _errors
26 except ImportError:
27 import os.path as _op
File h5py/_errors.pyx:1, in init h5py._errors()
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
What is gridding and why is it important?#
Antenna vs. Cartesian Coordinates#
Radar data, by default, is stored in a polar (or antenna) coordinate system, with the data coordinates stored as an angle (ranging from 0 to 360 degrees with 0 == North), and a radius from the radar, and an elevation which is the angle between the ground and the ground.
This format can be challenging to plot, since it is scan/radar specific. Also, it can make comparing with model data, which is on a lat/lon grid, challenging since one would need to transform the model daa cartesian coordinates to polar/antenna coordiantes.
Fortunately, PyART has a variety of gridding routines, which can be used to grid your data to a Cartesian grid. Once it is in this new grid, one can easily slice/dice the dataset, and compare to other data sources.
Why is Gridding Important?#
Gridding is essential to combining multiple data sources (ex. multiple radars), and comparing to other data sources (ex. model data). There are also decisions that are made during the gridding process that have a large impact on the regridded data - for example:
What resolution should my grid be?
Which interpolation routine should I use?
How smooth should my interpolated data be?
While there is not always a right or wrong answer, it is important to understand the options available, and document which routine you used with your data! Also - experiment with different options and choose the best for your use case!
An overview of gridding with Py-ART#
Let’s dig into the regridding process with PyART!
Read in and Visualize a Test Dataset#
Let’s start with the same file used in the previous notebook (PyART Basics), which is a radar file from Northern Oklahoma.
file = get_test_data('swx_20120520_0641.nc')
radar = pyart.io.read(file)
Let’s plot up quick look of reflectivity, at the lowest elevation scan (closest to the ground)
fig = plt.figure(figsize=[12, 12])
display = pyart.graph.RadarDisplay(radar)
display.plot_ppi('corrected_reflectivity_horizontal',
cmap='pyart_HomeyerRainbow')
As mentioned before, the dataset is currently in the antenna coordinate system measured as distance from the radar
Setup our Gridding Routine with pyart.map.grid_from_radars()#
Py-ART has the Grid object which has characteristics similar to that of the Radar object, except that the data are stored in Cartesian coordinates instead of the radar’s antenna coordinates.
pyart.core.Grid?
We can transform our data into this grid object, from the radars, using pyart.map.grid_from_radars().
Beforing gridding our data, we need to make a decision about the desired grid resolution and extent. For example, one might imagine a grid configuration of:
Grid extent/limits
20 km in the x-direction (north/south)
20 km in the y-direction (west/east)
15 km in the z-direction (vertical)
500 m spatial resolution
The pyart.map.grid_from_radars() function takes the grid shape and grid limits as input, with the order (z, y, x).
Let’s setup our configuration, setting our grid extent first, with the distance measured in meters
z_grid_limits = (500.,15_000.)
y_grid_limits = (-20_000.,20_000.)
x_grid_limits = (-20_000.,20_000.)
Now that we have our grid limits, we can set our desired resolution (again, in meters)
grid_resolution = 500
Let’s compute our grid shape - using the extent and resolution to compute the number of grid points in each direction.
def compute_number_of_points(extent, resolution):
return int((extent[1] - extent[0])/resolution)
Now that we have a helper function to compute this, let’s apply it to our vertical dimension
z_grid_points = compute_number_of_points(z_grid_limits, grid_resolution)
z_grid_points
We can apply this to the horizontal (x, y) dimensions as well.
x_grid_points = compute_number_of_points(x_grid_limits, grid_resolution)
y_grid_points = compute_number_of_points(y_grid_limits, grid_resolution)
print(z_grid_points,
y_grid_points,
x_grid_points)
Use our configuration to grid the data!#
Now that we have the grid shape and grid limits, let’s grid up our radar!
grid = pyart.map.grid_from_radars(radar,
grid_shape=(z_grid_points,
y_grid_points,
x_grid_points),
grid_limits=(z_grid_limits,
y_grid_limits,
x_grid_limits),
)
grid
We now have a pyart.core.Grid object!
Plot up the Grid Object#
Plot a horizontal view of the data#
We can use the GridMapDisplay from pyart.graph to visualize our regridded data, starting with a horizontal view (slice along a single vertical level)
display = pyart.graph.GridMapDisplay(grid)
display.plot_grid('corrected_reflectivity_horizontal',
level=0,
vmin=-20,
vmax=60,
cmap='pyart_HomeyerRainbow')
Plot a Latitudinal Slice#
We can also slice through a single latitude or longitude!
display.plot_latitude_slice('corrected_reflectivity_horizontal',
lat=36.5,
vmin=-20,
vmax=60,
cmap='pyart_HomeyerRainbow')
plt.xlim([-20, 20]);
Plot with Xarray#
Another neat feature of the Grid object is that we can transform it to an xarray.Dataset!
ds = grid.to_xarray()
ds
Now, our plotting routine is a one-liner, starting with the horizontal slice:
ds.isel(z=0).corrected_reflectivity_horizontal.plot(cmap='pyart_HomeyerRainbow',
vmin=-20,
vmax=60);
And a vertical slice at a given y dimension (latitude)
ds.sel(y=1300,
method='nearest').corrected_reflectivity_horizontal.plot(cmap='pyart_HomeyerRainbow',
vmin=-20,
vmax=60);
Summary#
Within this notebook, we covered the basics of gridding radar data using pyart, including:
What we mean by gridding and why is it matters
Configuring your gridding routine
Visualize gridded radar data
What’s Next#
In the next few notebooks, we walk through applying data cleaning methods, and advanced visualization methods!
Resources and References#
MeteoSwiss Py-ART essentials links: