Py-ART Basics

Py-ART Basics#

Overview#

Within this notebook, we will cover:

  1. General overview of Py-ART and its functionality

  2. Reading data using Py-ART

  3. An overview of the pyart.Radar object

  4. Create a Plot of our Radar Data

Prerequisites#

Concepts

Importance

Notes

Intro to Cartopy

Helpful

Basic features

Matplotlib Basics

Helpful

Basic plotting

NumPy Basics

Helpful

Basic arrays

  • Time to learn: 45 minutes

Imports#

import os
import warnings

import cartopy.crs as ccrs
import matplotlib.pyplot as plt


import pyart
from pyart.testing import get_test_data

warnings.filterwarnings('ignore')

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel_launcher.py", line 18, in <module>
    app.launch_new_instance()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/traitlets/config/application.py", line 1075, in launch_instance
    app.start()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 739, in start
    self.io_loop.start()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 205, in start
    self.asyncio_loop.run_forever()
  File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
    self._run_once()
  File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
    handle._run()
  File "/srv/conda/envs/notebook/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
    await self.process_one()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 534, in process_one
    await dispatch(*args)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
    await result
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
    await super().execute_request(stream, ident, parent)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
    reply_content = await reply_content
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
    res = shell.run_cell(
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
    return super().run_cell(*args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3048, in run_cell
    result = self._run_cell(
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3103, in _run_cell
    result = runner(coro)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
    coro.send(None)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3308, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3490, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3550, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/tmp/ipykernel_289/2480179226.py", line 8, in <module>
    import pyart
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/__init__.py", line 11, in <module>
    from . import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/__init__.py", line 70, in <module>
    from .pattern import read_pattern #noqa
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/pattern.py", line 23, in <module>
    from ..core.radar import Radar
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/__init__.py", line 40, in <module>
    from .grid import Grid #noqa
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/grid.py", line 26, in <module>
    import xarray
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/__init__.py", line 3, in <module>
    from xarray import groupers, testing, tutorial
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/groupers.py", line 15, in <module>
    import pandas as pd
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/__init__.py", line 26, in <module>
    from pandas.compat import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/compat/__init__.py", line 27, in <module>
    from pandas.compat.pyarrow import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/compat/pyarrow.py", line 8, in <module>
    import pyarrow as pa
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyarrow/__init__.py", line 65, in <module>
    import pyarrow.lib as _lib

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
AttributeError: _ARRAY_API not found

A module that was compiled using NumPy 1.x cannot be run in
NumPy 2.0.1 as it may crash. To support both 1.x and 2.x
versions of NumPy, modules must be compiled with NumPy 2.0.
Some module may need to rebuild instead e.g. with 'pybind11>=2.12'.

If you are a user of the module, the easiest solution will be to
downgrade to 'numpy<2' or try to upgrade the affected module.
We expect that some modules will need time to support NumPy 2.

Traceback (most recent call last):  File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/srv/conda/envs/notebook/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel_launcher.py", line 18, in <module>
    app.launch_new_instance()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/traitlets/config/application.py", line 1075, in launch_instance
    app.start()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelapp.py", line 739, in start
    self.io_loop.start()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 205, in start
    self.asyncio_loop.run_forever()
  File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 601, in run_forever
    self._run_once()
  File "/srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py", line 1905, in _run_once
    handle._run()
  File "/srv/conda/envs/notebook/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 545, in dispatch_queue
    await self.process_one()
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 534, in process_one
    await dispatch(*args)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 437, in dispatch_shell
    await result
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 362, in execute_request
    await super().execute_request(stream, ident, parent)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/kernelbase.py", line 778, in execute_request
    reply_content = await reply_content
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/ipkernel.py", line 449, in do_execute
    res = shell.run_cell(
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/ipykernel/zmqshell.py", line 549, in run_cell
    return super().run_cell(*args, **kwargs)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3048, in run_cell
    result = self._run_cell(
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3103, in _run_cell
    result = runner(coro)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/async_helpers.py", line 129, in _pseudo_sync_runner
    coro.send(None)
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3308, in run_cell_async
    has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3490, in run_ast_nodes
    if await self.run_code(code, result, async_=asy):
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3550, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/tmp/ipykernel_289/2480179226.py", line 8, in <module>
    import pyart
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/__init__.py", line 11, in <module>
    from . import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/__init__.py", line 70, in <module>
    from .pattern import read_pattern #noqa
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/pattern.py", line 23, in <module>
    from ..core.radar import Radar
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/__init__.py", line 40, in <module>
    from .grid import Grid #noqa
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/core/grid.py", line 26, in <module>
    import xarray
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/__init__.py", line 3, in <module>
    from xarray import groupers, testing, tutorial
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/xarray/groupers.py", line 15, in <module>
    import pandas as pd
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/__init__.py", line 49, in <module>
    from pandas.core.api import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/core/api.py", line 9, in <module>
    from pandas.core.dtypes.dtypes import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pandas/core/dtypes/dtypes.py", line 24, in <module>
    from pandas._libs import (
  File "/srv/conda/envs/notebook/lib/python3.9/site-packages/pyarrow/__init__.py", line 65, in <module>
    import pyarrow.lib as _lib

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
AttributeError: _ARRAY_API not found

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 8
      4 import cartopy.crs as ccrs
      5 import matplotlib.pyplot as plt
----> 8 import pyart
      9 from pyart.testing import get_test_data
     11 warnings.filterwarnings('ignore')

File /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/__init__.py:11
      7 from os import environ as _environ
      9 # import subpackages
     10 # print out helpful message if build fails or importing from source tree
---> 11 from . import (
     12     __check_build,  # noqa
     13     aux_io,  # noqa
     14     bridge,  # noqa
     15     config,  # noqa
     16     core,  # noqa
     17     correct,  # noqa
     18     filters,  # noqa
     19     graph,  # noqa
     20     io,  # noqa
     21     map,  # noqa
     22     retrieve,  # noqa
     23     testing,  # noqa
     24     util,  # noqa
     25 )
     26 from ._debug_info import _debug_info  # noqa
     28 # root level functions

File /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/__init__.py:76
     74 from .arm_vpt import read_kazr #noqa
     75 from .edge_netcdf import read_edge_netcdf #noqa
---> 76 from .odim_h5 import read_odim_h5, read_odim_grid_h5, read_odim_vp_h5 #noqa
     77 from .odim_h5_writer import write_odim_h5, write_odim_grid_h5 #noqa
     78 from .gamic_hdf5 import read_gamic #noqa

File /srv/conda/envs/notebook/lib/python3.9/site-packages/pyart/aux_io/odim_h5.py:24
     21 import numpy as np
     23 try:
---> 24     import h5py
     25     _H5PY_AVAILABLE = True
     26 except ImportError:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/h5py/__init__.py:25
     19 # --- Library setup -----------------------------------------------------------
     20 
     21 # When importing from the root of the unpacked tarball or git checkout,
     22 # Python sees the "h5py" source directory and tries to load it, which fails.
     23 # We tried working around this by using "package_dir" but that breaks Cython.
     24 try:
---> 25     from . import _errors
     26 except ImportError:
     27     import os.path as _op

File h5py/_errors.pyx:1, in init h5py._errors()

ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject

An Overview of Py-ART#

History of the Py-ART#

  • Development began to address the needs of ARM with the acquisition of a number of new scanning cloud and precipitation radar as part of the American Recovery Act.

  • The project has since expanded to work with a variety of weather radars and a wider user base including radar researchers and climate modelers.

  • The software has been released on GitHub as open source software under a BSD license. Runs on Linux, OS X. It also runs on Windows with more limited functionality.

What can PyART Do?#

Py-ART can be used for a variety of tasks from basic plotting to more complex processing pipelines. Specific uses for Py-ART include:

  • Reading radar data in a variety of file formats.

  • Creating plots and visualization of radar data.

  • Correcting radar moments while in antenna coordinates, such as:

    • Doppler unfolding/de-aliasing.

    • Attenuation correction.

    • Phase processing using a Linear Programming method.

  • Mapping data from one or multiple radars onto a Cartesian grid.

  • Performing retrievals.

  • Writing radial and Cartesian data to NetCDF files.

Reading in Data Using Py-ART#

Reading data in using pyart.io.read#

When reading in a radar file, we use the pyart.io.read module.

pyart.io.read can read a variety of different radar formats, such as Cf/Radial, LASSEN, and more. The documentation on what formats can be read by Py-ART can be found here:

For most file formats listed on the page, using pyart.io.read should suffice since Py-ART has the ability to automatically detect the file format.

Let’s check out what arguments arguments pyart.io.read() takes in!

pyart.io.read?

Let’s use a sample data file from pyart - which is cfradial format.

When we read this in, we get a pyart.Radar object!

file = get_test_data('swx_20120520_0641.nc')
radar = pyart.io.read(file)
radar

Investigate the pyart.Radar object#

Within this pyart.Radar object object are the actual data fields.

This is where data such as reflectivity and velocity are stored.

To see what fields are present we can add the fields and keys additions to the variable where the radar object is stored.

radar.fields.keys()

Extract a sample data field#

The fields are stored in a dictionary, each containing coordinates, units and more. All can be accessed by just adding the fields addition to the radar object variable.

For an individual field, we add a string in brackets after the fields addition to see the contents of that field.

Let’s take a look at 'corrected_reflectivity_horizontal', which is a common field to investigate.

print(radar.fields['corrected_reflectivity_horizontal'])

We can go even further in the dictionary and access the actual reflectivity data.

We use add 'data' at the end, which will extract the data array (which is a masked numpy array) from the dictionary.

reflectivity = radar.fields['corrected_reflectivity_horizontal']['data']
print(type(reflectivity), reflectivity)

Lets’ check the size of this array…

reflectivity.shape

This reflectivity data array, numpy array, is a two-dimensional array with dimensions:

  • Gates (number of samples away from the radar)

  • Rays (direction around the radar)

print(radar.nrays, radar.ngates)

If we wanted to look the 300th ray, at the second gate, we would use something like the following:

print(reflectivity[300, 2])

Plotting our Radar Data#

An Overview of Py-ART Plotting Utilities#

Now that we have loaded the data and inspected it, the next logical thing to do is to visualize the data! Py-ART’s visualization functionality is done through the objects in the pyart.graph module.

In Py-ART there are 4 primary visualization classes in pyart.graph:

Plotting grid data

Use the RadarMapDisplay with our data#

For the this example, we will be using RadarMapDisplay, using Cartopy to deal with geographic coordinates.

We start by creating a figure first.

fig = plt.figure(figsize=[10, 10])

Once we have a figure, let’s add our RadarMapDisplay

fig = plt.figure(figsize=[10, 10])
display = pyart.graph.RadarMapDisplay(radar)

Adding our map display without specifying a field to plot won’t do anything we need to specifically add a field to field using .plot_ppi_map()

display.plot_ppi_map('corrected_reflectivity_horizontal')

By default, it will plot the elevation scan, the the default colormap from Matplotlib… let’s customize!

We add the following arguements:

  • sweep=3 - The fourth elevation scan (since we are using Python indexing)

  • vmin=-20 - Minimum value for our plotted field/colorbar

  • vmax=60 - Maximum value for our plotted field/colorbar

  • projection=ccrs.PlateCarree() - Cartopy latitude/longitude coordinate system

  • cmap='pyart_HomeyerRainbow' - Colormap to use, selecting one provided by PyART

fig = plt.figure(figsize=[12, 12])
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('corrected_reflectivity_horizontal',
                     sweep=3,
                     vmin=-20,
                     vmax=60,
                     projection=ccrs.PlateCarree(),
                     cmap='pyart_HomeyerRainbow')
plt.show()

You can change many parameters in the graph by changing the arguments to plot_ppi_map. As you can recall from earlier. simply view these arguments in a Jupyter notebook by typing:

display.plot_ppi_map?

For example, let’s change the colormap to something different

fig = plt.figure(figsize=[12, 12])
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('corrected_reflectivity_horizontal',
                     sweep=3,
                     vmin=-20,
                     vmax=60,
                     projection=ccrs.PlateCarree(),
                     cmap='pyart_Carbone42')
plt.show()

Or, let’s view a different elevation scan! To do this, change the sweep parameter in the plot_ppi_map function.

fig = plt.figure(figsize=[12, 12])
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('corrected_reflectivity_horizontal',
                     sweep=0,
                     vmin=-20,
                     vmax=60,
                     projection=ccrs.PlateCarree(),
                     cmap='pyart_Carbone42')
plt.show()

Let’s take a look at a different field - for example, correlation coefficient (corr_coeff)

fig = plt.figure(figsize=[12, 12])
display = pyart.graph.RadarMapDisplay(radar)
display.plot_ppi_map('copol_coeff',
                     sweep=0,
                     vmin=0.8,
                     vmax=1.,
                     projection=ccrs.PlateCarree(),
                     cmap='pyart_Carbone42')
plt.show()

Summary#

Within this notebook, we covered the basics of working with radar data using pyart, including:

  • Reading in a file using pyart.io

  • Investigating the Radar object

  • Visualizing radar data using the RadarMapDisplay

What’s Next#

In the next few notebooks, we walk through gridding radar data, applying data cleaning methods, and advanced visualization methods!

Resources and References#

MeteoSwiss Py-ART essentials links:

Contents