ADD: Cfradial2 Reader #288

syedhamidali · 2025-07-13T21:31:35Z

Closes can xradar read Cfradial2 netcdf files? #287, Implement open_cfradial2_datatree #93
Tests added
Changes are documented in history.md

codecov · 2025-07-13T21:39:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.93%. Comparing base (3f6ebc0) to head (3bf46ec).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #288   +/-   ##
=======================================
  Coverage   93.92%   93.93%           
=======================================
  Files          27       28    +1     
  Lines        5533     5539    +6     
=======================================
+ Hits         5197     5203    +6     
  Misses        336      336

Flag	Coverage Δ
notebooktests	`79.94% <83.33%> (+<0.01%)`	⬆️
unittests	`93.51% <100.00%> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

docs/history.md

kmuehlbauer

@syedhamidali Just some minor suggestions, but looks good to me. We can enhance the compatibility layer later on, when users request it. Thanks!

tests/io/test_cfradial2.py

Co-authored-by: Kai Mühlbauer <[email protected]>

kmuehlbauer · 2025-07-14T11:06:34Z

tests/io/test_cfradial2.py

+import xradar as xd
+
+
+@pytest.fixture


@syedhamidali Maybe my suggestion was a bit brief, but this fixture is already defined in conftest.py as well as the tmp_file fixture. You should be able to just use them here and can remove the re-definitions here.

kmuehlbauer · 2025-07-14T11:07:57Z

tests/io/test_cfradial2.py

+    dtree = xd.io.open_cfradial1_datatree(cfradial1_file)
+
+    # Write to CfRadial2 NetCDF format
+    dtree.to_netcdf(temp_file)


I'm not really sure, but you might have to add kwarg engine="netcdf4" here, since the tmp_file doesn't have an extension.

aladinor · 2025-07-14T13:39:35Z

Hi everyone,

Thanks for putting all of this together. I was looking at the cfradial2 data, and I can see it has a hierarchical structure with a root dataset as well as the sweep nodes. However, is this file fully FM301 compliant? I notice something like the root contains some attributes, but does it contain all the required attributes under the FM301 data model?

print(dtree['/'].ds)

<xarray.Dataset> Size: 40B
Dimensions:           (nsweep: 1)
Dimensions without coordinates: nsweep
Data variables:
    sweep_group_name  (nsweep) <U10 40B ...
Attributes:
    Conventions:  CF-1.7, Cf/Radial
    version:      2.0
    title:        Radar CfRadial2 Dataset
    history:      Updated CfRadial2 on 2025-05-01T00:46:09.040110Z
    RadarName:    denton.tx
    Latitude:     33.25355
    Longitude:    -97.15204
    Height:       224.0

On the other hand, I realized the dataset contains sweeps numbered as sweep_xxxx, is this the "correct" way to number the sweep under the WMO data model? Finally, I also realized this dataset contains coordinates and dimensions that I am not sure are cfradrial-FM301 compliant.

print(dtree['/sweep_0000']. ds 

<xarray.Dataset> Size: 27MB
Dimensions:                       (ntime: 723, nrange: 1003)
Dimensions without coordinates: ntime, nrange
Data variables: (12/19)
    time_us                       (ntime) datetime64[ns] 6kB ...
    range_m                       (nrange) float64 8kB ...
    azimuth_deg                   (ntime) float64 6kB ...
    elevation_deg                 (ntime) float64 6kB ...
    range_gate_width_mm           (ntime) float64 6kB ...
    start_range_mm                (ntime) float64 6kB ...
    ...                            ...
    specific_differential_phase   (ntime, nrange) float32 3MB ...
    radial_velocity_mps           (ntime, nrange) float32 3MB ...
    signal_to_noise_ratio_db      (ntime, nrange) float32 3MB ...
    spectral_width_mps            (ntime, nrange) float32 3MB ...
    filtering                     (ntime, nrange) int8 725kB ...
    rain_rate_mmhr                (ntime, nrange) float32 3MB ...
Attributes:
    NumGates:          1003
    ScanType:          2
    AntennaGain:       33.0
    AntennaBeamwidth:  2.7
    PosVelocity:       15.0

Please let me know your thoughts

kmuehlbauer · 2025-07-14T13:48:33Z

@aladinor See https://library.wmo.int/idviewer/35625/1042 for a description of attributes.

The thing is FM301 differs slightly from CfRadial2. How should we deal with this?

aladinor · 2025-07-14T13:55:55Z

Indeed, multiple aspects make this dataset quite different from the data standard. However, data generation/storage is something each institution decides. In this case, I had to do a data ingestion pipeline to put this dataset in a close version of standard-compliant format.

<xarray.DataTree>
Group: /
│   Dimensions:              ()
│   Data variables:
│       latitude             float64 8B 33.25
│       longitude            float64 8B -97.15
│       altitude             float64 8B 224.0
│       sweep_group_name     <U7 28B 'sweep_0'
│       time_coverage_start  |S32 32B b'2025-05-01T00:46:09Z'
│       time_coverage_end    |S32 32B b'2025-05-01T00:58:11Z'
│       sweep_fixed_angle    float64 8B 2.0
│   Attributes:
│       Conventions:  CF-1.7, Cf/Radial
│       version:      2.0
│       title:        Radar CfRadial2 Dataset
│       history:      Updated CfRadial2 on 2025-05-01T00:46:09.040110Z
│       scan_name:    2
├── Group: /sweep_0
│       Dimensions:                       (azimuth: 723, range: 1003)
│       Coordinates:
│         * azimuth                       (azimuth) float64 6kB 0.37 0.85 ... 359.8
│           elevation                     (azimuth) float64 6kB 2.0 2.0 2.0 ... 2.0 2.0
│           latitude                      float64 8B 33.25
│           longitude                     float64 8B -97.15
│           altitude                      float64 8B 224.0
│           crs_wkt                       int64 8B 0
│           x                             (azimuth, range) float64 6MB 0.0 ... -2.796
│           y                             (azimuth, range) float64 6MB 0.0 ... 1.001e+03
│           z                             (azimuth, range) float64 6MB 224.0 ... 259.0
│       Dimensions without coordinates: range
│       Data variables: (12/17)
│           time_us                       (azimuth) datetime64[ns] 6kB ...
│           range_m                       (range) float64 8kB ...
│           range_gate_width_mm           (azimuth) float64 6kB ...
│           start_range_mm                (azimuth) float64 6kB ...
│           carrier_frequency_hz          (azimuth) float64 6kB ...
│           pulse_length_sec              (azimuth) float64 6kB ...
│           ...                            ...
│           specific_differential_phase   (azimuth, range) float32 3MB ...
│           radial_velocity_mps           (azimuth, range) float32 3MB ...
│           signal_to_noise_ratio_db      (azimuth, range) float32 3MB ...
│           spectral_width_mps            (azimuth, range) float32 3MB ...
│           filtering                     (azimuth, range) int8 725kB ...
│           rain_rate_mmhr                (azimuth, range) float32 3MB ...
├── Group: /radar_parameters
├── Group: /georeferencing_correction
└── Group: /radar_calibration

Therefore, I think we can read "cfradial2" hierarchical files, but we can not guarantee consistency as in the other backends. Any ideas/suggestions on how to deal with this?

kmuehlbauer · 2025-07-14T13:58:55Z

Therefore, I think we can read "cfradial2" hierarchical files, but we can not guarantee consistency as in the other backends. Any ideas/suggestions on how to deal with this?

I'd implement a compatibility layer, a function which takes the CfRadial2 DataTree and transforms it to the wanted FM301 compatible DataTree. This can happen in the open_cfradial2_datatree function.

aladinor · 2025-07-14T14:09:19Z

Thanks @kmuehlbauer for mentioning this. In this draft, I created a sort of wrapper, but I think this is particular to any institution and their way to create their cfradial2 data. @syedhamidali could check it out, and perhaps we can get a nice solution for this.

aladinor · 2025-07-17T19:17:01Z

I feel like we can merge this, but putting a warning. Something like "cfradial2 format might not be totally FM301 compliant". This will allow xradar to read hierarchical data. Let me know your thoughts.

syedhamidali added 2 commits July 14, 2025 03:00

ADD: Cfradial2 Reader

5095f58

FIX: PR serial fix

ececf42

syedhamidali added 4 commits July 14, 2025 03:35

FIX: Fix Tests

9a2e654

FIX: Run Black

5ecd3aa

FIX: Add zarr to req

c173790

FIX: write to nc instead

4a22a79

kmuehlbauer reviewed Jul 14, 2025

View reviewed changes

docs/history.md Outdated Show resolved Hide resolved

kmuehlbauer requested changes Jul 14, 2025

View reviewed changes

tests/io/test_cfradial2.py Outdated Show resolved Hide resolved

tests/io/test_cfradial2.py Outdated Show resolved Hide resolved

syedhamidali and others added 3 commits July 14, 2025 16:15

FIX: Use fixture in pytest

61ef0fd

FIX: Use fixture in pytest

b96f791

Update docs/history.md

3bf46ec

Co-authored-by: Kai Mühlbauer <[email protected]>

kmuehlbauer reviewed Jul 14, 2025

View reviewed changes

aladinor mentioned this pull request Jul 14, 2025

can xradar read Cfradial2 netcdf files? #287

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ADD: Cfradial2 Reader #288

ADD: Cfradial2 Reader #288

Uh oh!

syedhamidali commented Jul 13, 2025 •

edited by kmuehlbauer

Loading

Uh oh!

codecov bot commented Jul 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

kmuehlbauer left a comment

Uh oh!

Uh oh!

Uh oh!

kmuehlbauer Jul 14, 2025

Uh oh!

kmuehlbauer Jul 14, 2025

Uh oh!

aladinor commented Jul 14, 2025

Uh oh!

kmuehlbauer commented Jul 14, 2025

Uh oh!

aladinor commented Jul 14, 2025 •

edited

Loading

Uh oh!

kmuehlbauer commented Jul 14, 2025 •

edited

Loading

Uh oh!

aladinor commented Jul 14, 2025 •

edited

Loading

Uh oh!

aladinor commented Jul 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

		import xradar as xd


		@pytest.fixture

ADD: Cfradial2 Reader #288

Are you sure you want to change the base?

ADD: Cfradial2 Reader #288

Uh oh!

Conversation

syedhamidali commented Jul 13, 2025 • edited by kmuehlbauer Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jul 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

kmuehlbauer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kmuehlbauer Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

kmuehlbauer Jul 14, 2025

Choose a reason for hiding this comment

Uh oh!

aladinor commented Jul 14, 2025

Uh oh!

kmuehlbauer commented Jul 14, 2025

Uh oh!

aladinor commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kmuehlbauer commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aladinor commented Jul 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aladinor commented Jul 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

syedhamidali commented Jul 13, 2025 •

edited by kmuehlbauer

Loading

codecov bot commented Jul 13, 2025 •

edited

Loading

aladinor commented Jul 14, 2025 •

edited

Loading

kmuehlbauer commented Jul 14, 2025 •

edited

Loading

aladinor commented Jul 14, 2025 •

edited

Loading