support CoverageJSON

[tomkralidis]

First off great work on leaflet-velocity -- fantastic plugin!

CoverageJSON is a format for publishing geotemporal data to the web. As part of various OGC API efforts, various API servers return CoverageJSON as an output format (pygeoapi is an example).

As the default input format for leaflet-velocity seems to the the result of grib2json, it would be valuable to have read support for CoverageJSON. This would open the door for OGC APIs emitting CoverageJSON of wind data to integrate cleanly into the plugin without the extra hop of the grib2json format.

Having said this, any guidance on how to implement in the design of the plugin would also be valued.

[brentfraser]

Hi @tomkralidis ,

The author @danwild would be the best person to supply guidance, but I'll share what I know from my work last year adding a little data checking and map projection generalization last year.

One approach would be to add a function to transform a CoverageJSON data file to windy format JSON. Just where this function would be called might depend on what the CoverageJSON contains.

Currently the data percolates thru the system like so:

1. User's JavaScript get the data from the server with something like

       $.getJSON("mydatafile.json", function(mydata) {
             var velocityLayer = L.velocityLayer({ data: mydata });

2. L.velocityLayer just passes the data object along: setOptions ( options.data ), then _startWindy

3. Windy: start, then buildGrid(data,callback)

The data is untouched until buildGrid (I think) so the data transform function could be inserted in any of those steps. buildGrid starts examining the members of the JSON to see if the expected parts are there, then the data values are read and manipulated, so it needs to be in "windy" format by the start of buildGrid.

[danwild]

Hi @tomkralidis,

Very open to the idea of CoverageJSON / or other OGC compliant format.
From memory I don't think I saw a decent OGC candidate when I started this project.

Unfortunately it doesn't looks like there's been much activity around CoverageJSON since Reading-eScience-Centre put up a few WIP's 5 years ago? (personally I'd love it if pycovjson were a real thing, I have a bespoke python Web Processing Service that translates NetCDF data for use with this plugin, but would much prefer to defer to a more widely supported lib).

As an aside, I've been considering also putting together a WebGL version of this plugin.. if someone were to provide a convincing argument for an OGC input format I'd certainly consider it.

[chris-little]

@danwild @tomkralidis We recommended CoverageJSON as a response format for the OGC approved, and just about to be published, Environmental Data Retrieval (EDR) API, because of widespread implementation and usage.
CoverageJSON is not (yet) an OGC standard, but Scott Simmons, the OGC CTO, suggested progressing it as an OGC adopted Community Standard, which I, and Jon Blower of Reading University, support and will try to progress.

There may be some push back within OGC from the Coverages Standard WG when we later try to converge CoverageJSON and the OGC CIS-JSON for later versions. (e.g. JSON-LD version update)

I've noted your wish that pycovjson does something more, presumably not just NetCDF.

HTH

[brentfraser]

Does anyone have a link to a public server (e.g NOAA, NASA or other) currently serving daily U/V wind or wave vector data in a JSON format? Preferably in CoverageJSON, but I'm interested in any JSON format.

[tomkralidis]

Thanks all for the discussion. Quasi-summary:

• CoverageJSON is quietly evolving
• we can consider a workflow to convert CoverageJSON into the windy JSON format. Sample attempt below in [1]
• @brentfraser we (MSC) don't have any wind JSON per se, but you can use the following workflow (using our 15km global model as an example):

# download sample GDPS 1000.0 mb UGRD and VGRD GRIB2 data from MSC Datamart (i.e. single timestep of U and V)
# note the timestamps below have a shelf life but the basic pattern is (first forecast hour of 00z model run):
# `https://dd.weather.gc.ca/model_gem_global/15km/grib2/lat_lon/00/000/CMC_glb_UGRD_ISBL_1000_latlon.15x.15_*_P000.grib2`
# `https://dd.weather.gc.ca/model_gem_global/15km/grib2/lat_lon/00/000/CMC_glb_VGRD_ISBL_1000_latlon.15x.15_*_P000.grib2`

curl -O https://dd.weather.gc.ca/model_gem_global/15km/grib2/lat_lon/00/000/CMC_glb_UGRD_ISBL_1000_latlon.15x.15_2021042000_P000.grib2
curl -O https://dd.weather.gc.ca/model_gem_global/15km/grib2/lat_lon/00/000/CMC_glb_VGRD_ISBL_1000_latlon.15x.15_2021042000_P000.grib2
# combine the 2 GRIB2 files
cat *.grib2 > combined.grib2
# convert GRIB2 data into JSON via grib2json
grib2json --compact --data combined.grib2 > combined.min.json

[1] sample attempt (hacky and can definitely be improved!)

diff --git a/src/js/windy.js b/src/js/windy.js
index c132ac6..aa117a2 100644
--- a/src/js/windy.js
+++ b/src/js/windy.js
@@ -10,6 +10,39 @@
  interpolation and animation process.
  */
 
+function CoverageJSON2Windy(data) {
+  var c = [];
+  var pn = 2;
+  for (var key in data.parameters) {
+    var p = {
+        header: {
+            gridDefinitionTemplate: 0,
+            lo1: Math.abs(data.domain.axes.x.start),
+            lo2: data.domain.axes.x.stop,
+            la1: data.domain.axes.y.stop,
+            la2: data.domain.axes.y.start,
+            nx: data.ranges[key].shape[1],
+            ny: data.ranges[key].shape[0],
+            dx: (Math.abs(data.domain.axes.x.start) + Math.abs(data.domain.axes.x.stop)) / data.ranges[key].shape[1],
+            dy: (Math.abs(data.domain.axes.y.start) + Math.abs(data.domain.axes.y.stop)) / data.ranges[key].shape[0],
+            parameterCategory: 2,
+            parameterNumber: pn++,
+            winds: "true",
+            scanMode: 64,
+            refTime: data.domain.axes.time.values[0],
+            forecastTime: 0
+        },
+        data: data.ranges[key].values
+    };
+    c.push(p);
+  }
+
+  console.log(c[0].header);
+  console.log(c[1].header);
+
+  return c;
+}
+
 var Windy = function(params) {
   var MIN_VELOCITY_INTENSITY = params.minVelocity || 0; // velocity at which particle intensity is minimum (m/s)
   var MAX_VELOCITY_INTENSITY = params.maxVelocity || 10; // velocity at which particle intensity is maximum (m/s)
@@ -52,6 +85,10 @@ var Windy = function(params) {
   var date;
   var λ0, φ0, Δλ, Δφ, ni, nj;
 
+  if (gridData.hasOwnProperty("type") && gridData["type"] === "Coverage") {
+    gridData = CoverageJSON2Windy(gridData);
+  }
+
   var setData = function(data) {
     gridData = data;
   };

[brentfraser]

@tomkralidis Thanks for the info. I'm already using wind-js-server to do basically the same thing (but it's good to know how to do it with MSC data!).

I was hoping there was a well-oiled environmental data ecosystem out there delivering near-real-time data in JSON format, but I guess not yet.

Your initial question got me thinking about possible enhancements:

1. Making a general leaflet-velocity demo loading some STAC-like JSON of metadata describing available velocity data servers (NOAA, MSC, etc) and their data sets with links to load data
2. Data providers having a "latest" or "current" data set so the client code did not have to fabricate a vendor-specific date-time based URI (with an "next-update-at" UTC property)
3. As the clock tics over to the next-update-at time, the client code could load the new data set.

Thanks!

[zakjan]

Pardon me for a shameless plug, I maintain a STAC Catalog with pre-processed environmental data for visualization, with deck.gl layers for simple integration with common map libraries. Basic subscription covers the server costs. Provided formats are PNG and GeoTIFF currently, but could be extended easily. https://weatherlayers.com/

There is an open-sourced deck.gl-particle as well, which can be used in Leaflet together with deck.gl-leaflet.