Issue using a dataset router with mutliple datasets

[Jakidxav]

I am relatively new to Xpublish and am trying to follow the Serving multiple datasets tutorial. The docs say that using custom routes for multiple datasets should work just like they do for single datasets. However, I can't seem to get the example to work.

Possibly related:

• Flexible routes #25
• Publishing a collection of datasets #23
• structuring an Xpublish application #63

Here is the code that I adapted from the Xpublish tutorial.

import xarray as xr
import xpublish
from xpublish.dependencies import get_dataset
from fastapi import APIRouter, Depends, HTTPException

ds = xr.tutorial.open_dataset('air_temperature', chunks=dict(lat=5, lon=5),)
ds2 = xr.tutorial.open_dataset('rasm')

ds_collection = {
    'air_temperature': ds,
    'rasm': ds2,
}

router = APIRouter()

@router.get('/{var_name}/mean')
def get_mean(var_name: str, dataset: xr.Dataset = Depends(get_dataset)):
    if var_name not in dataset.variables:
        raise HTTPException(
            status_code=404, detail=f"Variable '{var_name}' not found in dataset"
        )

    return float(dataset[var_name].mean())

rest_collection = xpublish.Rest(ds_collection, routers=[router])
rest_collection.serve()

When I try and run this example, however, I get the following error:

ValueError: Duplicated param name dataset_id at path /datasets/{dataset_id}/datasets/{dataset_id}/{var_name}

Am I doing something wrong here? I seem to be triggering the error raised in the xpublish.utils.api.check_route_conflicts() method because the /datasets/{datasets_id}/ path is repeated. If I am understanding other Github issues and discussions, it seems like I may need to serve up separate datasets on individual ports within the same app, is that right?

Any help or advice would be much appreciated!

[abkfenris]

Good catch, I think we might be normalizing the xpublish.Rest(routers) twice.

First here:

xpublish/xpublish/rest.py

Lines 104 to 109 in bec4efa

	normalized_routers: List[Tuple[APIRouter, Dict]] = normalize_app_routers(
	routers or [],
	self._dataset_route_prefix,
	)
	check_route_conflicts(normalized_routers)
	self._routers = normalized_routers

Then:

xpublish/xpublish/rest.py

Lines 282 to 286 in bec4efa

	app_routers.extend(
	normalize_app_routers(
	plugin_dataset_routers + (dataset_routers or []), self._dataset_route_prefix
	)
	)

Have you tried implementing the router as a plugin? https://xpublish.readthedocs.io/en/0.4.1/getting-started/tutorial/dataset-router-plugin.html I think most folks using Xpublish these days are setting up new routes in plugins instead, which would have kept us from catching that (and I'm not sure we're testing that either).

We actually are treating single datasets as the oddity these days, and are defaulting to multi-dataset support, and none of that should require separate ports.

[Jakidxav]

Thanks @abkfenris !

Using the instructions in Creating a dataset router plugin, I can now access the /{var_name}/mean route for multiple datasets via the same port. I'll take a closer look at the plugins page in the docs for more.