-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Open
Labels
bugBug reportBug report
Description
Describe the bug
Currently, Power BI assets in our DataHub instance are not linked to a specific PlatformInstance, which makes the structure less organized and harder to manage.
We attempted to migrate these assets to include the PlatformInstance name (e.g., abc) using the CLI command:
datahub migrate dataplatform2instance --platform powerbi --instance abcHowever, the migration fails immediately with a 404 Not Found error when trying to fetch relationships from GMS (/openapi/relationships/v1), preventing the process from completing.
To Reproduce
- Run:
migrate dataplatform2instance --platform powerbi --instance musement
- Confirm prompt.
- Observe error after 0% progress.
Expected behavior
Assets should be migrated so their URNs include the PlatformInstance prefix (e.g., abc), improving structure.
Actual behavior
Migration aborts with:
ERROR: Unable to get metadata from DataHub (404 Not Found at /openapi/relationships/v1)
Full error message:
Starting migration: platform:powerbi, instance=<instance-name>, force=False, dry-run=False
Will migrate 665 urns such as ['urn:li:dataset:(urn:li:dataPlatform:powerbi,<dataset-name-1>,PROD)', 'urn:li:dataset:(urn:li:dataPlatform:powerbi,<dataset-name-2>,PROD)', 'urn:li:dataset:(urn:li:dataPlatform:powerbi,<dataset-name-3>,PROD)', ...]
New urns will look like ['urn:li:dataset:(urn:li:dataPlatform:powerbi,<instance-name>.<dataset-name-1>,PROD)', 'urn:li:dataset:(urn:li:dataPlatform:powerbi,<instance-name>.<dataset-name-2>,PROD)', ...]
Ok to proceed? [y/N]: y
0% (0 of 665) | | Elapsed Time: 0:00:00 ETA: --:--:--[2025-12-09 10:41:25,274] ERROR {datahub.entrypoints:247} - Command failed: ('Unable to get metadata from DataHub', {'timestamp': 1765276884670, 'status': 404, 'error': 'Not Found', 'path': '/openapi/relationships/v1'})
Traceback (most recent call last):
File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 262, in _send_restli_request
response.raise_for_status()
File "/home/datahub/.venv/lib/python3.10/site-packages/requests/models.py", line 1026, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://<datahub-api-endpoint>/openapi/relationships/v1?urn=urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3Apowerbi%2C<dataset-name>,PROD%29&direction=INCOMING&relationshipTypes=DownstreamOf&relationshipTypes=Consumes&relationshipTypes=Produces&relationshipTypes=ForeignKeyToDataset&relationshipTypes=DerivedFrom&relationshipTypes=IsPartOf&start=0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/metadata-ingestion/src/datahub/entrypoints.py", line 234, in main
sys.exit(datahub(standalone_mode=False, **kwargs))
File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1462, in __call__
return self.main(*args, **kwargs)
File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1383, in main
rv = self.invoke(ctx)
File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1850, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1850, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1246, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 814, in invoke
return callback(*args, **kwargs)
File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 485, in wrapper
raise e
File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 433, in wrapper
res = func(*args, **kwargs)
File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 485, in wrapper
raise e
File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 433, in wrapper
res = func(*args, **kwargs)
File "/metadata-ingestion/src/datahub/upgrade/upgrade.py", line 491, in async_wrapper
ret = func(*args, **kwargs)
File "/metadata-ingestion/src/datahub/cli/migrate.py", line 134, in dataplatform2instance
dataplatform2instance_func(instance, platform, dry_run, env, force, hard, keep)
File "/metadata-ingestion/src/datahub/cli/migrate.py", line 242, in dataplatform2instance_func
for relationship in relationships:
File "/metadata-ingestion/src/datahub/cli/migration_utils.py", line 279, in get_incoming_relationships
yield from client.get_related_entities(
File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 1272, in get_related_entities
response = self._get_generic(
File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 277, in _get_generic
return self._send_restli_request("GET", url, params=params)
File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 267, in _send_restli_request
datahub.configuration.common.OperationalError: ('Unable to get metadata from DataHub', {'timestamp': 1765276884670, 'status': 404, 'error': 'Not Found', 'path': '/openapi/relationships/v1'})
Additional context
- Goal: Better structure by linking assets to a specific PlatformInstance.
- Environment: DataHub version and ingestion image
v1.3.0. - Likely cause: Missing or incompatible GMS endpoint for relationships.
Metadata
Metadata
Assignees
Labels
bugBug reportBug report