Skip to content

'dataplatform2instance' Migration Command Fails When Linking Assets to PlatformInstance #15538

@Useserall

Description

@Useserall

Describe the bug

Currently, Power BI assets in our DataHub instance are not linked to a specific PlatformInstance, which makes the structure less organized and harder to manage.
We attempted to migrate these assets to include the PlatformInstance name (e.g., abc) using the CLI command:

datahub migrate dataplatform2instance --platform powerbi --instance abc

However, the migration fails immediately with a 404 Not Found error when trying to fetch relationships from GMS (/openapi/relationships/v1), preventing the process from completing.


To Reproduce

  1. Run:
    migrate dataplatform2instance --platform powerbi --instance musement
  2. Confirm prompt.
  3. Observe error after 0% progress.

Expected behavior

Assets should be migrated so their URNs include the PlatformInstance prefix (e.g., abc), improving structure.


Actual behavior

Migration aborts with:

ERROR: Unable to get metadata from DataHub (404 Not Found at /openapi/relationships/v1)

Full error message:

Starting migration: platform:powerbi, instance=<instance-name>, force=False, dry-run=False
Will migrate 665 urns such as ['urn:li:dataset:(urn:li:dataPlatform:powerbi,<dataset-name-1>,PROD)', 'urn:li:dataset:(urn:li:dataPlatform:powerbi,<dataset-name-2>,PROD)', 'urn:li:dataset:(urn:li:dataPlatform:powerbi,<dataset-name-3>,PROD)', ...]
New urns will look like ['urn:li:dataset:(urn:li:dataPlatform:powerbi,<instance-name>.<dataset-name-1>,PROD)', 'urn:li:dataset:(urn:li:dataPlatform:powerbi,<instance-name>.<dataset-name-2>,PROD)', ...]
Ok to proceed? [y/N]: y
  0% (0 of 665) |                                                                                                                                                         | Elapsed Time: 0:00:00 ETA:  --:--:--[2025-12-09 10:41:25,274] ERROR    {datahub.entrypoints:247} - Command failed: ('Unable to get metadata from DataHub', {'timestamp': 1765276884670, 'status': 404, 'error': 'Not Found', 'path': '/openapi/relationships/v1'})
Traceback (most recent call last):
  File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 262, in _send_restli_request
    response.raise_for_status()
  File "/home/datahub/.venv/lib/python3.10/site-packages/requests/models.py", line 1026, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://<datahub-api-endpoint>/openapi/relationships/v1?urn=urn%3Ali%3Adataset%3A%28urn%3Ali%3AdataPlatform%3Apowerbi%2C<dataset-name>,PROD%29&amp;direction=INCOMING&amp;relationshipTypes=DownstreamOf&amp;relationshipTypes=Consumes&amp;relationshipTypes=Produces&amp;relationshipTypes=ForeignKeyToDataset&amp;relationshipTypes=DerivedFrom&amp;relationshipTypes=IsPartOf&amp;start=0
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/metadata-ingestion/src/datahub/entrypoints.py", line 234, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
  File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1462, in __call__
    return self.main(*args, **kwargs)
  File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1383, in main
    rv = self.invoke(ctx)
  File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1850, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 1246, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/datahub/.venv/lib/python3.10/site-packages/click/core.py", line 814, in invoke
    return callback(*args, **kwargs)
  File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 485, in wrapper
    raise e
  File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 433, in wrapper
    res = func(*args, **kwargs)
  File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 485, in wrapper
    raise e
  File "/metadata-ingestion/src/datahub/telemetry/telemetry.py", line 433, in wrapper
    res = func(*args, **kwargs)
  File "/metadata-ingestion/src/datahub/upgrade/upgrade.py", line 491, in async_wrapper
    ret = func(*args, **kwargs)
  File "/metadata-ingestion/src/datahub/cli/migrate.py", line 134, in dataplatform2instance
    dataplatform2instance_func(instance, platform, dry_run, env, force, hard, keep)
  File "/metadata-ingestion/src/datahub/cli/migrate.py", line 242, in dataplatform2instance_func
    for relationship in relationships:
  File "/metadata-ingestion/src/datahub/cli/migration_utils.py", line 279, in get_incoming_relationships
    yield from client.get_related_entities(
  File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 1272, in get_related_entities
    response = self._get_generic(
  File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 277, in _get_generic
    return self._send_restli_request("GET", url, params=params)
  File "/metadata-ingestion/src/datahub/ingestion/graph/client.py", line 267, in _send_restli_request
datahub.configuration.common.OperationalError: ('Unable to get metadata from DataHub', {'timestamp': 1765276884670, 'status': 404, 'error': 'Not Found', 'path': '/openapi/relationships/v1'})

Additional context

  • Goal: Better structure by linking assets to a specific PlatformInstance.
  • Environment: DataHub version and ingestion image v1.3.0.
  • Likely cause: Missing or incompatible GMS endpoint for relationships.

Metadata

Metadata

Assignees

Labels

bugBug report

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions