Test failure: test_hiveserde_table_in_place_migration_job[migrate-external-hiveserde-tables-in-place-experimental] #4667
Description
❌ test_hiveserde_table_in_place_migration_job[migrate-external-hiveserde-tables-in-place-experimental]: AssertionError: Workflow failed: migrate-external-hiveserde-tables-in-place-experimental (25m9.315s)
AssertionError: Workflow failed: migrate-external-hiveserde-tables-in-place-experimental
assert False
+ where False = validate_step('migrate-external-hiveserde-tables-in-place-experimental')
+ where validate_step = <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f32d35c7670>.validate_step
+ where <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f32d35c7670> = <tests.integration.conftest.MockInstallationContext object at 0x7f32d34bb070>.deployed_workflows
[gw9] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python
14:24 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wx4d/config.yml) doesn't exist.
14:24 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
14:24 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
14:24 INFO [databricks.labs.ucx.install] Fetching installations...
14:24 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wx4d is corrupted. Skipping...
14:24 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
14:24 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:24 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:24 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+None20251008142444
14:24 INFO [databricks.labs.ucx.install] Creating ucx schemas...
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
14:25 INFO [databricks.labs.ucx.install] Creating dashboards...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wx4d/README for the next steps.
14:25 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
14:25 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/915916323696872
14:25 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
14:25 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/915916323696872/runs/793195742329525
14:25 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/915916323696872
14:25 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:28 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/480454692702817
14:28 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-hiveserde-tables-in-place-experimental job: None
14:28 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/480454692702817/runs/455825969774779
14:28 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-hiveserde-tables-in-place-experimental workflow: https://DATABRICKS_HOST#job/480454692702817
14:28 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:24 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wx4d/config.yml) doesn't exist.
14:24 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
14:24 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
14:24 INFO [databricks.labs.ucx.install] Fetching installations...
14:24 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wx4d is corrupted. Skipping...
14:24 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
14:24 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:24 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:24 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+None20251008142444
14:24 INFO [databricks.labs.ucx.install] Creating ucx schemas...
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
14:24 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
14:25 INFO [databricks.labs.ucx.install] Creating dashboards...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
14:25 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
14:25 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:25 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.wx4d/README for the next steps.
14:25 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
14:25 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/915916323696872
14:25 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
14:25 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/915916323696872/runs/793195742329525
14:25 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/915916323696872
14:25 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:28 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/480454692702817
14:28 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-hiveserde-tables-in-place-experimental job: None
14:28 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-hiveserde-tables-in-place-experimental job: https://DATABRICKS_HOST#job/480454692702817/runs/455825969774779
14:28 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-hiveserde-tables-in-place-experimental workflow: https://DATABRICKS_HOST#job/480454692702817
14:28 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:48 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+None20251008142444 from https://DATABRICKS_HOST
14:48 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sgdi1
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=558405894925704, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1112251496649431, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=782064413591367, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=740705972254206, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=895417481587588, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=542343504811529, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=965567296529961, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=466319586539897, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=915916323696872, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=480454692702817, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1123279207384585, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=685069235027590, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=895623118719479, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=601612444264450, as it is no longer needed
14:49 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=305217178599129, as it is no longer needed
14:49 INFO [databricks.labs.ucx.install] Deleting cluster policy
14:49 INFO [databricks.labs.ucx.install] Deleting secret scope
14:49 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw9] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python
Running from nightly #654