Test failure: `test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]`

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: AssertionError: Workflow failed: migrate-external-tables-ctas (23m44.652s)

AssertionError: Workflow failed: migrate-external-tables-ctas
assert False
 +  where False = validate_step('migrate-external-tables-ctas')
 +    where validate_step = <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f32d3687f70>.validate_step
 +      where <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f32d3687f70> = <tests.integration.conftest.MockInstallationContext object at 0x7f32d03c3b80>.deployed_workflows
[gw9] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python
14:49 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.JH55/config.yml) doesn't exist.
14:49 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
14:49 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
14:49 INFO [databricks.labs.ucx.install] Fetching installations...
14:49 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.JH55 is corrupted. Skipping...
14:49 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
14:49 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:49 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:49 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+None20251008144937
14:49 INFO [databricks.labs.ucx.install] Creating ucx schemas...
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
14:49 INFO [databricks.labs.ucx.install] Creating dashboards...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:50 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.JH55/README for the next steps.
14:50 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
14:50 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/734169767783335
14:50 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
14:50 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/734169767783335/runs/227758706409028
14:50 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/734169767783335
14:50 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:52 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/510903761013040
14:52 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-tables-ctas job: None
14:52 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/510903761013040/runs/421830018507561
14:52 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/510903761013040
14:52 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:49 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.JH55/config.yml) doesn't exist.
14:49 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
14:49 INFO [databricks.labs.ucx.installer.hms_lineage] HMS Lineage feature creates one system table named system.hms_to_uc_migration.table_access and helps in your migration process from HMS to UC by allowing you to programmatically query HMS lineage data.
14:49 INFO [databricks.labs.ucx.install] Fetching installations...
14:49 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.JH55 is corrupted. Skipping...
14:49 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
14:49 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:49 DEBUG [tests.integration.conftest] Waiting for clusters to start...
14:49 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+None20251008144937
14:49 INFO [databricks.labs.ucx.install] Creating ucx schemas...
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
14:49 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
14:49 INFO [databricks.labs.ucx.install] Creating dashboards...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
14:49 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
14:49 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:49 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
14:50 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.JH55/README for the next steps.
14:50 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
14:50 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/734169767783335
14:50 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
14:50 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/734169767783335/runs/227758706409028
14:50 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/734169767783335
14:50 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
14:52 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/510903761013040
14:52 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-tables-ctas job: None
14:52 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/510903761013040/runs/421830018507561
14:52 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/510903761013040
14:52 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
15:12 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+None20251008144937 from https://DATABRICKS_HOST
15:12 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sjxut
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1119050742590867, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=376092817040291, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=352371646271522, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=228433785314310, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=656494453602878, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=92750377353105, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=960511650217905, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=510903761013040, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=734169767783335, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=510257304452223, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=22040315077686, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=50304736398432, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=678879442367396, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=171294184768743, as it is no longer needed
15:12 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=168394723116742, as it is no longer needed
15:12 INFO [databricks.labs.ucx.install] Deleting cluster policy
15:12 INFO [databricks.labs.ucx.install] Deleting secret scope
15:12 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw9] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #654}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='f9ff2166-6903-4dc0-87ef-27b0840f3500'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: f9ff2166-6903-4dc0-87ef-27b0840f3500 Correlation ID: e3e34577-56b0-4476-ba79-7a3fe893f03b Timestamp: 2025-10-18 05:08:37Z","error_codes":[7000215],"timestamp":"2025-10-18 05:08:37Z","trace_id":"f9ff2166-6903-4dc0-87ef-27b0840f3500","correlation_id":"e3e34577-56b0-4476-ba79-7a3fe893f03b","error_uri":"https://login.microsoftonline.com/error?code=7000215"}' (10.693s)

databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='f9ff2166-6903-4dc0-87ef-27b0840f3500'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: f9ff2166-6903-4dc0-87ef-27b0840f3500 Correlation ID: e3e34577-56b0-4476-ba79-7a3fe893f03b Timestamp: 2025-10-18 05:08:37Z","error_codes":[7000215],"timestamp":"2025-10-18 05:08:37Z","trace_id":"f9ff2166-6903-4dc0-87ef-27b0840f3500","correlation_id":"e3e34577-56b0-4476-ba79-7a3fe893f03b","error_uri":"https://login.microsoftonline.com/error?code=7000215"}'
05:08 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.o02L/config.yml) doesn't exist.
05:08 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:08 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:08 INFO [databricks.labs.ucx.install] Fetching installations...
05:08 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:08 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:08 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:08 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251018050844 from https://DATABRICKS_HOST
05:08 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_s9p69xhje
05:08 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:08 INFO [databricks.labs.ucx.install] Deleting secret scope
05:08 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw9] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #664}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='79d64d08-6893-4dfd-b83b-285cb1e05600'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: 79d64d08-6893-4dfd-b83b-285cb1e05600 Correlation ID: 785e8d63-03c1-4ca7-84d6-06df7646be17 Timestamp: 2025-10-19 05:10:43Z","error_codes":[7000215],"timestamp":"2025-10-19 05:10:43Z","trace_id":"79d64d08-6893-4dfd-b83b-285cb1e05600","correlation_id":"785e8d63-03c1-4ca7-84d6-06df7646be17","error_uri":"https://login.microsoftonline.com/error?code=7000215"}' (10.966s)

databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='79d64d08-6893-4dfd-b83b-285cb1e05600'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: 79d64d08-6893-4dfd-b83b-285cb1e05600 Correlation ID: 785e8d63-03c1-4ca7-84d6-06df7646be17 Timestamp: 2025-10-19 05:10:43Z","error_codes":[7000215],"timestamp":"2025-10-19 05:10:43Z","trace_id":"79d64d08-6893-4dfd-b83b-285cb1e05600","correlation_id":"785e8d63-03c1-4ca7-84d6-06df7646be17","error_uri":"https://login.microsoftonline.com/error?code=7000215"}'
05:10 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.sCLH/config.yml) doesn't exist.
05:10 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:10 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:10 INFO [databricks.labs.ucx.install] Fetching installations...
05:10 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:10 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:10 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:10 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251019051051 from https://DATABRICKS_HOST
05:10 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_spbwpebsm
05:10 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:10 INFO [databricks.labs.ucx.install] Deleting secret scope
05:10 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw0] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #665}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='fe4ff917-24a5-49bb-ab76-eccf81385a00'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: fe4ff917-24a5-49bb-ab76-eccf81385a00 Correlation ID: 940da817-2480-43fc-937b-add4b908ca41 Timestamp: 2025-10-20 05:11:51Z","error_codes":[7000215],"timestamp":"2025-10-20 05:11:51Z","trace_id":"fe4ff917-24a5-49bb-ab76-eccf81385a00","correlation_id":"940da817-2480-43fc-937b-add4b908ca41","error_uri":"https://login.microsoftonline.com/error?code=7000215"}' (25.541s)

databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='fe4ff917-24a5-49bb-ab76-eccf81385a00'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: fe4ff917-24a5-49bb-ab76-eccf81385a00 Correlation ID: 940da817-2480-43fc-937b-add4b908ca41 Timestamp: 2025-10-20 05:11:51Z","error_codes":[7000215],"timestamp":"2025-10-20 05:11:51Z","trace_id":"fe4ff917-24a5-49bb-ab76-eccf81385a00","correlation_id":"940da817-2480-43fc-937b-add4b908ca41","error_uri":"https://login.microsoftonline.com/error?code=7000215"}'
05:12 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.Dn6a/config.yml) doesn't exist.
05:12 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:12 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:12 INFO [databricks.labs.ucx.install] Fetching installations...
05:12 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:12 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:12 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:12 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251020051212 from https://DATABRICKS_HOST
05:12 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_s09f3oaoe
05:12 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:12 INFO [databricks.labs.ucx.install] Deleting secret scope
05:12 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw0] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #666}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='70e36592-8402-457e-afeb-d56f850d6000'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: 70e36592-8402-457e-afeb-d56f850d6000 Correlation ID: 8ba64b41-4370-414a-98e1-0486aacc34c6 Timestamp: 2025-10-21 05:09:54Z","error_codes":[7000215],"timestamp":"2025-10-21 05:09:54Z","trace_id":"70e36592-8402-457e-afeb-d56f850d6000","correlation_id":"8ba64b41-4370-414a-98e1-0486aacc34c6","error_uri":"https://login.microsoftonline.com/error?code=7000215"}' (16.724s)

databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='70e36592-8402-457e-afeb-d56f850d6000'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: 70e36592-8402-457e-afeb-d56f850d6000 Correlation ID: 8ba64b41-4370-414a-98e1-0486aacc34c6 Timestamp: 2025-10-21 05:09:54Z","error_codes":[7000215],"timestamp":"2025-10-21 05:09:54Z","trace_id":"70e36592-8402-457e-afeb-d56f850d6000","correlation_id":"8ba64b41-4370-414a-98e1-0486aacc34c6","error_uri":"https://login.microsoftonline.com/error?code=7000215"}'
05:09 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.EMBY/config.yml) doesn't exist.
05:09 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:09 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:09 INFO [databricks.labs.ucx.install] Fetching installations...
05:09 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:10 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:10 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:10 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251021051001 from https://DATABRICKS_HOST
05:10 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_ssgoql99l
05:10 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:10 INFO [databricks.labs.ucx.install] Deleting secret scope
05:10 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw9] linux -- Python 3.10.19 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #667}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='77990cf2-b050-467c-86b2-1a89e970a500'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: 77990cf2-b050-467c-86b2-1a89e970a500 Correlation ID: b68aa1cb-87af-48be-9af4-930759cc5f6e Timestamp: 2025-10-22 05:10:12Z","error_codes":[7000215],"timestamp":"2025-10-22 05:10:12Z","trace_id":"77990cf2-b050-467c-86b2-1a89e970a500","correlation_id":"b68aa1cb-87af-48be-9af4-930759cc5f6e","error_uri":"https://login.microsoftonline.com/error?code=7000215"}' (16.504s)

databricks.sdk.errors.platform.BadRequest: HTTP Error 401; url='https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token' AADToken: HTTP connection to https://login.microsoftonline.com/9f37a392-f0ae-4280-9796-f1864a10effc/oauth2/token failed for getting token from AzureAD.; requestId='77990cf2-b050-467c-86b2-1a89e970a500'; contentType='application/json; charset=utf-8'; response '{"error":"invalid_client","error_description":"AADSTS7000215: Invalid client secret provided. Ensure the secret being sent in the request is the client secret value, not the client secret ID, for a secret added to app '225cf71d-897e-4a73-b561-ee32487761be'. Trace ID: 77990cf2-b050-467c-86b2-1a89e970a500 Correlation ID: b68aa1cb-87af-48be-9af4-930759cc5f6e Timestamp: 2025-10-22 05:10:12Z","error_codes":[7000215],"timestamp":"2025-10-22 05:10:12Z","trace_id":"77990cf2-b050-467c-86b2-1a89e970a500","correlation_id":"b68aa1cb-87af-48be-9af4-930759cc5f6e","error_uri":"https://login.microsoftonline.com/error?code=7000215"}'
05:10 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.zFe2/config.yml) doesn't exist.
05:10 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:10 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:10 INFO [databricks.labs.ucx.install] Fetching installations...
05:10 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:10 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:10 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:10 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251022051021 from https://DATABRICKS_HOST
05:10 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_skh1r5faq
05:10 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:10 INFO [databricks.labs.ucx.install] Deleting secret scope
05:10 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw5] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #668}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: AssertionError: Workflow failed: migrate-external-tables-ctas (24m11.446s)

AssertionError: Workflow failed: migrate-external-tables-ctas
assert False
 +  where False = validate_step('migrate-external-tables-ctas')
 +    where validate_step = <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f0f9ff942b0>.validate_step
 +      where <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f0f9ff942b0> = <tests.integration.conftest.MockInstallationContext object at 0x7f0f9fd3eb30>.deployed_workflows
[gw3] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python
20:47 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lfGx/config.yml) doesn't exist.
20:47 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
20:47 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
20:47 INFO [databricks.labs.ucx.install] Fetching installations...
20:47 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lfGx is corrupted. Skipping...
20:47 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
20:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:47 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+220251022204717
20:47 INFO [databricks.labs.ucx.install] Creating ucx schemas...
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
20:47 INFO [databricks.labs.ucx.install] Creating dashboards...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lfGx/README for the next steps.
20:47 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
20:47 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/82985803367093
20:47 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
20:47 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/82985803367093/runs/622785641493937
20:47 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/82985803367093
20:47 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
20:50 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/392975107986736
20:50 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-tables-ctas job: None
20:50 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/392975107986736/runs/282116444515754
20:50 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/392975107986736
20:50 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
20:47 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lfGx/config.yml) doesn't exist.
20:47 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
20:47 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
20:47 INFO [databricks.labs.ucx.install] Fetching installations...
20:47 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lfGx is corrupted. Skipping...
20:47 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
20:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:47 DEBUG [tests.integration.conftest] Waiting for clusters to start...
20:47 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+220251022204717
20:47 INFO [databricks.labs.ucx.install] Creating ucx schemas...
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
20:47 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
20:47 INFO [databricks.labs.ucx.install] Creating dashboards...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
20:47 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
20:47 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
20:47 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.lfGx/README for the next steps.
20:47 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
20:47 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/82985803367093
20:47 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
20:47 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/82985803367093/runs/622785641493937
20:47 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/82985803367093
20:47 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
20:50 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/392975107986736
20:50 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-tables-ctas job: None
20:50 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/392975107986736/runs/282116444515754
20:50 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/392975107986736
20:50 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
21:10 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251022204717 from https://DATABRICKS_HOST
21:10 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sntntmnmz
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=259085579978543, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=215579208094549, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=59834943381888, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=82985803367093, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=517862989011931, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=330461365829562, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1070269004515664, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=253742088034067, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=127366820062624, as it is no longer needed
21:10 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=392975107986736, as it is no longer needed
21:11 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=812996602701078, as it is no longer needed
21:11 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=86470238710903, as it is no longer needed
21:11 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=591234230882682, as it is no longer needed
21:11 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=882554918928725, as it is no longer needed
21:11 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=466963160857519, as it is no longer needed
21:11 INFO [databricks.labs.ucx.install] Deleting cluster policy
21:11 INFO [databricks.labs.ucx.install] Deleting secret scope
21:11 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw3] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #668}

[github-actions]

❌ test_hiveserde_table_in_place_migration_job[migrate-external-tables-ctas]: AssertionError: Workflow failed: migrate-external-tables-ctas (23m7.469s)

AssertionError: Workflow failed: migrate-external-tables-ctas
assert False
 +  where False = validate_step('migrate-external-tables-ctas')
 +    where validate_step = <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f01fdf7b5b0>.validate_step
 +      where <databricks.labs.ucx.installer.workflows.DeployedWorkflows object at 0x7f01fdf7b5b0> = <tests.integration.conftest.MockInstallationContext object at 0x7f01fdd61450>.deployed_workflows
[gw8] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python
05:33 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NJuq/config.yml) doesn't exist.
05:33 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:33 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:33 INFO [databricks.labs.ucx.install] Fetching installations...
05:33 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NJuq is corrupted. Skipping...
05:33 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+220251024053357
05:33 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:34 INFO [databricks.labs.ucx.install] Creating dashboards...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NJuq/README for the next steps.
05:34 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
05:34 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/393115615566467
05:34 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
05:34 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/393115615566467/runs/908938256480167
05:34 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/393115615566467
05:34 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:36 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/1096363014015846
05:36 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-tables-ctas job: None
05:36 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/1096363014015846/runs/528673281351816
05:36 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/1096363014015846
05:36 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:33 DEBUG [databricks.labs.ucx.install] Cannot find previous installation: Path (/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NJuq/config.yml) doesn't exist.
05:33 INFO [databricks.labs.ucx.install] Please answer a couple of questions to configure Unity Catalog migration
05:33 INFO [databricks.labs.ucx.installer.hms_lineage] HMS lineage init script already exists and enabled
05:33 INFO [databricks.labs.ucx.install] Fetching installations...
05:33 WARNING [databricks.labs.ucx.install] Existing installation at /Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NJuq is corrupted. Skipping...
05:33 INFO [databricks.labs.ucx.installer.policy] Creating UCX cluster policy.
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 DEBUG [tests.integration.conftest] Waiting for clusters to start...
05:33 INFO [databricks.labs.ucx.install] Installing UCX v0.60.2+220251024053357
05:33 INFO [databricks.labs.ucx.install] Creating ucx schemas...
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-data-reconciliation
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migration-progress-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables-in-mounts-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=failing
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-tables
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-hiveserde-tables-in-place-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=convert-wasbs-to-adls-gen2-experimental
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assessment
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=assess-workflows
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-external-tables-ctas
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=remove-workspace-local-backup-groups
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=validate-groups-permissions
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=migrate-groups-legacy
05:34 INFO [databricks.labs.ucx.installer.workflows] Creating new job configuration for step=scan-tables-in-mounts-experimental
05:34 INFO [databricks.labs.ucx.install] Creating dashboards...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/views...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress...
05:34 DEBUG [databricks.labs.ucx.install] Reading step folder /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/groups...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/migration/main...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/progress/main...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/CLOUD_ENV...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/main...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/estimates...
05:34 INFO [databricks.labs.ucx.install] Creating dashboard in /home/runner/work/ucx/ucx/src/databricks/labs/ucx/queries/assessment/interactive...
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.installer.mixins] Fetching warehouse_id from a config
05:34 INFO [databricks.labs.ucx.install] Installation completed successfully! Please refer to the https://DATABRICKS_HOST/#workspace/Users/0a330eb5-dd51-4d97-b6e4-c474356b1d5d/.NJuq/README for the next steps.
05:34 INFO [databricks.labs.ucx.progress.install] Installation completed successfully!
05:34 DEBUG [databricks.labs.ucx.installer.workflows] starting assessment job: https://DATABRICKS_HOST#job/393115615566467
05:34 INFO [databricks.labs.ucx.installer.workflows] Named parameters for assessment job: None
05:34 INFO [databricks.labs.ucx.installer.workflows] Started assessment job: https://DATABRICKS_HOST#job/393115615566467/runs/908938256480167
05:34 DEBUG [databricks.labs.ucx.installer.workflows] Validating assessment workflow: https://DATABRICKS_HOST#job/393115615566467
05:34 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:36 DEBUG [databricks.labs.ucx.installer.workflows] starting migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/1096363014015846
05:36 INFO [databricks.labs.ucx.installer.workflows] Named parameters for migrate-external-tables-ctas job: None
05:36 INFO [databricks.labs.ucx.installer.workflows] Started migrate-external-tables-ctas job: https://DATABRICKS_HOST#job/1096363014015846/runs/528673281351816
05:36 DEBUG [databricks.labs.ucx.installer.workflows] Validating migrate-external-tables-ctas workflow: https://DATABRICKS_HOST#job/1096363014015846
05:36 INFO [databricks.labs.ucx.installer.workflows] Identified a run in progress waiting for run completion
05:56 INFO [databricks.labs.ucx.install] Deleting UCX v0.60.2+220251024053357 from https://DATABRICKS_HOST
05:56 INFO [databricks.labs.ucx.install] Deleting inventory database dummy_sfjywuhq7
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=627292071691038, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=747454052252107, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=466765801882822, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=701217766884680, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=325517875710487, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1066330672596382, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=158709561365981, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=358465344634788, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=393115615566467, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=996360605913606, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=1096363014015846, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=251478681307662, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=627327141086064, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=895512520803300, as it is no longer needed
05:56 INFO [databricks.labs.ucx.installer.workflows] Removing job_id=828400402460708, as it is no longer needed
05:56 INFO [databricks.labs.ucx.install] Deleting cluster policy
05:56 INFO [databricks.labs.ucx.install] Deleting secret scope
05:56 INFO [databricks.labs.ucx.install] UnInstalling UCX complete
[gw8] linux -- Python 3.10.18 /home/runner/work/ucx/ucx/.venv/bin/python

_{Running from nightly #670}