✨ Add application skeleton #551

renardeinside · 2025-08-28T18:25:33Z

This PR introduces the application skeleton, based on React, Vite, TailwindCSS + ShadCN (frontend) and FastAPI in the backend.

github-actions · 2025-08-28T18:50:12Z

❌ 384/389 passed, 12 flaky, 5 failed, 17 skipped, 4h46m4s total

❌ test_save_and_load_checks_from_lakebase_table_with_user_installation: databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase. (2.049s)

databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase.
[gw9] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python
09:12 INFO [databricks.labs.dqx.installer.install] Please answer a couple of questions to provide TEST_SCHEMA DQX run configuration. The configuration can also be updated manually after the installation.
09:12 INFO [databricks.labs.dqx.installer.install] DQX will be installed in the TEST_SCHEMA location: '/Users/<your_user>/.dqx'
09:12 INFO [databricks.labs.dqx.installer.install] Deleting DQX v0.9.4+2220251016091215 from https://DATABRICKS_HOST
09:12 INFO [databricks.labs.dqx.installer.install] Uninstalling DQX complete
[gw9] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python

❌ test_save_and_load_checks_from_lakebase_table_with_output_modes: databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase. (380ms)

databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase.
[gw6] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python
[gw6] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python

❌ test_profiler_workflow_save_to_lakebase: databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase. (15.633s)

databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase.
[gw9] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python
09:12 INFO [databricks.labs.dqx.installer.install] Please answer a couple of questions to provide TEST_SCHEMA DQX run configuration. The configuration can also be updated manually after the installation.
09:12 INFO [databricks.labs.dqx.installer.install] DQX will be installed in the TEST_SCHEMA location: '/Users/<your_user>/.dqx'
09:12 INFO [databricks.labs.dqx.installer.install] Installing DQX v0.9.4+2220251016091217
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Creating dashboards...
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Reading dashboard assets from /home/runner/work/dqx/dqx/src/databricks/labs/dqx/queries/quality/dashboard...
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Using 'main.dqx_test.output_table' output table as the source table for the dashboard...
09:12 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
09:12 WARNING [databricks.labs.lsql.dashboards] Parsing unsupported field in dashboard.yml: tiles.00_2_dq_error_types.hidden
09:12 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Installing 'DQX_Quality_Dashboard' dashboard in '/Users/3fe685a1-96cc-4fec-8cdb-6944f5c9787e/.r4Jk/dashboards'
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Creating new job configuration for step=profiler
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Creating new job configuration for step=quality-checker
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Creating new job configuration for step=e2e
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Updating configuration for step=e2e job_id=332899495174561
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Updating configuration for step=e2e job_id=332899495174561
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Updating configuration for step=e2e job_id=332899495174561
09:12 INFO [databricks.labs.dqx.installer.install] Installation completed successfully!
09:12 INFO [databricks.labs.dqx.installer.install] Please answer a couple of questions to provide TEST_SCHEMA DQX run configuration. The configuration can also be updated manually after the installation.
09:12 INFO [databricks.labs.dqx.installer.install] DQX will be installed in the TEST_SCHEMA location: '/Users/<your_user>/.dqx'
09:12 INFO [databricks.labs.dqx.installer.install] Installing DQX v0.9.4+2220251016091217
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Creating dashboards...
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Reading dashboard assets from /home/runner/work/dqx/dqx/src/databricks/labs/dqx/queries/quality/dashboard...
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Using 'main.dqx_test.output_table' output table as the source table for the dashboard...
09:12 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
09:12 WARNING [databricks.labs.lsql.dashboards] Parsing unsupported field in dashboard.yml: tiles.00_2_dq_error_types.hidden
09:12 WARNING [databricks.labs.lsql.dashboards] Parsing : No expression was parsed from ''
09:12 INFO [databricks.labs.dqx.installer.dashboard_installer] Installing 'DQX_Quality_Dashboard' dashboard in '/Users/3fe685a1-96cc-4fec-8cdb-6944f5c9787e/.r4Jk/dashboards'
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Creating new job configuration for step=profiler
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Creating new job configuration for step=quality-checker
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Creating new job configuration for step=e2e
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Updating configuration for step=e2e job_id=332899495174561
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Updating configuration for step=e2e job_id=332899495174561
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Updating configuration for step=e2e job_id=332899495174561
09:12 INFO [databricks.labs.dqx.installer.install] Installation completed successfully!
09:12 INFO [databricks.labs.dqx.installer.install] Deleting DQX v0.9.4+2220251016091217 from https://DATABRICKS_HOST
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Removing job_id=896055926390403, as it is no longer needed
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Removing job_id=949082591269897, as it is no longer needed
09:12 INFO [databricks.labs.dqx.installer.workflow_installer] Removing job_id=332899495174561, as it is no longer needed
09:12 INFO [databricks.labs.dqx.installer.install] Uninstalling DQX complete
[gw9] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python

❌ test_save_and_load_checks_from_lakebase_table_with_run_config: databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase. (389ms)

databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase.
[gw3] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python
[gw3] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python

❌ test_save_and_load_checks_from_lakebase_table: databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase. (319ms)

databricks.sdk.errors.platform.BadRequest: Failed to create database instance because you have hit the workspace limit. Contact Databricks for quota increase.
[gw8] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python
[gw8] linux -- Python 3.12.11 /home/runner/work/dqx/dqx/.venv/bin/python

Flaky tests:

🤪 test_profiler_workflow_class (12.11s)
🤪 test_profiler_workflow_with_custom_install_folder (57.496s)
🤪 test_e2e_workflow (6m9.372s)
🤪 test_profiler_workflow (1m6.09s)
🤪 test_profiler_workflow_for_multiple_run_configs (1m5.733s)
🤪 test_profiler_workflow_for_patterns (1m19.444s)
🤪 test_quality_checker (50.36s)
🤪 test_apply_checks_and_save_in_tables_for_patterns_exclude_no_tables_matching (9m52.752s)
🤪 test_quality_checker_workflow_for_multiple_run_configs_table_checks_storage (1m11.813s)
🤪 test_profiler_workflow_for_patterns_table_checks_storage (1m30.422s)
🤪 test_quality_checker_workflow_for_patterns (1m12.099s)
🤪 test_quality_checker_workflow_table_checks_storage (1m11.307s)

_{Running from acceptance #2813}

Copilot

Pull Request Overview

This PR introduces the application skeleton for a full-stack web application with a React frontend and FastAPI backend. The implementation includes a modern tech stack with TailwindCSS, ShadCN UI components, and comprehensive development tooling.

Adds React-based frontend with TanStack Router, React Query, and ShadCN UI components
Implements FastAPI backend with user authentication, database configuration, and API endpoints
Sets up development environment with TypeScript, ESLint, Prettier, and comprehensive build tooling

Reviewed Changes

Copilot reviewed 43 out of 48 changed files in this pull request and generated 2 comments.

File	Description
ui/*	Complete React frontend setup with routing, state management, and UI components
src/databricks/labs/dqx/app/*	FastAPI backend with authentication, configuration, and API endpoints
pyproject.toml	Added app dependencies for FastAPI, uvicorn, and database libraries
docs/dqx/docs/dev/contributing.mdx	Added development setup documentation for the new app

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

ui/src/main.tsx

src/databricks/labs/dqx/app/dependencies.py

mwojtyczka · 2025-08-29T10:29:54Z

docs/dqx/docs/dev/contributing.mdx

+
+## Contributing to the DQX App 
+
+DQX App is a web application that provides a user interface for DQX. It is built with React and FastAPI.


should we mention Lakehouse app as deployment mechanism to give a bit more context?

mwojtyczka · 2025-08-29T10:32:23Z

pyproject.toml

    "pylint~=3.3.1",
    "pylint-per-file-ignores~=1.3",
    "pylint-pytest==2.0.0a0",
+    "pylint_pydantic",


pin the version

src/databricks/labs/dqx/app/app.py

mwojtyczka · 2025-08-29T10:49:57Z

src/databricks/labs/dqx/app/config.py

+        return f"postgresql://{self.user}:{self.password}@{self.host}:{self.port}/{self.database}?sslmode=require"
+
+
+class DatabaseManager(BaseModel):


should not belong to config

src/databricks/labs/dqx/app/dependencies.py

mwojtyczka · 2025-08-29T10:53:13Z

ui/src/assets/Logo.tsx

@@ -0,0 +1,5 @@
+import logoUrl from "/logo.svg";


can we use existing logo from docs, or does it have to live inside the app?

ui/src/main.tsx

fmt

mwojtyczka · 2025-09-01T15:05:39Z

docs/dqx/docs/dev/contributing.mdx

+
+Add a new `.env` file in the root of the project with the following content:
+```
+DQX_DEV_TOKEN=<your-databricks-token>


we should recommend using oauth. Earlier in the docs we have a note on this.
Is this DQX_DEV_TOKEN correct? it's not a standard evn var and it is not used anywhere in this PR

mwojtyczka · 2025-09-01T15:15:31Z

pyproject.toml

    # The models cannot be delcared as dependency here buecase PyPI does not support URL-based dependencies which would prevent releases.
 ]
+app = [
+    "fastapi",


we should pin the versions

mwojtyczka · 2025-09-01T15:22:10Z

docs/dqx/docs/dev/contributing.mdx

+
+N.b. - The `DQX_DEV_TOKEN` is a Databricks token that you can generate in the Databricks UI or via the Databricks CLI:
+```bash
+databricks token create --lifetime-seconds 3600 --comment "DQX App Development Token"


Suggested change

databricks token create --lifetime-seconds 3600 --comment "DQX App Development Token"

databricks tokens create --lifetime-seconds 3600 --comment "DQX App Development Token"

mwojtyczka · 2025-09-01T15:24:14Z

docs/dqx/docs/dev/contributing.mdx

+
+2. Run the backend in development mode:
+```bash
+uvicorn src.databricks.labs.dqx.app.app:app --reload


The only way i could make it work on my machine was to enter the shell hatch shell or run it, or hatch run python -m uvicorn src.databricks.labs.dqx.app.app:app --reload, otherwise i was getting zsh: command not found: uvicorn

Co-authored-by: Copilot <[email protected]>

initial app skeleton

e4b6289

renardeinside requested a review from a team as a code owner August 28, 2025 18:25

renardeinside requested review from nehamilak-db and removed request for a team August 28, 2025 18:25

renardeinside had a problem deploying to tool August 28, 2025 18:25 — with GitHub Actions Error

fix pylint

26f6408

renardeinside temporarily deployed to tool August 28, 2025 18:32 — with GitHub Actions Inactive

mwojtyczka requested a review from Copilot August 29, 2025 10:28

Copilot AI reviewed Aug 29, 2025

View reviewed changes

ui/src/main.tsx Outdated Show resolved Hide resolved

src/databricks/labs/dqx/app/dependencies.py Outdated Show resolved Hide resolved

mwojtyczka requested changes Aug 29, 2025

View reviewed changes

Update src/databricks/labs/dqx/app/app.py

38b1546

fmt

mwojtyczka temporarily deployed to tool August 29, 2025 10:55 — with GitHub Actions Inactive

mwojtyczka reviewed Sep 1, 2025

View reviewed changes

Update src/databricks/labs/dqx/app/dependencies.py

003bd44

Co-authored-by: Copilot <[email protected]>

mwojtyczka temporarily deployed to tool September 1, 2025 15:44 — with GitHub Actions Inactive

Merge branch 'main' into feat/app-skeleton

a93f5b0

mwojtyczka temporarily deployed to tool October 2, 2025 22:02 — with GitHub Actions Inactive

mwojtyczka force-pushed the feat/app-skeleton branch from 6dc1723 to a93f5b0 Compare October 3, 2025 13:21

mwojtyczka temporarily deployed to tool October 3, 2025 13:21 — with GitHub Actions Inactive

mwojtyczka had a problem deploying to tool October 3, 2025 13:21 — with GitHub Actions Failure

mwojtyczka temporarily deployed to tool October 3, 2025 13:21 — with GitHub Actions Inactive

mwojtyczka had a problem deploying to tool October 3, 2025 13:21 — with GitHub Actions Error

Merge branch 'main' into feat/app-skeleton

e9949e9

mwojtyczka temporarily deployed to tool October 3, 2025 14:15 — with GitHub Actions Inactive

mwojtyczka had a problem deploying to tool October 3, 2025 14:15 — with GitHub Actions Failure

mwojtyczka temporarily deployed to tool October 3, 2025 14:15 — with GitHub Actions Inactive

Merge branch 'main' into feat/app-skeleton

0f158d3

mwojtyczka temporarily deployed to tool October 7, 2025 07:38 — with GitHub Actions Inactive

mwojtyczka had a problem deploying to tool October 7, 2025 07:38 — with GitHub Actions Failure

mwojtyczka temporarily deployed to tool October 7, 2025 07:38 — with GitHub Actions Inactive

mwojtyczka had a problem deploying to tool October 7, 2025 07:38 — with GitHub Actions Failure

Merge branch 'main' into feat/app-skeleton

4de2ee6

mwojtyczka temporarily deployed to tool October 16, 2025 08:52 — with GitHub Actions Inactive

mwojtyczka had a problem deploying to tool October 16, 2025 08:52 — with GitHub Actions Failure

mwojtyczka temporarily deployed to tool October 16, 2025 08:52 — with GitHub Actions Inactive


		## Contributing to the DQX App

		DQX App is a web application that provides a user interface for DQX. It is built with React and FastAPI.

		return f"postgresql://{self.user}:{self.password}@{self.host}:{self.port}/{self.database}?sslmode=require"


		class DatabaseManager(BaseModel):

	databricks token create --lifetime-seconds 3600 --comment "DQX App Development Token"
	databricks tokens create --lifetime-seconds 3600 --comment "DQX App Development Token"

✨ Add application skeleton #551

Are you sure you want to change the base?

✨ Add application skeleton #551

Uh oh!

Conversation

renardeinside commented Aug 28, 2025

Uh oh!

github-actions bot commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

mwojtyczka Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

mwojtyczka Aug 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mwojtyczka Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mwojtyczka Aug 29, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mwojtyczka Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mwojtyczka Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

mwojtyczka Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

mwojtyczka Sep 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

github-actions bot commented Aug 28, 2025 •

edited

Loading

mwojtyczka Aug 29, 2025 •

edited

Loading

mwojtyczka Sep 1, 2025 •

edited

Loading

mwojtyczka Sep 1, 2025 •

edited

Loading