profiler documentation

[sundarshankar89]

Changes

What does this PR do?

Adds documentation about profilers

• Source Pre-requisites
• Configuration Process
• Execution Process

Relevant implementation details

Pre-req

Caveats/things to watch out for when reviewing:

Linked issues

Resolves #..

Functionality

• added relevant user documentation
• added new CLI command
• modified existing command: databricks labs lakebridge ...
• ... +add your own

Tests

• manually tested
• added unit tests
• added integration tests

[github-actions]

✅ 51/51 passed, 10 flaky, 3m6s total

Flaky tests:

• 🤪 test_validate_mixed_checks (161ms)
• 🤪 test_validate_successful_schema_check (159ms)
• 🤪 test_validate_invalid_schema_path (1ms)
• 🤪 test_validate_invalid_schema_check (1ms)
• 🤪 test_validate_non_empty_tables (7ms)
• 🤪 test_stores_and_fetches_config (1.851s)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[False] (13.649s)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[True] (13.98s)
• 🤪 test_transpile_teradata_sql_non_interactive[True] (5.479s)
• 🤪 test_transpiles_informatica_to_sparksql (9.856s)

_{Running from acceptance #2880}

[gueniai]

We need to add docs for deploying the local dashboard as well.

The section of "The profiler will" should be more prominent, customers will have questions regarding runtimes, impact to their system, and probably some details about the output.

[sundarshankar89]

We need to add docs for deploying the local dashboard as well.

The section of "The profiler will" should be more prominent, customers will have questions regarding runtimes, impact to their system, and probably some details about the output.

That portion is still in development, can we iterate on that once @goodwillpunning is finished withat that?

[gueniai]

Yes, of course, but that will be done before we release the docs, this was just a note so that we don't forget to add functionality too before merging this branch

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.61%. Comparing base (7be6421) to head (80c2135).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2098      +/-   ##
==========================================
- Coverage   64.87%   63.61%   -1.26%     
==========================================
  Files          96       98       +2     
  Lines        7895     8210     +315     
  Branches      821      839      +18     
==========================================
+ Hits         5122     5223     +101     
- Misses       2593     2807     +214     
  Partials      180      180              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.