feat(databricks): support notebook-scoped Python packages

[sd-db]

Resolves #15688

Problem

Fusion does not support dbt-databricks' notebook_scoped_libraries configuration for Python models.

Solution

• Add v1-compatible configuration propagation and boolean validation.
• Prepend notebook-scoped %pip install and Python restart cells.
• Preserve package order and duplicates.
• Exclude scoped packages from job-level libraries while preserving additional libraries.
• Use the Jobs Reset API correctly for existing workflows.
• Add schema, adapter, and parity regression coverage.

Validation

• Fusion schema tests: 3 passed.
• Fusion Databricks adapter tests: 9 passed.
• v1 recording, Fusion replay, and Fusion live parity: 6/6 passed.
• Format, clippy, build, and diff checks passed.

Checklist

• I have read the contributing guide and understand what's expected of me.
• I have run this code in development, and it appears to resolve the stated issue.
• This PR includes tests, or tests are not required or relevant for this PR.
• This PR has no interface changes (e.g., macros, CLI, logs, JSON artifacts, config files, adapter interface, etc.) or this PR has already received feedback and approval from Product or DX.
• This PR includes type annotations for new and modified functions.

[codescene-delta-analysis]

Code Health Improved (1 files improve in Code Health)

Gates Failed
Enforce advisory code health rules (2 files with Large Method)

Our agent can fix these. Install it.

Gates Passed
3 Quality Gates Passed

Reason for failure

Enforce advisory code health rules	Violations	Code Health Impact
model_config.rs	1 advisory rule	4.42 → 4.41	Suppress
manifest_nodes.rs	1 advisory rule	6.97 → 6.97	Suppress

See analysis details in CodeScene

View Improvements

File	Code Health Impact	Categories Improved
mod.rs	9.01 → 9.02	Large Method, String Heavy Function Arguments

Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.