refactor: consolidate datafusion session setup

[roeap]

Description

We are still creating sessions in several places in the codebase. With this PR, we create a single code path (or at least module) to create datafusion sessions. We also further generalise some APIs to use the Session trait.

The main code paths where we still use SessionState whenever we need to use the DataFrame abstraction. These might disappear once we start planning more based on data.

Closing the related issue for now, since further improvements are part of larger refactorings.

Related Issue(s)

• closes: Consolidate Datafusion session management #3799

[codecov]

Codecov Report

❌ Patch coverage is 85.31469% with 21 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.76%. Comparing base (09e2a9b) to head (17ba5f7).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/core/src/operations/delete.rs	87.75%	0 Missing and 6 partials ⚠️
crates/core/src/operations/write/execution.rs	64.70%	0 Missing and 6 partials ⚠️
crates/core/src/operations/constraints.rs	57.14%	0 Missing and 3 partials ⚠️
crates/core/src/operations/load.rs	60.00%	0 Missing and 2 partials ⚠️
crates/core/src/operations/write/mod.rs	75.00%	0 Missing and 2 partials ⚠️
crates/core/src/delta_datafusion/session.rs	97.87%	0 Missing and 1 partial ⚠️
crates/core/src/operations/update.rs	66.66%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3860      +/-   ##
==========================================
- Coverage   73.79%   73.76%   -0.03%     
==========================================
  Files         151      152       +1     
  Lines       39149    39122      -27     
  Branches    39149    39122      -27     
==========================================
- Hits        28891    28860      -31     
- Misses       8987     8989       +2     
- Partials     1271     1273       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.