feat: support SparkJDBCDataset JDBC URLs in credentials

[Shizoqua]

Description

Closes #1286.

This PR allows spark.SparkJDBCDataset to use credentials.url as the JDBC URL when the explicit url argument is not provided.

Development notes

• Added fallback URL resolution from credentials["url"].
• Preserved explicit url precedence over credentials.
• Avoided mutating the caller's credentials dictionary.
• Excluded url from the Spark JDBC properties passed to load/save.
• Added focused tests for credentials URL usage, explicit URL precedence, and the missing URL error path.
• Updated the docstring example and kedro-datasets/RELEASE.md.

Checked locally:

• python -m ruff check kedro_datasets/spark/spark_jdbc_dataset.py tests/spark/test_spark_jdbc_dataset.py
• git diff --check
• python -m compileall kedro_datasets/spark/spark_jdbc_dataset.py tests/spark/test_spark_jdbc_dataset.py

Attempted locally:

• python -m pytest tests/spark/test_spark_jdbc_dataset.py did not run because the local environment is missing aiobotocore, which is imported by tests/conftest.py.

Developer Certificate of Origin

The commit includes a Signed-off-by line.

Checklist

• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Updated jsonschema/kedro-catalog-X.XX.json if necessary (not necessary)
• Added a description of this change in the relevant RELEASE.md file
• Added tests to cover my changes
• Received approvals from at least half of the TSC (required for adding a new, non-experimental dataset)

[ravi-kumar-pilla]

Kedro PR Review

Generated with review-kedro-pr skill.

Overview

• Critical: 1 | Suggestions: 3

Thanks @Shizoqua — this aligns SparkJDBCDataset with how SQLTableDataset handles connection strings via credentials, and the implementation looks solid (explicit url precedence, no credentials mutation, url excluded from JDBC properties).

Critical (must fix before merge)

• CI — detect-secrets / merge-gatekeeper are failing. The hook reports that .secrets.baseline needs updating after the docstring changes. Please run the repo's detect-secrets hook (or detect-secrets scan --update .secrets.baseline) and commit the baseline update.

Suggestions (consider improving)

• Add @Shizoqua under Community contributions in RELEASE.md.
• Add a symmetric load() precedence test to match test_save_explicit_url_takes_precedence_over_credentials_url.
• Optional: cross-reference the SQLTableDataset credentials.con pattern in the docstring.

Notes

• General review: Focused PR, good tests, backward compatible for existing catalogs with explicit url.
• PR checklist: Docs and RELEASE updated; tests added; DCO signed.
• API usage: Correct AbstractDataset usage; no breaking changes for current callers.

[ravi-kumar-pilla]

Hi @Shizoqua ,

Thanks for the contribution. The code looks good to me. Left minor comments and a required CI fix to go in before we merge this. Otherwise looks nice !! 💯

[Shizoqua]

thanks @ravi-kumar-pilla i have addressed Suggestions and fixed CI failure

[ravi-kumar-pilla]

LGTM...thank you @Shizoqua

[lrcouto]

Thanks for the PR @Shizoqua !

One small thing I'd like to point out is that the database URL passed in the credentials is being exposed by the describe() function. You can see it by running:

    db_credentials = {
        "url": "jdbc:postgresql://prod-db.internal:5432/table_name",
        "user": "johndoe",
        "password": "super-safe-password",
    }

    dataset = SparkJDBCDataset(table="table_name", credentials=db_credentials)

    described = dataset._describe()
    print(described)

If you're using pandas.SQLTableDataset as a reference, I think you could just drop the url key from describe() entirely.

Other than this, I think it looks good!

[Shizoqua]

Thanks for catching that! I’ve updated _describe() to omit the url from credentials entirely and added a test to cover it.