Validate Dataset modules for the Kedro HTTP server

[lrcouto]

Description

Addresses a possible RCE vulnerability on catalog type resolution through non-trusted runtime_params on the HTTP server.

Development notes

This PR was reworked after initial review. The first version added a dataset_modules_whitelist allowlist to OmegaConfigLoader based on #5568. We decided to not use an allowlist for this specific issue (an unified allowlist mechanism for Kedro is being currently discussed here: #5652)

For this current version, OmegaConfigLoader gains restrict_runtime_params_type_selection: bool = False. When set, any catalog type field actually resolved via runtime_params (at any nesting depth) is simply rejected.

Instead, KedroServiceSession.run() / .load_context() / ._get_config_loader() gains a trusted_runtime_params: bool = True parameter. trusted_runtime_params=False is passed by http_server.py's _execute_pipeline, since request.params is where untrusted and possibly malicious input is going to come from.

Impact to existing projects

Only projects whose catalog uses the runtime_params: resolver to set a dataset's type field, and run that catalog through the Kedro HTTP server, are affected. For example:

source_data:
  type: "${runtime_params:dataset.type}"
  filepath: "data/01_raw/companies.csv"

If the catalog does not contain this pattern, no changes need to be made.

If it does and it serves it over the HTTP server, POST /run requests that receive type from params will now fail with InterpolationResolutionError,

## Developer Certificate of Origin
All commits must be signed off to comply with the [DCO](https://developercertificate.org/). If your PR is blocked due to unsigned commits, follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR.

## Checklist

- [ ] Read the [contributing](https://github.com/kedro-org/kedro/blob/main/CONTRIBUTING.md) guidelines
- [ ] Linked to a relevant GitHub issue
- [ ] Signed off each commit with a [DCO](https://developercertificate.org/)
- [ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
- [ ] Updated the documentation to reflect the code changes
- [ ] Added a description of this change in the [`RELEASE.md`](https://github.com/kedro-org/kedro/blob/main/RELEASE.md) file
- [ ] Added tests to cover my changes

[ankatiyar]

Thanks @lrcouto! I think before we add allowlisting mechanism here, we should wait to finalise the protocol re: #5652
In either case, I think we don't need to complicate the solution with allow listing at all. Is there a way we can check if the runtime_params passed by the user is an executable string (similar to the logging config bug we solved) if it goes into the type: field in general. Maybe we can centralise some utility functions and use them in the OmegaConfigLoader to check when catalog is being resolved?

[lrcouto]

Thanks @lrcouto! I think before we add allowlisting mechanism here, we should wait to finalise the protocol re: #5652 In either case, I think we don't need to complicate the solution with allow listing at all. Is there a way we can check if the runtime_params passed by the user is an executable string (similar to the logging config bug we solved) if it goes into the type: field in general. Maybe we can centralise some utility functions and use them in the OmegaConfigLoader to check when catalog is being resolved?

I'm thinking on how this could be done, because any dataset type entry is "executable" as in "running the module code", as it resolves to the dataset object. But not all of those have the possibility of being vulnerable to RCE. In the example of the PickleDataset, it'll load and arbitrary python object, which can be dangerous, but that same danger is not going to happen on a CSV dataset. The idea of using a whitelist would be to add that element of human oversight to the type selection before the dataset object itself is loaded.

So I'm considering blocking dataset selection via runtime_params only when the runtime_params themselves come from an untrusted source, i.e. from a request from outside. Lemme see if I can make that work.

[lrcouto]

Pushed a different solution. A little less flexible than the allowlist, but simpler.

[ankatiyar]

Thanks @lrcouto this solution looks better to me - left some suggestions
Also wondering if this can be done in a slightly cleaner way without adding too much code in OmegaConfigLoader. I don't have a clean solution yet myself, will think about it some more and get back to you about this.

[ankatiyar]

Is this necessary here and on line 435?

[lrcouto]

Mypy complains about the big Union that returns from OmegaConf.to_container. It's the same reason why this decorator is being used for load_and_merge_dir_config, for example.

[lrcouto]

Thank you @ankatiyar ! Applied the suggested changes.

[merelcht]

The approach looks good to me. My main suggestion is to clean up the docstring/comments a bit, because they're quite verbose.

[ankatiyar]

Thanks @lrcouto the method looks good to me! I agree with merel's comments about making things less verbose!

[lrcouto]

Thank you @merelcht ! Applied the suggested changes.

[merelcht]

Thanks Laura, this looks good to me now! 👍