Add PUT request support at `/api/jobs/{job_id}/files` to `FastAPIJobFiles` #20353

kysrpex · 2025-05-26T13:26:00Z

⚠️ This PR requires #20235, it will remain as a draft until the latter is merged. Have a look at 47f9617 to see a diff showing what truly changed. This is one of the tweaks that are needed for the development of an integration of Galaxy with ARC (Advanced Resource Connector) as a Pulsar job runner.

This path already supports GET, HEAD and POST requests; add support for PUT requests. There is a significant difference in behavior between POST and PUT requests:

POST requests take path and job_key both as query parameters or as body parameters belonging to a multipart request. PUT requests take them only as query parameters (just like GET and HEAD).
POST requests submit a file as one of the fields of the multipart request, whereas the submitted file is the whole body of the request for PUT requests.
POST requests can append to the tool_stdout and tool_stderr, PUT requests can only create new files or overwrite whole files.
POST requests support resumable uploads but PUT requests do not.
POST requests take the form parameters __file_path (path of a file uploaded via the nginx upload module) and __file but PUT requests do not.

How to test the changes?

(Select all options that apply)

I've included appropriate automated tests.
This is a refactoring of components with existing test coverage.
Instructions for manual testing are as follows:
1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

`FastAPIJobFiles` is the new, FastAPI version of `JobFilesAPIController`. The endpoints that have been migrated should exhibit exactly the same behavior as the old ones from `FastAPIJobFiles`. Something to keep in mind is that while FastAPI has some extra built-in features that the legacy WSGI system did not have, such as answering HEAD requests, those do not work because of the way legacy WSGI endpoints are injected into the FastAPI app (using `app.mount("/", wsgi_handler)`), meaning that for example, HEAD requests are passed to the `wsgi_handler` sub-application. Endpoints dedicated to TUS uploads work in tandem with the WSGI middleware `TusMiddleware` from the `tuswsgi` package. As explained above, WSGI middlewares and endpoints are injected into the FastAPI app after FastAPI routes as a single sub-application `wsgi_handler` using `app.mount("/", wsgi_handler)`, meaning that requests are passed to the `wsgi_handler` sub-application (and thus to `TusMiddleware`) only if there was no FastAPI endpoint defined to handle them. Therefore, they cannot be migrated to FastAPI unless `TusMiddleware` is also migrated to ASGI.

Work around a bug in FastAPI (fastapi/fastapi#13175) that assigns the same operation id to both request methods GET and HEAD of the endpoint `/api/jobs/{job_id}/files` when using the `@router.api_route()` decorator with `methods=["GET", "HEAD"]` as keyword argument.

…T requests to `/api/jobs/{job_id}/files` Pulsar formats the `path` and `job_key` parameters as query parameters when submitting POST requests to `/api/jobs/{job_id}/files`. However, many Galaxy tests format them as form parameters. The only way to keep the endpoint working as it should (as it worked before the migration to FastAPI) is to accept both query and form parameters.

… requests to `/api/jobs/{job_id}/files` FastAPI will not use the parameter aliases of form parameters in the OpenAPI docs, but the name of their Python variables. Therefore, the API docs show `path_form` and `job_key_form`. Rename them so that the API docs show the correct parameter names.

…iles` This path already supports GET, HEAD and POST requests; add support for PUT requests. There is a significant difference in behavior between POST and PUT requests: - POST requests take `path` and `job_key` both as query parameters or as body parameters belonging to a multipart request. PUT requests take them only as query parameters (just like GET and HEAD). - POST requests submit a file as one of the fields of the multipart request, whereas the submitted file is the whole body of the request for PUT requests. - POST requests can append to the `tool_stdout` and `tool_stderr`, PUT requests can only create new files or overwrite whole files. - POST requests support resumable uploads but PUT requests do not. - POST requests take the form parameters `__file_path` (path of a file uploaded via the nginx upload module) and `__file` but PUT requests do not.