Skip to content

build(python): upgrade to Flask 3.x#678

Open
tiborsimko wants to merge 2 commits intoreanahub:masterfrom
tiborsimko:invenio-upgrade
Open

build(python): upgrade to Flask 3.x#678
tiborsimko wants to merge 2 commits intoreanahub:masterfrom
tiborsimko:invenio-upgrade

Conversation

@tiborsimko
Copy link
Copy Markdown
Member

Upgrade Flask from <2.3.0 to >=3.0.0 and Werkzeug from <2.3.0 to >=3.0.0, in line with the Invenio package upgrade in reana-server. Bump marshmallow from <3.0.0 to >=3.5.0,<4.0.0 and webargs to >=8.0, since the new Invenio stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8 APIs. Replace the deprecated missing= keyword on schema fields with load_default=. Add unknown=marshmallow.EXCLUDE to all @use_kwargs and @use_args decorators that parse query parameters, so that REANA-specific extras such as access_token are silently ignored, matching the pre-upgrade webargs 5 behaviour.

Replace all remaining Model.query call sites in the REST handlers, the workflow run manager, and the test suite with explicit Session.query(Model) queries, since SQLAlchemy 2.x no longer supports the Base.query shortcut.

Replace the deprecated FLASK_ENV setting with FLASK_DEBUG, both in the runtime checks of the workflow run manager and in the DEBUG_ENV_VARS injected into spawned workflow engine containers. Update the test configuration to use DEBUG instead of FLASK_ENV.

Remove pinned reana-commons, reana-db, and kubernetes entries from requirements.txt so that pip install . resolves them freshly from setup.py against the locally-mounted modules/reana-db and modules/reana-commons shared sources during development, avoiding version conflicts with PyPI versions that still carry the old pins.

Closes reanahub/reana-server#770.

@tiborsimko tiborsimko self-assigned this Apr 26, 2026
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 26, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
@michaelbuchar
Copy link
Copy Markdown

AI Review

Workspace-limit tests still mock the pre-migration query API

_mock_workspace_access() still monkeypatches workflows_workspace.User.query, but the migrated handlers now look users up via Session.query(User). On this branch that means the helper no longer intercepts anything, and the PR's python-tests job fails with five tests/test_workspace_limits.py cases returning 500 instead of their expected 200/400 responses. Please update the helper to stub Session.query(...) or use a real session-backed user fixture.

reana-workflow-controller/tests/test_workspace_limits.py:45-55

tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Replace `docker stop` with `docker rm -f` when cleaning the old
Postgres test container. Stopping a `--rm` container triggers
asynchronous removal, so on faster machines the next `docker
run --rm` could fire before the old container was gone, leading
to a name conflict. `docker rm -f` stops and removes
synchronously, so the subsequent `docker run` always succeeds.
Replace `docker stop` with `docker rm -f` when cleaning the old
Postgres test container. Stopping a `--rm` container triggers
asynchronous removal, so on faster machines the next `docker
run --rm` could fire before the old container was gone, leading
to a name conflict. `docker rm -f` stops and removes
synchronously, so the subsequent `docker run` always succeeds.
tiborsimko added a commit to tiborsimko/reana-workflow-controller that referenced this pull request Apr 28, 2026
Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 28, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.51%. Comparing base (dc8bb22) to head (324f4ad).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #678      +/-   ##
==========================================
+ Coverage   74.46%   74.51%   +0.04%     
==========================================
  Files          17       17              
  Lines        2044     2048       +4     
==========================================
+ Hits         1522     1526       +4     
  Misses        522      522              
Files with missing lines Coverage Δ
reana_workflow_controller/config.py 95.45% <ø> (ø)
reana_workflow_controller/rest/utils.py 84.70% <100.00%> (ø)
reana_workflow_controller/rest/workflows.py 68.83% <100.00%> (+0.10%) ⬆️
...eana_workflow_controller/rest/workflows_session.py 72.34% <100.00%> (+0.60%) ⬆️
reana_workflow_controller/rest/workflows_status.py 83.90% <100.00%> (+0.18%) ⬆️
...na_workflow_controller/rest/workflows_workspace.py 63.57% <100.00%> (+0.26%) ⬆️
reana_workflow_controller/workflow_run_manager.py 80.32% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Upgrade Flask from `<2.3.0` to `>=3.0.0` and Werkzeug from
`<2.3.0` to `>=3.0.0`, in line with the Invenio package upgrade
in reana-server. Bump marshmallow from `<3.0.0` to
`>=3.5.0,<4.0.0` and webargs to `>=8.0`, since the new Invenio
stack requires marshmallow 3 and the corresponding webargs 8.

Adapt the REST endpoints to the new marshmallow 3 and webargs 8
APIs. Replace the deprecated `missing=` keyword on schema fields
with `load_default=`. Add `unknown=marshmallow.EXCLUDE` to all
`@use_kwargs` and `@use_args` decorators that parse query
parameters, so that REANA-specific extras such as `access_token`
are silently ignored, matching the pre-upgrade webargs 5
behaviour.

Replace all remaining `Model.query` call sites in the REST
handlers, the workflow run manager, and the test suite with
explicit `Session.query(Model)` queries, since SQLAlchemy 2.x
no longer supports the `Base.query` shortcut.

Replace the deprecated `FLASK_ENV` setting with `FLASK_DEBUG`,
both in the runtime checks of the workflow run manager and in
the `DEBUG_ENV_VARS` injected into spawned workflow engine
containers. Update the test configuration to use `DEBUG` instead
of `FLASK_ENV`.

Remove pinned `reana-commons`, `reana-db`, and `kubernetes`
entries from `requirements.txt` so that `pip install .` resolves
them freshly from `setup.py` against the locally-mounted
`modules/reana-db` and `modules/reana-commons` shared sources
during development, avoiding version conflicts with PyPI
versions that still carry the old pins.

Closes reanahub/reana-server#770
@tiborsimko
Copy link
Copy Markdown
Member Author

_mock_workspace_access() still monkeypatches workflows_workspace.User.query, but the migrated handlers now look users up via Session.query(User). On this branch that means the helper no longer intercepts anything, and the PR's python-tests job fails with five tests/test_workspace_limits.py cases returning 500 instead of their expected 200/400 responses.

Thanks — fixed in this branch. _mock_workspace_access() now patches workflows_workspace.Session.query (returning a fake query that yields a stub user) instead of the obsolete User.query, so the helper intercepts the migrated Session.query(User).filter(...).first() lookup. The five tests/test_workspace_limits.py cases pass locally and in CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

build(python): upgrade to latest Invenio packages, Flask 3.x, SQLAlchemy 2.x

2 participants