Skip to content

Add rate limiting middleware example and introduce tests for examples#11969

Open
rodrigobnogueira wants to merge 46 commits intoaio-libs:masterfrom
rodrigobnogueira:add-rate-limit-middleware-example
Open

Add rate limiting middleware example and introduce tests for examples#11969
rodrigobnogueira wants to merge 46 commits intoaio-libs:masterfrom
rodrigobnogueira:add-rate-limit-middleware-example

Conversation

@rodrigobnogueira
Copy link
Member

@rodrigobnogueira rodrigobnogueira commented Jan 17, 2026

What do these changes do?

Adds a new example (examples/rate_limit_middleware.py) demonstrating rate limiting middleware for the aiohttp client using the token bucket algorithm.

Introduced tests/test_examples.py with subprocess-based smoke tests for self-contained examples (e.g., middlewares) and in-process functional tests for importable server examples. This validates examples run without errors, addressing potential bit rot.

Refactoring of examples to make them pytestable.

Are there changes in behavior for the user?

No changes to the aiohttp library code.

Is it a substantial burden for the maintainers to support this?

No.

Related issue number

N/A — New example contribution.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist
  • Documentation reflects the changes
  • If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • Add a new news fragment into the CHANGES/ folder

rodrigobnogueira pushed a commit to rodrigobnogueira/aiohttp that referenced this pull request Jan 17, 2026
@psf-chronographer psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Jan 17, 2026
@codspeed-hq
Copy link

codspeed-hq bot commented Jan 17, 2026

Merging this PR will not alter performance

✅ 59 untouched benchmarks


Comparing rodrigobnogueira:add-rate-limit-middleware-example (d2d96e9) with master (a640f4f)

Open in CodSpeed

@codecov
Copy link

codecov bot commented Jan 17, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.77%. Comparing base (0cba798) to head (d2d96e9).
⚠️ Report is 21 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff            @@
##           master   #11969    +/-   ##
========================================
  Coverage   98.76%   98.77%            
========================================
  Files         127      128     +1     
  Lines       44655    44881   +226     
  Branches     2367     2382    +15     
========================================
+ Hits        44102    44329   +227     
  Misses        393      393            
+ Partials      160      159     -1     
Flag Coverage Δ
CI-GHA 98.62% <ø> (+<0.01%) ⬆️
OS-Linux 98.36% <ø> (-0.01%) ⬇️
OS-Windows 96.71% <ø> (+<0.01%) ⬆️
OS-macOS 97.62% <ø> (+0.02%) ⬆️
Py-3.10.11 97.16% <ø> (+0.01%) ⬆️
Py-3.10.19 97.64% <ø> (+<0.01%) ⬆️
Py-3.11.14 97.84% <ø> (+0.01%) ⬆️
Py-3.11.9 97.37% <ø> (+0.02%) ⬆️
Py-3.12.10 97.45% <ø> (+0.01%) ⬆️
Py-3.12.12 97.93% <ø> (+0.01%) ⬆️
Py-3.13.11 ?
Py-3.13.12 98.18% <ø> (?)
Py-3.14.2 ?
Py-3.14.2t ?
Py-3.14.3 98.15% <ø> (?)
Py-3.14.3t 97.23% <ø> (?)
Py-pypy3.11.13-7.3.20 97.38% <ø> (-0.03%) ⬇️
VM-macos 97.62% <ø> (+0.02%) ⬆️
VM-ubuntu 98.36% <ø> (-0.01%) ⬇️
VM-windows 96.71% <ø> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@webknjaz webknjaz requested review from Dreamsorcerer and removed request for asvetlov January 21, 2026 13:51
@webknjaz
Copy link
Member

@Dreamsorcerer do we have any guidance on the examples folder? Some of those files are decade-old and I, for example, never really look in there.

@rodrigobnogueira would you be interested in trying to get pytest to run the examples and getting code coverage as acceptance tests?

@rodrigobnogueira
Copy link
Member Author

rodrigobnogueira commented Jan 21, 2026

@Dreamsorcerer do we have any guidance on the examples folder? Some of those files are decade-old and I, for example, never really look in there.

@rodrigobnogueira would you be interested in trying to get pytest to run the examples and getting code coverage as acceptance tests?

Yes @webknjaz , I can try that to get pytest to run the examples, but I suppose the examples guidance should be the first step, because some examples may not comply.


Guidance draft:

  • No external dependencies: Examples should not rely on dependencies outside the repository or special local configurations unless clearly stated.

Potential tweaks of the rule above: if an example must use an external dep (e.g., for integrating with a database like asyncpg in a real-world demo), it should include a clear setup section at the top, like a requirements.txt snippet or install command.

  • Imports: Examples should not contain relative imports (e.g., from .. import aiohttp).
  • Minimal but complete: They should demonstrate one concept clearly without excessive boilerplate, but include enough code to actually run.
  • User input: The example should not require user input for the test to run.
  • Documentation: Each example should include a docstring explaining what it demonstrates and how to run it.

@rodrigobnogueira

This comment was marked as outdated.

@Dreamsorcerer
Copy link
Member

@Dreamsorcerer do we have any guidance on the examples folder? Some of those files are decade-old and I, for example, never really look in there.

Not particularly. This PR is still open on my computer to review, will get back to it soon enough.

@rodrigobnogueira would you be interested in trying to get pytest to run the examples and getting code coverage as acceptance tests?

I guess the question is whether all examples should be cluttered with tests? We have mypy running on them which atleast avoids regular API breakage like we often had in the past.

@webknjaz
Copy link
Member

@rodrigobnogueira would you be interested in trying to get pytest to run the examples and getting code coverage as acceptance tests?

I guess the question is whether all examples should be cluttered with tests? We have mypy running on them which atleast avoids regular API breakage like we often had in the past.

@Dreamsorcerer I was mostly thinking of having them run as something like subprocess invocations (maybe in containers). The tests would be just invoking them and checking they completed fine, didn't have warnings, smoke mainly, not adding assertions inside. We could have them have own pytest mark so we'd only invoke them in a single CI job.

rodrigo.nogueira added 6 commits January 30, 2026 13:31
This example demonstrates how to implement client-side rate limiting
using the token bucket algorithm with support for:

- Configurable rate and burst size
- Per-domain rate limiting for multi-host scenarios
- Automatic Retry-After header handling

Includes a self-contained test server for demonstration.
Per maintainer feedback, add subprocess-based smoke tests for examples.
Tests verify examples complete without errors.

- Add 'example' pytest marker excluded from default run
- Create tests/test_examples.py with parametrized tests
- Test 7 self-contained middleware examples
- Tests must run with --numprocesses=0 due to port conflicts
@rodrigobnogueira rodrigobnogueira force-pushed the add-rate-limit-middleware-example branch from 9f7a307 to 4d6c2c4 Compare January 30, 2026 18:25
@rodrigobnogueira

This comment was marked as outdated.

@rodrigobnogueira
Copy link
Member Author

rodrigobnogueira commented Jan 31, 2026

The regex test issue (#11992) has just shown up here.

macos, python 3.10

FAILED tests/test_client_middleware_digest_auth.py::test_regex_performance - assert (216.348098916 - 216.338082958) < 0.01

rodrigo.nogueira added 2 commits February 11, 2026 22:12
Fixed flaky test where websocket client closed before server echo
responses arrived. Added asyncio.sleep(0.1) to allow pending messages
to be received before assertions.
@rodrigobnogueira
Copy link
Member Author

rodrigobnogueira commented Feb 12, 2026

Standardizing Example Tests to Self-Contained Smoke Tests

Every example now follows a 3-function pattern:

run_test_server() -> tuple[AppRunner, int]
Starts the application on a dynamic port (port 0) for testing.
Returns the runner (for cleanup) and the bound port.

run_tests(port: int) -> None
Runs assertions against the running server using ClientSession.
Prints "OK: ..." for each passing check.

main() -> None
Orchestrates: starts server, runs tests, cleans up.
The main block simply calls asyncio.run(main()).

For server-only examples (web_srv, server_simple, web_cookies, etc.), the test code exercises the existing routes using aiohttp.ClientSession.

For client examples (client_json, client_auth, curl, client_ws), we embed a lightweight mock server inside the example itself, then point the existing client logic at it.

@rodrigobnogueira
Copy link
Member Author

All test logic that previously lived in tests/test_examples.py as functional tests (using aiohttp_client and importing from examples/) was moved INTO each example file.

tests/test_examples.py now has a single parametrized test that runs each example as a subprocess.

I think having the test logic embedded in each example file adds value:

  • A user can run python examples/web_cookies.py, for example, and see it work end-to-end, with clear "OK: ..." output showing what was tested.
  • There's no need to understand pytest, aiohttp_client, or any test infrastructure. The example is the demo.
  • The previous structure was confusing: you had an example that did web.run_app(init()) — which blocks forever — and then a separate functional test in a completely different file that imported the example's init() function. A newcomer looking at the example wouldn't know how to actually test it. Now it's all in one place.

@rodrigobnogueira
Copy link
Member Author

rodrigobnogueira commented Feb 12, 2026

@webknjaz , I've gone through the example files in this PR and checked them against the originals on master. I believe they correspond in intent to the original examples — the core handler logic, route definitions, and middleware implementations are preserved across the board. 🙏

I'm aware this PR has grown quite a bit and may be harder to review than ideal. I think there's room for simplification. In particular, there is a ~12-line boilerplate block (run_test_server with AppRunner/TCPSite on port 0, and the main() try/finally pattern) that repeats across 15+ example files. Extracting that into a small common helper module under examples/ (something like _helpers.py) would reduce the diff size. The trade-off is that currently users can have a complete example in one file.

I hope this is headed in the right direction. If you have suggestions on how we could wrap this PR up sooner, I'm happy to work on improvements in follow-up PRs. For example, I still haven't worked on exposing the actual bound port through the runners/sites public interface, as you previously mentioned. I can try this here as well. Let me know your thoughts.

Thanks

Copy link
Member

@Dreamsorcerer Dreamsorcerer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this makes the examples too complex for my liking.

If we want to proceed with actual tests for the examples, I would suggest adding separate test files under examples/tests/. These could atleast provide examples to users on how to do proper testing of their application. If we go with this, I'd suggest a separate PR for each example, it will significantly accelerate the speed we can review and merge the changes.

At the same time, I kind of feel that these examples should be small and trivial enough, that they don't really need tests. Anything complex enough to need proper testing likely goes to https://github.com/aio-libs/aiohttp-demos/

rodrigo.nogueira and others added 4 commits February 14, 2026 23:41
- Strip rate_limit_middleware.py to just TokenBucket + RateLimitMiddleware
  classes and a minimal main() demo using base_url
- Add examples/tests/test_rate_limit_middleware.py with unit tests
- Add examples/tests/pytest.ini for pythonpath config
- Update CHANGES/11969.doc.rst to reflect simplified scope
…demo output to include request elapsed time.
@rodrigobnogueira
Copy link
Member Author

Hey @Dreamsorcerer @webknjaz, the PR looks manageable now. I've kept just my original example with some improvements and created a test file for it in examples/tests/test_rate_limit_middleware.py

Added examples/tests/pytest.ini so tests are runnable locally without affecting project config.

Kept scope minimal—no touches to existing examples.

This keeps examples clear for learning while providing dedicated tests in a separate dir.

Let me know if we can apply the same idea to other examples. This way I could check the other examples and submit maybe 2 or 3 adjusted examples with tests in subsequent PRs.

🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

Comments