Allow JSONEncoder to return bytes directly

[kevinpark1217]

Summary

Add explicit APIs for bytes-returning JSON serializers (like orjson), addressing maintainer feedback to avoid isinstance() checks in hot paths.

Changes

1. Kept JSONEncoder unchanged - still returns str only
2. Added new JSONBytesEncoder type - Callable[[Any], bytes]
3. No default bytes encoder - users must explicitly provide their encoder (e.g., orjson.dumps)
4. No isinstance() checks in hot paths - separate explicit APIs instead
5. Uses object instead of Any for data parameters in new APIs
6. dumps is keyword-only in send_json_bytes() methods, matching send_json() signature

New APIs

• JSONBytesEncoder type in typedefs.py
• JsonBytesPayload class - requires dumps parameter
• json_bytes_response() function - requires dumps parameter
• send_json_bytes() methods on WebSocketResponse and ClientWebSocketResponse - sends as binary frames, requires dumps keyword argument
• ClientSession(json_serialize_bytes=...) parameter - None by default, falls back to json_serialize if not set

Documentation

• Added Sphinx doc entries for json_bytes_response(), WebSocketResponse.send_json_bytes(), and ClientWebSocketResponse.send_json_bytes()
• Changelog uses proper RST cross-reference roles

Benefits

• Avoids runtime overhead of isinstance() checks
• Makes WebSocket frame type explicit (binary for bytes)
• Provides clear API separation for different use cases
• No behavioral changes to existing code

Closes #11988

[codspeed-hq]

Merging this PR will not alter performance

✅ 59 untouched benchmarks

---

_{Comparing kevinpark1217:allow-jsonencoder-return-bytes (4972fcb) with master (a640f4f)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.77%. Comparing base (a640f4f) to head (4972fcb).
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #11989    +/-   ##
========================================
  Coverage   98.76%   98.77%            
========================================
  Files         128      128            
  Lines       44881    45006   +125     
  Branches     2382     2385     +3     
========================================
+ Hits        44328    44453   +125     
  Misses        393      393            
  Partials      160      160            

Flag	Coverage Δ
CI-GHA	98.62% <100.00%> (+<0.01%)	⬆️
OS-Linux	98.36% <100.00%> (+<0.01%)	⬆️
OS-Windows	96.72% <100.00%> (+<0.01%)	⬆️
OS-macOS	97.62% <100.00%> (+0.01%)	⬆️
Py-3.10.11	97.17% <100.00%> (+<0.01%)	⬆️
Py-3.10.19	97.64% <100.00%> (+<0.01%)	⬆️
Py-3.11.14	97.84% <100.00%> (+<0.01%)	⬆️
Py-3.11.9	97.37% <100.00%> (+<0.01%)	⬆️
Py-3.12.10	97.46% <100.00%> (+<0.01%)	⬆️
Py-3.12.12	97.94% <100.00%> (+<0.01%)	⬆️
Py-3.13.12	98.19% <100.00%> (+<0.01%)	⬆️
Py-3.14.3	98.15% <100.00%> (+<0.01%)	⬆️
Py-3.14.3t	97.24% <100.00%> (+<0.01%)	⬆️
Py-pypy3.11.13-7.3.20	97.39% <100.00%> (+<0.01%)	⬆️
VM-macos	97.62% <100.00%> (+0.01%)	⬆️
VM-ubuntu	98.36% <100.00%> (+<0.01%)	⬆️
VM-windows	96.72% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Dreamsorcerer]

@bdraco Do the benchmarks already cover these cases?

[webknjaz]

IIRC, there was a similar request years ago, rejected.

[webknjaz]

I think it was #4482

[webknjaz]

No tests? How do we know this keeps working?

[Dreamsorcerer]

IIRC, there was a similar request years ago, rejected.

With ujson dead, I think we're ready to change this: #10795 (comment)

My only question is whether the isinstance() calls here are fine, or we should add a new parameter and avoid the isinstance() checks. I suspect this is one of the performance hot paths for some cases (like websockets on homeassistant?).

[webknjaz]

Ah, fair. I also had a feeling that I'd prefer having a new API rather than overloading the existing one..

[bdraco]

Thanks for working on this! The use case makes sense.

I agree with webknjaz about preferring a new API over isinstance(). A few concerns:

1. isinstance() in the hot path. Minor, but runtime overhead on every call when the encoder type is fixed for the session lifetime.

2. WebSocket frame type changes silently. With a bytes encoder, send_json() now calls send_bytes() (binary frame) instead of send_str() (text frame). JSON is text, so this could break clients expectng text frames.

3. Harder to reason about. Union return types that branch at runtime are messier long-term.

Alternative: Add explicit parallel methods like JsonBytesPayload, json_bytes_response(), send_json_bytes(), and json_serialize_bytes param on ClientSession. No isinstance checks, clear contracts, existing code untouched.

[kevinpark1217]

@webknjaz @bdraco I have updated the PR. Now it's all new parallel API with bytes serialization option, rather than having isinstance() in the hot-path. Can you guys take a look again?

[Copilot]

Pull request overview

Adds explicit, bytes-oriented JSON serialization APIs (for serializers like orjson) so callers can avoid str→bytes conversion overhead and keep hot paths free of runtime isinstance() checks.

Changes:

• Introduces JSONEncoderBytes and new bytes-specific primitives: JsonBytesPayload and web.json_bytes_response().
• Adds send_json_bytes() to server/client WebSocket responses to transmit JSON as binary frames.
• Adds ClientSession(json_serialize_bytes=...) to send request JSON bodies using a bytes-returning encoder, plus corresponding test coverage.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
aiohttp/typedefs.py	Adds JSONEncoderBytes callable type alias.
aiohttp/payload.py	Adds JsonBytesPayload for bytes-returning JSON dumps.
aiohttp/client.py	Adds json_serialize_bytes session option and uses JsonBytesPayload when set.
aiohttp/web_response.py	Adds json_bytes_response() and exports it.
aiohttp/web.py	Re-exports json_bytes_response() from aiohttp.web.
aiohttp/web_ws.py	Adds WebSocketResponse.send_json_bytes() for binary JSON frames.
aiohttp/client_ws.py	Adds ClientWebSocketResponse.send_json_bytes() for binary JSON frames.
tests/test_payload.py	Adds unit tests for JsonBytesPayload.
tests/test_web_response.py	Adds tests for web.json_bytes_response().
tests/test_web_websocket.py	Adds error-path tests for WebSocketResponse.send_json_bytes().
tests/test_client_ws_functional.py	Adds functional tests asserting send_json_bytes() uses binary frames.
tests/test_client_functional.py	Adds functional test for ClientSession(json_serialize_bytes=...).
CHANGES/11989.feature.rst	Adds a towncrier feature fragment documenting the new APIs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Dreamsorcerer]

I think this looks reasonable to me.

[kevinpark1217]

I think this looks reasonable to me.

@Dreamsorcerer Thanks! It's ready to be merged at you convenience

[bdraco]

would be nice to make these real rst links