Allow JSONEncoder to return bytes directly

[kevinpark1217]

Summary

Add explicit APIs for bytes-returning JSON serializers (like orjson), addressing maintainer feedback to avoid isinstance() checks in hot paths.

Changes

1. Kept JSONEncoder unchanged - still returns str only
2. Added new JSONBytesEncoder type - Callable[[Any], bytes]
3. No default bytes encoder - users must explicitly provide their encoder (e.g., orjson.dumps)
4. No isinstance() checks in hot paths - separate explicit APIs instead
5. Uses object instead of Any for data parameters in new APIs
6. dumps is keyword-only in send_json_bytes() methods, matching send_json() signature

New APIs

• JSONBytesEncoder type in typedefs.py
• JsonBytesPayload class - requires dumps parameter
• json_bytes_response() function - requires dumps parameter
• send_json_bytes() methods on WebSocketResponse and ClientWebSocketResponse - sends as binary frames, requires dumps keyword argument
• ClientSession(json_serialize_bytes=...) parameter - None by default, falls back to json_serialize if not set

Documentation

• Added Sphinx doc entries for json_bytes_response(), WebSocketResponse.send_json_bytes(), and ClientWebSocketResponse.send_json_bytes()
• Changelog uses proper RST cross-reference roles

Benefits

• Avoids runtime overhead of isinstance() checks
• Makes WebSocket frame type explicit (binary for bytes)
• Provides clear API separation for different use cases
• No behavioral changes to existing code

Closes #11988

[codspeed-hq]

Merging this PR will not alter performance

✅ 59 untouched benchmarks

---

_{Comparing kevinpark1217:allow-jsonencoder-return-bytes (4972fcb) with master (a640f4f)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.77%. Comparing base (a640f4f) to head (4972fcb).
⚠️ Report is 17 commits behind head on master.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #11989      +/-   ##
==========================================
- Coverage   98.77%   98.77%   -0.01%     
==========================================
  Files         128      128              
  Lines       44881    45006     +125     
  Branches     2382     2385       +3     
==========================================
+ Hits        44332    44453     +121     
- Misses        390      393       +3     
- Partials      159      160       +1     

Flag	Coverage Δ
CI-GHA	98.62% <100.00%> (-0.01%)	⬇️
OS-Linux	98.36% <100.00%> (+<0.01%)	⬆️
OS-Windows	96.72% <100.00%> (+<0.01%)	⬆️
OS-macOS	97.62% <100.00%> (+0.01%)	⬆️
Py-3.10.11	97.17% <100.00%> (+<0.01%)	⬆️
Py-3.10.19	97.64% <100.00%> (+<0.01%)	⬆️
Py-3.11.14	97.84% <100.00%> (+<0.01%)	⬆️
Py-3.11.9	97.37% <100.00%> (-0.01%)	⬇️
Py-3.12.10	97.46% <100.00%> (+<0.01%)	⬆️
Py-3.12.12	97.94% <100.00%> (+<0.01%)	⬆️
Py-3.13.12	98.19% <100.00%> (+<0.01%)	⬆️
Py-3.14.3	98.15% <100.00%> (-0.01%)	⬇️
Py-3.14.3t	97.24% <100.00%> (+<0.01%)	⬆️
Py-pypy3.11.13-7.3.20	97.39% <100.00%> (+<0.01%)	⬆️
VM-macos	97.62% <100.00%> (+0.01%)	⬆️
VM-ubuntu	98.36% <100.00%> (+<0.01%)	⬆️
VM-windows	96.72% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Dreamsorcerer]

@bdraco Do the benchmarks already cover these cases?

[webknjaz]

IIRC, there was a similar request years ago, rejected.

[webknjaz]

I think it was #4482

[webknjaz]

No tests? How do we know this keeps working?

[Dreamsorcerer]

IIRC, there was a similar request years ago, rejected.

With ujson dead, I think we're ready to change this: #10795 (comment)

My only question is whether the isinstance() calls here are fine, or we should add a new parameter and avoid the isinstance() checks. I suspect this is one of the performance hot paths for some cases (like websockets on homeassistant?).

[webknjaz]

Ah, fair. I also had a feeling that I'd prefer having a new API rather than overloading the existing one..

[bdraco]

Thanks for working on this! The use case makes sense.

I agree with webknjaz about preferring a new API over isinstance(). A few concerns:

1. isinstance() in the hot path. Minor, but runtime overhead on every call when the encoder type is fixed for the session lifetime.

2. WebSocket frame type changes silently. With a bytes encoder, send_json() now calls send_bytes() (binary frame) instead of send_str() (text frame). JSON is text, so this could break clients expectng text frames.

3. Harder to reason about. Union return types that branch at runtime are messier long-term.

Alternative: Add explicit parallel methods like JsonBytesPayload, json_bytes_response(), send_json_bytes(), and json_serialize_bytes param on ClientSession. No isinstance checks, clear contracts, existing code untouched.

[kevinpark1217]

@webknjaz @bdraco I have updated the PR. Now it's all new parallel API with bytes serialization option, rather than having isinstance() in the hot-path. Can you guys take a look again?

[Copilot]

Pull request overview

Adds explicit, bytes-oriented JSON serialization APIs (for serializers like orjson) so callers can avoid str→bytes conversion overhead and keep hot paths free of runtime isinstance() checks.

Changes:

• Introduces JSONEncoderBytes and new bytes-specific primitives: JsonBytesPayload and web.json_bytes_response().
• Adds send_json_bytes() to server/client WebSocket responses to transmit JSON as binary frames.
• Adds ClientSession(json_serialize_bytes=...) to send request JSON bodies using a bytes-returning encoder, plus corresponding test coverage.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
aiohttp/typedefs.py	Adds JSONEncoderBytes callable type alias.
aiohttp/payload.py	Adds JsonBytesPayload for bytes-returning JSON dumps.
aiohttp/client.py	Adds json_serialize_bytes session option and uses JsonBytesPayload when set.
aiohttp/web_response.py	Adds json_bytes_response() and exports it.
aiohttp/web.py	Re-exports json_bytes_response() from aiohttp.web.
aiohttp/web_ws.py	Adds WebSocketResponse.send_json_bytes() for binary JSON frames.
aiohttp/client_ws.py	Adds ClientWebSocketResponse.send_json_bytes() for binary JSON frames.
tests/test_payload.py	Adds unit tests for JsonBytesPayload.
tests/test_web_response.py	Adds tests for web.json_bytes_response().
tests/test_web_websocket.py	Adds error-path tests for WebSocketResponse.send_json_bytes().
tests/test_client_ws_functional.py	Adds functional tests asserting send_json_bytes() uses binary frames.
tests/test_client_functional.py	Adds functional test for ClientSession(json_serialize_bytes=...).
CHANGES/11989.feature.rst	Adds a towncrier feature fragment documenting the new APIs.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Dreamsorcerer]

I think this looks reasonable to me.

[kevinpark1217]

I think this looks reasonable to me.

@Dreamsorcerer Thanks! It's ready to be merged at you convenience

[patchback]

Backport to 3.14: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 67fa1f5 on top of patchback/backports/3.14/67fa1f5e64a95a5e081f566c4718bc7062c7dbdb/pr-11989

Backporting merged PR #11989 into master

1. Ensure you have a local repo clone of your fork. Unless you cloned it
   from the upstream, this would be your origin remote.
2. Make sure you have an upstream repo added as a remote too. In these
   instructions you'll refer to it by the name upstream. If you don't
   have it, here's how you can add it:

   $ git remote add upstream https://github.com/aio-libs/aiohttp.git

3. Ensure you have the latest copy of upstream and prepare a branch
   that will hold the backported code:

   $ git fetch upstream
   $ git checkout -b patchback/backports/3.14/67fa1f5e64a95a5e081f566c4718bc7062c7dbdb/pr-11989 upstream/3.14

4. Now, cherry-pick PR Allow JSONEncoder to return bytes directly #11989 contents into that branch:

   $ git cherry-pick -x 67fa1f5e64a95a5e081f566c4718bc7062c7dbdb

   If it'll yell at you with something like fatal: Commit 67fa1f5e64a95a5e081f566c4718bc7062c7dbdb is a merge but no -m option was given., add -m 1 as follows instead:

   $ git cherry-pick -m1 -x 67fa1f5e64a95a5e081f566c4718bc7062c7dbdb

5. At this point, you'll probably encounter some merge conflicts. You must
   resolve them in to preserve the patch from PR Allow JSONEncoder to return bytes directly #11989 as close to the
   original as possible.
6. Push this branch to your fork on GitHub:

   $ git push origin patchback/backports/3.14/67fa1f5e64a95a5e081f566c4718bc7062c7dbdb/pr-11989

7. Create a PR, ensure that the CI is green. If it's not — update it so that
   the tests and any other checks pass. This is it!
   Now relax and wait for the maintainers to process your pull request
   when they have some cycles to do reviews. Don't worry — they'll tell you if
   any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.