Skip to content

fix(ai): normalize messages in truncate_messages_by_size to support Pydantic (#5350) #5730

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
harryautomazione wants to merge 6 commits into
base: master
Choose a base branch
from
+8 −7
Open

fix(ai): normalize messages in truncate_messages_by_size to support Pydantic (#5350) #5730

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bd0b816
fix(ai): remove setting of GEN_AI_AGENT_NAME from langchain
harryautomazione Mar 20, 2026
85279b5
Fix Pydantic AI docs: update class docstring with correct install com…
harryautomazione Mar 20, 2026
bcce29d
Fix Langchain GEN_AI_AGENT_NAME regression by adding contextvar propa…
harryautomazione Mar 20, 2026
556b6d5
Fix _wrap_agent_executor_stream token reset leak on exception during …
harryautomazione Mar 20, 2026
d0299c8
fix(ai): normalize messages before tracking truncation size in OpenAI…
harryautomazione Mar 22, 2026
dbb209e
chore: remove accidental changes from this PR
harryautomazione Mar 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions sentry_sdk/ai/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -682,18 +682,19 @@ def truncate_messages_by_size(
In the single message case, the serialized message size may exceed `max_bytes`, because
truncation is based only on character count in that case.
"""
serialized_json = json.dumps(messages, separators=(",", ":"))
normalized_messages = _normalize_data(messages, unpack=False)
Copy link

@cursor cursor bot Mar 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normalization silently converts None values to string "None"

Medium Severity

_normalize_data converts None values to the string "None" (line 488's fallback: str(data)). OpenAI assistant messages with tool calls commonly have content: None. After this change, those become content: "None", altering the JSON representation from null to "None" — a semantic difference that also slightly skews the byte-size calculation used for truncation decisions.

Additional Locations (1)
Fix in Cursor Fix in Web

serialized_json = json.dumps(normalized_messages, separators=(",", ":"))
current_size = len(serialized_json.encode("utf-8"))

if current_size <= max_bytes:
return messages, 0
return normalized_messages, 0

truncation_index = _find_truncation_index(messages, max_bytes)
if truncation_index < len(messages):
truncated_messages = messages[truncation_index:]
truncation_index = _find_truncation_index(normalized_messages, max_bytes)
if truncation_index < len(normalized_messages):
truncated_messages = normalized_messages[truncation_index:]
else:
truncation_index = len(messages) - 1
truncated_messages = messages[-1:]
truncation_index = len(normalized_messages) - 1
truncated_messages = normalized_messages[-1:]

if len(truncated_messages) == 1:
truncated_messages[0] = _truncate_single_message_content_if_present(
Expand Down
Loading