Draft
Conversation
d33b0e2 to
fe2c84b
Compare
MSK event headers delivered by the Java Lambda runtime use a JSON object with numeric string keys and decimal string values rather than an array of integers. Records are similarly delivered as an object with numeric string keys instead of an array. Update deserialization and carrier extraction to support both formats, and update the fixture and tests to reflect the real-world payload shape. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace `n as u8` cast with `u8::try_from(n).ok()` to avoid truncation - Collapse nested `if let` blocks into a single `if let ... && let ...` - Replace redundant closure `|o| o.len()` with `serde_json::Map::len` Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
https://datadoghq.atlassian.net/browse/SLES-2739
In Kafka's wire protocol (KIP-82), header values are always byte[]. Every Kafka client library enforces this:
All three tracers accept string trace context values from the propagation layer, convert to UTF-8 bytes at the carrier adapter boundary, and hand byte[] to the Kafka client.
This isn't a quirk of Java's getBytes() — it's the only way Kafka headers work.
What MSK Lambda does
When MSK triggers a Lambda, AWS serializes the Kafka record to JSON. Since header values are byte[] on the wire, AWS encodes them as decimal byte values. However, the exact JSON
shape depends on the Lambda runtime:
"headers": [{"x-datadog-trace-id": [51, 54, 57, ...]}]
decimal strings
"records": {
"topic-0": {
"0": {
"headers": {
"0": {"someOtherHeader": ["70", "114", ...]},
"2": {"x-datadog-trace-id": {"0":"52","1":"54",...}},
"4": {"x-datadog-sampling-priority": ["49"]}
}
}
}
}
What's the difference between the msk_event.json and the newly added
msk_event_with_headers.jsonhere?as Kafka headers. It includes non-Datadog headers at lower indices to verify that the carrier extraction correctly finds Datadog headers regardless of their position. (source: I did a real world example and below is the evidence of testing)