Skip to content

feat: extract trace context from MSK events - #830

Open
lucassarcanjo wants to merge 3 commits into
DataDog:mainfrom
lucassarcanjo:feat/msk-trace-extraction
Open

feat: extract trace context from MSK events#830
lucassarcanjo wants to merge 3 commits into
DataDog:mainfrom
lucassarcanjo:feat/msk-trace-extraction

Conversation

@lucassarcanjo

Copy link
Copy Markdown

What does this PR do?

Adds automatic trace context extraction for MSK-triggered Lambdas. Kafka header byte arrays are decoded as UTF-8 and passed to the existing tracer, preserving Datadog and W3C propagation context.

For batches, the Lambda span uses the first record with valid trace context across the topic-partition groups. Malformed headers are skipped, and headers from different records are never combined.

Motivation

Fixes #829. An instrumented producer's trace headers reach the Lambda event, but the existing dispatcher does not extract them, so the Lambda starts a separate trace.

Testing Guidelines

  • yarn test --runInBand: 691 tests and 3 snapshots passed on Node.js 22.
  • yarn lint and formatting checks passed.
  • Local Node.js 24 check with real dd-trace@5.118.0: the Lambda tracing lifecycle preserves the producer parent ID, sampling priority, and 128-bit trace ID with Datadog-only, W3C-only, and combined headers. The same reproduction fails with the original dispatcher.
  • Live AWS/MSK and Docker integration validation are pending; this PR is a draft.

Types of Changes

  • New feature

Check all that apply

  • This PR's description is comprehensive
  • This PR's changes are covered by the automated tests
  • This PR passes the integration tests (ask a Datadog member to run the tests)

Comment thread src/trace/context/extractors/msk.ts
@lucassarcanjo
lucassarcanjo marked this pull request as ready for review September 9, 2026 12:11
@lucassarcanjo
lucassarcanjo requested review from a team as code owners September 9, 2026 12:11
@lucassarcanjo
lucassarcanjo requested a review from lym953 September 9, 2026 12:11

@BridgeAR BridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR! I just left a few suggestions to make the code a bit faster :)

Comment thread src/trace/context/extractors/msk.ts Outdated
Comment thread src/trace/context/extractors/msk.ts Outdated
Comment thread src/trace/context/extractors/msk.ts Outdated
Comment thread src/trace/context/extractors/msk.ts Outdated
- Guard event.records with an early return instead of an empty fallback
- Wrap the record loop in a single try/catch so errors log once
- Return null from getParsedRecordHeaders when nothing decodes, lazily
  creating the headers map
- Drop per-byte validation and rely on Buffer.from

@BridgeAR BridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the quick follow-ups!

Code wise it seems fine to me.

@lucassarcanjo

Copy link
Copy Markdown
Author

Hi @lym953, could you take a look at this PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Built-in trace context extraction for MSK-triggered Lambdas

3 participants