Skip to content

feat(trace-utils)!: change buffer implementation#2055

Open
yannham wants to merge 6 commits into
yannham/change-buffer-foundationfrom
yannham/change-buffer-state
Open

feat(trace-utils)!: change buffer implementation#2055
yannham wants to merge 6 commits into
yannham/change-buffer-foundationfrom
yannham/change-buffer-state

Conversation

@yannham
Copy link
Copy Markdown
Contributor

@yannham yannham commented May 28, 2026

What does this PR do?

This PR backports the change buffer implementation required for dd-trace-js adoption of native spans. Follow-up of #2046.

The change buffer is a contiguous region of memory where span operations are encoded from the JS runtime, similar to a byte code. The implementation decode all those span events and reconstruct the corresponding span objects.

Motivation

Required to land native spans dd-trace-js.

How to test the change?

tests included.

yannham and others added 4 commits May 28, 2026 19:29
Add the main ChangeBufferState<T> struct and its full implementation:
span creation/pooling, change buffer interpretation (opcodes), deferred
tag resolution, span header materialization, flush logic, and string
table management. This is the core state machine that processes batched
span mutations from JS/WASM via a shared memory buffer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@yannham yannham requested review from a team as code owners May 28, 2026 17:35
@github-actions
Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 605 documentation warning(s) found

📦 libdd-trace-utils - 605 warning(s)


Updated: 2026-05-28 17:36:56 UTC | Commit: df918d8 | missing-docs job results

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9f874ddf2f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +501 to +505
let num: u32 = self.change_buffer.read_unchecked(index);
self.string_table
.get_unchecked(num as usize)
.clone()
.unwrap_unchecked()
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep FFI buffer decoding checked

When a cached non-create operation such as SetName or SetServiceName is decoded from the shared FFI buffer, the caller reaches this helper without validating either that the string id is still present in string_table or that the argument bytes are in bounds. An out-of-sync/evicted string id that used to produce ChangeBufferError::StringNotFound now hits unwrap_unchecked(), which is undefined behavior across the FFI-facing change-buffer path rather than a recoverable error.

Useful? React with 👍 / 👎.

@github-actions
Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

Summary by Rule

Rule Base Branch PR Branch Change
unwrap_used 0 1 ⚠️ +1 (N/A)
Total 0 1 ⚠️ +1 (N/A)

Annotation Counts by File

File Base Branch PR Branch Change
libdd-trace-utils/src/change_buffer/mod.rs 0 1 ⚠️ +1 (N/A)

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 21 21 No change (0%)
datadog-live-debugger 6 6 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-remote-config 3 3 No change (0%)
datadog-sidecar 57 57 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 3 3 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 12 13 ⚠️ +1 (+8.3%)
Total 195 196 ⚠️ +1 (+0.5%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@github-actions
Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 4 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-trace-utils - 4 error(s)

Show output
error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:177:1
    │
177 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by @lopopolo) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v4.1.0
      │   ├── libdd-capabilities-impl v2.0.0
      │   │   └── libdd-trace-utils v5.0.0
      │   │       └── (dev) libdd-trace-utils v5.0.0 (*)
      │   └── libdd-trace-utils v5.0.0 (*)
      ├── (dev) libdd-trace-normalization v2.0.0
      │   └── libdd-trace-utils v5.0.0 (*)
      ├── libdd-trace-utils v5.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v5.0.0 (*)

error[vulnerability]: Name constraints for URI names were incorrectly accepted
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:200:1
    │
200 │ rustls-webpki 0.103.10 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0098
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0098
    ├ Name constraints for URI names were ignored and therefore accepted.
      
      Note this library does not provide an API for asserting URI names, and URI name constraints are otherwise not implemented.  URI name constraints are now rejected unconditionally.
      
      Since name constraints are restrictions on otherwise properly-issued certificates, this bug is reachable only after signature verification and requires misissuance to exploit.
      
      This vulnerability is identified as [GHSA-965h-392x-2mh5](https://github.com/rustls/webpki/security/advisories/GHSA-965h-392x-2mh5). Thank you to @1seal for the report.
    ├ Solution: Upgrade to >=0.103.12, <0.104.0-alpha.1 OR >=0.104.0-alpha.6 (try `cargo update -p rustls-webpki`)
    ├ rustls-webpki v0.103.10
      └── rustls v0.23.37
          ├── hyper-rustls v0.27.7
          │   └── libdd-common v4.1.0
          │       ├── libdd-capabilities-impl v2.0.0
          │       │   └── libdd-trace-utils v5.0.0
          │       │       └── (dev) libdd-trace-utils v5.0.0 (*)
          │       └── libdd-trace-utils v5.0.0 (*)
          ├── libdd-common v4.1.0 (*)
          └── tokio-rustls v0.26.0
              ├── hyper-rustls v0.27.7 (*)
              └── libdd-common v4.1.0 (*)

error[vulnerability]: Name constraints were accepted for certificates asserting a wildcard name
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:200:1
    │
200 │ rustls-webpki 0.103.10 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0099
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0099
    ├ Permitted subtree name constraints for DNS names were accepted for certificates asserting a wildcard name.
      
      This was incorrect because, given a name constraint of `accept.example.com`, `*.example.com` could feasibly allow a name of `reject.example.com` which is outside the constraint.
      This is very similar to [CVE-2025-61727](https://go.dev/issue/76442).
      
      Since name constraints are restrictions on otherwise properly-issued certificates, this bug is reachable only after signature verification and requires misissuance to exploit.
      
      This vulnerability is identified as [GHSA-xgp8-3hg3-c2mh](https://github.com/rustls/webpki/security/advisories/GHSA-xgp8-3hg3-c2mh). Thank you to @1seal for the report.
    ├ Solution: Upgrade to >=0.103.12, <0.104.0-alpha.1 OR >=0.104.0-alpha.6 (try `cargo update -p rustls-webpki`)
    ├ rustls-webpki v0.103.10
      └── rustls v0.23.37
          ├── hyper-rustls v0.27.7
          │   └── libdd-common v4.1.0
          │       ├── libdd-capabilities-impl v2.0.0
          │       │   └── libdd-trace-utils v5.0.0
          │       │       └── (dev) libdd-trace-utils v5.0.0 (*)
          │       └── libdd-trace-utils v5.0.0 (*)
          ├── libdd-common v4.1.0 (*)
          └── tokio-rustls v0.26.0
              ├── hyper-rustls v0.27.7 (*)
              └── libdd-common v4.1.0 (*)

error[vulnerability]: Reachable panic in certificate revocation list parsing
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:200:1
    │
200 │ rustls-webpki 0.103.10 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0104
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0104
    ├ A panic was reachable when parsing certificate revocation lists via [`BorrowedCertRevocationList::from_der`]
      or [`OwnedCertRevocationList::from_der`].  This was the result of mishandling a syntactically valid empty
      `BIT STRING` appearing in the `onlySomeReasons` element of a `IssuingDistributionPoint` CRL extension.
      
      This panic is reachable prior to a CRL's signature being verified.
      
      Applications that do not use CRLs are not affected.
      
      Thank you to @tynus3 for the report.
    ├ Solution: Upgrade to >=0.103.13, <0.104.0-alpha.1 OR >=0.104.0-alpha.7 (try `cargo update -p rustls-webpki`)
    ├ rustls-webpki v0.103.10
      └── rustls v0.23.37
          ├── hyper-rustls v0.27.7
          │   └── libdd-common v4.1.0
          │       ├── libdd-capabilities-impl v2.0.0
          │       │   └── libdd-trace-utils v5.0.0
          │       │       └── (dev) libdd-trace-utils v5.0.0 (*)
          │       └── libdd-trace-utils v5.0.0 (*)
          ├── libdd-common v4.1.0 (*)
          └── tokio-rustls v0.26.0
              ├── hyper-rustls v0.27.7 (*)
              └── libdd-common v4.1.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-05-28 17:38:57 UTC | Commit: df918d8 | dependency-check job results

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 578 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.47%. Comparing base (8431f28) to head (9f874dd).

Additional details and impacted files
@@                         Coverage Diff                          @@
##           yannham/change-buffer-foundation    #2055      +/-   ##
====================================================================
- Coverage                             73.03%   72.47%   -0.56%     
====================================================================
  Files                                   465      465              
  Lines                                 76881    77436     +555     
====================================================================
- Hits                                  56147    56122      -25     
- Misses                                20734    21314     +580     
Components Coverage Δ
libdd-crashtracker 65.45% <ø> (+0.01%) ⬆️
libdd-crashtracker-ffi 38.78% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 85.52% <ø> (ø)
libdd-data-pipeline-ffi 75.28% <ø> (ø)
libdd-common 79.89% <ø> (ø)
libdd-common-ffi 74.41% <ø> (ø)
libdd-telemetry 73.37% <ø> (-0.03%) ⬇️
libdd-telemetry-ffi 31.36% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 74.75% <ø> (-1.47%) ⬇️
libdd-profiling 81.69% <ø> (-0.02%) ⬇️
libdd-profiling-ffi 64.79% <ø> (ø)
libdd-sampling 97.46% <ø> (ø)
datadog-sidecar 30.22% <ø> (ø)
datdog-sidecar-ffi 15.32% <ø> (ø)
spawn-worker 48.86% <ø> (ø)
libdd-tinybytes 93.80% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 87.30% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 82.89% <0.00%> (-5.41%) ⬇️
libdd-tracer-flare 86.88% <ø> (ø)
libdd-log 74.83% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-datadog-prod-us1-2

This comment has been minimized.

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 28, 2026

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 82.94 MB 83.14 MB +.24% (+206.37 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.63 MB 7.63 MB +0% (+24 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 94.02 MB 94.23 MB +.21% (+210.75 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.25 MB 10.26 MB +.04% (+4.92 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.54 MB 24.67 MB +.55% (+140.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 83.96 KB 83.96 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 178.23 MB 179.34 MB +.62% (+1.10 MB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 915.25 MB 917.61 MB +.25% (+2.36 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.03 MB 8.03 MB +.05% (+4.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 83.96 KB 83.96 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 23.77 MB 23.80 MB +.13% (+32.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 47.43 MB 47.50 MB +.16% (+80.93 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.27 MB 21.37 MB +.47% (+103.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 85.29 KB 85.29 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 182.23 MB 183.28 MB +.57% (+1.04 MB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 908.37 MB 910.73 MB +.25% (+2.35 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.20 MB 6.20 MB +.06% (+4.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 85.29 KB 85.29 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 25.48 MB 25.52 MB +.15% (+40.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 45.07 MB 45.15 MB +.17% (+80.50 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 73.95 MB 74.12 MB +.23% (+176.21 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.52 MB 8.54 MB +.13% (+12.03 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 89.36 MB 89.54 MB +.19% (+180.32 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.36 MB 10.38 MB +.12% (+12.90 KB) 🔍

@yannham yannham requested a review from a team as a code owner June 2, 2026 08:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants