WIP: feat(transport): TUS upload for large attachments by jpnurmi · Pull Request #1545 · getsentry/sentry-native

jpnurmi · 2026-02-27T17:30:45Z

Note

Depends on:

feat(transport): add HTTP retry with exponential backoff #1520

Large attachments (>=100 MB) are stored as attachment-ref envelope items
Database layer stores large attachment files separately in db/attachments/<uuid>/
Transport layer streams each attachment-ref to the server via TUS before sending the envelope

See also:

#skip-changelog (wip)

jpnurmi · 2026-03-02T09:51:11Z

@sentry review

jpnurmi · 2026-03-02T09:51:18Z

@cursor review

src/sentry_envelope.c

src/sentry_database.c

src/transports/sentry_http_transport.c

jpnurmi · 2026-03-02T11:18:47Z

@cursor review

src/transports/sentry_http_transport.c

src/sentry_database.c

src/sentry_envelope.c

jpnurmi · 2026-03-02T15:32:51Z

@cursor review

src/transports/sentry_http_transport.c

cursor · 2026-03-02T15:48:20Z

src/transports/sentry_http_transport_curl.c

+        }
+    }
+    return 0;
+}


Debug function writes directly to stderr bypassing logging

Low Severity

The new debug_function added as a CURLOPT_DEBUGFUNCTION callback writes HTTP traffic details (headers, JSON bodies) directly to stderr via fprintf, bypassing the SDK's SENTRY_DEBUG/SENTRY_WARN logging infrastructure. Given the PR is titled "WIP", this appears to be development debugging code — it uses non-standard => / <= prefixes and selectively prints JSON-looking payloads, which is useful during development but inconsistent with the rest of the codebase.

jpnurmi · 2026-03-03T08:44:58Z

@cursor review

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

✅ Fixed: copy_file_range fallback misses common filesystem errors
- Expanded the copy_file_range fallback condition to also break on EXDEV, EOPNOTSUPP, and EINVAL so the read/write fallback handles these recoverable cases.

Or push these changes by commenting:

@cursor push 76476611c4

Preview (76476611c4)

diff --git a/src/path/sentry_path_unix.c b/src/path/sentry_path_unix.c
--- a/src/path/sentry_path_unix.c
+++ b/src/path/sentry_path_unix.c
@@ -362,7 +362,8 @@
             goto done;
         } else if (errno == EAGAIN || errno == EINTR) {
             continue;
-        } else if (errno == ENOSYS) {
+        } else if (errno == ENOSYS || errno == EXDEV || errno == EOPNOTSUPP
+            || errno == EINVAL) {
             break;
         } else {
             rv = 1;

_{This Bugbot Autofix run was free. To enable autofix for future PRs, go to the Cursor dashboard.}

src/path/sentry_path_unix.c

jpnurmi · 2026-03-03T12:58:20Z

@cursor review

src/sentry_database.c

jpnurmi · 2026-03-05T09:22:29Z

@cursor review

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

src/sentry_envelope.c

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add body_path to prepared HTTP requests for streaming file uploads from disk. Implement TUS protocol request formatting, upload URL construction, response cleanup helper, Location header parsing, and curl debug logging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Cache attachments >= 100 MB to disk at envelope preparation time, then upload them via TUS protocol before sending the envelope. The envelope carries lightweight attachment-ref items that point to the uploaded location instead of embedding the full payload. Key changes: - Add attachment helpers (get_size, get_filename, type_to_string) - Add envelope attachment-ref item type with location/content_type - Cache large attachments to <db>/cache/<uuid>/ with refs.json metadata - TUS upload with 404 fallback, retry integration, cache cleanup - Skip large attachments in envelope serialization (size threshold) - Fix envelope deserialization deadlock on bgworker thread Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…hment() Restore the helper that was dropped during the tus-separate refactor, keeping the commented-out testing hook for small .dmp files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Switch from TUS "Creation With Upload" (single POST with body) to TUS "Creation" (POST without body) + upload (PATCH with body). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This temporarily reverts commit 5a4097f1db4ae749370b0d82a2dc447ce5e8fc78 until it's supported on the server side.

The cache directory will contain actual attachment files alongside this metadata file, so use a namespaced name to prevent collisions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

…od at transport time Replace sentry__is_large_attachment(path, size) with sentry__attachment_is_external(att) which returns true for MINIDUMP type or size >= SENTRY_LARGE_ATTACHMENT_SIZE. Minidumps are always cached as external files; other large attachments continue to be cached by size. Rename sentry__cache_large_attachments to sentry__cache_external_attachments. At transport time, resolve_and_send_external_attachments checks each cached attachment: large files with TUS support go via TUS upload, smaller files are inlined into the envelope. For raw envelopes (read from disk on restart), sentry__envelope_append_raw_attachment builds the item bytes and appends them directly to the raw payload buffer. External attachment cleanup is deferred until after successful send so files survive for retry on network failure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This reverts commit 4f3fa6d.

…pty POST - Resolve relative Location header against DSN origin (sentry__dsn_resolve_url) - Add x-sentry-auth header to TUS upload PATCH request - Use CURLOPT_CUSTOMREQUEST for bodyless POST to avoid curl's default Content-Type: application/x-www-form-urlencoded Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Add append_raw_attachment_ref() to append attachment-ref items to raw (serialized) envelopes by directly manipulating the payload buffer. sentry__envelope_add_attachment_ref() dispatches to this for raw envelopes, so callers don't need to know about the envelope format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Instead of sending attachment-refs in a separate envelope, append them to the event envelope so the server can associate them in one request. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Show upload progress (MB uploaded, percentage, speed) via the progress callback. Suppress per-chunk "=> Send" noise for non-JSON data since the progress callback covers that. Keep recv logging for all responses. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

jpnurmi force-pushed the jpnurmi/feat/tus branch 2 times, most recently from da2ce2d to 0946169 Compare March 2, 2026 09:32

cursor bot reviewed Mar 2, 2026

View reviewed changes

src/sentry_envelope.c Outdated Show resolved Hide resolved

src/sentry_database.c Show resolved Hide resolved

src/transports/sentry_http_transport.c Outdated Show resolved Hide resolved

jpnurmi mentioned this pull request Mar 2, 2026

ref: add SENTRY_MAKE_0 macro for zero-initialized allocation #1546

Draft

cursor bot reviewed Mar 2, 2026

View reviewed changes

src/transports/sentry_http_transport.c Show resolved Hide resolved

src/sentry_database.c Outdated Show resolved Hide resolved

src/sentry_envelope.c Outdated Show resolved Hide resolved

cursor bot reviewed Mar 2, 2026

View reviewed changes

cursor bot reviewed Mar 3, 2026

View reviewed changes

src/path/sentry_path_unix.c Show resolved Hide resolved

jpnurmi force-pushed the jpnurmi/feat/tus branch from a69528d to 44d6ecc Compare March 3, 2026 09:27

cursor bot reviewed Mar 3, 2026

View reviewed changes

src/sentry_database.c Outdated Show resolved Hide resolved

jpnurmi force-pushed the jpnurmi/feat/tus branch from 3bc1ee2 to 37004bf Compare March 5, 2026 08:03

cursor bot reviewed Mar 5, 2026

View reviewed changes

src/sentry_envelope.c Outdated Show resolved Hide resolved

jpnurmi force-pushed the jpnurmi/feat/tus branch 3 times, most recently from eb0ead2 to 4ad89c0 Compare March 16, 2026 13:03

jpnurmi force-pushed the jpnurmi/feat/http-retry branch from b133fd6 to 93db900 Compare March 17, 2026 16:15

jpnurmi force-pushed the jpnurmi/feat/tus branch from 4ad89c0 to 417fe23 Compare March 19, 2026 10:29

jpnurmi and others added 7 commits March 19, 2026 17:00

fix(curl): trim HTTP response header values

b16281e

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(path): add sentry__path_copy with unit test

f18a4a3

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

feat(path): add sentry__path_get_dir_size helper

e0894ae

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

refactor: centralize large attachment check in sentry__is_large_attac…

3928f05

…hment() Restore the helper that was dropped during the tus-separate refactor, keeping the commented-out testing hook for small .dmp files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

feat(tus): split into separate creation and upload requests

a9e5b92

Switch from TUS "Creation With Upload" (single POST with body) to TUS "Creation" (POST without body) + upload (PATCH with body). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

jpnurmi and others added 6 commits March 19, 2026 17:00

Revert "feat(tus): split into separate creation and upload requests"

c5ea059

This temporarily reverts commit 5a4097f1db4ae749370b0d82a2dc447ce5e8fc78 until it's supported on the server side.

rename refs.json to __sentry-attachments.json to avoid conflicts

0b609aa

The cache directory will contain actual attachment files alongside this metadata file, so use a namespaced name to prevent collisions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Reapply "feat(tus): split into separate creation and upload requests"

0f27df0

This reverts commit 4f3fa6d.

jpnurmi force-pushed the jpnurmi/feat/tus branch from 417fe23 to 6d860cd Compare March 19, 2026 16:29

jpnurmi and others added 2 commits March 19, 2026 17:32

refactor(tus): append attachment-refs to event envelope

8e3b656

Instead of sending attachment-refs in a separate envelope, append them to the event envelope so the server can associate them in one request. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

jpnurmi force-pushed the jpnurmi/feat/tus branch from 6d860cd to a07948d Compare March 19, 2026 16:33

Uh oh!

Conversation

jpnurmi commented Feb 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jpnurmi commented Mar 2, 2026

Uh oh!

jpnurmi commented Mar 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jpnurmi commented Mar 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jpnurmi commented Mar 2, 2026

Uh oh!

Uh oh!

cursor bot Mar 2, 2026

Choose a reason for hiding this comment

Debug function writes directly to stderr bypassing logging

Uh oh!

jpnurmi commented Mar 3, 2026

Uh oh!

cursor bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jpnurmi commented Mar 3, 2026

Uh oh!

Uh oh!

jpnurmi commented Mar 5, 2026

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jpnurmi commented Feb 27, 2026 •

edited

Loading

cursor bot left a comment •

edited

Loading