feat: add Content-Encoding option for file operations#1125
Open
mishushakov wants to merge 17 commits intomainfrom
Open
feat: add Content-Encoding option for file operations#1125mishushakov wants to merge 17 commits intomainfrom
mishushakov wants to merge 17 commits intomainfrom
Conversation
Add optional `contentEncoding` parameter to file read/write methods in both JS and Python SDKs. When set to 'gzip', the SDK will set the Content-Encoding header on requests. Downloads automatically decompress via native HTTP client support (fetch/httpx). Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 3b0ff29 The changes in this PR will be included in the next version bump. This PR includes changesets to release 2 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
TextIOBase is a subclass of IOBase, so a single IOBase check with an isinstance(raw, str) guard handles both cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b15a8f7115
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
packages/python-sdk/tests/sync/sandbox_sync/files/test_content_encoding.py
Outdated
Show resolved
Hide resolved
read(..., format="bytes") returns bytearray, not bytes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
packages/python-sdk/tests/sync/sandbox_sync/files/test_content_encoding.py
Outdated
Show resolved
Hide resolved
Content-Encoding describes the encoding of the request body, which is absent in GET requests. Accept-Encoding is the correct header to signal that the client wants a compressed response. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Member
Author
|
@jakubno lmk if we should default all upload/download ops to gzip once the infra's rolled out? |
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Encode str to bytes unconditionally before passing to gzip.compress, and restore TextIOBase handling for IO streams. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Contributor
Package ArtifactsBuilt from b872f85. Download artifacts from this workflow run. JS SDK ( npm install ./e2b-2.14.2-mishushakov-content-encoding.0.tgzCLI ( npm install ./e2b-cli-2.8.1-mishushakov-content-encoding.0.tgzPython SDK ( pip install ./e2b-2.15.1+mishushakov.content.encoding-py3-none-any.whl |
Move encoding from base FilesystemRequestOpts to a separate FilesystemEncodingOpts interface, applied only to read/write/writeFiles. Extract FilesystemFormatOpts for the read format option. Export both new interfaces from the package entry point. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
mishushakov
force-pushed
the
mishushakov/content-encoding
branch
from
7a82326 to
caa470b
Compare
mishushakov
force-pushed
the
mishushakov/content-encoding
branch
from
caa470b to
4c543a7
Compare
The backend decompresses the full request body before parsing multipart, so Content-Encoding: gzip must apply to the whole serialized form — not individual file blobs within it. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Send raw file bytes with Content-Type: application/octet-stream per file, with gzip applied conditionally via Content-Encoding header. This removes all multipart/form-data encoding from the write path, eliminating the need for dummy httpx.Request objects and manual boundary construction. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
For IOBase/ReadableStream inputs, data is now streamed directly to the HTTP connection without reading the entire file into memory first. Gzip compression also streams via zlib.compressobj (Python) and CompressionStream (JS) instead of buffering the whole file. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Move upload body preparation logic into shared helpers: - Python: to_upload_body() in e2b.sandbox.filesystem.filesystem - JS: toUploadBody() in utils.ts Remove unused gzipCompress and toBlob from JS utils. Remove duplicated _gzip_stream from async/sync filesystem modules. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Change `encoding="gzip"` / `encoding: 'gzip'` to `gzip=True` / `gzip: true` across all SDK methods (read, write, write_files) and tests. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Waiting on e2b-dev/infra#1811
Summary
Add optional
contentEncoding: 'gzip'parameter to sandbox file read/write operations across JS and Python SDKs. When enabled, uploads are gzip-compressed and downloads request compressed responses. Both HTTP clients (fetch/httpx) automatically decompress responses, so no manual decompression is needed.Changes
contentEncodingtoFilesystemRequestOpts, gzip compression in write ops, header passing in read/writecontent_encodingparameter, gzip compression viagzip.compress(), header passingTesting
Format and lint pass. JS build succeeds. Ready for integration testing with envd backend.
Note
Medium Risk
Changes the HTTP upload/download behavior for sandbox filesystem operations (new headers, optional gzip compression, and multi-file uploads now issue multiple requests instead of one multipart request), which could impact performance or backend compatibility.
Overview
Adds an optional
gzipflag to sandbox filesystemread/writeoperations in both the JS and Python SDKs to request gzip-encoded downloads (Accept-Encoding: gzip) and gzip-compress uploads (Content-Encoding: gzip).Reworks file uploads to send raw
application/octet-streambodies (optionally streaming-compressed) instead of multipart form uploads, and includes new JS/Python tests plus patch version bumps via a changeset.Written by Cursor Bugbot for commit 3b0ff29. This will update automatically on new commits. Configure here.