Skip to content

feat(server): Add multipart upload endpoints#463

Merged
lcian merged 22 commits into
mainfrom
lcian/feat/multipart-upload-endpoints
May 29, 2026
Merged

feat(server): Add multipart upload endpoints#463
lcian merged 22 commits into
mainfrom
lcian/feat/multipart-upload-endpoints

Conversation

@lcian
Copy link
Copy Markdown
Member

@lcian lcian commented May 7, 2026

⚠️ Stacked on #458

Final part of the Multipart uploads implementation server-side, this implements the endpoints on top of the Service.

Close FS-341
Close FS-359

@lcian lcian changed the base branch from main to lcian/feat/multipart-upload-tiered May 7, 2026 12:31
@lcian

This comment was marked as outdated.

@lcian

This comment was marked as outdated.

Comment thread objectstore-server/src/endpoints/multipart.rs
Comment thread objectstore-server/src/endpoints/multipart.rs
@lcian lcian changed the title lcian/feat/multipart upload endpoints feat(server): Add multipart upload endpoints May 7, 2026
@linear-code

This comment was marked as off-topic.

@lcian

This comment was marked as outdated.

@lcian

This comment was marked as outdated.

Comment thread objectstore-server/src/endpoints/multipart.rs
@lcian

This comment was marked as outdated.

cursor[bot]

This comment was marked as outdated.

@lcian

This comment was marked as resolved.

@lcian

This comment was marked as resolved.

cursor[bot]

This comment was marked as spam.

Comment thread objectstore-service/src/backend/common.rs Outdated
headers: HeaderMap,
) -> ApiResult<Response> {
let mut metadata = Metadata::from_headers(&headers, "").map_err(ServiceError::from)?;
// TODO: Do this in `complete` instead, when we have a Service API to mutate metadata.
Copy link
Copy Markdown
Member Author

@lcian lcian May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is easy to do but we would need a new method on Backend, plus all the implementations and plumbing through Service, filed a follow-up: https://linear.app/getsentry/issue/FS-356/set-correct-metadatatime-created-for-multipart-uploads.

Or, we can just accept and document that time_created refers to the time the upload was initiated, but that sounds wrong, it should probably be the time the logical object is created, which is the time the multipart upload is completed.

@lcian lcian marked this pull request as ready for review May 8, 2026 14:08
@lcian lcian requested a review from a team as a code owner May 8, 2026 14:08
@lcian lcian requested a review from matt-codecov May 8, 2026 17:19
Comment thread objectstore-server/src/auth/service.rs Outdated
@lcian lcian force-pushed the lcian/feat/multipart-upload-tiered branch from 409be20 to d6eb979 Compare May 12, 2026 11:32
@lcian lcian requested review from matt-codecov and removed request for matt-codecov May 15, 2026 16:37
@lcian lcian force-pushed the lcian/feat/multipart-upload-endpoints branch from 5babbfb to c33ba6f Compare May 15, 2026 16:57
@codecov

This comment was marked as spam.

Comment thread objectstore-server/src/endpoints/multipart.rs
Copy link
Copy Markdown
Contributor

@matt-codecov matt-codecov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

accept w comments

Comment thread objectstore-server/src/auth/service.rs Outdated
Comment thread objectstore-server/src/endpoints/common.rs
Comment thread objectstore-service/src/error.rs Outdated
Comment thread objectstore-service/src/service.rs Outdated
Comment thread objectstore-server/src/endpoints/multipart.rs Outdated
Comment thread objectstore-server/src/endpoints/multipart.rs
Comment thread objectstore-server/src/endpoints/multipart.rs Outdated
cursor[bot]

This comment was marked as outdated.

@lcian lcian force-pushed the lcian/feat/multipart-upload-endpoints branch from ab5d578 to a2496b3 Compare May 29, 2026 08:20
Comment thread objectstore-server/src/endpoints/multipart.rs
Comment thread objectstore-service/src/backend/local_fs.rs
Validate that upload_id contains only normal path components before
joining it into the multipart directory path. Rejects segments
containing .., /, or other traversal patterns.
Comment thread objectstore-service/src/backend/local_fs.rs Outdated
Replace `type UploadId = String` with a newtype struct that validates
on construction: non-empty and no path-traversal components (.., /, \).

This moves the path-safety guarantee from LocalFsBackend's safe_join
helper to the type itself, protecting all backends. Invalid upload_id
values in query params are now rejected at deserialization time (400).
Comment thread objectstore-service/src/backend/gcs.rs
@lcian lcian merged commit b84e11f into main May 29, 2026
25 checks passed
@lcian lcian deleted the lcian/feat/multipart-upload-endpoints branch May 29, 2026 09:28
lcian added a commit that referenced this pull request May 29, 2026
⚠️ Stacked on #463

Implements a multipart upload API for the Rust client.
Instead of going for a low-level API where calls match exactly HTTP
calls as I've planned initially, I've decided to abstract some things,
for example, initiating a multipart upload returns a `MultipartUpload`
that represents it, much like the existing `Session` we have.
Please look at the README to see the intended usage.

Also makes a couple changes that aren't strictly related to the
multipart upload impl:
- Moves multipart upload-related types from `objectstore-server` to
`objectstore-types`, to make them usable from the client too, given that
they both deal with the wire format.
- Makes the server return a vec of completed parts, each with their own
`part_number` when calling `ListParts`, as opposed to a `BTreeMap`.
- Makes `Usecase::compression` take an `Option<Compression>`. This is a
breaking change that makes it possible to disable default-compression on
a usecase, which wasn't possible before.

Close FS-357

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants