Skip to content

Recursive s3 cp fails with "Need to rewind the stream <botocore.httpchecksum.AwsChunkedWrapper object at 0x10c8762b0>, but stream is not seekable." #10026

@corneliusroemer

Description

@corneliusroemer

Describe the bug

When doing recursive upload of around 8 files with multiple files of order 100MB I get upload errors:

$ aws s3 cp /Users/cr/Downloads/ \
  s3://nextstrain-ncov-private/gisaid-tars/unprocessed/ \
  --recursive \
  --exclude "*" \
  --include "gisaid_auspice_input_hcov-19_2026_01_27*.tar"

upload failed: Downloads/gisaid_auspice_input_hcov-19_2026_01_27_19 (4).tar to s3://nextstrain-ncov-private/gisaid-tars/unprocessed/gisaid_auspice_input_hcov-19_2026_01_27_19 (4).tar Need to rewind the stream <botocore.httpchecksum.AwsChunkedWrapper object at 0x10c8762b0>, but stream is not seekable.

Profile:

[profile default]
s3 =
  max_concurrent_requests = 200
  max_queue_size = 5000
  multipart_threshold = 4MB
  multipart_chunksize = 4MB

Regression Issue

  • Select this option if this issue appears to be a regression.

Maybe a regression, I didn't see this error until today.

Expected Behavior

No error

Current Behavior

upload failed: Downloads/gisaid_auspice_input_hcov-19_2026_01_27_19 (4).tar to s3://nextstrain-ncov-private/gisaid-tars/unprocessed/gisaid_auspice_input_hcov-19_2026_01_27_19 (4).tar Need to rewind the stream <botocore.httpchecksum.AwsChunkedWrapper object at 0x10c8762b0>, but stream is not seekable.

Reproduction Steps

Can't share the files due to privacy.

Possible Solution

No response

Additional Information/Context

No response

CLI version used

aws-cli/2.33.7 Python/3.13.11 Darwin/25.2.0 source/arm64

Environment details (OS name and version, etc.)

Darwin/25.2.0 source/arm64

Metadata

Metadata

Assignees

Labels

bugThis issue is a bug.p3This is a minor priority issueresponse-requestedWaiting on additional info and feedback. Will move to "closing-soon" in 7 days.s3

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions