fix: no retry when remote table metadata not found#1386
Conversation
Treat object-store metadata 404s as permanent so download skips the affected table immediately instead of exhausting the retry backoff.
…ermes-auto/clickhouse-backup-1379-skip-missing-metadata
…ermes-auto/clickhouse-backup-1379-skip-missing-metadata # Conflicts: # pkg/backup/download.go # pkg/backup/download_test.go
A missing table .json on remote storage is a permanent broken-backup condition, not a transient one. Return a clear error instead of silently skipping the table (a silent skip restores fewer tables than the backup claims, with no signal to the operator). Detection now covers all backends' not-found phrasings (S3 NoSuchKey, GCS doesn't exist/404, Azure BlobNotFound, FTP "No such file or directory", SFTP "file does not exist") so the 404 breaks out of the retry loop immediately instead of burning the ~35s exponential backoff. Add per-backend integration coverage (S3/SFTP/FTP/GCS/GCS-emulator/ AZBLOB/COS) asserting download fails fast with a not-found error and without retrying. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Changed the missing- Rationale: a missing table Kept the no-retry fix (the whole point of #1379): the not-found 404 still breaks out of the retry loop immediately instead of burning the ~35s exponential backoff. Detection now covers all backends' phrasings (S3 Added per-backend integration tests ( Note: the optional |
Summary
.sqlhandling for incremental/embedded metadata and return non-not-found errors directlyTests
go test ./pkg/backup -run TestIsRemoteMetadataNotFound -count=1go test ./pkg/backup -count=1Closes #1379