Open
Conversation
Implements a new cachier backend backed by AWS S3 (or any S3-compatible service such as MinIO or localstack). Key changes: - src/cachier/cores/s3.py: new _S3Core implementing _BaseCore; stores one pickled CacheEntry per key under <prefix>/<func_str>/<key>.pkl; supports direct boto3 client, callable factory, or auto-created client via region / endpoint_url / Config options; syncs async callers via thread delegation from _BaseCore defaults (boto3 is sync-only). - src/cachier/_types.py: add S3Client type alias; extend Backend literal with "s3". - src/cachier/core.py: wire s3_bucket, s3_prefix, s3_client, s3_client_factory, s3_region, s3_endpoint_url, s3_config decorator parameters; import and instantiate _S3Core. - pyproject.toml: add "s3" pytest marker; add [project.optional-dependencies] with per-backend extras (mongo, redis, sql, s3, all). - tests/s3_tests/: 18 tests covering basic caching, skip/overwrite, stale_after, next_time, allow_none, entry_size_limit, clear_cache, clear_being_calculated, delete_stale_entries, client factory, thread safety, and error handling. Uses moto[s3] for offline testing with no real AWS account needed. - tests/requirements_s3.txt: boto3 + moto[s3] test deps. - examples/s3_example.py: runnable demo for basic caching, stale_after, client factory, and cache management. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #338 +/- ##
===========================================
- Coverage 100.00% 90.57% -9.43%
===========================================
Files 11 12 +1
Lines 1477 1666 +189
Branches 185 209 +24
===========================================
+ Hits 1477 1509 +32
- Misses 0 156 +156
- Partials 0 1 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #41.
Adds a new
s3backend to cachier, allowing function results to be persistently cached in AWS S3 (or any S3-compatible service: MinIO, localstack, etc.).@cachier(backend="s3", s3_bucket="my-bucket")- same decorator API as all other coress3_client), a callable factory (s3_client_factory), or let the core auto-create one froms3_region/s3_endpoint_url/s3_configCacheEntryper cache key, stored at<s3_prefix>/<func_str>/<cache_key>.pklasyncio.to_threaddelegation (boto3 is sync-only)Files changed
src/cachier/cores/s3.py_S3Coreclass implementing the full_BaseCoreinterfacesrc/cachier/_types.pyS3Clienttype alias; extendBackendliteral with"s3"src/cachier/core.py_S3Corepyproject.tomls3pytest marker; add[project.optional-dependencies]extrastests/s3_tests/moto[s3](no real AWS account required)tests/requirements_s3.txtboto3+moto[s3]test depsexamples/s3_example.pyTest plan
pytest -m s3- all 18 S3 tests pass (uses moto for offline mocking)pytest -m "not (mongo or redis or sql or s3)"- 219 existing tests pass, 0 regressionsruff check- no linting errors on changed filesmypy src/cachier/- only pre-existing SQL overload error; boto3 stubs handled with# type: ignore[import-untyped]Open questions (from issue)
The following points from the original issue are left for follow-up if desired:
s3_client_factory(currently sync-only, matching boto3's nature)🤖 Generated with Claude Code