-
-
Notifications
You must be signed in to change notification settings - Fork 16
feat: Add PSR-16 caching support to JsonLoader #1295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
jaapio
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This idea has been on my list for a while. Thanks for this improvement.
I would like to see usage of a psr interface for the cache storage. This could be either psr 6 or 16. This removes the file dependencies from this implementation, and makes it more flexible for different purposes.
We could use symfony cache by default in the cli application. To provide a simple usage for the people using this as a tool. While people using the library can still chose their preferred implementation. Maybe we could even use an in memory adapter by default?
|
you are absolutely right, dunno why I missed using PSR here ... ;-) I tried to keep it as simple/small as possible, without further dependencies. "just-a-cli-tool"-approach. |
|
Updated to use PSR-16 Changes:
Usage: // With Symfony Cache (filesystem)
$pool = new FilesystemAdapter('inventory', 3600, $cacheDir);
$cache = new Psr16Cache($pool);
$loader = new JsonLoader($httpClient, $cache);
// Without caching (network every time)
$loader = new JsonLoader($httpClient);The CLI application can wire up Symfony's FilesystemAdapter by default, while library users can provide any PSR-16 implementation. |
|
Implemented your suggestion to use PSR-16 exclusively (removed the custom Changes:
Multi-tier caching now possible: use Symfony\Component\Cache\Adapter\ArrayAdapter;
use Symfony\Component\Cache\Adapter\ChainAdapter;
use Symfony\Component\Cache\Adapter\FilesystemAdapter;
use Symfony\Component\Cache\Psr16Cache;
// L1: Memory (fast) -> L2: Filesystem (persistent)
$chain = new ChainAdapter([
new ArrayAdapter(),
new FilesystemAdapter('inventory', 3600, $cacheDir),
]);
$loader = new JsonLoader($httpClient, new Psr16Cache($chain));This gives users full flexibility to configure any caching strategy while keeping the default simple (in-memory deduplication). |
1a2c132 to
731319c
Compare
2ee5d29 to
0c34b0c
Compare
| public function __construct( | ||
| private readonly HttpClientInterface $client, | ||
| CacheInterface|null $cache = null, | ||
| private readonly int $cacheTtl = self::DEFAULT_TTL, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should leave this out, and configure the value on the adapter. This class does not have to do anything with the caching,
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
stupid oversight ... :-/
Adds PSR-16 caching to JsonLoader for inventory JSON files. By default, uses an in-memory ArrayAdapter that deduplicates HTTP requests within a process. For persistent caching across CLI invocations, inject any PSR-16 compatible cache implementation. Features: - PSR-16 interface: Use any compatible cache implementation - In-memory default: Automatic request deduplication - Multi-tier support: Use Symfony ChainAdapter for memory + filesystem - Configurable TTL: Default 1 hour, customizable per use case - Hash-based keys: Uses xxh128 for URL to cache key mapping Performance impact: ~53% faster rendering for interlink-heavy documentation by avoiding repeated HTTP requests.
0c34b0c to
859c968
Compare
jaapio
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I notice that the number of PRs and the size of them are taking a lot of time. I do appreciate all the work you spend, but please have a look at the content you are pushing.
I'm trying to read all you comments, and posts but some of them are so obvious AI generated that I'm starting to ignore your messages. I don't think that's worth my time, but also not yours. So I would like to ask you to carefully review yourself what ever is posted. Otherwise I will have to close the PR without comments.
| * By default, uses an in-memory ArrayAdapter for request deduplication. | ||
| * For persistent caching across requests, inject a FilesystemAdapter or RedisAdapter. | ||
| * For multi-tier caching (memory + disk), use Symfony's ChainAdapter. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is so specific that I don't think we should have this in here. I can agree with the description about the default, however that's already written down on the constructor.
| return $cached; | ||
| } | ||
|
|
||
| // Fetch from network |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why those inline comments? Do they add value?
| For the TYPO3 documentation, inventory caching reduced render times by up to 53% | ||
| when referencing the PHP and TYPO3 core inventories. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not a Typo3 project, I don't think this belongs here.
| ======= | ||
| Caching | ||
| ======= | ||
|
|
||
| The guides library supports caching to improve performance when rendering | ||
| documentation repeatedly. This is particularly useful for development workflows | ||
| and CI/CD pipelines where the same documentation is rendered multiple times. | ||
|
|
||
| Inventory Caching | ||
| ================= | ||
|
|
||
| When using intersphinx-style cross-references between documentation projects, | ||
| the guides library fetches inventory files (``objects.inv.json``) from remote URLs. | ||
| These HTTP requests can be cached to avoid repeated network fetches. | ||
|
|
||
| The ``JsonLoader`` uses PSR-16 (Simple Cache) for all caching, defaulting to an | ||
| in-memory ``ArrayAdapter`` for request deduplication within a single process. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding docs for all new features you add is great. But please make sure that the docs actually explain what your PR focuses on. I don't think we should include all kind of symfony examples. I think we can assume that when people want to do something with Cache that they understand the way they can configure the cache adapters of their choice.
It might be better of explain where to find PSR-16 compatible, and what to expect.
Decreasing render time from minutes to seconds isn't worth your time. Ok. That's your decissions, sorry to hear that. I am not doing this for my personal pleasure, I did this because render time time was way too long for my projects. And I wanted to give back. I already invested a lot of work to please you. If you do not like it. So tell form the start. Saves us both a lot of time. Taking you comment seriously, I will stop pushing PRs. Doesn't want ot waste your time.
Yes, of course. Blaming me for this? And do you know why? AI is not biased. It does not write documentation or comments with a bias about who may read it or who may use it. I t does not judge about it's users ir there intends. Feel free to close. I have my fast rendering for local checks, suites me. CU. |
|
I want to be very clear here. From the moment you opened your first PR, I indicated that I wanted to discuss the overall approach before large or complex changes were implemented. That expectation hasn’t changed. This is now the second time that caution around complexity, long-term maintenance, and ecosystem impact is being interpreted as something personal or discouraging. It isn’t. My responsibility as a maintainer is to be careful with changes that affect the core architecture, especially when they introduce new subsystems with far-reaching consequences. I’m glad you’ve found a setup that works for your use case. However, my role is to ensure that changes also work for other users, downstream integrations, and for the maintainers who will be responsible for this code years from now. That responsibility outweighs the fact that something works well in a single environment. You may find this article helpful in understanding the maintainer perspective on large contributions: |
Summary
Adds PSR-16 caching to
JsonLoaderfor inventory JSON files. By default, uses an in-memoryArrayAdapterthat deduplicates HTTP requests within a process. For persistent caching across CLI invocations, inject any PSR-16 compatible cache.Changes
Behavior
$cachevaluenull(default)FilesystemAdapterChainAdapterFeatures
Performance Impact
~53% faster rendering for interlink-heavy documentation by avoiding repeated HTTP requests for inventory files.
Backward Compatibility
This is fully backward compatible - new optional parameter with sensible default.
Usage Examples
Test Plan