Skip to content

Conversation

@CybotTM
Copy link
Contributor

@CybotTM CybotTM commented Jan 23, 2026

Summary

Adds PSR-16 caching to JsonLoader for inventory JSON files. By default, uses an in-memory ArrayAdapter that deduplicates HTTP requests within a process. For persistent caching across CLI invocations, inject any PSR-16 compatible cache.

Changes

// Constructor signature
public function __construct(
    private readonly HttpClientInterface $client,
    CacheInterface|null $cache = null,  // PSR-16 cache (default: ArrayAdapter)
)

Behavior

$cache value Behavior
null (default) In-memory ArrayAdapter - deduplicates within same process
FilesystemAdapter Persistent caching across CLI invocations
ChainAdapter Multi-tier caching (memory + filesystem)

Features

  • PSR-16 interface: Use any compatible cache implementation
  • In-memory default: Automatic request deduplication within same process
  • Multi-tier support: Use Symfony ChainAdapter for L1 memory + L2 filesystem
  • Hash-based keys: Uses xxh128 for URL → cache key mapping

Performance Impact

~53% faster rendering for interlink-heavy documentation by avoiding repeated HTTP requests for inventory files.

Backward Compatibility

This is fully backward compatible - new optional parameter with sensible default.

Usage Examples

// Default - in-memory deduplication
$loader = new JsonLoader($httpClient);

// Persistent filesystem cache (TTL configured on adapter)
$pool = new FilesystemAdapter('inventory', 3600, '/tmp/cache');
$loader = new JsonLoader($httpClient, new Psr16Cache($pool));

// Multi-tier: memory + filesystem
$chain = new ChainAdapter([
    new ArrayAdapter(),
    new FilesystemAdapter('inventory', 3600, '/tmp/cache'),
]);
$loader = new JsonLoader($httpClient, new Psr16Cache($chain));

Test Plan

  • All existing tests pass
  • New tests for cache hit/miss, deduplication, deterministic keys
  • PHPStan passes
  • CI validation

Copy link
Member

@jaapio jaapio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This idea has been on my list for a while. Thanks for this improvement.

I would like to see usage of a psr interface for the cache storage. This could be either psr 6 or 16. This removes the file dependencies from this implementation, and makes it more flexible for different purposes.

We could use symfony cache by default in the cli application. To provide a simple usage for the people using this as a tool. While people using the library can still chose their preferred implementation. Maybe we could even use an in memory adapter by default?

@CybotTM CybotTM marked this pull request as draft January 26, 2026 16:47
@CybotTM
Copy link
Contributor Author

CybotTM commented Jan 26, 2026

you are absolutely right, dunno why I missed using PSR here ... ;-)

I tried to keep it as simple/small as possible, without further dependencies. "just-a-cli-tool"-approach.

@CybotTM
Copy link
Contributor Author

CybotTM commented Jan 26, 2026

Updated to use PSR-16 CacheInterface as suggested:

Changes:

  • Added psr/simple-cache ^3.0 dependency
  • JsonLoader constructor now accepts CacheInterface|null $cache
  • Optional int $cacheTtl parameter (default: 3600s)
  • In-memory deduplication always active regardless of PSR-16 cache
  • Updated documentation with examples for different backends (Filesystem, Redis, In-Memory)
  • Tests now use mock CacheInterface

Usage:

// With Symfony Cache (filesystem)
$pool = new FilesystemAdapter('inventory', 3600, $cacheDir);
$cache = new Psr16Cache($pool);
$loader = new JsonLoader($httpClient, $cache);

// Without caching (network every time)
$loader = new JsonLoader($httpClient);

The CLI application can wire up Symfony's FilesystemAdapter by default, while library users can provide any PSR-16 implementation.

@CybotTM
Copy link
Contributor Author

CybotTM commented Jan 26, 2026

Implemented your suggestion to use PSR-16 exclusively (removed the custom $memoryCache array):

Changes:

  • Default is now ArrayAdapter (in-memory) when no cache provided
  • Removed custom private array $memoryCache in favor of PSR-16
  • Added symfony/cache dependency for the default adapter

Multi-tier caching now possible:

use Symfony\Component\Cache\Adapter\ArrayAdapter;
use Symfony\Component\Cache\Adapter\ChainAdapter;
use Symfony\Component\Cache\Adapter\FilesystemAdapter;
use Symfony\Component\Cache\Psr16Cache;

// L1: Memory (fast) -> L2: Filesystem (persistent)
$chain = new ChainAdapter([
    new ArrayAdapter(),
    new FilesystemAdapter('inventory', 3600, $cacheDir),
]);

$loader = new JsonLoader($httpClient, new Psr16Cache($chain));

This gives users full flexibility to configure any caching strategy while keeping the default simple (in-memory deduplication).

@CybotTM CybotTM force-pushed the perf/inventory-caching branch from 1a2c132 to 731319c Compare January 26, 2026 18:55
@CybotTM CybotTM changed the title feat: Add optional caching support to JsonLoader feat: Add PSR-16 caching support to JsonLoader Jan 26, 2026
@CybotTM CybotTM force-pushed the perf/inventory-caching branch 2 times, most recently from 2ee5d29 to 0c34b0c Compare January 26, 2026 19:16
@CybotTM CybotTM marked this pull request as ready for review January 26, 2026 19:16
public function __construct(
private readonly HttpClientInterface $client,
CacheInterface|null $cache = null,
private readonly int $cacheTtl = self::DEFAULT_TTL,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should leave this out, and configure the value on the adapter. This class does not have to do anything with the caching,

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stupid oversight ... :-/

Adds PSR-16 caching to JsonLoader for inventory JSON files. By default,
uses an in-memory ArrayAdapter that deduplicates HTTP requests within
a process. For persistent caching across CLI invocations, inject any
PSR-16 compatible cache implementation.

Features:
- PSR-16 interface: Use any compatible cache implementation
- In-memory default: Automatic request deduplication
- Multi-tier support: Use Symfony ChainAdapter for memory + filesystem
- Configurable TTL: Default 1 hour, customizable per use case
- Hash-based keys: Uses xxh128 for URL to cache key mapping

Performance impact: ~53% faster rendering for interlink-heavy
documentation by avoiding repeated HTTP requests.
@CybotTM CybotTM force-pushed the perf/inventory-caching branch from 0c34b0c to 859c968 Compare January 26, 2026 20:25
Copy link
Member

@jaapio jaapio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I notice that the number of PRs and the size of them are taking a lot of time. I do appreciate all the work you spend, but please have a look at the content you are pushing.

I'm trying to read all you comments, and posts but some of them are so obvious AI generated that I'm starting to ignore your messages. I don't think that's worth my time, but also not yours. So I would like to ask you to carefully review yourself what ever is posted. Otherwise I will have to close the PR without comments.

Comment on lines +32 to +34
* By default, uses an in-memory ArrayAdapter for request deduplication.
* For persistent caching across requests, inject a FilesystemAdapter or RedisAdapter.
* For multi-tier caching (memory + disk), use Symfony's ChainAdapter.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is so specific that I don't think we should have this in here. I can agree with the description about the default, however that's already written down on the constructor.

return $cached;
}

// Fetch from network
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why those inline comments? Do they add value?

Comment on lines +161 to +155
For the TYPO3 documentation, inventory caching reduced render times by up to 53%
when referencing the PHP and TYPO3 core inventories.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a Typo3 project, I don't think this belongs here.

Comment on lines +1 to +17
=======
Caching
=======

The guides library supports caching to improve performance when rendering
documentation repeatedly. This is particularly useful for development workflows
and CI/CD pipelines where the same documentation is rendered multiple times.

Inventory Caching
=================

When using intersphinx-style cross-references between documentation projects,
the guides library fetches inventory files (``objects.inv.json``) from remote URLs.
These HTTP requests can be cached to avoid repeated network fetches.

The ``JsonLoader`` uses PSR-16 (Simple Cache) for all caching, defaulting to an
in-memory ``ArrayAdapter`` for request deduplication within a single process.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding docs for all new features you add is great. But please make sure that the docs actually explain what your PR focuses on. I don't think we should include all kind of symfony examples. I think we can assume that when people want to do something with Cache that they understand the way they can configure the cache adapters of their choice.

It might be better of explain where to find PSR-16 compatible, and what to expect.

@CybotTM
Copy link
Contributor Author

CybotTM commented Jan 26, 2026

I don't think that's worth my time,

Decreasing render time from minutes to seconds isn't worth your time. Ok. That's your decissions, sorry to hear that. I am not doing this for my personal pleasure, I did this because render time time was way too long for my projects. And I wanted to give back. I already invested a lot of work to please you. If you do not like it. So tell form the start. Saves us both a lot of time.

Taking you comment seriously, I will stop pushing PRs. Doesn't want ot waste your time.

but some of them are so obvious AI generated

Yes, of course. Blaming me for this? And do you know why? AI is not biased. It does not write documentation or comments with a bias about who may read it or who may use it. I t does not judge about it's users ir there intends.

Feel free to close.

I have my fast rendering for local checks, suites me.

CU.

@jaapio
Copy link
Member

jaapio commented Jan 27, 2026

I want to be very clear here. From the moment you opened your first PR, I indicated that I wanted to discuss the overall approach before large or complex changes were implemented. That expectation hasn’t changed.

This is now the second time that caution around complexity, long-term maintenance, and ecosystem impact is being interpreted as something personal or discouraging. It isn’t. My responsibility as a maintainer is to be careful with changes that affect the core architecture, especially when they introduce new subsystems with far-reaching consequences.

I’m glad you’ve found a setup that works for your use case. However, my role is to ensure that changes also work for other users, downstream integrations, and for the maintainers who will be responsible for this code years from now. That responsibility outweighs the fact that something works well in a single environment.

You may find this article helpful in understanding the maintainer perspective on large contributions:
https://dev.to/matks/your-pull-request-is-not-a-gift-237m

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants