Skip to content

Conversation

@bitfaster
Copy link
Owner

@bitfaster bitfaster commented Jan 9, 2026

This PR implements events for ConcurrentLfu as a switchable policy. When disabled all event code is fully elided at runtime by the JIT compiler.

Events are considered perf critical in ConcurrentLfu because the ItemRemoved logic is invoked as part of the maintenance cycle, this introduces overhead even when there are no events registered. The Maintenance method latency determines cache throughput at the limit, so any overhead here is not desired. Later, these events could be captured in a list and processed asynchronously via the scheduler.

This PR was produced with Claude Code using the following prompts:

Prompt 1:

Implement the ItemRemoved and ItemUpdated events for ConcurrentLfu. Do not modify the LFU builder classes yet. Follow the pattern of ITelemetryPolicy in ConcurrentLru, but do not incorporate the cache hit metrics because they are already implemented in ConcurrentLfu. Instead of ITelemetryPolicy, for ConcurrentLfu define a new interface IEventPolicy with the ItemRemoved and ItemUpdated events. IEventPolicy should be implemented as a struct to enable the JIT to elide unused code when events are not yet enabled.

Prompt 2:

Implement the same IEventPolicy changes made in ConcurrentLfu in the ConcurrentTLfu class.

Prompt 3:

Fix INodePolicy and the classes derived from it to accept the event policy generic parameter.

Fixed manually: reviewed code to determine correct approach to distinguish between evicted vs removed.

Fixed manually: test build and run.

Prompt 4:

Add unit tests to ConcurrentLfuTests for the ItemRemoved and ItemUpdated events. For ItemRemoved, explicitly test
different removed reasons. When calling TryRemove, reason should be Removed. When the cache reaches bounded size and
the policy evicts an item, reason should be evicted. When calling Trim and Clear, reason should be set accordingly.

Prompt 5:

For the ConcurrentLfu and ConcurrentTLfu event properties, replicate the Proxy inner class from ConcurrentLruCore that enables the telemetry policy mutable struct to function correctly.

Prompt 6:

In ConcurrentLfuCore, modify the Evict method to accept an ItemRemovedReason argument. When the Maintenance method is invoked from Trim, ItemRemovedReason should be trimmed. When Maintenance is invoked from Clear, ItemRemovedReason should be cleared. In all other cases, Evict should use ItemRemovedReason Evicted. Do not introduce a class level flag to track ItemRemovedReason, instead propagate the reason as a method argument from the call site where the reason is known. The existing unit tests cover each of these cases, and the tests for cleared and trimmed are failing. This change should fix those two test cases.

Prompt 7:

Add explicit unit tests for the NoEventPolicy and EventPolicy policy classes. The tests should exercise all methods and properties, similar to the TelemetryPolicyTests used to test the TelemetryPolicy class.

@coveralls
Copy link

coveralls commented Jan 9, 2026

Coverage Status

coverage: 98.843% (-0.2%) from 99.072%
when pulling 7f86b54 on users/alexpeck/lfu4
into 20f02a0 on main.

@bitfaster bitfaster changed the title Implement Lfu events: attempt 3 Implement ConcurrentLfu events: attempt 3 Jan 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants