[API Proposal]: Metrics for M.E.Caching.MemoryCache

[cincuranet]

Background and motivation

Microsoft.Extensions.Caching.Memory.IMemoryCache is the primary in-process cache for modern .NET apps. Compared to System.Runtime.Caching, it lacks a standard, built-in metrics story. That gap makes it hard to:

• Migrate from System.Runtime.Caching while keeping comparable observability.
• Validate that caches are actually saving work in production.
• Detect thrashing due to memory pressure or aggressive expirations.

.NET 8 added MemoryCacheOptions.TrackStatistics and IMemoryCache.GetCurrentStatistics(), which expose hit/miss and size data. However, each team still has to:

• Invent its own metric names and tag schema.
• Integrate manually with Meter / OpenTelemetry.
• Handle multiple caches and naming conventions.
• Be careful not to add hot-path overhead in the cache.

API Proposal

namespace Microsoft.Extensions.Caching.Memory;

public class MemoryCacheStatistics
{
    long TotalHits { get; init; }
    long TotalMisses { get; init; }
    long CurrentEntryCount { get; init; }
    long CurrentEstimatedSize { get; init; }
+   long TotalEvictions { get; init; }
}

TotalEvictions is total number of entries evicted by the cache implementation since the cache was created. It includes removals due to size limits, expirations, memory pressure, but not i.e. explicit user removals.

namespace Microsoft.Extensions.Caching.Memory;

public class MemoryCacheOptions
{
    // ...
    public bool TrackStatistics { get; set; }
+   public string Name { get; set; } = "Default";
}

The name for this cache. Will be used as dimension in OTEL instruments or as an opaque identifier to distinguish between two or more caches.

namespace Microsoft.Extensions.Caching.Memory;

public class MemoryCache
{
    public MemoryCache(IOptions<MemoryCacheOptions> optionsAccessor)
    public MemoryCache(IOptions<MemoryCacheOptions> optionsAccessor, ILoggerFactory loggerFactory)
+   public MemoryCache(IOptions<MemoryCacheOptions> optionsAccessor, ILoggerFactory loggerFactory, IMeterFactory? meterFactory)
    // ...
}

Optional IMeterFactory that will be used to create Meter and instruments. I.e. WebApplication already contains registration for DefaultMeterFactory so it will work out of the box. If not provided, implementation will use shared meter like in i.e. S.N.Http.

The actual implementation will use underlying MemoryCacheStatistics and will just publish those values. If and only if TrackStatistics is true.

OTEL names and instruments:

• Meter: Microsoft.Extensions.Caching.Memory
• TotalHits+TotalMisses => ObservableCounter<long>: cache.requests
  • with attribute cache.request.type Hit, Miss
• CurrentEntryCount => ObservableUpDownCounter<long>: cache.entries
• CurrentEstimatedSize => ObservableGauge<long>: cache.estimated_size
• TotalEvictions => ObservableCounter<long>: cache.evictions
• dimension for all => cache.name

All these instruments are observable to not create any bottleneck in hot path.

API Usage

With DI (most of the usages):

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddMemoryCache(options =>
{
    options.TrackStatistics = true;
    options.Name = "my-cache";
});
// Optional: hook into OpenTelemetry metrics
builder.Services.AddOpenTelemetry()
    .WithMetrics(metrics =>
    {
        metrics.AddMeter("Microsoft.Extensions.Caching.Memory");
        // configure exporters, views, etc.
    });

Or when used manually:

var options = new MemoryCacheOptions
{
    TrackStatistics = true,
    Name = "my-cache",
};
var cache = new MemoryCache(Options.Create(options));

Alternative Designs

Alternative design could be to use - instead of "eager" publishing of meter/instruments - "late" bound publishing where developer is required to explicitly enable the publishing (and also optionally stop the publishing).

public interface IMemoryCacheMetrics
{
    void AddCache(string name, IMemoryCache cache);
    void RemoveCache(IMemoryCache cache);
}

This is non-intuitive. Requires developer to do extra steps and does not follow what we do in i.e. System.Runtime or System.Net.Http where metrics are available without extra steps. And also bloats the API surface area. We can consider this in the future as another way, if we see strong demand for this kind of flexibility.

Risks

• Incorrect implementation of metrics polling could increase latency or allocations. But it is the same as calling GetCurrentStatistics extensively.
• Name relies on callers to supply stable, unique names. Duplicate names or name changes over time can make metrics harder to interpret.

These risks are considered manageable and can be mitigated with clear documentation, testing, and early feedback.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-extensions-caching
See info in area-owners.md if you want to be subscribed.

[cincuranet]

cc @rjmurillo @JamesNK @noahfalk @mrek-msft @rosebyte @karelz

[cincuranet]

Related to #122470, #67770.

[julealgon]

What happens if any of the values in MemoryCacheStatistics overflows? I see they are all long-typed, but since this tracks metrics for a process, if the process itself is very long lived, I could actually see overflowing these values being a not-so-rare situation. (EDIT: seeing the numbers below, this statement now feels quite dumb 😅)

Is there anything that can be done to prevent that issue?

[cincuranet]

Let's do some quick computation: long.MaxValue / (365 * 2 * 86400) ~= 146_235_604_338. That means you'd have to be doing ~146 billion hits per second sustained continuously for 2 years to overflow. Not a realistic concern to me. :)

[AguirreMoy]

Regarding the proposed cache.hits and cache.misses instruments:

Would it be possible to combine these into a single metric (e.g., cache.lookups or cache.requests) that uses a result attribute to distinguish between hit and miss?

This dimensional approach is the standard pattern for OpenTelemetry and is already used by other major ecosystem receivers. It simplifies downstream data analysis (like calculating hit ratios) and reduces the number of top-level instruments.

See similar implementations for reference:

• Memcached: memcached.operations
  (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/memcachedreceiver/documentation.md#memcachedoperations)
• Elasticsearch: elasticsearch.node.cache.count
  (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/elasticsearchreceiver/documentation.md#elasticsearchnodecachecount)
• MySQL: mysql.table_open_cache
  (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/mysqlreceiver/documentation.md#mysqltable_open_cache)