Redis backed Caching of Internal Embeddings

[ajtiwari07]

1. Embeddings-Specific Cache Configuration

• New EmbeddingsCacheOptions class for embeddings-specific cache settings
• Configurable TTL (Time-To-Live) with default of 24 hours (optimized for deterministic embeddings)
• Independent cache configuration from global runtime cache
• Support for enabling/disabling caching at the embeddings level

2. Level 2 (L2) Distributed Redis Cache Support

• New EmbeddingsCacheLevel2Options class for Redis-specific configuration
• Dedicated Redis connection for embeddings cache
• Independent from global runtime L2 cache (separate connection string)
• FusionCache integration with Redis backplane for distributed cache invalidation
• Graceful fallback to L1-only if L2 is not configured or fails

3. Enhanced Configuration Schema

• Updated JSON schema with new cache section under embeddings
• Support for environment variable substitution in Redis connection strings
• Clear documentation and examples in schema
• Validation for required fields when L2 is enabled

4. Dedicated FusionCache Instance for Embeddings

• Separate FusionCache instance named "EmbeddingsCache"
• Independent cache lifecycle from global DAB cache
• Optimized entry options for embedding workloads
• Error resilience with exception suppression for distributed operations

5. Comprehensive Test Coverage

• Unit tests for EmbeddingsCacheOptions class (205 lines)
• Unit tests for EmbeddingsCacheLevel2Options class (64 lines)
• Serialization tests for cache options (233 lines)
• Runtime cache L2 options tests (241 lines)
• Redis connection startup tests (138 lines)

  "$schema": "https://github.com/Azure/data-api-builder/releases/latest/download/dab.draft.schema.json",
  "data-source": {
    "database-type": "mssql",
    "connection-string": "@env('DATABASE_CONNECTION_STRING')"
  },
  "runtime": {
    "embeddings": {
      "enabled": true,
      "provider": "azure-openai",
      "base-url": "https://my-openai.openai.azure.com",
      "api-key": "@env('AZURE_OPENAI_API_KEY')",
      "model": "text-embedding-ada-002",
      "dimensions": 1536,
      "cache": {
        "enabled": true,
        "ttl-hours": 72,
        "level-2": {
          "enabled": true,
          "connection-string": "@env('EMBEDDINGS_REDIS_CONNECTION_STRING')"
        }
      }
    }
  }```

[JimRoberts-MS]

for log messages and such not sure if it would be less confusing to refer to it as 'Remote' cache or something in case people are running outside of Azure

[ajtiwari07]

Technically yes, I was hoping it is our value prop going into L2 caching. Will Modify it.

[JimRoberts-MS]

for this particular api do we not need to .Apply() or something to signal that we're done configuring?

[ajtiwari07]

Nope, FusionCacheBuilder is itself registered in the DI container.