Redis-backed storage in the embedded authorization server

[tgrunnagle]

Summary

Add RFC for Redis-backed storage in the embedded authorization server to enable horizontal scaling and high-availability deployments.

Addresses a portion of https://github.com/stacklok/stacklok-epics/issues/197

Why

The current in-memory storage prevents running multiple ToolHive replicas since they cannot share authentication state. Production Kubernetes deployments need HA support.

What

This RFC proposes a RedisStorage implementation that:

• Implements the existing Storage interface with Redis as the backend
• Supports standalone, Sentinel, and Cluster deployment modes
• Uses key partitioning for multi-tenancy
• Remains fully backward compatible (memory storage stays the default)

[jhrozek]

some comments inline, but I feel like I'm too much of a redis n00b to be able to review the deployment options properly

[JAORMX]

I'm thinking that having a model of one MCP server per Valkey/Redis deployment is ideal for us. This way, you don't need to deal with multi-tenancy, nor scaling the valkey deployment globally. Instead, the valkey deployment can scale as the MCP server traffic increases. I'd also suggest doing something very opinionated to begin with, e.g. having one way of authenticating and one way to operate. This way, the deployment is heavily targeted towards the use case(s) instead of needing to provide a lot of flexibility. We can provide flexible alternatives as we see fit. wdyt?

[JAORMX]

e.g. my recommendation would be to go sentinel mode, and have one primary and two replicas for seamless(-ish) rollovers.