[ARCHITECTURE] Storage persistence prevents multi-replica (HA) deployments without RWX volumes

[bernardgut]

Problem

The current chart uses Deployment + PVC for the OpenCloud pod, creating two PersistentVolumeClaims (config and data) that default to ReadWriteOnce access mode. This architecture makes multi-replica (HA) deployments impossible without ReadWriteMany (RWX) volumes.

While the values.yaml comments document this limitation:

# Number of replicas (Note: When using multiple replicas, persistence should be disabled
# or use a storage class that supports ReadWriteMany access mode)
replicas: 1

...there is no mechanism in the chart to properly decouple these concerns. The only options today are:

1. Single replica (no HA) — works with RWO ✅
2. Multiple replicas + RWX volumes — requires NFS or similar (antipattern for production K8s) ⚠️
3. Multiple replicas + persistence disabled — loses config and metadata across restarts ❌

None of these options provide a production-grade HA deployment.

Root Cause Analysis

The two PVCs serve fundamentally different purposes that should be handled differently:

1. Config PVC (/etc/opencloud) — Can be externalized

The config volume stores:

• Auto-generated secrets: JWT secret, WOPI secret, machine auth API key, transfer secret, URL signing secret, system user API key
• Service configuration: Generated by opencloud init on startup
• Web UI themes and customizations

These secrets are generated randomly on first startup. When config persistence is disabled and multiple replicas exist, each pod generates different secrets — causing authentication failures between services (e.g., the Collaboration pod cannot validate JWT tokens from the OpenCloud pod).

Current workaround: Inject secrets via environment variables (OC_JWT_SECRET, OC_MACHINE_AUTH_API_KEY, etc.) and disable config persistence. This works but is undocumented and requires creating all internal secrets manually.

Proposed fix:

• Support injecting all init secrets via Kubernetes Secrets / env vars in the chart
• Make opencloud init --force-overwrite use ENV-provided secrets instead of generating new ones
• This fully decouples config from the filesystem — no PVC needed

2. Data PVC (/var/lib/opencloud) — Application-level limitation

The data volume stores S3ng metadata (file metadata, sharing data, user settings) even when blobs are stored on S3. This is the same issue documented in owncloud/ocis-charts#489, where @wkloucek confirmed:

"storageusers, storagesystem need RWX volumes because this is where the metadata is stored for the S3ng storage driver"

"Maybe there's gonna be another storage driver in the future that can live without RWX for storing metadata and leveraging one of those two key-value-stores..."

This is tracked upstream as owncloud/ocis#4594. Until the OpenCloud application supports storing metadata in a key-value store (Redis/NATS), this remains an application-level requirement.

However, the chart can still improve the situation by clearly separating the concerns:

• Config → externalized via ENV/Secrets (no PVC needed)
• Data (metadata) → clearly documented as requiring RWX if using multiple replicas with S3ng

Impact on Services

The chart's monolithic OpenCloud deployment bundles multiple internal services that have different storage requirements:

Data Category	Current Storage	Can Be Externalized?	Proposed Solution
Init secrets (JWT, WOPI, etc.)	Config PVC	✅ Yes	ENV vars from K8s Secrets
Service registry	Config PVC	✅ Yes	NATS/Redis (already supported)
Cache (thumbnails, etc.)	Config PVC / memory	✅ Yes	Redis Sentinel
Persistent store (notifications, activity)	Config PVC / NATS	✅ Yes	Redis Sentinel / NATS JetStream
S3ng file metadata	Data PVC	❌ Not yet	Needs upstream app changes
Sharing data / user settings	Data PVC	❌ Not yet	Needs upstream app changes

Prior Art

This issue was first raised in the original (now archived) ownCloud chart:

• owncloud/ocis-charts#489 — "No HA documentation. Is HA supported?"
• owncloud/ocis-charts#129 — "persistence as a global parameter?"
• Upstream tracking: owncloud/ocis#4594 — storage driver improvements

Proposed Changes (Chart-Level)

I propose to make the following changes, that can be made at the chart level to improve the HA story, without requiring upstream application changes:

PR 1: Externalize init secrets (Config PVC decoupling)

• Add support for injecting all internal secrets via existingSecret references
• Add ENV var injection for JWT, WOPI, machine-auth, transfer, URL-signing secrets
• Allow config persistence to be disabled without breaking multi-replica setups
• Document the required secrets and their purpose

PR 2: Redis Sentinel integration for cache and persistent stores

• Add values for configuring external Redis Sentinel as the cache and persistent store backend
• Inject OC_CACHE_STORE, OC_PERSISTENT_STORE, OC_CACHE_STORE_NODES, etc. via ENV
• This moves thumbnails, notifications, activity logs, and other ephemeral/persistent data off the filesystem
• Reference: OpenCloud cache/store documentation

Remaining limitation (requires upstream app changes)

• S3ng metadata still requires a shared filesystem (RWX PVC) for multi-replica deployments
• This should be tracked upstream as a request for a key-value metadata storage driver

Environment

• Chart version: 2.0.1 (Tim-herbie/opencloud-helm)
• OpenCloud version: 5.2.0
• Storage driver: decomposeds3 (S3ng)
• Tested with: external NATS, external PostgreSQL (CNPG), external S3

[Tim-herbie]

Thank you for opening this issue — I really like your vision of making this Helm chart highly available.

About two months ago, I experimented with a similar setup and managed to build a more or less HA OpenCloud environment using external services like Collabora, NATS, OpenLDAP, S3, Keycloak, and CNPG.

I haven’t specifically looked into removing the config PVC yet, but if it’s feasible, it definitely seems like a worthwhile improvement.

[bernardgut]

I am working KVFS-driver, a NATS JetStream KV + S3 storage driver for OpenCloud, to replace POSIX filesystem dependencies for horizontally-scalable container-native file storage, im currently at alpha deployment stage. This is why I need to change a bit the helm chart. I am not sure upstream opencloud is open to these kinds of contribution so I might have to fork it. In any case this is why I want to move the chart torwards a more cloud-native deployment with my PRs...
But yeah glad to hear I am not the only one who needs this.

[Tim-herbie]

The first PR that you named PR 1: Externalize init secrets (Config PVC decoupling) is merged into main.