Skip to content

Improve self-hosting docs #650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
nik13 wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Improve self-hosting docs #650

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/lib/navigation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ export const tabNavigation: NavTab[] = [
{ title: 'Docker Compose', href: '/docs/self-hosting/docker-compose' },
{ title: 'Environment Variables', href: '/docs/self-hosting/environment' },
{ title: 'System Configuration', href: '/docs/self-hosting/configuration' },
{ title: 'AgentCC Gateway', href: '/docs/self-hosting/gateway' },
{ title: 'User Management', href: '/docs/self-hosting/user-management' },
{ title: 'Production', href: '/docs/self-hosting/production' },
{ title: 'Troubleshooting', href: '/docs/self-hosting/troubleshooting' },
Expand Down
125 changes: 61 additions & 64 deletions src/pages/docs/self-hosting.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,102 +1,99 @@
---
title: "Self-Hosting Future AGI: Deploy on Your Own Infrastructure"
description: "Deploy the full Future AGI platform on your own infrastructure using Docker Compose. Follow the step-by-step guide to get all services running locally."
description: "Deploy Future AGI on your own infrastructure with the self-hosted installer and Docker Compose."
---

## About

Future AGI is fully open-source. Self-hosting runs the entire stack on your machines — all traces, datasets, evaluations, and model calls stay within your network. Backend is Django, frontend is React + Vite, LLM gateway is Go.
Self-host Future AGI when you need control over data residency, network access, model credentials, and operational policy. The open-source stack runs the product UI, backend APIs, tracing and evaluation pipelines, object storage, analytics storage, background workers, and AgentCC Gateway inside your infrastructure.

Not sure if you need this? The hosted version at [app.futureagi.com](https://app.futureagi.com) is easier to operate. Self-host when you need **data residency**, **air-gapped environments**, **cost control at scale**, or **deep customization**.
The fastest path is `./bin/install`. It prepares `.env`, generates local secrets, starts Docker Compose, pulls the published Future AGI images, waits for the backend health check, and can create your first user.

## What you get

| Capability | Included in the self-hosted stack |
|---|---|
| Observability | Trace ingestion, sessions, spans, annotations, ClickHouse-backed analytics |
| Evaluation | Eval execution, background workflows, datasets, prompt and simulation workflows |
| AgentCC Gateway | Self-hosted gateway for provider routing, retries, failover, rate limits, caching, budgets, and virtual keys |
| Data ownership | Postgres, ClickHouse, Redis, and MinIO run in your environment |
| Provider control | Bring your own OpenAI, Anthropic, Gemini, Bedrock, Ollama, vLLM, or internal model endpoints |
| Deployment control | Local laptop, VM, or bare metal with Docker Compose today; Kubernetes and Helm charts are coming soon |

## Recommended paths

| Goal | Start here | Expected outcome |
|---|---|---|
| Try the platform locally | [Docker Compose](/docs/self-hosting/docker-compose) | Installer-driven setup on `localhost` |
| Size infrastructure | [Requirements](/docs/self-hosting/requirements) | Hardware tier, port map, platform compatibility |
| Configure model access | [AgentCC Gateway](/docs/self-hosting/gateway) | Provider keys, health check, test model call |
| Prepare for users | [Production](/docs/self-hosting/production) | TLS, secrets, backups, monitoring, upgrade runbook |
| Fix a failed boot | [Troubleshooting](/docs/self-hosting/troubleshooting) | Symptom-based commands and recovery steps |

## Quick start

```bash
git clone https://github.com/future-agi/future-agi.git
cd future-agi
cp .env.example .env
docker pull futureagi/future-agi:v1.8.19_base
docker compose up
./bin/install
```

First boot builds from source (~10–15 min). After `Application startup complete`:

| Service | URL |
|---|---|
| Frontend | http://localhost:3000 |
| Backend API | http://localhost:8000 |
| PeerDB UI | http://localhost:3001 — `peerdb` / `peerdb` |
The installer starts the stack from published images, waits for `http://localhost:8000/health/`, and prompts for the first user unless you pass `--skip-user-creation`. The standard self-hosting path does not build application images locally.

## Deployment options
| Service | URL | Notes |
|---|---|---|
| Frontend | [http://localhost:3000](http://localhost:3000) | Product UI |
| Backend API | [http://localhost:8000](http://localhost:8000) | API, SDK ingestion, admin |
| AgentCC Gateway | [http://localhost:8090](http://localhost:8090) | Provider routing gateway mapped from the container |

| Option | Status |
|---|---|
| Docker Compose | Available |
| Helm / Kubernetes | Coming soon |
| Air-gapped | Coming soon |
<Warning>
Before sharing the instance with anyone else, review `.env`, configure TLS, and restrict access to backend and data-store ports. `./bin/install` generates local secrets for you; manually created `.env` files must not keep `CHANGEME` values.
</Warning>

## Architecture

21 containers across four layers.

```
```text
Browser
└─ frontend (React/nginx)
└─ backend (Django) ──── gateway (Go) ──── OpenAI · Anthropic · Gemini · Bedrock
├── postgres primary DB + WAL replication
├── clickhouse analytics store
├── redis cache / pub-sub
├── minio object storage
└── temporal ──── worker background jobs / eval pipelines

postgres ──── PeerDB CDC ──── clickhouse (continuous replication)
-> frontend (React/nginx)
-> backend (Django API, SDK ingestion, admin)
-> agentcc-gateway (AgentCC Gateway, Go)
-> OpenAI, Anthropic, Gemini, Bedrock, Ollama, vLLM, internal models
-> temporal -> worker (evals, agent jobs, data jobs)
-> serving (embeddings and local model utilities)
-> code-executor (sandboxed eval code)

backend -> postgres primary transactional store
backend -> redis cache, locks, pub/sub
backend -> minio S3-compatible object storage
backend -> clickhouse analytics and trace query store
```

**Application** — `frontend` · `backend` · `worker` · `gateway` · `serving` · `code-executor`
| Layer | Services | Production note |
|---|---|---|
| Application | `frontend`, `backend`, `worker`, `serving`, `code-executor` | Scale backend and workers independently as usage grows |
| AgentCC Gateway | `agentcc-gateway` | Keep provider keys here; expose publicly only if external apps call it directly |
| Data | `postgres`, `clickhouse`, `redis`, `minio` | Use managed services for stronger durability and backup controls |
| Workflow | `temporal` | Keep close to Postgres and workers for low-latency task execution |

**Data** — `postgres` · `clickhouse` · `redis` · `minio`
## Security boundaries

**Workflow** — `temporal`
Only the frontend and backend normally need to be reachable by users. The Compose file publishes AgentCC Gateway for local testing, but production deployments should keep it private unless external applications call it directly. Keep Postgres, ClickHouse, Redis, MinIO, and Temporal on private networks unless you have a deliberate admin access path such as VPN or bastion.

**CDC (PeerDB)** — `peerdb-catalog` · `peerdb-temporal` · `peerdb-minio` · `peerdb-flow-api` · `peerdb-flow-worker` · `peerdb-flow-snapshot-worker` · `peerdb-server` · `peerdb-ui` · `peerdb-temporal-init` · `peerdb-init`

| Layer | Service | Purpose |
|---|---|---|
| App | `frontend` | React SPA served by nginx |
| App | `backend` | Django REST + gRPC + WebSocket API |
| App | `worker` | Temporal worker — evals, agent loops, data jobs |
| App | `gateway` | Go LLM proxy — routing, retries, rate limits, logging |
| App | `serving` | Embeddings and small model inference |
| App | `code-executor` | nsjail-sandboxed eval code runner (`privileged: true` required) |
| Data | `postgres` | Primary DB — users, traces, datasets, evals, prompts |
| Data | `clickhouse` | Analytics DB — replicated from Postgres via PeerDB |
| Data | `redis` | Cache, rate limits, WebSocket pub/sub |
| Data | `minio` | S3-compatible object storage (swap for S3 in prod) |
| Workflow | `temporal` | Durable workflow engine — shares main Postgres |
| CDC | PeerDB stack | Continuous Postgres → ClickHouse replication (10 services) |
If you expose AgentCC Gateway for external applications, put it behind TLS and use gateway virtual keys instead of provider keys in client applications. Provider keys should stay in `.env`, `config.yaml`, or your secret manager.

## Next Steps

<CardGroup cols={2}>
<Card title="Requirements" icon="server" href="/docs/self-hosting/requirements">
Hardware tiers, platform compatibility, ports reference.
Hardware sizing, supported platforms, and network ports.
</Card>
<Card title="Docker Compose" icon="docker" href="/docs/self-hosting/docker-compose">
Setup, deployment modes, day-to-day operations.
</Card>
<Card title="Environment Variables" icon="settings" href="/docs/self-hosting/environment">
Full `.env` reference — secrets, ports, flags, keys.
Run the full stack, verify health, and operate it day to day.
</Card>
<Card title="System Configuration" icon="sliders" href="/docs/self-hosting/configuration">
LLM gateway providers, PeerDB mirrors, Temporal workers.
</Card>
<Card title="User Management" icon="user" href="/docs/self-hosting/user-management">
Create accounts via email or Django shell.
<Card title="AgentCC Gateway" icon="route" href="/docs/self-hosting/gateway">
Configure self-hosted model routing, provider keys, and test calls.
</Card>
<Card title="Production" icon="shield" href="/docs/self-hosting/production">
Hardening, backups, monitoring, upgrades.
</Card>
<Card title="Troubleshooting" icon="wrench" href="/docs/self-hosting/troubleshooting">
Solutions for every known error.
Harden the deployment before real users or production workloads.
</Card>
</CardGroup>
162 changes: 68 additions & 94 deletions src/pages/docs/self-hosting/configuration.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,139 +1,113 @@
---
title: "Self-Hosting System Configuration"
description: "Configure the LLM gateway config.yaml with provider API keys, set up PeerDB Postgres-to-ClickHouse CDC mirrors, and tune Temporal worker concurrency."
description: "Configure AgentCC Gateway config.yaml with provider API keys, frontend API URL, and Temporal worker concurrency."
---

## About

Configure the moving parts that aren't covered by `.env` alone: provider entries in the LLM gateway's `config.yaml`, the PeerDB Postgres → ClickHouse replication mirrors, and Temporal worker concurrency.
`.env` controls ports, secrets, and service-level runtime flags. A complete self-hosted deployment also needs three system-level checks:

## LLM gateway
- AgentCC Gateway has provider configuration and keys
- Temporal workers have enough concurrency for your evaluation and tracing workload

<Warning>
The LLM gateway requires additional configuration before model calls will work. You must create a `config.yaml` and provide your provider API keys — see the setup steps below.
</Warning>
## Configuration map

The gateway is a Go LLM proxy that routes all model calls. It ships with `config.example.yaml` — OpenAI enabled by default.
| Area | Where to configure | When to change |
|---|---|---|
| Secrets and ports | `.env` | Before first boot, when rotating secrets, or when ports conflict |
| AgentCC Gateway providers | `agentcc-gateway/config.yaml` and `.env` | Before running model calls through AgentCC Gateway |
| Worker concurrency | `.env` | When evals, traces, or long-running jobs queue up |
| Frontend API URL | `VITE_HOST_API` in `.env` | When moving from localhost to a public backend URL |

### Setup
## AgentCC Gateway

```bash
# 1. Copy the example
cp futureagi/agentcc-gateway/config.example.yaml \
futureagi/agentcc-gateway/config.yaml
AgentCC Gateway runs as the `agentcc-gateway` Compose service. It routes model calls from the Future AGI backend and, if you expose it, from your own applications.

# 2. Edit config.yaml — uncomment providers, set keys via ${VAR} interpolation
# 3. Set matching keys in .env (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.)
For the complete setup, provider examples, self-hosted models, and test calls, see [AgentCC Gateway](/docs/self-hosting/gateway).

# 4. Point the gateway volume at your config.yaml (in docker-compose.yml)
# volumes:
# - ./futureagi/agentcc-gateway/config.yaml:/app/config.yaml:ro
Quick setup:

# 5. Restart
docker compose up -d --force-recreate gateway
```bash
cp agentcc-gateway/config.example.yaml \
agentcc-gateway/config.yaml
```

`config.yaml` is gitignored. Treat it as a secret.

### Provider config examples
Point the Compose mount at your config in `.env`:

<Tabs>
<Tab title="OpenAI / Anthropic / Gemini">
```yaml
providers:
openai:
api_key: "${OPENAI_API_KEY}"
api_format: "openai"
models: [gpt-4o, gpt-4o-mini]
```bash
AGENTCC_CONFIG_PATH=agentcc-gateway/config.yaml
```

anthropic:
api_key: "${ANTHROPIC_API_KEY}"
api_format: "anthropic"
models: [claude-opus-4-5, claude-sonnet-4-5]
Add or update provider keys in `.env`:

gemini:
api_key: "${GOOGLE_API_KEY}"
api_format: "gemini"
models: [gemini-2.0-flash, gemini-1.5-pro]
```
</Tab>
<Tab title="AWS Bedrock">
```yaml
providers:
bedrock:
api_key: "${AWS_SECRET_ACCESS_KEY}"
api_format: "bedrock"
region: "${AWS_REGION}"
access_key: "${AWS_ACCESS_KEY_ID}"
models: [anthropic.claude-3-5-sonnet-20241022-v2:0]
```bash
OPENAI_API_KEY=sk-...
ANTHROPIC_API_KEY=sk-ant-...
GOOGLE_API_KEY=...
```
</Tab>
<Tab title="Vertex AI">
```yaml
providers:
vertex:
base_url: "https://us-central1-aiplatform.googleapis.com"
api_key: "${GOOGLE_ACCESS_TOKEN}"
api_format: "gemini"
headers:
x-gcp-project: "${GCP_PROJECT_ID}"
x-gcp-location: "us-central1"
models: [gemini-2.0-flash-001]

Then recreate AgentCC Gateway:

```bash
docker compose up -d --force-recreate agentcc-gateway
```
Vertex uses a Bearer token, not an API key. Rotate `GOOGLE_ACCESS_TOKEN` via a sidecar calling `gcloud auth print-access-token`.
</Tab>
</Tabs>

For routing rules, rate limits, caching, and the full config reference — see [Agent Command Center → Self-hosted](/docs/command-center/deployment/self-hosted).
The Compose file passes `GOOGLE_API_KEY` into the AgentCC Gateway container as `GEMINI_API_KEY`, matching the AgentCC Gateway example config.

---
Verify:

## PeerDB (Postgres → ClickHouse CDC)
```bash
curl http://localhost:8090/healthz
docker compose logs --tail=100 agentcc-gateway
```

PeerDB continuously replicates Postgres tables into ClickHouse so trace and eval analytics stay fast.
## Temporal workers

**First-boot timing issue**: `peerdb-init` runs immediately on startup, before Django migrations may have completed. If mirrors show "not started" in the PeerDB UI:
The default self-hosted stack uses one all-queue worker:

```bash
# 1. Wait until backend logs "Application startup complete"
docker compose logs -f backend

# 2. Re-run init
docker compose run --rm peerdb-init bash /setup.sh
TEMPORAL_ALL_QUEUES=true
TEMPORAL_MAX_CONCURRENT_ACTIVITIES=50
TEMPORAL_MAX_CONCURRENT_WORKFLOW_TASKS=50
```

Verify at [http://localhost:3001](http://localhost:3001) — mirrors should show `running` within seconds.
Increase these when jobs are healthy but queued for too long. Keep them lower when the host is CPU or memory constrained.

After upgrades that touch replicated tables, re-run the same init command.
The dev overlay runs dedicated workers by queue:

---
| Service | Queue | Typical use |
|---|---|---|
| `worker-default` | `default` | General workflows |
| `worker-tasks-s` | `tasks_s` | Short tasks |
| `worker-tasks-l` | `tasks_l` | Larger eval and simulation tasks |
| `worker-tasks-xl` | `tasks_xl` | Expensive long-running tasks |
| `worker-trace-ingestion` | `trace_ingestion` | Trace ingestion pipeline |
| `worker-agent-compass` | `agent_compass` | Agent Command Center jobs |

## Temporal workers
Use the dev overlay for debugging queue-specific behavior:

**Default (all-queue)** — one worker polls all task queues. Controlled by `TEMPORAL_ALL_QUEUES=true` in `.env`. Good for self-hosted deployments.
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up
```

**Per-queue workers** (dev mode) — six dedicated workers via the dev overlay:
## Frontend URL changes

| Service name | Queue | Typical concurrency |
|---|---|---|
| `worker-default` | `default` | 100 |
| `worker-tasks-s` | `tasks_s` | 200 |
| `worker-tasks-l` | `tasks_l` | 50 |
| `worker-tasks-xl` | `tasks_xl` | 10 |
| `worker-trace-ingestion` | `trace_ingestion` | 100 |
| `worker-agent-compass` | `agent_compass` | 50 |
`VITE_HOST_API` controls the backend URL used by the browser. When you move from local URLs to a public backend domain, update `.env` or `deploy/.env.production` and recreate the frontend container:

Tune concurrency in `.env` via `TEMPORAL_MAX_CONCURRENT_ACTIVITIES` and `TEMPORAL_MAX_CONCURRENT_WORKFLOW_TASKS`.
```bash
VITE_HOST_API=https://api.yourcompany.com
docker compose up -d --force-recreate frontend
```

Temporal UI (dev mode): [http://localhost:8085](http://localhost:8085)
If the UI loads but API calls fail with CORS or network errors, this is the first value to check.

## Next Steps

<CardGroup cols={2}>
<Card title="Production" icon="shield" href="/docs/self-hosting/production">
Hardening, backups, and monitoring before going live.
<Card title="AgentCC Gateway" icon="route" href="/docs/self-hosting/gateway">
Configure provider access and test model calls.
</Card>
<Card title="Troubleshooting" icon="wrench" href="/docs/self-hosting/troubleshooting">
Solutions for common configuration errors.
<Card title="Production" icon="shield" href="/docs/self-hosting/production">
Harden networking, backups, monitoring, and upgrades.
</Card>
</CardGroup>
Loading
Loading