feat(ansible): add simple pairing flow

.env.example

@@ -126,9 +126,14 @@ POSTGRES_PASSWORD=
 # INGEST_IMAGE=ghcr.io/carrtech-dev/ct-ops/ingest@sha256:...
 # ANSIBLE_API_IMAGE=ghcr.io/carrtech-dev/ct-ops/ansible-api@sha256:...
 
-# Optional Ansible module service-token settings for manually run ansible-api
-# containers. Configure matching values in Administration -> Integrations ->
-# Automation when using Service token HMAC authentication.
+# Optional Ansible API pairing credentials. The ansible-api container uses
+# these to issue or rotate CT-Ops generated service tokens, which are stored in
+# the ansible_api_data volume and in CT-Ops encrypted settings.
+# ANSIBLE_API_PAIRING_USERNAME=ctops
+# ANSIBLE_API_PAIRING_PASSWORD=
+
+# Optional legacy Ansible module service-token settings for manually run
+# ansible-api containers. Prefer pairing credentials for new installs.
 # ANSIBLE_API_SERVICE_TOKEN_ID=ansible-api
 # ANSIBLE_API_SERVICE_TOKEN_SECRET=
 

PROGRESS.md

@@ -15,6 +15,32 @@
 
 ## What Has Been Built
 
+### Session 136 — Simplified Ansible pairing
+
+**URL, username, password Ansible setup** (`apps/ansible-api/server.py`, `apps/web/lib/automation/ansible-api.ts`, `apps/web/lib/actions/automation.ts`, `apps/web/app/(dashboard)/settings/integrations/automation/automation-settings-client.tsx`, `docker-compose.single.yml`)
+- Added an Ansible API pairing endpoint that accepts initial environment-backed
+  credentials, generates a service-token secret, persists it in the Ansible API
+  data volume, and uses it for ongoing HMAC verification.
+- Allowed later rotation by re-pairing with the same initial credentials; static
+  legacy `ANSIBLE_API_SERVICE_TOKEN_*` environment variables remain supported
+  but are treated as externally managed.
+- Simplified the CT-Ops automation settings screen so admins pair an Ansible
+  connection with only URL, username, and password. CT-Ops stores the generated
+  service secret encrypted and clears the initial password after pairing.
+- Added bundled compose/start-script defaults for `ANSIBLE_API_PAIRING_USERNAME`
+  and generated `ANSIBLE_API_PAIRING_PASSWORD`, plus persistent Ansible token
+  storage and updated docs/tests.
+
+**Validation**
+- `python3 -m unittest tests/test_contract.py` from `apps/ansible-api`
+- `node --experimental-strip-types --test lib/automation/ansible-pairing-core.test.mjs lib/automation/ansible-ui-gating.test.mjs lib/actions/mutation-authz.test.mjs` from `apps/web`
+- `pnpm --dir apps/web type-check`
+- `pnpm --dir apps/web db:validate`
+- `pnpm --dir apps/web lint 'app/(dashboard)/settings/integrations/automation/automation-settings-client.tsx' lib/actions/automation.ts lib/automation/ansible-api.ts lib/automation/ansible-pairing-core.ts lib/automation/ansible-pairing-core.test.mjs lib/automation/ansible-ui-gating.test.mjs lib/actions/mutation-authz.test.mjs tests/e2e/settings/automation.spec.ts`
+- `pnpm --dir apps/web test:unit`
+- `pnpm --dir apps/web test:e2e tests/e2e/settings/automation.spec.ts`
+- `bash deploy/scripts/test-ansible-profile-wiring.sh`
+
 ### Session 135 — Module connection contract framework
 
 **External module contract and Ansible conversion** (`apps/web/lib/modules/*`, `apps/web/lib/db/schema/module-connections.ts`, `apps/web/lib/automation/ansible-api.ts`, `apps/ansible-api/server.py`)

apps/ansible-api/Dockerfile

@@ -2,7 +2,9 @@ FROM python:3.13-alpine
 
 RUN apk add --no-cache ansible openssh-client \
   && addgroup -S ansible \
-  && adduser -S -G ansible ansible
+  && adduser -S -G ansible ansible \
+  && mkdir -p /var/lib/ct-ops/ansible-api \
+  && chown -R ansible:ansible /var/lib/ct-ops
 
 WORKDIR /app
 COPY server.py /app/server.py

apps/ansible-api/server.py

@@ -5,8 +5,10 @@
 import hashlib
 import hmac
 import os
+import secrets
 import subprocess
 import tempfile
+import threading
 import time
 from dataclasses import dataclass
 from datetime import datetime, timezone
@@ -19,6 +21,12 @@
 MAX_CLOCK_SKEW_SECONDS = 300
 NONCE_TTL_SECONDS = 600
 _seen_nonces: dict[str, float] = {}
+PAIRING_RATE_LIMIT_WINDOW_SECONDS = 300
+PAIRING_RATE_LIMIT_ATTEMPTS = 10
+DEFAULT_SERVICE_TOKEN_FILE = "/var/lib/ct-ops/ansible-api/service-token.json"
+DEFAULT_SERVICE_TOKEN_ID = "ansible-api"
+_pairing_lock = threading.Lock()
+_pairing_attempts: dict[str, list[float]] = {}
 
 
 def ansible_version() -> str:
@@ -58,6 +66,7 @@ def capabilities_payload() -> dict[str, Any]:
             "streamingLogs": False,
             "jobCancel": False,
             "dryRun": False,
+            "pairing": True,
         },
     }
 
@@ -97,11 +106,25 @@ def openapi_payload() -> dict[str, Any]:
                     },
                 },
             },
+            "/api/v1/pairing/claim": {
+                "post": {
+                    "summary": "Exchange initial pairing credentials for a CT-Ops service token",
+                    "responses": {
+                        "200": {"description": "Generated service-token credentials"},
+                        "401": {"description": "Invalid pairing credentials"},
+                        "409": {"description": "Ansible API token is managed by environment variables"},
+                    },
+                },
+            },
         },
     }
 
 
-def _configured_service_token() -> tuple[str, str] | None:
+def _service_token_file() -> str:
+    return os.environ.get("ANSIBLE_API_SERVICE_TOKEN_FILE", DEFAULT_SERVICE_TOKEN_FILE).strip()
+
+
+def _configured_env_service_token() -> tuple[str, str] | None:
     token_id = os.environ.get("ANSIBLE_API_SERVICE_TOKEN_ID", "").strip()
     token_secret = os.environ.get("ANSIBLE_API_SERVICE_TOKEN_SECRET", "")
     if not token_id and not token_secret:
@@ -111,6 +134,101 @@ def _configured_service_token() -> tuple[str, str] | None:
     return token_id, token_secret
 
 
+def _load_service_token_file() -> tuple[str, str] | None:
+    token_file = _service_token_file()
+    if not token_file or not os.path.exists(token_file):
+        return None
+    try:
+        with open(token_file, "r", encoding="utf-8") as handle:
+            payload = json.load(handle)
+    except Exception as err:
+        raise PermissionError("Ansible API service token file is not readable") from err
+
+    token_id = str(payload.get("tokenId", "")).strip()
+    token_secret = str(payload.get("tokenSecret", ""))
+    if not token_id or len(token_secret.encode("utf-8")) < 32:
+        raise PermissionError("Ansible API service token file is not configured correctly")
+    return token_id, token_secret
+
+
+def _configured_service_token() -> tuple[str, str] | None:
+    return _configured_env_service_token() or _load_service_token_file()
+
+
+def _write_service_token_file(token_id: str, token_secret: str) -> None:
+    token_file = _service_token_file()
+    if not token_file:
+        raise PermissionError("Ansible API service token file is not configured")
+    directory = os.path.dirname(token_file)
+    if directory:
+        os.makedirs(directory, mode=0o700, exist_ok=True)
+    payload = {
+        "tokenId": token_id,
+        "tokenSecret": token_secret,
+        "createdAt": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
+    }
+    fd, tmp_path = tempfile.mkstemp(prefix=".service-token-", dir=directory or None)
+    try:
+        with os.fdopen(fd, "w", encoding="utf-8") as handle:
+            json.dump(payload, handle, sort_keys=True)
+            handle.write("\n")
+        os.chmod(tmp_path, 0o600)
+        os.replace(tmp_path, token_file)
+    except Exception:
+        try:
+            os.unlink(tmp_path)
+        except FileNotFoundError:
+            pass
+        raise
+
+
+def _pairing_credentials() -> tuple[str, str]:
+    username = os.environ.get("ANSIBLE_API_PAIRING_USERNAME", "").strip()
+    password = os.environ.get("ANSIBLE_API_PAIRING_PASSWORD", "")
+    if not username or not password:
+        raise PermissionError("Ansible API pairing is not configured")
+    return username, password
+
+
+def _check_pairing_rate_limit(client_id: str | None) -> None:
+    key = client_id or "local"
+    now = time.time()
+    cutoff = now - PAIRING_RATE_LIMIT_WINDOW_SECONDS
+    attempts = [value for value in _pairing_attempts.get(key, []) if value >= cutoff]
+    if len(attempts) >= PAIRING_RATE_LIMIT_ATTEMPTS:
+        _pairing_attempts[key] = attempts
+        raise PermissionError("too many pairing attempts")
+    attempts.append(now)
+    _pairing_attempts[key] = attempts
+
+
+def claim_pairing_token(payload: dict[str, Any], client_id: str | None = None) -> dict[str, Any]:
+    if _configured_env_service_token() is not None:
+        raise PermissionError("Ansible API service token is managed by environment variables")
+
+    configured_username, configured_password = _pairing_credentials()
+    _check_pairing_rate_limit(client_id)
+
+    username = payload.get("username")
+    password = payload.get("password")
+    if not isinstance(username, str) or not isinstance(password, str):
+        raise ValueError("username and password are required")
+    if not hmac.compare_digest(username, configured_username) or not hmac.compare_digest(password, configured_password):
+        raise PermissionError("invalid pairing credentials")
+
+    with _pairing_lock:
+        if _configured_env_service_token() is not None:
+            raise PermissionError("Ansible API service token is managed by environment variables")
+        token_id = DEFAULT_SERVICE_TOKEN_ID
+        token_secret = secrets.token_urlsafe(48)
+        _write_service_token_file(token_id, token_secret)
+        return {
+            "ok": True,
+            "tokenId": token_id,
+            "tokenSecret": token_secret,
+        }
+
+
 def _header(headers: Any, name: str) -> str:
     if hasattr(headers, "get"):
         value = headers.get(name)
@@ -394,6 +512,32 @@ def do_GET(self) -> None:
         self.write_json({"error": "not_found"}, status=404)
 
     def do_POST(self) -> None:
+        if self.path == "/api/v1/pairing/claim":
+            try:
+                length = int(self.headers.get("content-length", "0"))
+                body = self.rfile.read(min(length, 65_536))
+                payload = json.loads(body.decode("utf-8"))
+                if not isinstance(payload, dict):
+                    raise ValueError("request body must be an object")
+                client_id = self.client_address[0] if self.client_address else None
+                self.write_json(claim_pairing_token(payload, client_id=client_id))
+            except ValueError as err:
+                self.write_json({"error": str(err)}, status=400)
+            except PermissionError as err:
+                message = str(err)
+                if "managed by environment" in message:
+                    status = 409
+                elif "not configured" in message:
+                    status = 503
+                elif "too many" in message:
+                    status = 429
+                else:
+                    status = 401
+                self.write_json({"error": message}, status=status)
+            except Exception:
+                self.write_json({"error": "pairing failed"}, status=500)
+            return
+
         if self.path == "/api/v1/runs/ansible-ping":
             try:
                 length = int(self.headers.get("content-length", "0"))

apps/ansible-api/tests/test_contract.py

@@ -2,6 +2,7 @@
 import hashlib
 import hmac
 import base64
+import tempfile
 import unittest
 from unittest import mock
 
@@ -43,6 +44,63 @@ def test_verify_service_request_allows_unsigned_when_no_token_configured(self):
         with mock.patch.dict(server.os.environ, {}, clear=True):
             self.assertIsNone(server.verify_service_request("POST", "/api/v1/runs/ansible-ping", b"{}", {}))
 
+    def test_pairing_claim_requires_configured_initial_credentials(self):
+        with mock.patch.dict(server.os.environ, {}, clear=True):
+            with self.assertRaisesRegex(PermissionError, "not configured"):
+                server.claim_pairing_token({"username": "ctops", "password": "secret"})
+
+    def test_pairing_claim_generates_persisted_service_token(self):
+        with tempfile.TemporaryDirectory() as tmpdir:
+            token_file = f"{tmpdir}/service-token.json"
+            with mock.patch.dict(server.os.environ, {
+                "ANSIBLE_API_PAIRING_USERNAME": "ctops",
+                "ANSIBLE_API_PAIRING_PASSWORD": "initial password",
+                "ANSIBLE_API_SERVICE_TOKEN_FILE": token_file,
+            }, clear=True):
+                payload = server.claim_pairing_token({
+                    "username": "ctops",
+                    "password": "initial password",
+                })
+
+                self.assertEqual(payload["ok"], True)
+                self.assertEqual(payload["tokenId"], "ansible-api")
+                self.assertGreaterEqual(len(payload["tokenSecret"]), 32)
+                configured = server._configured_service_token()
+                self.assertEqual(configured, ("ansible-api", payload["tokenSecret"]))
+
+    def test_pairing_claim_rotates_existing_generated_service_token(self):
+        with tempfile.TemporaryDirectory() as tmpdir:
+            token_file = f"{tmpdir}/service-token.json"
+            with mock.patch.dict(server.os.environ, {
+                "ANSIBLE_API_PAIRING_USERNAME": "ctops",
+                "ANSIBLE_API_PAIRING_PASSWORD": "initial password",
+                "ANSIBLE_API_SERVICE_TOKEN_FILE": token_file,
+            }, clear=True):
+                first = server.claim_pairing_token({
+                    "username": "ctops",
+                    "password": "initial password",
+                })
+                second = server.claim_pairing_token({
+                    "username": "ctops",
+                    "password": "initial password",
+                })
+
+                self.assertNotEqual(first["tokenSecret"], second["tokenSecret"])
+                self.assertEqual(server._configured_service_token(), ("ansible-api", second["tokenSecret"]))
+
+    def test_pairing_claim_rejects_static_environment_service_token(self):
+        with mock.patch.dict(server.os.environ, {
+            "ANSIBLE_API_PAIRING_USERNAME": "ctops",
+            "ANSIBLE_API_PAIRING_PASSWORD": "initial password",
+            "ANSIBLE_API_SERVICE_TOKEN_ID": "ansible-api",
+            "ANSIBLE_API_SERVICE_TOKEN_SECRET": "ansible signing secret with enough entropy",
+        }, clear=True):
+            with self.assertRaisesRegex(PermissionError, "managed by environment"):
+                server.claim_pairing_token({
+                    "username": "ctops",
+                    "password": "initial password",
+                })
+
     def test_verify_service_request_accepts_valid_hmac_token(self):
         body = b'{"ok":true}'
         timestamp = "2026-05-15T12:00:00Z"

apps/docs/docs/deployment/docker-compose.md

@@ -130,9 +130,10 @@ The remaining ports are bound to `127.0.0.1` only, so `web:3000`, `ingest:8080`,
 
 The optional `ansible-api` service is not published to the host and CT-Ops does
 not start it automatically. Run the Ansible module yourself, then configure its
-URL and service-token settings in **Settings → Integrations → Automation**. The
-web app never mounts the Docker socket and does not orchestrate module
-containers.
+URL plus the initial pairing username and password in **Settings → Integrations
+→ Automation**. The Ansible API generates the service-token secret used for
+ongoing signed requests and stores it in its persistent data volume. The web app
+never mounts the Docker socket and does not orchestrate module containers.
 
 When installing inside a VM, LXC, or Incus instance that sits behind a NAT or
 private bridge, forward the external HTTPS port and `9443` to the instance.

apps/docs/docs/features/feature-flags.md

@@ -34,8 +34,11 @@ Administrators enable it from **Settings → Integrations → Automation**. CT-O
 ```
 
 CT-Ops does not start Docker containers from the web app or from `./start.sh`.
-Administrators configure the Ansible API URL and optional service-token HMAC
-settings on the same Automation page. The `ansible-api` container can run on
-the CT-Ops host, on a different host, or behind a reverse proxy.
+Administrators pair the Ansible API URL with the initial username and password
+configured on the `ansible-api` container. The container then generates a
+service-token secret for ongoing signed requests; CT-Ops stores that generated
+secret encrypted and does not retain the initial password. The `ansible-api`
+container can run on the CT-Ops host, on a different host, or behind a reverse
+proxy.
 
 When the API is healthy, administrators can save encrypted SSH private-key credential profiles and run an Ansible ping task from host or host-group task views. CT-Ops stores task state and redacted output in its task history; the Ansible container only executes the requested ping operation.