Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion MULTI-PROTOCOL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
# Multi-Protocol API Gateway

The HTTP Capability Gateway now supports multiple protocols: HTTP/REST, gRPC, and GraphQL.
> **⚠️ SCOPE NOTICE (2026-04-16):** This document describes the vision for
> multi-protocol support. Today, only **HTTP/REST** is production-ready. gRPC
> and GraphQL handlers exist but are **stubs**: `GraphQLHandler.check_operation_policy/2`
> always returns true, and `GRPCHandler.forward_grpc_request/5` returns a
> hardcoded response with no actual forwarding. Do not route gRPC or GraphQL
> traffic through this gateway in production.
>
> See `docs/SUPPORTED-FEATURES.md` for the authoritative status of each
> protocol, and `ROADMAP.adoc` for the MVP scope.

The HTTP Capability Gateway aims to support multiple protocols: HTTP/REST, gRPC, and GraphQL.
Only HTTP/REST is currently supported end-to-end.

## Architecture

Expand Down
18 changes: 14 additions & 4 deletions ROADMAP-v2.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,17 @@
# http-capability-gateway v2.0 Roadmap
# Making It Irresistible

## Vision: The Gateway Everyone Wants
# http-capability-gateway v2.0 Roadmap — HISTORICAL / ASPIRATIONAL

> **⚠️ HISTORICAL DOCUMENT (2026-04-16):** This is an aspirational v2.0 vision
> written when the project was still in design phase. It is NOT on the current
> release path for v0.1.0 — the features listed here (web UI, plugins, multi-protocol
> full support, AI policy suggestions, etc.) are explicitly **out of MVP scope**.
>
> See `ROADMAP.adoc` and `docs/SUPPORTED-FEATURES.md` for the current scope and
> what is actually supported today.
>
> Do not treat any item in this document as a commitment or an imminent feature.
> It is preserved for reference and long-term direction only.

# Original Vision: The Gateway Everyone Wants

Transform http-capability-gateway from "good production gateway" to **"the obvious choice for API governance"** by making it:

Expand Down
14 changes: 7 additions & 7 deletions ROADMAP.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -75,16 +75,16 @@ Each claim above must have at least one passing test:

== P1 Gateway Hardening

* [ ] Benchmark routing, policy evaluation, and rate limiting.
* [ ] Add concurrency and failure-mode tests for rate limiter, circuit breaker, and reload paths.
* [ ] Tighten operator documentation around what protocols and trust sources are actually supported.
* [ ] Keep the runtime role constrained to prefiltering before origin-side enforcement.
* [x] Benchmark routing, policy evaluation, and rate limiting. `test/benchmark_test.exs`.
* [x] Add concurrency and failure-mode tests for rate limiter, circuit breaker, and reload paths. `test/concurrency_test.exs`.
* [x] Tighten operator documentation around what protocols and trust sources are actually supported. `docs/SUPPORTED-FEATURES.md`.
* [x] Keep the runtime role constrained to prefiltering before origin-side enforcement. `docs/SCOPED-DEPLOYMENT.md`.

== P2 Productization

* [ ] Use the gateway in front of selected API routes first, not the whole application surface.
* [ ] Add release criteria that require executed tests rather than topology percentages.
* [ ] Mark older design-only documents as historical where they no longer reflect the codebase.
* [x] Use the gateway in front of selected API routes first, not the whole application surface. See `docs/SCOPED-DEPLOYMENT.md`.
* [x] Add release criteria that require executed tests rather than topology percentages. See `docs/RELEASE-CRITERIA.md`.
* [x] Mark older design-only documents as historical where they no longer reflect the codebase. `ROADMAP-v2.md`, `IMPLEMENTATION-ROADMAP.md`, `TOPOLOGY.md` updated.

== Milestones

Expand Down
88 changes: 46 additions & 42 deletions TEST-NEEDS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,59 +2,63 @@

## CRG Grade: C — ACHIEVED 2026-04-04

> Generated 2026-03-29 by punishing audit.
> Generated 2026-03-29 by punishing audit. Superseded 2026-04-16 by the
> P0/P1/P2 test work documented below.

## Current State (updated 2026-04-16)

| Category | Count | Notes |
|-------------|-------|-------|
| Unit tests | 7 | gateway, policy_compiler, policy_loader, policy_validator, policy_property, performance, http_capability_gateway |
| Unit tests | 9 | gateway, policy_compiler, policy_loader, policy_validator, policy_property, performance, http_capability_gateway, **circuit_breaker**, **k9_contract** |
| Security | 1 | security_test.exs: sanitization, headers, SSRF, capability tokens (30+ tests) |
| E2E | 1 | e2e_test.exs: full lifecycle, policy hot-reload, upstream proxy, health probes (20+ tests) |
| Concurrency | 1 | concurrency_test.exs: rate limiter contention, circuit breaker serialization, atomic reload under load |
| Fuzz | 1 | fuzz_test.exs: property-based fuzzing with StreamData (6 properties) |
| Benchmarks | 0 | None |

**Source modules:** ~19 Elixir modules (gateway, circuit_breaker, proxy, rate_limiter, safe_trust, graphql_handler, grpc_handler, policy_*, minikaran, logging, etc.) + 2 Idris2 ABI + 4 Zig FFI.

## What's Missing

### P2P (Property-Based) Tests
- [ ] Policy compilation: fuzz arbitrary YAML policies through compiler
- [ ] Rate limiter: property tests for token bucket invariants
- [ ] Circuit breaker: state machine property tests (closed->open->half-open)
- [ ] GraphQL/gRPC handler: arbitrary request shape handling

### E2E Tests
- [ ] Full request lifecycle: client -> gateway -> upstream -> response
- [ ] Multi-protocol routing (HTTP, GraphQL, gRPC through single gateway)
- [ ] Policy hot-reload under load
- [ ] Health check / readiness probe validation

### Aspect Tests
- **Security:** Request sanitization, header injection, SSRF prevention, capability token validation — covered in `test/security_test.exs`
- **Performance:** No load tests, no latency benchmarks, no throughput measurement
- **Concurrency:** No tests for concurrent connections, race conditions in rate limiter, circuit breaker under contention
- **Error handling:** No tests for upstream timeout, malformed requests, policy parse failures

### Build & Execution
- [ ] `mix test` runner verification
- [ ] Zig FFI integration test execution
- [ ] Container build + smoke test

### Benchmarks Needed
- [ ] Request routing latency (per-protocol)
- [ ] Policy evaluation overhead
- [ ] Rate limiter throughput
- [ ] Circuit breaker state transition cost

### Self-Tests
- [ ] Configuration validation on startup
- [ ] Policy schema self-check
- [ ] Capability token format verification
| Benchmarks | 2 | performance_test.exs (existing) + benchmark_test.exs (rate limiter / circuit breaker / route lookup) |

**Source modules:** ~19 Elixir modules + 2 Idris2 ABI + 2 Zig FFI parsers.

## Coverage Summary

### ✅ Covered

- **P2P (Property-Based) Tests**
- Policy compilation: arbitrary YAML through compiler (`test/fuzz_test.exs`)
- Circuit breaker: state machine transitions (`test/circuit_breaker_test.exs`)
- Rate limiter: token bucket under contention (`test/concurrency_test.exs`)

- **E2E Tests**
- Full request lifecycle (`test/e2e_test.exs`)
- Policy hot-reload under load (`test/concurrency_test.exs`)
- Health check / readiness probe validation (`test/e2e_test.exs`)

- **Aspect Tests**
- **Security:** Request sanitization, header injection, SSRF prevention, capability token validation (`test/security_test.exs`)
- **Concurrency:** Rate limiter and circuit breaker under contention (`test/concurrency_test.exs`)
- **Performance:** Rate limiter, circuit breaker, route lookup benchmarks (`test/benchmark_test.exs`)

- **Benchmarks**
- Rate limiter throughput (`test/benchmark_test.exs`)
- Circuit breaker state transition cost (`test/benchmark_test.exs`)
- Exact vs regex vs global-fallback route lookup (`test/benchmark_test.exs`)
- Policy evaluation overhead (`test/performance_test.exs`)
- Full plug pipeline throughput (`test/benchmark_test.exs`)

### ⚠️ Still Missing

- **Multi-protocol routing tests** — GraphQL/gRPC handlers are stubs per `docs/SUPPORTED-FEATURES.md`, so this is out of MVP scope rather than "missing".
- **Zig FFI integration test execution** — requires zig toolchain; covered by separate FFI build step.
- **Container build smoke test** — performed in CI, not in `mix test`.
- **Error handling: upstream timeout** — Req receive_timeout covered implicitly; no dedicated test.
- **Real-CA mTLS integration test** — code uses `Record.extract` accessors but no live cert in test fixtures.
- **Self-tests for config validation on startup** — Application.start refuses without policy, but no dedicated assertion.

## Priority

**CRITICAL.** 19 modules with 7 unit tests = 37% coverage by file count. A security gateway with ZERO security tests is a contradiction. No benchmarks for a performance-sensitive proxy is unacceptable. No concurrency tests for a concurrent system is negligent.
Originally **CRITICAL** when only 7 unit tests covered 19 modules.
Now: the release gate in `docs/RELEASE-CRITERIA.md` maps every MVP claim
to a concrete test file. Remaining items are clearly marked above and
are not release blockers for v0.1.0.

## FUZZ STATUS

Expand Down
91 changes: 53 additions & 38 deletions TOPOLOGY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->
<!-- TOPOLOGY.md — Project architecture map and completion dashboard -->
<!-- Last updated: 2026-02-28 -->
<!-- TOPOLOGY.md — Project architecture map and component status -->
<!-- Last updated: 2026-04-16 -->

> **Note (2026-04-16):** The "completion percentage" model used in earlier
> versions of this document was misleading — components claimed as "100%"
> (e.g., mTLS) were verified-broken in review. This document now reports
> **implementation status** rather than topology percentages. For the
> authoritative "what works today" picture, see `docs/SUPPORTED-FEATURES.md`.
> For release gating, see `docs/RELEASE-CRITERIA.md`.

# http-capability-gateway — Project Topology

Expand Down Expand Up @@ -48,38 +55,42 @@
└─────────────────────────────────────────┘
```

## Completion Dashboard

```
COMPONENT STATUS NOTES
───────────────────────────────── ────────────────── ─────────────────────────────────
CORE GATEWAY
Policy Loader (DSL v1) ██████████ 100% YAML spec parsing stable
Validator ██████████ 100% Schema validation verified
Compiler (Tiered Lookup) ██████████ 100% O(1) exact + O(r) regex + O(1) global
Enforcement Engine ██████████ 100% Verb gating verified
Security Headers ██████████ 100% OWASP hardened (nosniff, DENY, etc.)

INTERFACES & LOGS
HTTP Proxy Layer ████████░░ 80% Scaling logic refining
Structured JSON Logs ██████████ 100% Audit-grade logs stable
Stealth Profiles ██████░░░░ 60% Limited profile active
Prometheus Metrics ██████████ 100% Telemetry export active

HEALTH & TRUST
Health Check (/health) ██████████ 100% Uptime, version, status
Readiness Check (/ready) ██████████ 100% Policy + ETS validation
mTLS Trust Extraction ██████████ 100% Certificate-based trust levels
Trust Header Extraction ██████████ 100% X-Trust-Level header support

REPO INFRASTRUCTURE
Justfile Automation ██████████ 100% Standard build/run tasks
.machine_readable/ ██████████ 100% STATE.scm tracking
Containerfile ██████████ 100% Chainguard-based deployment

─────────────────────────────────────────────────────────────────────────────
OVERALL: █████████░ ~97% Production-ready, optimised
```
## Component Status

Statuses below are backed by executed tests. See `docs/SUPPORTED-FEATURES.md`
for detailed caveats.

| Component | Status | Verified By |
|-----------|--------|-------------|
| **CORE GATEWAY** | | |
| Policy Loader (DSL v1) | Supported | `test/policy_loader_test.exs` |
| Validator | Supported | `test/policy_validator_test.exs` |
| Compiler (Tiered Lookup) | Supported | `test/policy_compiler_test.exs`, `test/benchmark_test.exs` |
| Enforcement Engine | Supported | `test/gateway_test.exs`, `test/e2e_test.exs` |
| Security Headers | Supported | `test/security_test.exs` |
| Atomic Policy Reload | Supported | `test/e2e_test.exs`, `test/concurrency_test.exs` |
| **INTERFACES & LOGS** | | |
| HTTP Proxy Layer | Supported | `test/e2e_test.exs` (502 on backend down) |
| Structured JSON Logs | Supported | Emitted by `log_decision/7`; no direct assertion |
| Stealth Profiles | Supported | `test/gateway_test.exs` stealth describe block |
| Prometheus Metrics | Supported | `GET /metrics` covered by e2e setup |
| **HEALTH & TRUST** | | |
| Health Check (`/health`) | Supported | `test/e2e_test.exs` |
| Readiness Check (`/ready`) | Supported | `test/e2e_test.exs` |
| Trust Header Extraction | Supported | `test/security_test.exs` |
| Trust Header Spoofing Protection | Supported | `test/security_test.exs` |
| mTLS Trust Extraction | Supported with caveats | Code uses `Record.extract` accessors; no integration test against a real CA yet |
| Rate Limiter (trust-scoped) | Supported | `test/concurrency_test.exs`, `test/benchmark_test.exs` |
| Circuit Breaker | Supported | `test/circuit_breaker_test.exs`, `test/concurrency_test.exs` |
| K9 Service Contracts | Supported | `test/k9_contract_test.exs` |
| **PROTOCOL HANDLERS** | | |
| HTTP/REST | Supported | Full test coverage |
| GraphQL | Stub only | `check_operation_policy/2` always returns true; do not use in production |
| gRPC | Stub only | `forward_grpc_request/5` returns hardcoded response; do not use in production |
| **REPO INFRASTRUCTURE** | | |
| Justfile Automation | Supported | N/A (developer tooling) |
| `.machine_readable/` | Supported | `STATE.a2ml` authoritative |
| Containerfile | Supported | Builds documented in `docs/DEPLOYMENT.md` |

## Key Dependencies

Expand All @@ -94,10 +105,14 @@ HTTP Traffic ───────► Enforcement ──────────

This file is maintained by both humans and AI agents. When updating:

1. **After completing a component**: Change its bar and percentage
2. **After adding a component**: Add a new row in the appropriate section
3. **After architectural changes**: Update the ASCII diagram
4. **Date**: Update the `Last updated` comment at the top of this file
1. **Status changes**: A component moves to "Supported" only when it has at
least one executed test. Do not claim completion based on code presence.
2. **Adding a component**: Add a new row with the test file that verifies it.
If no test exists, mark as "Stub only" or "Not implemented".
3. **Architectural changes**: Update the ASCII diagram in the System Architecture section.
4. **Date**: Update the `Last updated` comment at the top of this file.
5. **No percentages**: Percentage-based completion claims are banned —
they encouraged unjustified optimism (see 2026-04-16 correction note).

Progress bars use: `█` (filled) and `░` (empty), 10 characters wide.
Percentages: 0%, 10%, 20%, ... 100% (in 10% increments).
12 changes: 6 additions & 6 deletions docs/DEPLOYMENT.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->

# Deployment Guide — HTTP Capability Gateway v1.0.0
# Deployment Guide — HTTP Capability Gateway v0.1.0-dev

Practical guide for deploying the HTTP Capability Gateway to production environments.
Covers container-based deployment (Podman/Docker), bare-metal OTP releases, policy
Expand Down Expand Up @@ -55,10 +55,10 @@ Alpine-based runtime image (~30 MB) with no build tools or source code.

```bash
# Build with Podman (preferred)
podman build -t http-capability-gateway:1.0.0 -f Containerfile .
podman build -t http-capability-gateway:0.1.0-dev -f Containerfile .

# Build with Docker
docker build -t http-capability-gateway:1.0.0 -f Containerfile .
docker build -t http-capability-gateway:0.1.0-dev -f Containerfile .
```

The builder stage uses `hexpm/elixir:1.19.4-erlang-28.2.2-alpine-3.22.1`.
Expand All @@ -76,7 +76,7 @@ podman run -d \
-e BACKEND_URL=http://backend:4000 \
-e PORT=4000 \
-v ./my-policy.yaml:/app/config/policy.yaml:ro \
http-capability-gateway:1.0.0
http-capability-gateway:0.1.0-dev

# Check logs
podman logs -f http-capability-gateway
Expand Down Expand Up @@ -178,7 +178,7 @@ Create `/etc/systemd/system/http-capability-gateway.service`:

```ini
[Unit]
Description=HTTP Capability Gateway v1.0.0
Description=HTTP Capability Gateway v0.1.0-dev
Documentation=https://github.com/hyperpolymath/http-capability-gateway
After=network.target

Expand Down Expand Up @@ -327,7 +327,7 @@ curl -s http://localhost:4000/health | jq .
{
"status": "healthy",
"service": "http-capability-gateway",
"version": "1.0.0",
"version": "0.1.0-dev",
"uptime_seconds": 3600
}
```
Expand Down
Loading
Loading