Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
fec4a8c
fix: require --section flag for ctx add task
parlakisik Apr 6, 2026
5ddedd0
feat: add hub foundation — store, auth, types
parlakisik Apr 6, 2026
77716a8
feat: add hub gRPC server with auth
parlakisik Apr 7, 2026
b85fb92
feat: add ctx serve --shared CLI command
parlakisik Apr 7, 2026
ff42a3e
feat: add hub client and ctx connect register
parlakisik Apr 7, 2026
adeebed
feat: add subscribe, sync commands and shared file renderer
parlakisik Apr 7, 2026
ddfd160
feat: add publish, listen, and status connect commands
parlakisik Apr 8, 2026
b6424ef
feat: add --include-shared flag to ctx agent
parlakisik Apr 8, 2026
842ba7f
feat: add --daemon and --stop flags to ctx serve --shared
parlakisik Apr 8, 2026
3f0331a
feat: add Raft leader election for hub clustering
parlakisik Apr 8, 2026
8bcb620
feat: add master-follower replication and client failover
parlakisik Apr 8, 2026
ef7eede
feat: add ctx hub status, peer, and stepdown commands
parlakisik Apr 8, 2026
0528ca0
test: add end-to-end integration tests for hub
parlakisik Apr 8, 2026
20c8c53
docs: add CLI reference for connect, serve --shared, and hub
parlakisik Apr 8, 2026
882d8e5
feat: add --share flag to ctx add for automatic hub publishing
parlakisik Apr 8, 2026
2aa2844
test: add unit tests for failover, fanout, and renderer
parlakisik Apr 9, 2026
df7dac6
feat: add auto-sync hook for shared hub entries
parlakisik Apr 9, 2026
173e1c3
fix: address code review findings for shared hub
parlakisik Apr 9, 2026
df82d33
docs: update connect and serve docs with review fixes
parlakisik Apr 9, 2026
9efe1a9
fix: reconcile hub code with main's audit tests after rebase
parlakisik Apr 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
130 changes: 124 additions & 6 deletions .context/TASKS.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,76 @@ TASK STATUS LABELS:

### Misc

- [ ] Improve hub failover client: distinguish auth errors (Unauthenticated/PermissionDenied) from connection errors. Fail fast on auth failures instead of cycling through all peers with the same invalid token. #priority:low #added:2026-04-08-194612

- [ ] Add file locking to ctx connect sync state to prevent concurrent sync races. Two sync processes (hook + manual) can both load the same LastSequence, process the same entries, and write duplicate content to .context/shared/. #priority:medium #added:2026-04-08-194557

- [ ] Fix fanout broadcast entry loss: non-blocking send drops entries to slow listeners silently. Log when entries are dropped. Consider per-listener backpressure or disconnect-on-lag. Buffer of 64 is too small for busy hubs. #priority:medium #added:2026-04-08-194542

- [ ] Prevent duplicate client registration in hub store: RegisterClient should reject if ProjectName already exists. Add token revocation support (delete client by ID/project). Currently tokens are valid forever with no way to disable compromised ones. #priority:medium #added:2026-04-08-194529

- [ ] Fix hub cluster: NewCluster result is discarded (not stored on Server), so Raft runs but leadership status is never queryable. Store cluster reference on Server, wire IsLeader/LeaderAddr into Status RPC and hub status command. #priority:medium #added:2026-04-08-194511

- [ ] Use crypto/subtle.ConstantTimeCompare for hub token validation instead of string equality. Current Store.ValidateToken uses == which is vulnerable to timing attacks. Also replace O(n) linear scan with a map[string]*ClientInfo for O(1) lookup. #priority:high #added:2026-04-08-194458

- [ ] Fix silent error suppression in hub: (1) ctx add --share silently ignores publish failures — warn user on failure, (2) hubsync hook swallows all errors — log to event system, (3) replication loop drops errors silently — add structured logging for debug. #priority:high #added:2026-04-08-194443

- [ ] Add input validation to hub Publish handler: reject empty ID, validate Type against allowed set (decision/learning/convention/task), enforce Content length limit (1MB), require non-empty Origin. Prevents garbage data and DoS via unbounded content. #priority:high #added:2026-04-08-194430

- [ ] Fix ctx connect listen: currently only does initial sync then blocks on ctx.Done() without ever calling the Listen RPC. Must stream entries in real-time via the server-streaming Listen RPC, writing to .context/shared/ as entries arrive. #priority:high #added:2026-04-08-194415

- [x] Remove any superpowers library references and implement all needed workflow mechanisms (brainstorm, plan, execute, review, subagent dispatch) natively in ctx. No external plugin libraries should be used — ctx must be self-contained. Clean up docs/superpowers/ directory and any remaining references. #priority:high #added:2026-04-06-121002 #done:2026-04-06

- [ ] SMB mount path support: add `CTX_BACKUP_MOUNT_PATH` env var so
`ctx backup` can use fstab/systemd automounts instead of requiring GVFS.
Spec: specs/smb-mount-path-support.md #priority:medium #added:2026-04-04-010000

### Architecture Docs

- [ ] Publish architecture docs to docs/: copy ARCHITECTURE.md,
DETAILED_DESIGN domain files, and CHEAT-SHEETS.md to docs/reference/.
Sanitize intervention points into docs/contributing/.
Exclude DANGER-ZONES.md and ARCHITECTURE-PRINCIPAL.md (internal only).
Spec: specs/publish-architecture-docs.md #priority:medium #added:2026-04-03-150000

- [ ] Update ctx-architecture skill to append discovered terms to GLOSSARY.md
during Phase 3. Additive only, max 10 terms per run, project-specific only,
alphabetical insertion, skip if GLOSSARY.md empty. Print added terms in
convergence report. Spec: specs/publish-architecture-docs.md #priority:low #added:2026-04-03-153000

### Code Cleanup Findings


- [x] Extend flagbind helpers (IntFlag, DurationFlag, DurationFlagP, StringP,
BoolP) and migrate ~50 call sites to unblock TestNoFlagBindOutsideFlagbind
#added:2026-04-01-233250

- [ ] Implement journal compaction: Elastic-style tiered storage with tar.gz
backup. Spec: specs/journal-compact.md #added:2026-03-31-110005

- [x] Refactor 28 grandfathered cmd/ purity violations found by
TestCmdDirPurity: move unexported helpers, exported non-Cmd/Run functions,
and types from cmd/ directories to core/. See grandfathered map in
compliance_test.go for the full list. #priority:medium #added:2026-03-31-005115


- [x] PD.4.5: Update AGENT_PLAYBOOK.md — add generic "check available skills"
instruction #priority:medium #added:2026-03-25-203340

**PD.5 — Validate:**


### Phase -3: DevEx

- [x] Plugin enablement gap: Ref:
`ideas/plugin-enablement-gap.md`. Local-installed plugins get
registered in `installed_plugins.json` but not auto-added to
`enabledPlugins`, so slash commands are invisible in non-ctx
projects.

- [x] Add cobra Example fields to CLI commands via
examples.yaml #added:2026-03-20-163413

- [x] Add CLI YAML drift detection test: verify flag names in
examples.yaml match actual registered flags, and Use: patterns
in commands.yaml match Use constants. Structural linkage is
Expand Down Expand Up @@ -165,13 +235,13 @@ Session-start checks, suppressibility, and registry for companion MCP tools.

### Phase CLI-FIX: CLI Infrastructure Fixes

- [ ] Bug: ctx add task appends to the last Phase section instead of a dedicated
- [x] Bug: ctx add task appends to the last Phase section instead of a dedicated
location. Tasks added via CLI land inside whatever Phase happens to be last in
TASKS.md, breaking Phase structure. Fix: add mandatory --phase flag to ctx add
task. If the named Phase section does not exist, create it. If --phase is
omitted, error with available Phase names. No fallback section — mandatory
placement forces intent at creation time.
#priority:high #added:2026-03-25-234813
TASKS.md, breaking Phase structure. Fix: add mandatory --section flag to ctx add
task. If the named section does not exist, create it. If --section is
omitted, error with message. Heading level fixed from ## to ### to match
TASKS.md structure.
#priority:high #added:2026-03-25-234813 #done:2026-04-06

### Phase BLOG: Blog Posts

Expand Down Expand Up @@ -951,3 +1021,51 @@ Not a fit (keep in `ctx`):
template placeholder instead of literal tool names. Define minimum interface
contract (query, context, impact). Spec:
`ideas/spec-mcp-warm-up-ceremony.md` #added:2026-03-25-120000

- [x] HUB-1: Define hub.proto — gRPC service definition with Register, Publish, Sync, Listen, Status RPCs. Generate Go code. Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-113020 #done:2026-04-06

- [x] HUB-2: Implement internal/hub/store.go — JSONL append-only entry storage with sequence assignment, type filtering, and since-sequence queries. Spec: specs/hub_implementation.md #priority:high #added:2026-04-06-113021 #done:2026-04-06

- [x] HUB-3: Implement internal/hub/auth.go — admin token generation on first run, client token issuance via Register RPC, gRPC interceptor for Bearer token validation. Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-113022 #done:2026-04-06

- [x] HUB-4: Implement internal/hub/server.go — gRPC server with Register, Publish, Sync RPCs. Wire auth interceptor, JSONL store, TLS support. Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-113024 #done:2026-04-06

- [x] HUB-5: Implement ctx serve --shared CLI command — starts gRPC hub server on specified port, generates admin token on first run, supports --tls-cert/--tls-key flags. Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-113030 #done:2026-04-06

- [x] HUB-6: Implement internal/hub/client.go — gRPC client with Register, Sync, Publish, Listen methods. Connection config encrypted storage via internal/crypto (same pattern as notify). Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-113032 #done:2026-04-06

- [x] HUB-7: Implement ctx connect register — one-time registration with hub, stores encrypted connection config in .context/.connect.enc. Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-113033 #done:2026-04-06

- [x] HUB-8: Implement ctx connect subscribe — set entry type filters (decisions, learnings, conventions), persist in local connection config. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113035 #done:2026-04-07

- [x] HUB-9: Implement ctx connect sync — initial full pull of matching entries from hub, write to .context/shared/ as markdown files with origin tags, record last-seen sequence in .sync-state.json. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113041 #done:2026-04-07

- [x] HUB-10: Implement ctx connect publish and --share flag — push local entries to hub. Add --share flag to ctx add so entries go to local file AND hub simultaneously. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113043 #done:2026-04-07

- [x] HUB-11: Implement Listen RPC with fan-out — server-streaming RPC that pushes new entries to connected clients in real-time. ctx connect listen with auto-reconnect on disconnect. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113044 #done:2026-04-07

- [x] HUB-12: Implement ctx connect status — show server address, connection state, last sync time, subscription config, entry counts by type. Includes hub-side Status RPC. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113046 #done:2026-04-07

- [x] HUB-13: Implement ctx agent --include-shared — add Tier 8 budget for shared knowledge in agent packet assembly. Shared entries from .context/shared/ included when --include-shared flag is passed. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113053 #done:2026-04-07

- [x] HUB-14: Implement --daemon flag for ctx serve --shared — background process with PID file, --stop to kill, graceful shutdown. Required for federation. Spec: specs/shared-hub-federation.md #priority:medium #added:2026-04-06-113054

- [x] HUB-15: Integrate hashicorp/raft for leader election — Raft-lite: use Raft ONLY for master election, not data consensus. --peers flag for cluster membership. Single-node mode auto-elects. Spec: specs/shared-hub-federation.md #priority:medium #added:2026-04-06-113056

- [x] HUB-16: Implement master-to-follower replication — master pushes entries to followers via gRPC stream. Followers catch up via sequence-based sync on reconnect. Spec: specs/shared-hub-federation.md #priority:medium #added:2026-04-06-113058

- [x] HUB-17: Implement client failover — clients maintain ordered peer list, auto-reconnect to new master on connection failure. Follower redirects client to current master address. Spec: specs/shared-hub-federation.md #priority:medium #added:2026-04-06-113104

- [x] HUB-18: Implement ctx hub status/peer/stepdown — cluster status display (role, peers, sync state, entries, uptime), runtime peer add/remove, graceful leadership transfer. Spec: specs/shared-hub-federation.md #priority:low #added:2026-04-06-113106

- [x] HUB-19: Update compliance test — add internal/hub/ to allowed-net-import list alongside internal/notify/. Core packages remain network-free. Spec: specs/hub_implementation.md #priority:high #added:2026-04-06-113107

- [x] HUB-20: End-to-end integration test — spin up hub, register 2 clients, publish from one, verify sync on other. Test --share flag, Listen stream, and reconnect behavior. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-113109

- [x] HUB-2a: Implement hub client registry and meta persistence — clients.json for registered client tokens/project names, meta.json for sequence counter and hub metadata. Separate from entries.jsonl. Spec: specs/shared-context-hub.md #priority:high #added:2026-04-06-114131

- [x] HUB-9a: Implement shared file renderer — convert Entry objects to markdown with origin tags and date headers, create/append to .context/shared/*.md files. Reused by both ctx connect sync and ctx connect listen. Spec: specs/shared-context-hub.md #priority:medium #added:2026-04-06-114131

- [x] HUB-21: Unit tests for internal/hub/ — store (append, query, rotation), auth (token generation, validation, interceptor), client (connect, reconnect), renderer (markdown output). Each package tested independently. Spec: specs/hub_implementation.md #priority:medium #added:2026-04-06-114131

- [x] HUB-22: Documentation — create docs/cli/connect.md and docs/cli/serve.md for new commands, update docs/cli/agent.md for --include-shared flag and --shared-budget option. Spec: specs/shared-context-hub.md #priority:low #added:2026-04-06-114131
120 changes: 120 additions & 0 deletions docs/cli/connect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
# / ctx: https://ctx.ist
# ,'`./ do you remember?
# `.,'\
# \ Copyright 2026-present Context contributors.
# SPDX-License-Identifier: Apache-2.0

title: Connect
icon: lucide/link
---

## `ctx connect`

Connect to a shared context hub for cross-project knowledge sharing.
Projects publish decisions, learnings, and conventions to a centralized
hub; other projects receive them alongside local context.

### `ctx connect register`

One-time registration with a shared hub. Requires the hub address and
admin token (printed by `ctx serve --shared` on first run).

```bash
ctx connect register localhost:9900 --token ctx_adm_7f3a...
```

On success, stores an encrypted connection config in
`.context/.connect.enc` for future RPCs.

### `ctx connect subscribe`

Set which entry types to receive from the hub. Only matching types
are returned by sync and listen.

```bash
ctx connect subscribe decision learning
ctx connect subscribe decision learning convention
```

### `ctx connect sync`

Pull matching entries from the hub and write them to
`.context/shared/` as markdown files with origin tags and date
headers. Tracks last-seen sequence for incremental sync.

```bash
ctx connect sync
```

### `ctx connect publish`

Push entries to the hub. Specify type and content as arguments.

```bash
ctx connect publish decision "Use UTC timestamps everywhere"
```

### `ctx connect listen`

Stream new entries from the hub in real-time. Writes to
`.context/shared/` as entries arrive. Press Ctrl-C to stop.

```bash
ctx connect listen
```

### `ctx connect status`

Show hub connection state and entry statistics.

```bash
ctx connect status
```

## Automatic sharing

Use `--share` on `ctx add` to write locally AND publish to the hub:

```bash
ctx add decision "Use UTC" --share \
--context "Need consistency" \
--rationale "Avoid timezone bugs" \
--consequence "UI does conversion"
```

If the hub is unreachable, the local write succeeds and a warning
is printed. The `--share` flag is best-effort — it never blocks
local context updates.

## Auto-sync

Once registered, the `check-hub-sync` hook automatically syncs
new entries from the hub at the start of each session (daily
throttled). No manual `ctx connect sync` needed.

## Shared files

Entries from the hub are stored in `.context/shared/`:

```
.context/shared/
decisions.md # Shared decisions with origin tags
learnings.md # Shared learnings
conventions.md # Shared conventions
.sync-state.json # Last-seen sequence tracker
```

These files are read-only (managed by sync/listen) and never
mixed with local context files.

## Agent integration

Include shared knowledge in agent context packets:

```bash
ctx agent --include-shared
```

Shared entries are included as Tier 8 in the budget-aware
assembly, scored by recency and type relevance.
40 changes: 40 additions & 0 deletions docs/cli/hub.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
# / ctx: https://ctx.ist
# ,'`./ do you remember?
# `.,'\
# \ Copyright 2026-present Context contributors.
# SPDX-License-Identifier: Apache-2.0

title: Hub
icon: lucide/network
---

## `ctx hub`

Manage a running shared context hub cluster.

### `ctx hub status`

Show cluster status: role, entry count, and connected projects.

```bash
ctx hub status
```

### `ctx hub peer`

Add or remove peers from the cluster at runtime.

```bash
ctx hub peer add host2:9901
ctx hub peer remove host2:9901
```

### `ctx hub stepdown`

Transfer leadership to another node gracefully. Use before
taking a node offline for maintenance.

```bash
ctx hub stepdown
```
4 changes: 3 additions & 1 deletion docs/cli/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ own guards and no-op gracefully.
| [`ctx learning`](context.md#ctx-learning) | Manage `LEARNINGS.md` (reindex) |
| [`ctx journal`](recall.md#ctx-recall) | Browse and export AI session history |
| [`ctx journal`](recall.md#ctx-journal) | Generate static site from journal entries |
| [`ctx serve`](recall.md#ctx-serve) | Serve any zensical directory (default: journal site) |
| [`ctx serve`](serve.md#ctx-serve) | Serve journal site or shared context hub |
| [`ctx watch`](tools.md#ctx-watch) | Auto-apply context updates from AI output |
| [`ctx setup`](tools.md#ctx-hook) | Generate AI tool integration configs |
| [`ctx loop`](tools.md#ctx-loop) | Generate autonomous loop script |
Expand All @@ -81,6 +81,8 @@ own guards and no-op gracefully.
| [`ctx hook`](tools.md#ctx-hook) | Manage lifecycle hooks (shell scripts for automation) |
| [`ctx skill`](tools.md#ctx-skill) | Manage reusable instruction bundles |
| [`ctx config`](config.md#ctx-config) | Manage runtime configuration profiles |
| [`ctx connect`](connect.md#ctx-connect) | Connect to a shared context hub |
| [`ctx hub`](hub.md#ctx-hub) | Manage hub cluster (status, peer, stepdown) |
| [`ctx system`](system.md#ctx-system) | System diagnostics and hook commands |

---
Expand Down
4 changes: 4 additions & 0 deletions docs/cli/init-status.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ ctx agent [flags]
| `--format` | md | Output format: `md` or `json` |
| `--cooldown` | 10m | Suppress repeated output within this duration (requires `--session`) |
| `--session` | (none) | Session ID for cooldown isolation (e.g., `$PPID`) |
| `--include-shared` | false | Include shared hub entries from `.context/shared/` |

**How budget works**:

Expand All @@ -112,6 +113,9 @@ in priority tiers:
3. **Conventions**: all conventions, up to 20% of budget
4. **Decisions**: scored by recency and relevance to active tasks
5. **Learnings**: scored by recency and relevance to active tasks
6. **Steering**: applicable steering file bodies
7. **Skill**: named skill content (from `--skill`)
8. **Shared**: hub entries from `.context/shared/` (with `--include-shared`)

Decisions and learnings are ranked by a combined score (how recent + how
relevant to your current tasks). High-scoring entries are included with
Expand Down
Loading
Loading