Skip to content

Commit 616b4d1

Browse files
authored
Feat/m365 mcp (#83)
* Add cross-platform M365 / Agent 365 HTTP MCP integration - Catalog of 21 Agent 365 MCP servers (scripts/m365-mcp-servers.json), populated from the live discoverToolServers endpoint with per-server url + scope baked in. URL pattern is /agents/servers/<name> (verified against discovery), not /agents/tenants/<tid>/servers/<name>. - Cross-platform setup scripts (TypeScript via tsx; run on Linux, macOS, WSL, Windows native, Git Bash): - scripts/setup-m365-app.ts: create / reuse / adopt single-tenant Entra app registration, declare per-server delegated scopes, attempt admin consent, persist state to ~/.hyperagent/m365.json. - scripts/m365-setup.ts: write one HTTP MCP entry per selected service into ~/.hyperagent/config.json with narrow per-server scopes (e.g. McpServers.Mail.All). - scripts/m365-refresh-servers.ts: refresh the catalog from the live discovery endpoint using a cached token. - scripts/m365-show.ts: print the saved app details. - scripts/mcp-add-http.ts: generic HTTP MCP entry writer (any vendor). - Justfile recipes are now plain delegates to tsx (no [unix] attribute, no inline bash) so they run identically across all supported shells. - src/agent/mcp/retry-fetch.ts: HTTP retry middleware (429/502/503/504 + network errors, exponential backoff, capped Retry-After) wired into StreamableHTTPClientTransport via the SDK's fetch option. - src/agent/mcp/session-cache.ts: Mcp-Session-Id persistence at ~/.hyperagent/mcp-sessions/<server>.json so reconnects can reattach to an existing session; cleared on connect failure. - extractContent now handles three Agent 365 response shapes via the new extractEmbeddedJson helper: clean JSON, status-prefixed JSON (Calendar), and {rawResponse: '...'} wrappers (Mail). - 16 new tests covering retry-fetch behaviour and the embedded-JSON extraction patterns. Full suite: 2197 passing. * Fix build-modules.js fresh-checkout failure scripts/build-modules.js was regenerating ha-modules.d.ts BEFORE running tsc. On a fresh checkout the gitignored builtin-modules/*.d.ts files don't exist, so the regenerated ha-modules.d.ts only contained the 4 native module declarations — wiping the up-to-date committed file. tsc then failed with cascading TS2305 errors: src/pptx-charts.ts: Module 'ha:ooxml-core' has no exported member '_createShapeFragment' (and similar for ShapeFragment, isShapeFragment, fragmentsToXml, MAX_CHARTS_PER_DECK, etc.) CI didn't catch this because it relied on cached / pre-built .d.ts files surviving between runs. The dts-sync test (which would catch drift) only runs after a successful build. Fix: swap the order. Run tsc first using the committed (correct) ha-modules.d.ts to emit fresh per-module .d.ts files, THEN regenerate ha-modules.d.ts from those fresh files. The regenerated output now matches committed byte-for-byte, and dts-sync.test.ts continues to catch drift. * Add cross-platform 'just clean' that wipes generated build outputs Previously 'just clean' only removed dist/ and node_modules/. The real landmines after a failed build are: - gitignored builtin-modules/*.{js,d.ts} (except the committed _save.js / _restore.js) - gitignored plugins/*/index.d.ts and plugins/shared/*.js - generated plugins/host-modules.d.ts and plugin-schema-types.d.ts - a clobbered (tracked) builtin-modules/src/types/ha-modules.d.ts — 'git pull' refuses to overwrite it because it has local changes from the broken build, so subsequent setups keep failing The new recipe wipes all of these and restores ha-modules.d.ts from HEAD, so a single 'just clean && just setup' recovers from any mid-build failure on every supported platform (unix + windows variants provided). * Fix M365 MCP URLs: inject tenantId into path The Agent 365 gateway requires a tenant-scoped path: https://<host>/agents/tenants/<tenantId>/servers/<name> The discoverToolServers endpoint returns un-tenanted URLs of the form https://<host>/agents/servers/<name> because the tenant comes from the caller's context. We were storing the discovery URL verbatim in ~/.hyperagent/config.json — which made the gateway respond: {"code":"EndpointInvalid", "message":"Tenant id is invalid.", "innererror":{"code":"TenantIdInvalid"}} (note the double space — empty tenantId substitution). Fix: m365-setup.ts now rewrites each catalog URL at config-write time via injectTenantIntoUrl(), splicing /tenants/<tenantId> into the path. The catalog itself still stores the canonical discovery URL so it stays tenant-agnostic and re-usable across users. Apologies for previously concluding the catalog URL pattern was correct based on raw curl tests — the SDK's full handshake reaches a deeper code path that surfaces the real tenant requirement. * feat(mcp): replace hand-rolled OAuth with MSAL, add device-code flow Replace the hand-rolled browser-oauth.ts and device-code-oauth.ts with @azure/msal-node's PublicClientApplication. MSAL handles PKCE, token caching, refresh, and redirect URIs correctly out of the box. Browser flow now uses http://localhost (ephemeral port, no /callback path) which matches the redirect URI registered on MSAL-compatible Entra apps (FOCI / VS Code / az CLI). This fixes AADSTS50011 redirect URI mismatch when using the VS Code app ID. Device-code flow uses acquireTokenByDeviceCode — prints verification URL + user code to stderr, no redirect URI needed. Changes: - Add @azure/msal-node dependency - New src/agent/mcp/auth/msal-oauth.ts (MSAL provider + file cache) - Delete browser-oauth.ts and device-code-oauth.ts - MCPOAuthConfig: flow is required (browser|device-code), callbackPort replaced with optional redirectUri - client-manager: single connectWithMsal replaces two methods - Scripts/Justfile: FLOW arg required, CALLBACK_PORT removed - Tests: MSAL provider tests, flow validation tests Tested: just check passes (2198/2198 tests, lint clean) * fix(mcp): use resource .default scope, print auth URL as fallback - Use ea9ffc3e-.../.default (Agent 365 resource) instead of per-server scopes. Per-server scopes aren't fully qualified so MSAL falls back to Graph, which breaks with FOCI apps. Matches a365cli behaviour. - Always print the auth URL to stderr so users can copy/paste when xdg-open isn't available (headless distros, SSH, no browser setup). - Remove callbackPort from catalog JSON. Tested: browser flow with VS Code FOCI app — works end-to-end. * fix: MSAL cache reader in refresh-servers, remove bogus plugin reconfigure hint - m365-refresh-servers: read MSAL .msal.json cache format only, drop legacy {savedAt, tokens} format (deleted provider wrote that) - slash-commands: remove hardcoded allowedContentTypes reconfigure suggestion from /plugin enable — was a fake example that doesn't exist on most plugins * feat(mcp): LLM-driven server connect, auto-enable gateway, MCP skill Enable the LLM to discover and connect MCP servers autonomously: 1. manage_mcp('connect') now works end-to-end: - Pre-approved servers connect silently - Unapproved + interactive TTY → prompts user for approval - Unapproved + no TTY → refuses with clear error - Mirrors the approval flow from /mcp enable 2. MCP gateway auto-enables on startup when servers are configured (the plugin is a boolean sentinel — zero risk, no audit needed) 3. list_mcp_servers() ungated — works before gateway is enabled so the LLM can discover servers without user running /plugin enable 4. System prompt: dynamic MCP section — concise hint when servers are configured, tells LLM to use tools for discovery instead of dumping a static docs block 5. Generic MCP skill (skills/mcp-services/SKILL.md) — teaches the full workflow: discover → connect → get schemas → import → call 6. mcp-setup-m365 pre-approves configured servers in the approval store so the LLM can connect them without interactive prompts Target flow: user asks 'What is in Teams?' → LLM discovers servers → connects work-iq-teams (pre-approved) → calls tool → returns data. Tested: just check passes (2198/2198 tests, lint clean) * fix(mcp): bump MAX_MCP_SERVERS from 20 to 50 M365 catalog has 21 servers alone — adding GitHub/filesystem pushed over the limit. Config parsing failed silently, gateway didn't auto- enable, LLM had no MCP awareness. * fix(mcp): don't block tool calls on interactive auth manage_mcp('connect') now checks if MSAL can acquire a token silently before attempting connection. If interactive auth (browser/device-code) would be needed, it returns immediately with an error telling the LLM to direct the user to /mcp enable <name> instead of hanging on a browser window inside a tool call. Once the user has authenticated once via /mcp enable, subsequent manage_mcp('connect') calls use the cached token and connect instantly. Added canAcquireSilently() to msal-oauth.ts — tries acquireTokenSilent and returns a boolean without triggering interactive flows. * fix(mcp): auto-approve gateway plugin on auto-enable The MCP gateway plugin source hash changes on every npm install (rebuild), invalidating the old approval. syncPluginsToSandbox then refuses to load it. Fix: when auto-enabling the gateway, set a synthetic audit result and approve with the current content hash. The plugin is a boolean sentinel (returns true from a single function) so auto-approving is zero risk. * fix(mcp): safe auth in yolo mode, /mcp enable as suggested command Three fixes: 1. /mcp enable refuses OAuth servers in --auto-approve/yolo mode when no cached silent token exists. Prevents opening a browser that nobody is watching in CI/pipeline scenarios. 2. Add /mcp enable to ACTIONABLE_COMMAND_PREFIXES so the LLM's suggestion to run '/mcp enable work-iq-teams' gets picked up by extractSuggestedCommands and offered as a one-click [Y/n] prompt. This closes the 'user has to manually type the command' gap. 3. Fix 'too many servers' test to match bumped MAX_MCP_SERVERS (50). Auth safety model: - manage_mcp tool call: canAcquireSilently check → refuses if interactive needed - /mcp enable (interactive): browser opens, user authenticates → fine - /mcp enable (yolo): canAcquireSilently check → refuses with clear message - Once authenticated: cached tokens → everything works silently * feat(mcp): write-safety gate + docs overhaul Write-safety gate: - Capture tool annotations (readOnlyHint, destructiveHint, etc.) from MCP listTools() into MCPToolSchema - Intercept non-read-only tool calls in plugin-adapter before execution - Interactive TTY: prompt user 'Allow? [y/n]' with tool name + args - --auto-approve: allow all operations (yolo is yolo) - No TTY + no auto-approve: refuse with clear error - Gate is transparent to the LLM — it sees results or error objects Docs (MCP.md): - Rewrite HTTP/OAuth section: MSAL, flow field, redirect URI, scopes - Rewrite M365 section: VS Code FOCI app, mcp-setup-m365 recipes, auth flows (browser/device-code), scope (.default), pre-approval - Add write-safety gate section with decision matrix - Update quick start: gateway auto-enables, LLM-driven discovery - Update architecture diagram with gate - Fix stale references (callbackPort, hand-rolled PKCE, old recipe names) - Bump max servers to 50 Tested: just check passes (2198/2198 tests, lint clean) * fix(mcp): session-remember tool approvals, fix gateway hash on startup 1. Write-safety gate now remembers approved tools for the session. First call to SearchMessages prompts [y/n], subsequent calls to the same tool skip the prompt. Avoids prompting on every read when servers don't provide readOnlyHint annotations. 2. Pre-approve MCP gateway at discovery time (before any sync), not just at auto-enable time. Fixes the 'REFUSING to load' error on startup when the plugin hash changed since last session. * fix(mcp): address PR #83 review feedback (4 comments) 1. Header validation: verify all header values are strings with non-empty keys, not just that headers is an object. 2. Config hash: include flow, tenantId, scopes, redirectUri, clientSecretEnv, and header keys in the SHA-256 hash. Approval is now invalidated on any meaningful HTTP config change. Updated m365-setup.ts hash computation to match. 3. Require scopes for OAuth: resolveScopes() throws if no scopes configured instead of silently using only offline_access. Config validator now rejects missing/empty scopes for OAuth. 4. Non-interactive connect: use canAcquireSilently() instead of acquireMsalToken() in non-interactive mode. Never falls back to interactive auth in headless/no-TTY scenarios. All test fixtures updated for required scopes. 2198/2198 pass.
1 parent 2ff2fcd commit 616b4d1

27 files changed

Lines changed: 5106 additions & 170 deletions

Justfile

Lines changed: 123 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -301,8 +301,36 @@ check: lint-all test-all
301301
@echo "✅ All checks passed — you may proceed to commit"
302302

303303
# Clean build artifacts (keeps deps/)
304+
#
305+
# Removes:
306+
# - node_modules and dist (npm/binary outputs)
307+
# - generated builtin-modules/*.{js,d.ts,d.ts.map} (preserves
308+
# _save.js / _restore.js which ARE committed)
309+
# - generated plugin .d.ts files and plugins/shared/*.js
310+
# - generated plugins/host-modules.d.ts
311+
#
312+
# Use this when a previous build failed mid-way and left stale
313+
# generated files that confuse `just setup` / `just build`.
314+
[unix]
304315
clean:
316+
#!/usr/bin/env bash
317+
set -euo pipefail
305318
rm -rf dist node_modules
319+
# Wipe gitignored builtin-modules build outputs (keep _save.js / _restore.js)
320+
find builtin-modules -maxdepth 1 \
321+
\( -name '*.js' -o -name '*.d.ts' -o -name '*.d.ts.map' \) \
322+
! -name '_save.js' ! -name '_restore.js' -delete 2>/dev/null || true
323+
# Wipe gitignored plugin build outputs
324+
find plugins -maxdepth 3 -name '*.d.ts' -delete 2>/dev/null || true
325+
find plugins/shared -maxdepth 1 -name '*.js' -delete 2>/dev/null || true
326+
rm -f plugins/host-modules.d.ts plugins/plugin-schema-types.d.ts
327+
# Restore committed ha-modules.d.ts in case a failed build clobbered it
328+
git checkout -- builtin-modules/src/types/ha-modules.d.ts 2>/dev/null || true
329+
echo "🧹 Cleaned build artefacts"
330+
331+
[windows]
332+
clean:
333+
if (Test-Path dist) { Remove-Item -Recurse -Force dist }; if (Test-Path node_modules) { Remove-Item -Recurse -Force node_modules }; Get-ChildItem builtin-modules -File | Where-Object { ($_.Extension -in '.js','.ts','.map') -and ($_.Name -notin '_save.js','_restore.js') } | Remove-Item -Force -ErrorAction SilentlyContinue; Get-ChildItem plugins -Recurse -Filter '*.d.ts' | Remove-Item -Force -ErrorAction SilentlyContinue; Get-ChildItem plugins/shared -Filter '*.js' -ErrorAction SilentlyContinue | Remove-Item -Force -ErrorAction SilentlyContinue; if (Test-Path plugins/host-modules.d.ts) { Remove-Item plugins/host-modules.d.ts }; if (Test-Path plugins/plugin-schema-types.d.ts) { Remove-Item plugins/plugin-schema-types.d.ts }; git checkout -- builtin-modules/src/types/ha-modules.d.ts 2>$null; Write-Output "🧹 Cleaned build artefacts"
306334

307335
# Clean everything including deps/ symlinks
308336
clean-all: clean
@@ -712,7 +740,12 @@ mcp-show-config:
712740
if (cfg.mcpServers) {
713741
console.log('Configured MCP servers:');
714742
for (const [name, s] of Object.entries(cfg.mcpServers)) {
715-
console.log(' ' + name + ': ' + (s.command || '?') + ' ' + (s.args || []).join(' '));
743+
if (s.type === 'http') {
744+
const auth = s.auth ? ' [' + s.auth.method + ']' : '';
745+
console.log(' ' + name + ': ' + s.url + auth);
746+
} else {
747+
console.log(' ' + name + ': ' + (s.command || '?') + ' ' + (s.args || []).join(' '));
748+
}
716749
}
717750
} else {
718751
console.log('No MCP servers configured.');
@@ -787,3 +820,92 @@ mcp-setup-workiq:
787820
echo " /mcp enable workiq"
788821
echo ""
789822
echo " First tool call opens a browser for Microsoft sign-in."
823+
824+
# ── Generic HTTP MCP server recipe ───────────────────────────────────
825+
#
826+
# Adds a single HTTP MCP server entry to ~/.hyperagent/config.json. Used
827+
# directly for ad-hoc HTTP MCP servers, and also called per-service by
828+
# `mcp-setup-m365` below.
829+
#
830+
# Args:
831+
# NAME Config key (becomes the alias for /mcp enable <NAME>).
832+
# URL HTTPS endpoint of the MCP server.
833+
# CLIENT_ID Optional. If set, OAuth is configured (and FLOW becomes required).
834+
# TENANT_ID Optional. Defaults to the auth-side default ('organizations').
835+
# SCOPES Optional, comma-separated. If empty + CLIENT_ID set,
836+
# defaults to '<URL-origin>/.default'.
837+
# FLOW REQUIRED when CLIENT_ID is set. "browser" or "device-code".
838+
#
839+
# Add an HTTP MCP server entry to ~/.hyperagent/config.json. Used by
840+
# `mcp-setup-m365` and intended for direct use when wiring custom HTTP
841+
# MCP servers (any vendor — not M365-specific).
842+
#
843+
# Examples:
844+
# just mcp-add-http example https://mcp.example.com/sse
845+
# just mcp-add-http work-iq-mail \
846+
# https://agent365.svc.cloud.microsoft/agents/servers/mcp_MailRemoteServer \
847+
# <client-id> <tenant-id> "" browser
848+
mcp-add-http NAME URL CLIENT_ID="" TENANT_ID="" SCOPES="" FLOW="":
849+
npx tsx scripts/mcp-add-http.ts "{{ NAME }}" "{{ URL }}" "{{ CLIENT_ID }}" "{{ TENANT_ID }}" "{{ SCOPES }}" "{{ FLOW }}"
850+
851+
# ── Microsoft 365 / Agent 365 HTTP MCP servers ───────────────────────
852+
#
853+
# Alternative to the stdio `mcp-setup-workiq` recipe above: direct
854+
# HTTP+OAuth to the Agent 365 per-service MCP endpoints (mail, calendar,
855+
# teams, sharepoint, onedrive, user, copilot, word, …). Requires either
856+
# a per-tenant Entra app registration (`mcp-m365-create-app`) or a
857+
# pre-existing client id passed explicitly.
858+
#
859+
# Flow:
860+
# 1. just mcp-m365-create-app # one-time: Entra app registration
861+
# 2. just mcp-m365-setup # writes one entry per M365 service
862+
# 3. just start → /plugin enable mcp → /mcp enable work-iq-<service>
863+
#
864+
# State lives at ~/.hyperagent/m365.json (clientId, tenantId).
865+
# The server catalog (alias → mcp_* id mapping) lives at
866+
# scripts/m365-mcp-servers.json — refresh via `just mcp-m365-refresh-servers`.
867+
868+
# Create (or reuse) the Entra app registration for the Agent 365 MCP servers.
869+
# Optional: --service-ref GUID for corporate tenants that require one.
870+
# Optional: --client-id ID to adopt an existing app.
871+
# Requires `az` CLI installed and `az login`'d. Cross-platform (Linux,
872+
# macOS, Windows native, Git Bash, WSL) — runs via tsx.
873+
mcp-m365-create-app *ARGS:
874+
npx tsx scripts/setup-m365-app.ts {{ ARGS }}
875+
876+
# Write the M365 HTTP MCP server entries into ~/.hyperagent/config.json
877+
# by looping over scripts/m365-mcp-servers.json. Reads clientId/tenantId
878+
# from ~/.hyperagent/m365.json by default; override with explicit args.
879+
#
880+
# Each server uses the URL and per-server scope discovered from
881+
# Agent 365 (see catalog file). The catalog stores the discovery URL
882+
# (/agents/servers/<name>); the setup script injects the caller's
883+
# tenantId at config-write time to produce the actual gateway URL
884+
# (/agents/tenants/<tid>/servers/<name>) that the gateway requires —
885+
# without it the server returns EndpointInvalid / TenantIdInvalid.
886+
#
887+
# Args:
888+
# SERVICES "all" (default), comma-separated alias list ("mail,teams"),
889+
# or "list" to print all known service aliases and exit.
890+
# CLIENT_ID Override Entra app client id
891+
# TENANT_ID Override Entra tenant id (used for OAuth authority)
892+
# SCOPE_OVERRIDE Optional: force a single scope for every server
893+
# (default: each server uses its catalogued scope)
894+
# FLOW REQUIRED. "browser" or "device-code". Picks which
895+
# user-interaction OAuth flow gets baked into every
896+
# server entry. There is no default — different
897+
# environments (laptop vs SSH vs FOCI app) need
898+
# different flows so the recipe forces an explicit
899+
# choice.
900+
mcp-setup-m365 SERVICES="all" CLIENT_ID="" TENANT_ID="" SCOPE_OVERRIDE="" FLOW="":
901+
npx tsx scripts/m365-setup.ts "{{ SERVICES }}" "{{ CLIENT_ID }}" "{{ TENANT_ID }}" "{{ SCOPE_OVERRIDE }}" "{{ FLOW }}"
902+
903+
# Refresh scripts/m365-mcp-servers.json from the live Agent 365 catalog.
904+
# Existing alias→server-id mappings are preserved; new server ids appear
905+
# under a derived alias.
906+
mcp-m365-refresh-servers *ARGS:
907+
npx tsx scripts/m365-refresh-servers.ts {{ ARGS }}
908+
909+
# Print the saved M365 app details (if any).
910+
mcp-m365-show:
911+
npx tsx scripts/m365-show.ts

0 commit comments

Comments
 (0)