fix: LLM provider works with subscription proxies + honor SUMMARIZE_CHUNK_SIZE from .env#742
Conversation
Some Anthropic-compatible proxies (e.g. Claude Code subscription bridges)
inject their own agent system prompt and override the caller's `system`
field. Structured-output instructions placed in `system` are then ignored
and the model replies conversationally ("I'll look at the files..."),
producing no parseable output — observed as empty_provider_response /
too_many_chunks_skipped on every summarize/compress call through such a proxy.
Fold the system prompt into the user message instead. It survives the
override and behaves identically against the official Anthropic API.
Signed-off-by: Caio Niehues <caioniehues@MacBook-Pro.local>
…emory/.env getChunkSize()/getChunkConcurrency() read process.env directly, so the documented SUMMARIZE_CHUNK_SIZE / SUMMARIZE_CHUNK_CONCURRENCY values set in ~/.agentmemory/.env were silently ignored — every other setting flows through getEnvVar()/getMergedEnv(). This matters for small-context local models, where the default 400-observation chunk overflows the window and all chunks fail to parse. Route both through getEnvVar() so the .env file is honored; process.env still wins (getMergedEnv spreads it last), so existing overrides and tests are unaffected. Signed-off-by: Caio Niehues <caioniehues@MacBook-Pro.local>
|
Someone is attempting to deploy a commit to the rohitg00's projects Team on Vercel. A member of the Team first needs to authorize it. |
📝 WalkthroughWalkthroughTwo independent configuration and integration changes: summarization configuration now respects the merged environment helper ( ChangesEnvironment Configuration Refactoring
Anthropic API Integration Fix
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure. 🔧 ESLint
ESLint skipped: no ESLint configuration detected in root package.json. To enable, add Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/functions/summarize.ts (1)
40-49: 💤 Low valueTrim the explanatory comment block.
The logic change is correct, and routing through
getEnvVarpreservesprocess.envprecedence (it is spread last ingetMergedEnv), so the existingprocess.env-based tests remain valid. However, the 4-line comment at Lines 41-44 narrates what the code does and its history; prefer lettinggetEnvVarconvey intent and drop the WHAT portion.♻️ Suggested trim
function getChunkSize(): number { - // Read via the merged env (file + process.env) so ~/.agentmemory/.env works, - // consistent with every other config var. process.env still wins (getMergedEnv - // spreads it last). Previously read process.env only, so the .env file value - // was silently ignored — a problem for small local-LLM context windows. const raw = getEnvVar("SUMMARIZE_CHUNK_SIZE");As per coding guidelines: "No code comments explaining WHAT — use clear naming instead".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/functions/summarize.ts` around lines 40 - 49, Remove the explanatory history-style comment inside getChunkSize and replace it with a single-line comment (or no comment) that states intent succinctly; specifically, trim the multi-line block that references merged env and process.env precedence and leave either nothing or a brief note like "Read SUMMARIZE_CHUNK_SIZE via getEnvVar" near the getChunkSize function which uses getEnvVar, parseInt, and CHUNK_SIZE_DEFAULT so the behavior remains unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/functions/summarize.ts`:
- Around line 40-49: Remove the explanatory history-style comment inside
getChunkSize and replace it with a single-line comment (or no comment) that
states intent succinctly; specifically, trim the multi-line block that
references merged env and process.env precedence and leave either nothing or a
brief note like "Read SUMMARIZE_CHUNK_SIZE via getEnvVar" near the getChunkSize
function which uses getEnvVar, parseInt, and CHUNK_SIZE_DEFAULT so the behavior
remains unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 76b485cb-3f65-4b7a-b0a6-c05119a9bfc9
📒 Files selected for processing (2)
src/functions/summarize.tssrc/providers/anthropic.ts
1 participant
Summary
Two independent robustness fixes for non-standard / local LLM endpoints, both surfaced while running agentmemory's compression + summarization through a self-hosted Claude Code subscription proxy and against local Ollama models.
1.
fix(providers)— fold the Anthropic system prompt into the user messageAnthropicProvider.call()passes the prompt via thesystemfield. Some Anthropic-compatible proxies — notably Claude Code subscription bridges — inject their own agent system prompt and override the caller'ssystem. The model then ignores agentmemory's structured-output instructions and replies conversationally (e.g. "I'll look at the auth code…"), socontent.find(b => b.type === 'text')returns prose with no<summary>→ everysummarize/compresscall fails asempty_provider_response/too_many_chunks_skipped.Folding the instructions into the user message survives the override and behaves identically against the official Anthropic API.
Evidence: the same bulk run that produced 0/84 parseable summaries through such a proxy produced 83/84 (quality 50–100) after this change.
2.
fix(summarize)— honorSUMMARIZE_CHUNK_SIZE/SUMMARIZE_CHUNK_CONCURRENCYfrom.envgetChunkSize()/getChunkConcurrency()readprocess.envdirectly, so the documented values placed in~/.agentmemory/.envare silently ignored — every other setting flows throughgetEnvVar()/getMergedEnv(). For small-context local models (e.g. an 8k-context Ollama model) the default 400-observation chunk (~44k tokens) overflows the window and all chunks fail to parse.Routing both through
getEnvVar()honors the file.process.envstill wins (getMergedEnvspreads it last), so existing overrides are unaffected.Test plan
test/summarize.test.tschunk-override tests pass unchanged (they setprocess.env, which still wins).SUMMARIZE_CHUNK_SIZEfrom.envafter fix feat: v0.2.0 -- 12 hooks, MCP tools, skills, pattern detection #2.Both fixes are small and independent; happy to split into two PRs if preferred.
Summary by CodeRabbit