Skip to content

Refactor agent to Groq LLM and replace web search with local directory search#1

Open
MaxAdmk wants to merge 2 commits intomainfrom
my-agent
Open

Refactor agent to Groq LLM and replace web search with local directory search#1
MaxAdmk wants to merge 2 commits intomainfrom
my-agent

Conversation

@MaxAdmk
Copy link
Copy Markdown
Owner

@MaxAdmk MaxAdmk commented Jan 21, 2026

This PR contains two commits:

  1. Refactor LLM usage to Groq (llama-3.3-70b-versatile) and stabilize the research flow.
  2. Replace external web search with a local filesystem markdown search (--dir).

…rch flow

- Migrate all LLM nodes (query generation, reflection, final answer) from Gemini to Groq using llama-3.3-70b-versatile
- Keep Gemini exclusively for Google Search grounding to preserve citation metadata
- Add graceful handling for Google Search API quota exhaustion (429 / RESOURCE_EXHAUSTED)
  - Return safe fallback state instead of crashing the graph
- Guard against missing or partial API responses
  - Handle empty candidates and absent grounding metadata
  - Fall back to plain-text extraction when citations cannot be generated
- Improve robustness and documentation
  - Defensive access to graph state keys
  - Fix docstring inconsistencies and remove redundant comments
The search component operates on a user-provided directory and retrieves
relevant documentation snippets directly from markdown files.

Architecture decision:
I chose an extended-context, file-based search approach instead of a
broad snippet search or vector database. Technical documentation often
contains long code examples that are easily truncated by short RAG-style
snippets.

To preserve full examples while respecting Groq token limits, the search
retrieves a small number of highly relevant results (top_k=2) with a
larger context window (~100 lines). Relevance is determined using a
deterministic keyword and phrase scoring mechanism without external
dependencies.

This design keeps the agent lightweight, reproducible, and suitable for
offline evaluation on local documentation archives.
text_lower = text.lower()

# Score file path matches (boost)
path_score = sum(path_lower.count(t) * 3 for t in terms) + sum(path_lower.count(p) * 5 for p in phrases)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Чому саме 3 і 5?

# Find best match position (prefer phrase matches, then term matches)
best_idx = len(text)
best_token = None
for p in phrases:
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Дуже заморочений алгоритм пошуку, ти бачив на тестах щоб він працював краще за якийсь примітивний фултекст?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants