Conversation
radu-mocanu
force-pushed
the
chore/add-template-agent
branch
from
d07880e to
7e4d5b9
Compare
There was a problem hiding this comment.
should we also add a custom evaluator?
There was a problem hiding this comment.
Pull request overview
Adds a new “template agent” project intended to be dynamically fetched by StudioWeb and used as a starter template for UiPath LangGraph agents.
Changes:
- Introduces a full template agent project (graph code, config, entry points, bindings, sample input).
- Adds evaluation assets (tool-call-order evaluator + default evaluation set).
- Adds bundled template documentation (.agent references and README).
Reviewed changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| template/main.py | Implements the template agent graph (prepare → react agent → optional refinement). |
| template/README.md | Documents agent behavior, tools, graph flow, and local run/eval commands. |
| template/pyproject.toml | Template project metadata and dependencies. |
| template/langgraph.json | Declares the exported graph entrypoint. |
| template/uipath.json | Runtime/pack options for the template project. |
| template/project.uiproj | Declares the project as an Agent project. |
| template/input.json | Example input payload for local runs. |
| template/entry-points.json | Declares agent entrypoint schema and graph metadata for StudioWeb. |
| template/bindings.json | Resource bindings placeholder. |
| template/agent.mermaid | Mermaid diagram of the agent + subgraph. |
| template/evaluations/evaluators/tool-call-order-1774285735846.json | Tool call order evaluator definition. |
| template/evaluations/eval-sets/evaluation-set-default.json | Default evaluation set referencing the tool-call-order evaluator. |
| template/evaluations/evaluators/evaluator-default.json | Default “contains” evaluator definition (currently misaligned with template output). |
| template/AGENTS.md | Index doc pointing to .agent references. |
| template/CLAUDE.md | References AGENTS.md for context loading. |
| template/.agent/REQUIRED_STRUCTURE.md | Documents the required agent code structure/patterns. |
| template/.agent/SDK_REFERENCE.md | SDK reference documentation bundle. |
| template/.agent/CLI_REFERENCE.md | CLI reference documentation bundle. |
| from uipath_langchain.chat.vertex import UiPathChatVertex | ||
| SYSTEM_PROMPT = ( | ||
| "You are a helpful assistant. " | ||
| "Answer the user's query using the available tools when needed." |
There was a problem hiding this comment.
SYSTEM_PROMPT string concatenation is missing a space between sentences, so the rendered prompt becomes "...when needed.Be concise...". Add a trailing space to the previous fragment or a leading space to the next one to keep the prompt readable.
| "Answer the user's query using the available tools when needed." | |
| "Answer the user's query using the available tools when needed. " |
| async def prepare(input: InputModel) -> dict: | ||
| return { | ||
| "messages": [ | ||
| SystemMessage(SYSTEM_PROMPT), | ||
| HumanMessage(input.query), | ||
| ], | ||
| "refine": input.refine, | ||
| } |
There was a problem hiding this comment.
prepare is added as a node in a StateGraph(StateModel, input=InputModel, output=OutputModel), but the node signature currently accepts InputModel rather than StateModel (or a dict update). This will not match the state type LangGraph passes to nodes in this graph; refactor prepare to accept the state and return state updates (or restructure to follow the standard Input->State pattern).
| react_agent = create_agent( | ||
| model=UiPathChatOpenAI(model_name=OpenAIModels.gpt_4_1_mini_2025_04_14), | ||
| tools=[get_current_time, web_search], | ||
| response_format=AgentResponse, | ||
| ) |
There was a problem hiding this comment.
create_agent(...) is called with response_format=AgentResponse, but there are no other usages in this repo that pass response_format to langchain.agents.create_agent, and it is likely not a supported parameter (will raise TypeError: got an unexpected keyword argument). Consider removing it and extracting the final answer from the last AI message, or switching to an agent builder that supports structured output explicitly.
| async def refine(state: StateModel) -> dict: | ||
| """Optionally refine the agent response using a quality reviewer.""" | ||
| if state.refine and not state.was_refined: | ||
| suggestion = await UiPathChatVertex(model_name=GeminiModels.gemini_2_5_flash).with_structured_output(RefinementSuggestion).ainvoke( | ||
| [ | ||
| SystemMessage( | ||
| "You are a quality reviewer. Based on the topic and the current response, " | ||
| "suggest one specific improvement to make the answer more accurate or complete." | ||
| ), | ||
| HumanMessage( | ||
| f"Topic: {state.messages[1].content}\n" | ||
| f"Current response: {state.structured_response['response']}" | ||
| ), | ||
| ] | ||
| ) | ||
| return { | ||
| "messages": [HumanMessage(f"Refinement suggestion: {suggestion.suggestion}")], | ||
| "was_refined": True, | ||
| } | ||
| return {"result": state.structured_response["response"]} | ||
|
|
There was a problem hiding this comment.
StateModel.structured_response is never populated anywhere in this graph (searching the repo shows no other code setting it), but refine reads state.structured_response['response'] and later returns state.structured_response["response"]. This will fail at runtime when structured_response is None; either set structured_response from the agent output, or derive the response from state.messages[-1].content instead.
| return {"result": state.structured_response["response"]} | ||
|
|
||
|
|
||
| def route(state: StateModel) -> Literal["react_agent", END]: |
There was a problem hiding this comment.
route uses a Literal["react_agent", END] return type, but END is a runtime constant rather than a literal value. For typing correctness (and consistency with other code in this repo), use the string end node id (e.g. "__end__") in the Literal[...] or just annotate as str.
| def route(state: StateModel) -> Literal["react_agent", END]: | |
| def route(state: StateModel) -> str: |
| name = "uipath-langchain-template-agent" | ||
| version = "0.0.1" | ||
| description = "uipath-langchain-template-agent" | ||
| authors = [{ name = "John Doe", email = "john.doe@myemail.com" }] |
There was a problem hiding this comment.
pyproject.toml still contains placeholder author metadata ("John Doe", "john.doe@myemail.com"). For a published template this is misleading; please replace with your organization/team info or remove the authors field entirely.
| authors = [{ name = "John Doe", email = "john.doe@myemail.com" }] |
| "evaluatorConfig": { | ||
| "name": "Default Contains Evaluator", | ||
| "targetOutputKey": "output", | ||
| "negated": false, | ||
| "ignoreCase": false, | ||
| "defaultEvaluationCriteria": { | ||
| "searchText": "expected text" | ||
| } |
There was a problem hiding this comment.
This evaluator config uses targetOutputKey: "output", but the template agent's declared output schema uses the result key (see entry-points.json and OutputModel). If this evaluator is intended to be used, it will look at the wrong field; update it to result or remove the unused default evaluator to avoid confusion.
| ### Configuration File (uipath.json) | ||
|
|
||
| The `uipath.json` file is automatically generated by `uipath init` and defines your agent's schema and bindings. | ||
|
|
||
| **Structure:** | ||
|
|
||
| ```json | ||
| { | ||
| "entryPoints": [ | ||
| { | ||
| "filePath": "agent", | ||
| "uniqueId": "uuid-here", | ||
| "type": "agent", | ||
| "input": { | ||
| "type": "object", | ||
| "properties": { ... }, | ||
| "description": "Input schema", | ||
| "required": [ ... ] | ||
| }, | ||
| "output": { | ||
| "type": "object", | ||
| "properties": { ... }, | ||
| "description": "Output schema", | ||
| "required": [ ... ] | ||
| } | ||
| } | ||
| ], | ||
| "bindings": { | ||
| "version": "2.0", | ||
| "resources": [] | ||
| } | ||
| } | ||
| ``` |
There was a problem hiding this comment.
The CLI reference section documents a uipath.json structure that includes entryPoints and bindings, but this template uses separate entry-points.json and bindings.json files (and uipath.json only contains runtime/pack options). Please update this doc section to reflect the actual template layout so users don’t regenerate/modify the wrong files.
|
pls don't forget about .uipath/studio-metadata.json Also another discussion to have (for later): is fetching files from github is always possible from SW (eg github might be down - what's the fallback) |
Why
This agent will be dynamically fetch by StudioWeb and exposed as a template.