Adds support for running Open Agent Spec agents as a new workflow type #1432
Conversation
Closes nvbugs-5425990 ## By Submitting this PR I confirm: - I am familiar with the [Contributing Guidelines](https://github.com/NVIDIA/NeMo-Agent-Toolkit/blob/develop/docs/source/resources/contributing.md). - We require that all contributors "sign-off" on their commits. This certifies that the contribution is your original work, or you have rights to submit it under the same license, or a compatible license. - Any contribution which contains commits that are not Signed-Off will not be accepted. - When the PR is ready for review, new or existing tests cover these changes. - When the PR is ready for review, the documentation is up to date with these changes. Authors: - Yuchen Zhang (https://github.com/yczhang-nv) Approvers: - Will Killian (https://github.com/willkill07) URL: NVIDIA#524
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: Yasha Pushak <yasha.pushak@oracle.com>
Signed-off-by: Yasha Pushak <yasha.pushak@oracle.com>
Signed-off-by: Yasha Pushak <yasha.pushak@oracle.com>
Signed-off-by: Yasha Pushak <yasha.pushak@oracle.com>
Signed-off-by: Yasha Pushak <yasha.pushak@oracle.com>
WalkthroughThis pull request adds Agent Spec integration to NeMo Agent Toolkit by introducing a new workflow type that loads and executes Agent Spec configurations via a LangGraph adapter. The implementation includes configuration models, workflow registration, documentation, and comprehensive tests. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant WorkflowBuilder
participant AgentSpecWorkflow
participant AgentSpecLoader
participant LangGraphComponent
participant LLM
Client->>WorkflowBuilder: build(config)
WorkflowBuilder->>AgentSpecWorkflow: initialize with tools
AgentSpecWorkflow->>AgentSpecLoader: load Agent Spec (YAML/JSON)
AgentSpecLoader->>LangGraphComponent: compile workflow
Client->>AgentSpecWorkflow: invoke(ChatRequest or string)
AgentSpecWorkflow->>AgentSpecWorkflow: normalize messages
AgentSpecWorkflow->>AgentSpecWorkflow: trim history (max_history)
AgentSpecWorkflow->>LangGraphComponent: invoke(input_state)
LangGraphComponent->>LLM: generate response
LLM-->>LangGraphComponent: assistant message
LangGraphComponent-->>AgentSpecWorkflow: result
AgentSpecWorkflow->>AgentSpecWorkflow: compute token usage
AgentSpecWorkflow-->>Client: ChatResponse (or string)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~22 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Fix all issues with AI agents
In `@docs/source/components/agents/agent-spec.md`:
- Line 49: Replace the typo string "Streaming: Non supported." with the correct
phrasing "Streaming: Not supported." in the agent specification docs (look for
the exact text "Streaming: Non supported." in agent-spec.md) so the sentence
reads correctly with proper grammar and punctuation.
- Line 20: Change the first mention of the product from "NeMo Agent Toolkit" to
the full product name "NVIDIA NeMo Agent toolkit" in the sentence starting "This
workflow allows running an [Agent Spec] configuration inside NeMo Agent
Toolkit..." and ensure all subsequent occurrences use lowercase "toolkit";
update the sentence text accordingly and scan the document for any other
instances of "NeMo Agent Toolkit" to correct their casing to "NeMo Agent
toolkit".
In `@pyproject.toml`:
- Around line 83-84: Update the agentspec extra list to use the patch-compatible
version specifier "~=0.1" for pyagentspec and langgraph-agentspec-adapter and
reorder the entries alphabetically by package name (e.g.,
langgraph-agentspec-adapter, nvidia-nat-langchain, pyagentspec) in the agentspec
extra; then run uv pip install langgraph-agentspec-adapter~=0.1 pyagentspec~=0.1
--sync (or run separate uv pip install <pkg> --sync for each) to update uv.lock
so the lockfile is in sync with the pyproject extras.
In `@src/nat/agent/agentspec/config.py`:
- Around line 44-50: The _validate_sources validator currently returns self but
lacks a type annotation; update the method signature of _validate_sources (the
function decorated with `@model_validator`) to include an explicit return type ->
Self and ensure Self is imported (from typing or typing_extensions as
appropriate for the project), keeping the body and the final "return self"
unchanged so static type checkers recognize the correct return type.
In `@tests/test_agentspec_smoke.py`:
- Around line 27-28: Remove the unnecessary `@pytest.mark.asyncio` decorator from
the async test function test_agentspec_adapter_smoke in
tests/test_agentspec_smoke.py; since pytest is configured with
asyncio_mode="auto", simply delete the decorator line above async def
test_agentspec_adapter_smoke(monkeypatch): so the coroutine test is
auto-detected and run without the decorator.
- Around line 54-59: Replace direct mutations of sys.modules with the pytest
monkeypatch fixture: instead of assigning
sys.modules["langgraph_agentspec_adapter"] = ... and
sys.modules["langgraph_agentspec_adapter.agentspecloader"] = fake_mod, call
monkeypatch.setitem(sys.modules, "langgraph_agentspec_adapter",
types.ModuleType("langgraph_agentspec_adapter")) and
monkeypatch.setitem(sys.modules, "langgraph_agentspec_adapter.agentspecloader",
fake_mod) so the changes are automatically reverted; ensure the test function
accepts the monkeypatch fixture and keep the existing fake_mod and StubLoader
setup.
🧹 Nitpick comments (5)
src/nat/agent/agentspec/config.py (1)
1-54: Add Google‑style docstrings for public module/class/function.This new public module lacks a module docstring, and the class/function docstrings are not in Google style nor wrapping code entities in backticks. Please align with the docstring requirements. As per coding guidelines, update docstrings accordingly.
scripts/agentspec_smoke.py (3)
29-29: Remove unnecessarynoqadirective.Static analysis indicates F401 rule is not enabled in ruff configuration, making this directive unnecessary.
♻️ Suggested fix
- import nat.agent.register # noqa: F401 + import nat.agent.register # Force registration
32-52: Consider stubbingload_jsonfor completeness.The
StubLoaderonly implementsload_yaml, but theAgentSpecLoaderinterface also hasload_json. While the smoke test only uses YAML, addingload_jsonwould make the stub more complete for future testing.♻️ Optional addition
class StubLoader: def __init__(self, *args, **kwargs): pass def load_yaml(self, _): return StubComponent() + + def load_json(self, _): + return StubComponent()
55-55: Remove unnecessarynoqadirective.Same as line 29 - F401 rule is not enabled.
♻️ Suggested fix
- import nat.agent.agentspec.register # noqa: F401 + import nat.agent.agentspec.register # Force Agent Spec registrationsrc/nat/agent/agentspec/register.py (1)
35-70: Consider narrowing the exception catch on line 62.The function handles polymorphic message types well. However, the bare
except Exceptionon line 62 catches all errors during content iteration. While this provides resilience, it could mask unexpected issues.♻️ Optional: narrow exception type
try: buf = io.StringIO() for part in c: if hasattr(part, "text"): buf.write(str(getattr(part, "text"))) else: buf.write(str(part)) content = buf.getvalue() - except Exception: + except (TypeError, AttributeError): content = str(c)Note: The
getattrwith constant attribute usage (B009 hints) is appropriate here for duck-typing objects of unknown types where direct attribute access would raiseAttributeError.
|
|
||
| # Agent Spec Workflow | ||
|
|
||
| This workflow allows running an [Agent Spec] configuration inside NeMo Agent Toolkit by converting it to a LangGraph component via the Agent Spec → LangGraph adapter. |
There was a problem hiding this comment.
Use the full product name on first mention and correct casing of “toolkit.”
First mention should be “NVIDIA NeMo Agent toolkit”, with lowercase “toolkit” in body text. As per coding guidelines, please adjust this line.
🔧 Suggested edit
-This workflow allows running an [Agent Spec] configuration inside NeMo Agent Toolkit by converting it to a LangGraph component via the Agent Spec → LangGraph adapter.
+This workflow allows running an [Agent Spec] configuration inside NVIDIA NeMo Agent toolkit by converting it to a LangGraph component via the Agent Spec → LangGraph adapter.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| This workflow allows running an [Agent Spec] configuration inside NeMo Agent Toolkit by converting it to a LangGraph component via the Agent Spec → LangGraph adapter. | |
| This workflow allows running an [Agent Spec] configuration inside NVIDIA NeMo Agent toolkit by converting it to a LangGraph component via the Agent Spec → LangGraph adapter. |
🤖 Prompt for AI Agents
In `@docs/source/components/agents/agent-spec.md` at line 20, Change the first
mention of the product from "NeMo Agent Toolkit" to the full product name
"NVIDIA NeMo Agent toolkit" in the sentence starting "This workflow allows
running an [Agent Spec] configuration inside NeMo Agent Toolkit..." and ensure
all subsequent occurrences use lowercase "toolkit"; update the sentence text
accordingly and scan the document for any other instances of "NeMo Agent
Toolkit" to correct their casing to "NeMo Agent toolkit".
|
|
||
| - Tools: NeMo Agent toolkit built-in tools provided in `tool_names` are exposed to the adapter `tool_registry` by name. If the Agent Spec also defines tools, the registries are merged; duplicate names are overwritten by built-in tools. | ||
| - I/O: Inputs are standard `ChatRequest` messages; the workflow returns a `ChatResponse`. | ||
| - Streaming: Non supported. |
There was a problem hiding this comment.
Fix typo: “Non supported” → “Not supported.”
This is a simple spelling/grammar correction. As per coding guidelines, fix the typo.
🔧 Suggested edit
-- Streaming: Non supported.
+- Streaming: Not supported.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| - Streaming: Non supported. | |
| - Streaming: Not supported. |
🧰 Tools
🪛 LanguageTool
[grammar] ~49-~49: Ensure spelling is correct
Context: ... returns a ChatResponse. - Streaming: Non supported. - Checkpointing: Not suppor...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🤖 Prompt for AI Agents
In `@docs/source/components/agents/agent-spec.md` at line 49, Replace the typo
string "Streaming: Non supported." with the correct phrasing "Streaming: Not
supported." in the agent specification docs (look for the exact text "Streaming:
Non supported." in agent-spec.md) so the sentence reads correctly with proper
grammar and punctuation.
| agentspec = ["nvidia-nat-langchain", "pyagentspec>=0.1", "langgraph-agentspec-adapter>=0.1"] # TODO: How do we actually reference the langgraph adapter as a dependency? | ||
| llama-index = ["nvidia-nat-llama-index"] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, examine the current state of pyproject.toml around lines 83-84
head -n 90 pyproject.toml | tail -n 20Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 952
🏁 Script executed:
# Check for the existence of these packages as workspace dependencies
fd -t d "nvidia-nat-langchain|pyagentspec|langgraph-agentspec-adapter" packages/Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 51
🏁 Script executed:
# Search for these packages in uv.lock to see if they're already listed
rg -n "pyagentspec|langgraph-agentspec-adapter|nvidia-nat-langchain" uv.lock | head -20Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 431
🏁 Script executed:
# Check the structure of packages directory to understand workspace deps
ls -la packages/ | head -20Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 1333
🏁 Script executed:
# Search for pyagentspec and langgraph-agentspec-adapter in all pyproject.toml files
rg -n "pyagentspec|langgraph-agentspec-adapter" --type tomlRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 256
🏁 Script executed:
# Check the full optional-dependencies section for context
sed -n '76,115p' pyproject.tomlRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 1431
Update agentspec optional dependencies to use correct version format and alphabetical ordering, then sync uv.lock.
External dependencies pyagentspec and langgraph-agentspec-adapter must use ~=0.1 instead of >=0.1 per project versioning standards, and should be alphabetically ordered within the extra list. These packages also need to be added to uv.lock via uv pip install <pkg> --sync.
🔧 Suggested edit
- agentspec = ["nvidia-nat-langchain", "pyagentspec>=0.1", "langgraph-agentspec-adapter>=0.1"] # TODO: How do we actually reference the langgraph adapter as a dependency?
+ agentspec = ["nvidia-nat-langchain", "langgraph-agentspec-adapter~=0.1", "pyagentspec~=0.1"] # TODO: How do we actually reference the langgraph adapter as a dependency?🤖 Prompt for AI Agents
In `@pyproject.toml` around lines 83 - 84, Update the agentspec extra list to use
the patch-compatible version specifier "~=0.1" for pyagentspec and
langgraph-agentspec-adapter and reorder the entries alphabetically by package
name (e.g., langgraph-agentspec-adapter, nvidia-nat-langchain, pyagentspec) in
the agentspec extra; then run uv pip install langgraph-agentspec-adapter~=0.1
pyagentspec~=0.1 --sync (or run separate uv pip install <pkg> --sync for each)
to update uv.lock so the lockfile is in sync with the pyproject extras.
| @model_validator(mode="after") | ||
| def _validate_sources(self): | ||
| provided = [self.agentspec_yaml, self.agentspec_json, self.agentspec_path] | ||
| cnt = sum(1 for v in provided if v) | ||
| if cnt != 1: | ||
| raise ValueError("Exactly one of agentspec_yaml, agentspec_json, or agentspec_path must be provided") | ||
| return self |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cat -n src/nat/agent/agentspec/config.py | head -60Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 3202
🏁 Script executed:
rg "def _" src/nat/agent/agentspec/ -A 1 | head -40Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 701
🏁 Script executed:
rg "@model_validator" src/nat/data_models/ -A 3 | head -60Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 3953
🏁 Script executed:
cat src/nat/data_models/agent.py | head -100Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 1698
Add an explicit return type for _validate_sources method.
As a validator returning self, add the return type annotation -> Self for consistency with type-checking requirements and to keep the method properly typed.
🔧 Suggested edit
+from typing import Self
@@
- def _validate_sources(self):
+ def _validate_sources(self) -> Self:
provided = [self.agentspec_yaml, self.agentspec_json, self.agentspec_path]
cnt = sum(1 for v in provided if v)
if cnt != 1:
raise ValueError("Exactly one of agentspec_yaml, agentspec_json, or agentspec_path must be provided")
return self🧰 Tools
🪛 Ruff (0.14.13)
49-49: Avoid specifying long messages outside the exception class
(TRY003)
🤖 Prompt for AI Agents
In `@src/nat/agent/agentspec/config.py` around lines 44 - 50, The
_validate_sources validator currently returns self but lacks a type annotation;
update the method signature of _validate_sources (the function decorated with
`@model_validator`) to include an explicit return type -> Self and ensure Self is
imported (from typing or typing_extensions as appropriate for the project),
keeping the body and the final "return self" unchanged so static type checkers
recognize the correct return type.
| @pytest.mark.asyncio | ||
| async def test_agentspec_adapter_smoke(monkeypatch): |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, check if the test file exists and view the relevant lines
cat -n tests/test_agentspec_smoke.py | sed -n '20,35p'Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 610
🏁 Script executed:
# Check pytest configuration in pyproject.toml
grep -A 20 "\[tool.pytest" pyproject.toml || echo "No pytest config found in pyproject.toml"Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 766
🏁 Script executed:
# Also check if pytest.ini exists
if [ -f pytest.ini ]; then cat pytest.ini; else echo "No pytest.ini found"; fiRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 88
Remove @pytest.mark.asyncio decorator; async tests are auto-detected.
The pytest configuration has asyncio_mode = "auto", which automatically detects and runs async tests. The decorator is unnecessary and violates the test guidelines.
🔧 Suggested edit
-@pytest.mark.asyncio
async def test_agentspec_adapter_smoke(monkeypatch):🧰 Tools
🪛 Ruff (0.14.13)
28-28: Unused function argument: monkeypatch
(ARG001)
🤖 Prompt for AI Agents
In `@tests/test_agentspec_smoke.py` around lines 27 - 28, Remove the unnecessary
`@pytest.mark.asyncio` decorator from the async test function
test_agentspec_adapter_smoke in tests/test_agentspec_smoke.py; since pytest is
configured with asyncio_mode="auto", simply delete the decorator line above
async def test_agentspec_adapter_smoke(monkeypatch): so the coroutine test is
auto-detected and run without the decorator.
| import types | ||
|
|
||
| fake_mod = types.ModuleType("langgraph_agentspec_adapter.agentspecloader") | ||
| fake_mod.AgentSpecLoader = StubLoader | ||
| sys.modules["langgraph_agentspec_adapter"] = types.ModuleType("langgraph_agentspec_adapter") | ||
| sys.modules["langgraph_agentspec_adapter.agentspecloader"] = fake_mod |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
cd /repo && find . -name "test_agentspec_smoke.py" -type fRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 124
🏁 Script executed:
cd /repo && cat -n tests/test_agentspec_smoke.py | head -70Repository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 124
🏁 Script executed:
find . -name "test_agentspec_smoke.py" -type fRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 100
🏁 Script executed:
git ls-files | grep -i test_agentspecRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 129
🏁 Script executed:
cat -n tests/test_agentspec_smoke.pyRepository: NVIDIA/NeMo-Agent-Toolkit
Length of output: 3072
Use monkeypatch for sys.modules edits to avoid cross-test leakage.
Direct mutation of sys.modules at lines 58–59 persists beyond this test and can cause failures in other tests; use monkeypatch.setitem() to auto-restore state after the test completes.
Suggested edit
fake_mod = types.ModuleType("langgraph_agentspec_adapter.agentspecloader")
fake_mod.AgentSpecLoader = StubLoader
- sys.modules["langgraph_agentspec_adapter"] = types.ModuleType("langgraph_agentspec_adapter")
- sys.modules["langgraph_agentspec_adapter.agentspecloader"] = fake_mod
+ monkeypatch.setitem(
+ sys.modules,
+ "langgraph_agentspec_adapter",
+ types.ModuleType("langgraph_agentspec_adapter"),
+ )
+ monkeypatch.setitem(sys.modules, "langgraph_agentspec_adapter.agentspecloader", fake_mod)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| import types | |
| fake_mod = types.ModuleType("langgraph_agentspec_adapter.agentspecloader") | |
| fake_mod.AgentSpecLoader = StubLoader | |
| sys.modules["langgraph_agentspec_adapter"] = types.ModuleType("langgraph_agentspec_adapter") | |
| sys.modules["langgraph_agentspec_adapter.agentspecloader"] = fake_mod | |
| import types | |
| fake_mod = types.ModuleType("langgraph_agentspec_adapter.agentspecloader") | |
| fake_mod.AgentSpecLoader = StubLoader | |
| monkeypatch.setitem( | |
| sys.modules, | |
| "langgraph_agentspec_adapter", | |
| types.ModuleType("langgraph_agentspec_adapter"), | |
| ) | |
| monkeypatch.setitem(sys.modules, "langgraph_agentspec_adapter.agentspecloader", fake_mod) |
🤖 Prompt for AI Agents
In `@tests/test_agentspec_smoke.py` around lines 54 - 59, Replace direct mutations
of sys.modules with the pytest monkeypatch fixture: instead of assigning
sys.modules["langgraph_agentspec_adapter"] = ... and
sys.modules["langgraph_agentspec_adapter.agentspecloader"] = fake_mod, call
monkeypatch.setitem(sys.modules, "langgraph_agentspec_adapter",
types.ModuleType("langgraph_agentspec_adapter")) and
monkeypatch.setitem(sys.modules, "langgraph_agentspec_adapter.agentspecloader",
fake_mod) so the changes are automatically reverted; ensure the test function
accepts the monkeypatch fixture and keep the existing fake_mod and StubLoader
setup.
| data-flywheel = ["nvidia-nat-data-flywheel"] | ||
| ingestion = ["nvidia-nat-ingestion"] # meta-package | ||
| langchain = ["nvidia-nat-langchain"] | ||
| agentspec = ["nvidia-nat-langchain", "pyagentspec>=0.1", "langgraph-agentspec-adapter>=0.1"] # TODO: How do we actually reference the langgraph adapter as a dependency? |
There was a problem hiding this comment.
To be updated with pyagentspec[langgraph] 26.1
3 participants
Description
Adds support for running Open Agent Spec agents as a new workflow type in NeMo Agent Toolkit.
Closes #1431
By Submitting this PR I confirm:
Summary by CodeRabbit
New Features
Documentation
Tests
✏️ Tip: You can customize this high-level summary in your review settings.