feat: Add LangChain middleware integration#69
feat: Add LangChain middleware integration#69dagardner-nv wants to merge 51 commits intoNVIDIA:mainfrom
Conversation
Signed-off-by: David Gardner <dagardner@nvidia.com>
…his version requires newer versions of langchain Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…ely this will be needed elsewhere, remove unused code that attempts to lazy-import nemo_flow Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
dagardner-nv
added
feat
WalkthroughAdds a standalone langchain_nemo_flow package (serialization, middleware, callback handler), upstream LangChain patches to route model/tool calls through NeMo Flow, a run_sync utility for sync↔async bridging with scope propagation, tests, CI updates to run integration tests and build wheels, and documentation updates. ChangesLangChain NeMo Flow Integration
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…his project doesn't have a rust extension Signed-off-by: David Gardner <dagardner@nvidia.com>
github-actions
Bot
added
size:XL
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
patches/langchain/0001-add-nemo-flow-integration.patch (2)
1108-1119:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftOpenAI
run_syncdoes not propagate scope stack, unlike Anthropic version.The OpenAI bridge's
run_sync(lines 1108-1119) does not propagatecontextvarsor the NeMo Flow scope stack to worker threads, while the Anthropic version (lines 976-1008) does. This inconsistency will cause NeMo Flow telemetry to be lost for sync OpenAI calls when an event loop is already running.Proposed fix - align with Anthropic version
def run_sync(coro: Any) -> Any: """Run *coro* synchronously, handling the case where an event loop is already running (e.g. Jupyter notebooks).""" try: asyncio.get_running_loop() except RuntimeError: # No loop running - we can just use asyncio.run. return asyncio.run(coro) # Loop already running - offload to a worker thread so we don't block. + import contextvars + ctx = contextvars.copy_context() + if _HAS_NEMO_FLOW: + try: + scope_stack = nemo_flow.get_scope_stack() + + def _run_with_scope_stack() -> Any: + nemo_flow.set_thread_scope_stack(scope_stack) + return asyncio.run(coro) + + with ThreadPoolExecutor(max_workers=1) as pool: + return pool.submit(ctx.run, _run_with_scope_stack).result() + except Exception: + pass with ThreadPoolExecutor(max_workers=1) as pool: - return pool.submit(asyncio.run, coro).result() + return pool.submit(ctx.run, asyncio.run, coro).result()Consider extracting
run_syncto a shared location (e.g.,langchain_core.utils._nemo_flow) to avoid this duplication.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@patches/langchain/0001-add-nemo-flow-integration.patch` around lines 1108 - 1119, The OpenAI run_sync implementation does not propagate contextvars/NeMo Flow scope stack to worker threads, causing telemetry loss when an event loop is already running; update run_sync to mirror the Anthropic version by capturing the current context and NeMo Flow scope stack before spawning the ThreadPoolExecutor and running the coroutine in that captured context (e.g., use contextvars.copy_context() and the NeMo Flow scope stack transfer mechanism used in the Anthropic run_sync), or factor the logic into a shared helper (e.g., langchain_core.utils._nemo_flow) and call that from both OpenAI and Anthropic bridges so both propagate context and scope consistently.
126-177:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftNeMo Flow streaming path buffers entire response before yielding.
The sync
_streamintegration (and async_astream) collects all events intocollected_eventslist, then iterates after the stream completes. This defeats the purpose of streaming - users won't receive chunks incrementally._nemo_flow.run_sync(_run()) # Blocks until entire stream collected # ... for event in collected_events: # Only then yields yield chunkThis is a significant UX regression when NeMo Flow is enabled. Consider restructuring to yield chunks as they arrive from
llm_stream_execute.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@patches/langchain/0001-add-nemo-flow-integration.patch` around lines 126 - 177, The current NeMo Flow branch buffers the whole response by collecting events in collected_events and calling _nemo_flow.run_sync(_run()) which blocks; instead, consume the async generator returned by _nemo_flow.llm_stream_execute directly and yield ChatGenerationChunk instances as events arrive. Modify the logic around _nemo_flow.make_request / _nemo_flow.llm_stream_execute so you do not populate collected_events and wait for _run() to finish; instead iterate the nemo_flow_stream (or its sync wrapper) event-by-event, call _make_message_chunk_from_anthropic_event for each event, emit the ChatGenerationChunk immediately, and invoke run_manager.on_llm_new_token when appropriate; keep the existing coerce_content_to_string computation and _finalizer semantics but avoid buffering the entire response before yielding.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@integrations/langchain/hatch_fecth_version.py`:
- Around line 1-2: Rename the module file from hatch_fecth_version.py to
hatch_fetch_version.py and update any import or packaging references
accordingly; specifically, rename the file in the integrations/langchain module
and update the corresponding entry in pyproject.toml (and any import paths or
setup references) so all references to hatch_fecth_version are changed to
hatch_fetch_version to avoid the typo.
In `@integrations/langchain/pyproject.toml`:
- Around line 34-41: The pyproject change must be paired with conditional
imports and runtime guards in the integration source: replace unconditional
top-level "import nemo_flow" in integrations/langchain/callbacks.py and
integrations/langchain/middleware.py with a try/except import (or lazy import
helper) that sets a module-level flag or None value, and update all places that
call nemo_flow.* (especially the direct calls in middleware.py and the
exception-wrapped calls in callbacks.py) to check that flag/None before invoking
NeMo Flow APIs (return early or noop when absent); ensure the import variable
name and the guard checks match the existing symbols used in those files so
tests that monkeypatch callbacks_module.nemo_flow still behave correctly.
In `@integrations/langchain/src/langchain_nemo_flow/_serialization.py`:
- Around line 94-99: The current except block around
split_system_message(messages_from_dict(raw_messages)) silently swallows all
errors and masks malformed NeMo Flow payloads; replace the bare except/pass in
the try/except that surrounds messages_from_dict and split_system_message with a
catch that captures the exception (e.g., except Exception as e:) and logs a
debug-level message including the exception and the offending raw_messages (use
the module logger or existing logger variable) before continuing, so overrides
remains unchanged but the failure is observable for debugging.
In `@integrations/langchain/src/langchain_nemo_flow/middleware.py`:
- Around line 91-97: The helper _prepare_model_call (and similarly
_prepare_tool_call) returns an anonymous tuple which forces positional
unpacking; define a small NamedTuple (e.g., _ModelCallPrep with fields
object_codec, llm_request, model_name, model_codec) and change
_prepare_model_call's return type to that NamedTuple and return an instance
instead, then update any call sites that unpack the tuple to use attribute
access (e.g., prep.object_codec) to improve readability.
In `@python/nemo_flow/utils.py`:
- Around line 40-44: Replace the bare "except Exception: pass" with catching the
exception as e, log the exception (add import logging if missing) and then
ensure the fallback uses the current contextvars context when running the
coroutine: capture ctx = contextvars.copy_context() before the try/except and in
the ThreadPoolExecutor.submit call run the coroutine inside that context (e.g.,
submit(lambda: ctx.run(asyncio.run, coro))). This preserves scope stack
propagation and records the original exception instead of swallowing it.
---
Outside diff comments:
In `@patches/langchain/0001-add-nemo-flow-integration.patch`:
- Around line 1108-1119: The OpenAI run_sync implementation does not propagate
contextvars/NeMo Flow scope stack to worker threads, causing telemetry loss when
an event loop is already running; update run_sync to mirror the Anthropic
version by capturing the current context and NeMo Flow scope stack before
spawning the ThreadPoolExecutor and running the coroutine in that captured
context (e.g., use contextvars.copy_context() and the NeMo Flow scope stack
transfer mechanism used in the Anthropic run_sync), or factor the logic into a
shared helper (e.g., langchain_core.utils._nemo_flow) and call that from both
OpenAI and Anthropic bridges so both propagate context and scope consistently.
- Around line 126-177: The current NeMo Flow branch buffers the whole response
by collecting events in collected_events and calling _nemo_flow.run_sync(_run())
which blocks; instead, consume the async generator returned by
_nemo_flow.llm_stream_execute directly and yield ChatGenerationChunk instances
as events arrive. Modify the logic around _nemo_flow.make_request /
_nemo_flow.llm_stream_execute so you do not populate collected_events and wait
for _run() to finish; instead iterate the nemo_flow_stream (or its sync wrapper)
event-by-event, call _make_message_chunk_from_anthropic_event for each event,
emit the ChatGenerationChunk immediately, and invoke
run_manager.on_llm_new_token when appropriate; keep the existing
coerce_content_to_string computation and _finalizer semantics but avoid
buffering the entire response before yielding.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 9106fde4-c48e-4da6-b6b8-34c523d462ae
⛔ Files ignored due to path filters (2)
integrations/langchain/uv.lockis excluded by!**/*.lockthird_party/sources.lockis excluded by!**/*.lock,!third_party/**
📒 Files selected for processing (16)
.github/ci-path-filters.yml.github/workflows/ci_python.yml.gitignoreintegrations/langchain/hatch_fecth_version.pyintegrations/langchain/pyproject.tomlintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/src/langchain_nemo_flow/middleware.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/tests/test_middleware.pyjustfilepatches/langchain/0001-add-nemo-flow-integration.patchpyproject.tomlpython/nemo_flow/__init__.pypython/nemo_flow/utils.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (15)
- GitHub Check: Python / Test (windows-arm64)
- GitHub Check: Python / Test (windows-amd64)
- GitHub Check: Python / Test (macos-arm64)
- GitHub Check: Python / Test (linux-arm64)
- GitHub Check: Python / Test (linux-amd64)
- GitHub Check: WebAssembly / Test (windows-arm64)
- GitHub Check: WebAssembly / Test (linux-amd64)
- GitHub Check: WebAssembly / Test (macos-arm64)
- GitHub Check: WebAssembly / Test (linux-arm64)
- GitHub Check: WebAssembly / Test (windows-amd64)
- GitHub Check: Node.js / Test (windows-arm64)
- GitHub Check: Go / Test (windows-amd64)
- GitHub Check: Rust / Test (windows-arm64)
- GitHub Check: Go / Test (windows-arm64)
- GitHub Check: Documentation / Build
🧰 Additional context used
📓 Path-based instructions (25)
{.github/**/*.{yml,yaml},*.patch,scripts/**/*,*.sh,*.bat,Dockerfile*}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update CI configuration, patch files, and build scripts with new functional identifiers after rename operations
Files:
.github/ci-path-filters.yml.github/workflows/ci_python.yml
{.github/**,.gitlab-ci.yml,.pre-commit-config.yaml,justfile,scripts/**}
⚙️ CodeRabbit configuration file
{.github/**,.gitlab-ci.yml,.pre-commit-config.yaml,justfile,scripts/**}: Review automation changes for reproducibility, pinned versions where appropriate, secret handling, and consistency with the documented validation matrix.
Pay attention to commands that need generated native artifacts, FFI libraries, or platform-specific environment variables.
Files:
.github/ci-path-filters.ymljustfile.github/workflows/ci_python.yml
**/*.toml
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license headers in TOML files using TOML comment syntax
Files:
pyproject.tomlintegrations/langchain/pyproject.toml
**/pyproject.toml
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency in Python packaging metadata in
pyproject.toml
Files:
pyproject.tomlintegrations/langchain/pyproject.toml
{Cargo.toml,setup.py,setup.cfg,pyproject.toml,go.mod,go.sum,package.json,package-lock.json,yarn.lock,pom.xml,*.gradle}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update repository manifest files (Cargo.toml, setup.py, go.mod, package.json, etc.) with new package/crate names during rename operations
Files:
pyproject.toml
**/*.{rs,py,js,ts,tsx,jsx,go,sh,yaml,yml,toml,json,md,txt}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.
Files:
pyproject.tomlpython/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/pyproject.tomlintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Format changed Python wrapper and test files with
uv run ruff format python
**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter for Python code with line length 120 and double quotes
Usetyfor Python type checking
Use snake_case naming convention for Python code
Include SPDX license headers in all Python source files using Python comment syntax
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use
SONAR_IGNORE_START/SONAR_IGNORE_ENDmarkers only for documented false positives that cannot be resolved in code; keep ignored blocks small, add explanatory comments, and require reviewer sign-off
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
python/nemo_flow/**/*.py
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Python API implementations and wrappersFollow Python binding naming conventions: use
snake_casefor variables, functions, and modules.
Files:
python/nemo_flow/__init__.pypython/nemo_flow/utils.py
**/*.{md,markdown,py,sh,bash,js,ts,java,cpp,go,rust}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current in documentation
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{md,markdown,py,sh,bash}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{md,markdown,py,sh,bash,js,ts,example}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Example commands must match current package names and paths
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
{docs/**,examples/**,crates/adaptive/**,python/nemo_flow/**,go/nemo_flow/**,**/{example,component}.{ts,tsx,js,rs,py,go}}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Any new adaptive component kind must have documentation, examples, and binding coverage across all supported languages
Files:
python/nemo_flow/__init__.pypython/nemo_flow/utils.py
**/*.{rs,py,go,js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,go,js,ts,jsx,tsx}: Format changed files with the language-native formatter before the final lint/test pass
Use language-native formatters:cargo fmt --allfor Rust,uv run ruff format pythonfor Python,go fmt ./...for Go,npm --prefix crates/node run formatfor Node.js, andnpm --prefix crates/node run precommit:format -- crates/wasm/wrappers crates/wasm/tests-js crates/wasm/scriptsfor WebAssembly
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{py,go,js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{js,ts,tsx,jsx,py,rs,go,java,c,cpp,h,cc,cxx,cs,rb,php,swift,kt}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changed files must be formatted with the language-native formatter
Files:
python/nemo_flow/__init__.pyintegrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
python/nemo_flow/**/*
⚙️ CodeRabbit configuration file
python/nemo_flow/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_flow/__init__.pypython/nemo_flow/utils.py
**/{integrations,integration,*-integration}/**
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
**/{integrations,integration,*-integration}/**: Keep NeMo Flow optional in framework integrations
Preserve the framework's original behavior when NeMo Flow is absent
Wrap tool and LLM paths at the correct framework boundary
Integration pattern must followdocs/integrate-frameworks/adding-scopes.md
Files:
integrations/langchain/src/langchain_nemo_flow/__init__.pyintegrations/langchain/hatch_fecth_version.pyintegrations/langchain/pyproject.tomlintegrations/langchain/tests/test_callbacks.pyintegrations/langchain/src/langchain_nemo_flow/callbacks.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/_serialization.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
justfile
📄 CodeRabbit inference engine (.agents/skills/update-project-version/SKILL.md)
justfile: Usejust set-version <version>command to update release-version source files including Cargo.toml workspace package version, workspace dependencies, and Node package metadata
Keep helper code functionsset_project_version,set_cargo_workspace_version, andset_node_package_versionaligned with the same version fields updated by the workflow
Files:
justfile
**/test_*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Run focused
pytestfirst when you know the affected area usingpytest -k "<pattern>"
Files:
integrations/langchain/tests/test_callbacks.pyintegrations/langchain/tests/test_middleware.py
.github/workflows/**/*.{yml,yaml}
📄 CodeRabbit inference engine (.agents/skills/maintain-ci/SKILL.md)
.github/workflows/**/*.{yml,yaml}: Putpermissions:on each job that needs token access in GitHub Actions workflows
Avoid workflow-level permissions unless the repository intentionally centralizes them and the inheritance tradeoff is documented
Keep third-party actions pinned to full commit SHAs and preserve the readable version comment after the SHA
Prefer action-native or ecosystem-native caching over genericactions/cachein GitHub Actions workflows
Use lockfiles or dependency manifests to drive cache invalidation in GitHub Actions workflows
Keep deploy and publish permissions isolated to the jobs that need them
Read both caller and callee when a workflow usesworkflow_callin GitHub Actions
Put release-tag validation in the earliest practical caller job when the pipeline has tag-based publish behavior
Keep release-tag policy aligned withRELEASING.md: raw SemVer tags only, no leadingv
contents: readis the default minimum for checkout-based build, test, docs, and packaging jobs
pull-requests: readis required for PR metadata lookup jobs in GitHub Actions workflows
pages: writeandid-token: writeshould be limited to Pages deployment jobs and any caller that invokes them through a reusable workflow
For reusable workflows, the caller must grant every permission the called jobs require and the callee cannot elevate beyond what the caller provides
Preferastral-sh/setup-uvcache support withcache-dependency-globanchored touv.lockfor Python dependency caching
PreferSwatinem/rust-cachewith explicitshared-keyandworkspacesinstead of ad hoc target-directory caching
Avoid caching generated outputs that can hide stale behavior unless the repo already relies on them deliberately
Ensure each job has the minimum permissions it needs during GitHub Actions CI review
Ensure reusable workflow callers grant only the scopes their callees require
Ensure every external action is pinned to a full SHA in GitHub Actions workflows
Ensure cache ...
Files:
.github/workflows/ci_python.yml
{.github/workflows/**/*.{yml,yaml},.gitlab-ci.yml}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Keep CI workflow package name references consistent with local package configurations
Files:
.github/workflows/ci_python.yml
**/{patches,patch}/**/*.patch
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
Keep the tracked patch artifact minimal and reproducible
Files:
patches/langchain/0001-add-nemo-flow-integration.patch
{./scripts/**,**/patches/**,**/*.patch}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
For third-party integration or patch changes, run patch validation with
./scripts/apply-patches.sh --checkand relevant integration tests, keeping root./scripts/*.shwrappers for third-party flows
Files:
patches/langchain/0001-add-nemo-flow-integration.patch
patches/**/*.patch
⚙️ CodeRabbit configuration file
patches/**/*.patch: Patch changes should remain reviewable as regenerated diffs against the matching third_party submodule.
Check that integration behavior, package metadata, and tests remain consistent with the main NeMo Flow API.
Files:
patches/langchain/0001-add-nemo-flow-integration.patch
🧠 Learnings (2)
📚 Learning: 2026-04-15T18:16:52.951Z
Learnt from: bbednarski9
Repo: NVIDIA/NeMo-Flow PR: 1
File: docs/atof-event-format.md:381-381
Timestamp: 2026-04-15T18:16:52.951Z
Learning: In docs/atof-event-format.md (and when reviewing references to this format across the NeMo-Flow repo), treat `AtifExporter.events_to_steps()` as the intended public/API method name. Do not flag it as inconsistent with internal Rust symbol names (e.g., `event_to_steps` in `crates/core/src/atif.rs`)—the documentation’s public-facing naming may differ intentionally from internal implementation details.
Applied to files:
.gitignore
📚 Learning: 2026-05-03T04:23:07.497Z
Learnt from: willkill07
Repo: NVIDIA/NeMo-Flow PR: 46
File: .github/workflows/ci_rust.yml:31-64
Timestamp: 2026-05-03T04:23:07.497Z
Learning: In GitHub Actions workflow YAML, it’s valid to conditionally disable a service container by setting the service container’s `image` to an empty string (`''`) via a matrix variable (e.g., `redis_service_image: ''`). This intentionally makes the runner skip service initialization for that matrix entry rather than failing the job. When reviewing workflows, don’t flag this as an error if the workflow uses an empty `image` to disable the service on specific matrix entries (e.g., OS-specific setups); verify the `image` is sourced from the matrix variable and that the service is only expected to be available when a non-empty image is provided.
Applied to files:
.github/workflows/ci_python.yml
🪛 Ruff (0.15.12)
python/nemo_flow/utils.py
[warning] 15-15: Dynamically typed expressions (typing.Any) are disallowed in coro
(ANN401)
[warning] 15-15: Dynamically typed expressions (typing.Any) are disallowed in run_sync
(ANN401)
[warning] 34-34: Dynamically typed expressions (typing.Any) are disallowed in _run_with_scope_stack
(ANN401)
[error] 40-41: try-except-pass detected, consider logging the exception
(S110)
[warning] 40-40: Do not catch blind exception: Exception
(BLE001)
🔇 Additional comments (16)
.gitignore (2)
77-77: LGTM!The
deepagentsignore entry is correctly placed in the third-party section and consistent with the other vendored dependency patterns.
95-96: LGTM!The VSCode ignore pattern is functional and correctly limits the match to the repository root.
.github/ci-path-filters.yml (1)
60-60: Python path filter update is correct.This ensures LangChain integration changes trigger Python CI as expected.
pyproject.toml (1)
92-93: Allowlist expansion is appropriate for integration-only imports.This keeps
tysignal clean without affecting base package behavior.python/nemo_flow/__init__.py (1)
186-187: Top-level export wiring forutilsis consistent.Import and
__all__were updated together, which keeps the public API coherent.Also applies to: 422-423
integrations/langchain/src/langchain_nemo_flow/__init__.py (1)
6-12: Public exports are clean and explicit.Good module boundary for the integration package entrypoints.
justfile (1)
769-775: Integration test target wiring looks good.
uv sync --lockedplus inclusion intest-allis a solid CI/local parity improvement.Also applies to: 906-906
integrations/langchain/tests/test_callbacks.py (1)
49-185: Lifecycle and failure-mode coverage is strong.The new tests exercise the key behavioral contracts (push/pop mapping, graceful no-op, and swallowed runtime errors).
.github/workflows/ci_python.yml (1)
84-87: CI integration coverage and packaging steps are correctly added.The workflow changes are coherent with the new package and remain aligned with pinned-action and least-permission patterns.
Also applies to: 114-117, 179-182, 240-246
integrations/langchain/src/langchain_nemo_flow/callbacks.py (2)
40-40: 💤 Low valuePotential IndexError if
serialized["id"]is an empty list.If
serialized.get("id", ["Unknown"])returns an empty list (unlikely but possible),[-1]will raiseIndexError. Since the outer try/except catches this, it won't crash, but the scope won't be created.
20-86: LGTM!Clean implementation of the callback handler. Error swallowing with debug logging is appropriate for framework integration stability. The scope handle lifecycle management is correct.
integrations/langchain/tests/test_middleware.py (2)
64-101: LGTM!Good test coverage for both sync and async model call wrapping. The
RecordingMiddlewarepattern effectively captures and verifies the middleware's interception behavior.
13-14: No action required. The importfrom langchain.agents.middleware import ModelRequest, ModelResponseis valid;langchain.agents.middlewareis a documented LangChain API available in the required version (>=1.2.9). Consistent usage across the codebase confirms the path is correct.integrations/langchain/src/langchain_nemo_flow/middleware.py (1)
31-183: LGTM!The middleware correctly routes LLM and tool calls through NeMo Flow while preserving LangChain's expected interfaces. The sync/async handling via
run_syncand the scope parenting logic are properly implemented.patches/langchain/0001-add-nemo-flow-integration.patch (1)
672-712: LGTM!The lazy import helper in
langchain_core.utils._nemo_flowcorrectly implements optional NeMo Flow loading with proper caching and graceful degradation.integrations/langchain/src/langchain_nemo_flow/_serialization.py (1)
46-56: LGTM!The codec inference logic correctly maps LangChain model types to NeMo Flow codecs, with appropriate handling for the OpenAI responses API variant.
There was a problem hiding this comment.
I am debating if we want to use extras to enable integration support -- e.g.
nemo-flow[langchain] rather than a separate package. Multiple uv.lock files aren't really fasible.
Then we could have a cononical nemo_flow.integrations.<vendor> 🤷🏻
I'm not sure what the best approach is here.
There was a problem hiding this comment.
The big advantage is that we could run a single pytest process to cover both the python extension and the integrations.
The down-side would be if these integrations end up introducing conflicting dependencies.
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Remove utils from being imported in __init__.py to avoid the costly import overhead Signed-off-by: David Gardner <dagardner@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (1)
integrations/langchain/pyproject.toml (1)
40-48:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftNeMo Flow is configured as required, which conflicts with integration optionality guidance.
nemo-flowis currently a hard runtime dependency (dependencies) and also forced viatool.uv.sources. For framework integrations, this should stay optional and preserve baseline framework behavior when NeMo Flow is absent.As per coding guidelines: "Keep NeMo Flow optional in framework integrations" and "Preserve the framework's original behavior when NeMo Flow is absent."
Also applies to: 74-75
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@integrations/langchain/pyproject.toml` around lines 40 - 48, The pyproject currently pins "nemo-flow == {version}" inside [tool.setuptools_dynamic_dependencies] dependencies which makes NeMo Flow a hard runtime dependency; remove "nemo-flow == {version}" from the dependencies array and instead declare it as an optional extra (e.g., under [project.optional-dependencies] or similar) so the integration remains optional, and update any references under tool.uv.sources that force inclusion of NeMo Flow so they only activate when the optional extra is requested; ensure all code paths in the integration check for the presence of NeMo Flow at runtime rather than assuming it is installed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@integrations/langchain/README.md`:
- Around line 14-21: The README's setup commands assume the repository root but
don't state it; update the README (integrations/langchain/README.md) to
explicitly instruct the user to run the following from the repository root (or
change commands to use full paths) so both `pip install -e .` and `pip install
-e integrations/langchain` succeed—add a short line like "Run these commands
from the repository root" above the existing setup block or rewrite the commands
to reference the repo root explicitly.
In `@integrations/langchain/tests/test_middleware.py`:
- Around line 30-53: The test fails because RecordingMiddleware overrides
llm_execute (public) but the middleware base now invokes _llm_execute (private),
so self.calls is never populated; update the test class to override the private
method name by renaming RecordingMiddleware.llm_execute to
RecordingMiddleware._llm_execute (preserve the same signature and body that
appends to self.calls and calls func with the intercepted request) so that
middleware.wrap_model_call / awrap_model_call will call the recording
implementation and assertions on middleware.calls[0] succeed.
In `@python/nemo_flow/utils.py`:
- Around line 37-41: The worker thread's Rust TLS is set by
nemo_flow.set_thread_scope_stack(scope_stack) but never restored, causing stale
scopes for later tasks; wrap the asyncio.run(coro) call inside a try/finally in
_run_with_scope_stack: capture the current thread scope (e.g., prev =
nemo_flow.get_thread_scope_stack() if available) before setting the new scope,
run the coroutine, then in finally restore the previous scope via
nemo_flow.set_thread_scope_stack(prev) (or clear it if no getter exists) so that
_run_with_scope_stack (invoked via _RUN_SYNC_EXECUTOR.submit(ctx.run,
_run_with_scope_stack).result()) always cleans up the thread-local scope stack.
---
Duplicate comments:
In `@integrations/langchain/pyproject.toml`:
- Around line 40-48: The pyproject currently pins "nemo-flow == {version}"
inside [tool.setuptools_dynamic_dependencies] dependencies which makes NeMo Flow
a hard runtime dependency; remove "nemo-flow == {version}" from the dependencies
array and instead declare it as an optional extra (e.g., under
[project.optional-dependencies] or similar) so the integration remains optional,
and update any references under tool.uv.sources that force inclusion of NeMo
Flow so they only activate when the optional extra is requested; ensure all code
paths in the integration check for the presence of NeMo Flow at runtime rather
than assuming it is installed.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 6f18e09b-f8d9-4af0-876d-40212b0dda78
⛔ Files ignored due to path filters (1)
third_party/README-langchain.mdis excluded by!third_party/**
📒 Files selected for processing (8)
AGENTS.mdREADME.mdintegrations/langchain/README.mdintegrations/langchain/pyproject.tomlintegrations/langchain/src/langchain_nemo_flow/middleware.pyintegrations/langchain/tests/test_middleware.pypython/nemo_flow/utils.pypython/tests/test_utils.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (19)
- GitHub Check: Rust / Test (windows-amd64)
- GitHub Check: Node.js / Test (windows-arm64)
- GitHub Check: WebAssembly / Test (linux-amd64)
- GitHub Check: Python / Test (linux-arm64)
- GitHub Check: Node.js / Test (windows-amd64)
- GitHub Check: WebAssembly / Test (macos-arm64)
- GitHub Check: Python / Test (windows-arm64)
- GitHub Check: Python / Test (windows-amd64)
- GitHub Check: WebAssembly / Test (windows-amd64)
- GitHub Check: WebAssembly / Test (linux-arm64)
- GitHub Check: Python / Test (linux-amd64)
- GitHub Check: WebAssembly / Test (windows-arm64)
- GitHub Check: Python / Test (macos-arm64)
- GitHub Check: Rust / Test (macos-arm64)
- GitHub Check: Rust / Test (windows-arm64)
- GitHub Check: Go / Test (windows-arm64)
- GitHub Check: Go / Test (macos-arm64)
- GitHub Check: Go / Test (windows-amd64)
- GitHub Check: Documentation / Build
🧰 Additional context used
📓 Path-based instructions (33)
**/*.{md,rst,html,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
**/*.{md,rst,html,txt}: Always spellNVIDIAin all caps. Do not useNvidia,nvidia,nVidia,nVIDIA, orNV.
Usean NVIDIAbefore a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol afterNVIDIAwhen referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names withNVIDIAon first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms withs, not an apostrophe, such asGPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such asCPU,GPU,PC,API, andUIusually do not need to be spelled out for developer audiences.
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/*.{md,rst,html}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)
Link the first mention of a product name when the destination helps the reader.
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/*.{md,rst,txt}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)
**/*.{md,rst,txt}: SpellNVIDIAin all caps. Do not useNvidia,nvidia, orNV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as 'here' or 'read more.'
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Usecanfor possibility and reservemayfor permission.
Useafterfor temporal relationships instead ofonce.
Preferrefer tooverseewhen the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not add trademark symbols to NeMo Flow learning documentation by default.
Do not rewrite API names, package names, command flags, or code literals for style reasons.
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/*.{md,markdown,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-technical-docs.md)
**/*.{md,markdown,rst}: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Use monospace formatting for code elements, commands, parameters, package names, and expressions
Use monospace formatting for directories, file names, and paths
Use angle brackets inside monospace for variables inside paths, such as/home/<username>/.login
Use quotation marks for error messages and strings in documentation
Use bold formatting for UI buttons, menus, fields, and labels in documentation
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use in documentation
Use italics for publication titles in documentation
Use plain text formatting for keyboard shortcuts in documentation
Prefer[NVIDIA/NeMo](link)format for GitHub repository references over generic phrases like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text in documentation
Avoid generic link anchors such as 'here,' 'this page,' and 'read more' in documentation
Include the acronym in link text if a linked term includes an acronym
Do not link long sentences or multiple sentences in documentation
Avoid links that pull readers away from a procedure unles...
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/*.{html,md}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license headers in HTML and Markdown files using HTML comment syntax
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update relevant crate or package README when that surface changed
Relevant package or crate README.md files must be updated when examples or binding guidance changes
Files:
integrations/langchain/README.mdREADME.md
**/*.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Run Markdown link checking via
lycheeforREADME.md,CONTRIBUTING.md, anddocs/through pre-commit hooks
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/*.{md,markdown,py,sh,bash,js,ts,java,cpp,go,rust}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current in documentation
Files:
integrations/langchain/README.mdpython/tests/test_utils.pyREADME.mdAGENTS.mdpython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{md,markdown,py,sh,bash}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
Files:
integrations/langchain/README.mdpython/tests/test_utils.pyREADME.mdAGENTS.mdpython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{md,markdown,py,sh,bash,js,ts,example}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Example commands must match current package names and paths
Files:
integrations/langchain/README.mdpython/tests/test_utils.pyREADME.mdAGENTS.mdpython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/{integrations,integration,*-integration}/**
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
**/{integrations,integration,*-integration}/**: Keep NeMo Flow optional in framework integrations
Preserve the framework's original behavior when NeMo Flow is absent
Wrap tool and LLM paths at the correct framework boundary
Integration pattern must followdocs/integrate-frameworks/adding-scopes.md
Files:
integrations/langchain/README.mdintegrations/langchain/pyproject.tomlintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Keep documentation and examples synchronized with current install, import, and build commands
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
**/{RELEASING,CHANGELOG,README}.{md,rst}
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Point release history and release notes to GitHub Releases, not to
CHANGELOG.mdor docs pages
Files:
integrations/langchain/README.mdREADME.md
**/*.{md,txt,rst}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
**/*.{md,txt,rst}: Ensure commands, package names, file paths, and APIs in documentation are correct and not stale; flag incorrect or outdated information as blocking issues
Ensure examples and procedures in documentation will execute successfully with current APIs and commands
Use consistent user-facing terminology throughout documentation that matches current repo terminology
Capitalize NVIDIA correctly in all documentation and public-facing text
Format code, commands, paths, and filenames as inline code (monospace) in documentation
Use descriptive anchor text for links instead of bare URLs or weak labels like 'here' in documentation
Prefer active voice, present tense, short sentences, and plain English in documentation
Structure documentation procedures as imperative steps that are easy to scan and not too long for a single sequence
Prefer 'after' instead of 'once' for temporal references in documentation
Use 'can' instead of 'may' when describing possibility (rather than permission) in documentation
Avoid ambiguous numeric dates and ordinal dates in documentation body text
Files:
integrations/langchain/README.mdREADME.mdAGENTS.md
{README.md,docs/index.md,**/README.md}
📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)
Update entry-point documentation (README.md, docs/index.md, package READMEs, binding-level source READMEs) whenever public behavior changes
Files:
integrations/langchain/README.mdREADME.md
**/*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Format changed Python wrapper and test files with
uv run ruff format python
**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter for Python code with line length 120 and double quotes
Usetyfor Python type checking
Use snake_case naming convention for Python code
Include SPDX license headers in all Python source files using Python comment syntaxFollow binding naming conventions: Python uses
snake_casefor naming.
Files:
python/tests/test_utils.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/test_*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Run focused
pytestfirst when you know the affected area usingpytest -k "<pattern>"
Files:
python/tests/test_utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use
SONAR_IGNORE_START/SONAR_IGNORE_ENDmarkers only for documented false positives that cannot be resolved in code; keep ignored blocks small, add explanatory comments, and require reviewer sign-off
Files:
python/tests/test_utils.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{rs,py,go,js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,go,js,ts,jsx,tsx}: Format changed files with the language-native formatter before the final lint/test pass
Use language-native formatters:cargo fmt --allfor Rust,uv run ruff format pythonfor Python,go fmt ./...for Go,npm --prefix crates/node run formatfor Node.js, andnpm --prefix crates/node run precommit:format -- crates/wasm/wrappers crates/wasm/tests-js crates/wasm/scriptsfor WebAssembly
Files:
python/tests/test_utils.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{py,go,js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change
Files:
python/tests/test_utils.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
**/*.{js,ts,tsx,jsx,py,rs,go,java,c,cpp,h,cc,cxx,cs,rb,php,swift,kt}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changed files must be formatted with the language-native formatter
Files:
python/tests/test_utils.pypython/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.pyintegrations/langchain/src/langchain_nemo_flow/middleware.py
{crates/**/tests/**,python/tests/**,go/nemo_flow/**/*_test.go}
⚙️ CodeRabbit configuration file
{crates/**/tests/**,python/tests/**,go/nemo_flow/**/*_test.go}: Tests should cover the behavior promised by the changed API surface, including error paths and cross-request isolation where relevant.
Prefer assertions on lifecycle events, scope stacks, middleware ordering, and binding parity over shallow smoke tests.
Files:
python/tests/test_utils.py
README.md
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Update README.md to reflect current workspace members and top-level documentation for changes affecting public behavior, bindings, examples, or workspace structure
Files:
README.md
{README.md,docs/index.md}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
{README.md,docs/index.md}: Update entry-point docs when examples or reading paths change
README.md or docs/index.md must be updated when entry points change
Files:
README.md
{README*,CHANGELOG*,docs/**/*.{md,rst,txt},examples/**/*,*.md}
📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)
Update documentation, examples, and getting-started guides with new package/module/crate names after rename operations
Files:
README.mdAGENTS.md
{docs/**/*.md,README.md,CONTRIBUTING.md}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
{docs/**/*.md,README.md,CONTRIBUTING.md}: For docs-only changes, run targeted checks only if commands, package names, or examples changed; usejust docsfor docs-site builds andjust docs-linkcheckwhen links changed
Verify that README and docs entry points match current package names and paths for large or public-facing changes
Files:
README.md
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}
⚙️ CodeRabbit configuration file
{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.
Files:
README.md
AGENTS.md
📄 CodeRabbit inference engine (CLAUDE.md)
Document agent implementations in AGENTS.md with clear descriptions of functionality and usage
Files:
AGENTS.md
python/nemo_flow/**/*.py
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Python API implementations and wrappers
Files:
python/nemo_flow/utils.py
{docs/**,examples/**,crates/adaptive/**,python/nemo_flow/**,go/nemo_flow/**,**/{example,component}.{ts,tsx,js,rs,py,go}}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Any new adaptive component kind must have documentation, examples, and binding coverage across all supported languages
Files:
python/nemo_flow/utils.py
python/nemo_flow/**/*
⚙️ CodeRabbit configuration file
python/nemo_flow/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_flow/utils.py
**/*.toml
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Include SPDX license headers in TOML files using TOML comment syntax
Files:
integrations/langchain/pyproject.toml
**/pyproject.toml
📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)
Maintain consistency in Python packaging metadata in
pyproject.toml
Files:
integrations/langchain/pyproject.toml
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Preserve the shared runtime model across bindings. Do not add behavior to one primary binding without considering Rust, Python, and Node.js parity.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Keep async behavior on the existing tokio-based model. Bindings should preserve callback and future lifetimes rather than blocking or hiding async work unexpectedly.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Do not hand-edit generated or packaged outputs unless the repository workflow expects them to be checked in. Regenerate through the documented recipe or script.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Prefer the repository `just` recipes over raw tool commands. Use raw `cargo`, `pytest`, `go test`, `npm`, or `wasm-pack` commands only for focused debugging or targeted single-test reruns that do not have a `just` recipe.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Run tests for every language affected by a change. If you touch the Rust core runtime, middleware semantics, event shape, scope behavior, typed codecs, plugins, or observability, expect to validate every affected binding because the bindings share the same runtime contract.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Use `uv run pre-commit run --all-files` before review when the change crosses languages or tooling. The hooks enforce SPDX headers, file hygiene, Ruff, `ty`, docs link checks, Cargo formatting/lints/audits, Go formatting/vet, Node formatting, and public docstring checks.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Use branch prefixes from the contributor docs: `feat/`, `fix/`, `docs/`, `test/`, or `refactor/`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Use signed-off commits for PR work: `git commit -s`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: PR descriptions should include what changed, why, how it was tested, and any breaking changes within the repository template format.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: If repo-local PR guidance such as the `prepare-pr` skill conflicts with generic GitHub connector or plugin guidance, follow the repo-local PR guidance for PR body format and review handoff details.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Rust is the source of truth for runtime behavior. Binding APIs should mirror the Rust semantics unless a language-specific wrapper intentionally improves ergonomics.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Go uses the C FFI and requires the FFI library build before tests; `just test-go` handles the library path setup.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: WebAssembly includes Rust wasm-bindgen tests plus JS wrapper/package tests; `just test-wasm` runs both paths.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: Use the stable root-level wrappers `./scripts/bootstrap-third-party.sh`, `./scripts/apply-patches.sh`, and `./scripts/generate-patches.sh` for third-party integration management.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:08:59.545Z
Learning: `apply-patches.sh` expects clean third-party checkouts. After editing an integration checkout, run `./scripts/generate-patches.sh` to regenerate patch files and verify they apply to a clean detached checkout.
🪛 markdownlint-cli2 (0.22.1)
README.md
[warning] 136-136: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 139-139: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
AGENTS.md
[warning] 209-209: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
[warning] 232-232: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
🪛 Ruff (0.15.12)
python/nemo_flow/utils.py
[warning] 17-17: Dynamically typed expressions (typing.Any) are disallowed in coro
(ANN401)
[warning] 17-17: Dynamically typed expressions (typing.Any) are disallowed in run_sync
(ANN401)
[warning] 37-37: Dynamically typed expressions (typing.Any) are disallowed in _run_with_scope_stack
(ANN401)
[error] 43-44: try-except-pass detected, consider logging the exception
(S110)
[warning] 43-43: Do not catch blind exception: Exception
(BLE001)
🔇 Additional comments (5)
python/nemo_flow/utils.py (1)
43-44: Silent exception swallowing — already noted in a previous review.The bare
except Exception: passstill makes it impossible to diagnose propagation failures; Ruff S110/BLE001 confirm this.AGENTS.md (1)
41-41: LGTM.New sections accurately describe the integration pattern and directory layout.
Also applies to: 209-236
README.md (1)
136-141: LGTM.python/tests/test_utils.py (1)
13-31: LGTM.The barrier-based synchronization reliably triggers the "loop already running" offload path and validates that both concurrent callers complete with correct values.
integrations/langchain/src/langchain_nemo_flow/middleware.py (1)
51-89: LGTM — private helpers correctly named with underscore prefix per reviewer feedback.
dagardner-nv
changed the title
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Signed-off-by: David Gardner <96306125+dagardner-nv@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
python/nemo_flow/utils.py (1)
34-49:⚠️ Potential issue | 🔴 Critical | ⚡ Quick win
except Exceptionswallows coroutine errors, then attempts to re-run an exhausted coroutine.The
tryat line 34 spans bothnemo_flow.get_scope_stack()and the.result()call at line 44.Future.result()re-raises exceptions from the worker, which includes any application-level error thrown insideasyncio.run(coro). That exception is silently swallowed byexcept Exception: pass, and then line 49 attempts to pass the same — already-started — coroutine object toasyncio.runa second time. Python will raiseRuntimeError: cannot reuse already awaited coroutine, permanently masking the original error.Two previously flagged issues also remain unresolved: the exception is swallowed without logging (Ruff S110/BLE001), and the fallback at line 49 drops the
ctxcaptured at line 33.The fix is to narrow the
tryto cover only the scope-stack lookup, then propagatectxon all paths:🐛 Proposed fix
+import logging + +_LOG = logging.getLogger(__name__) _RUN_SYNC_EXECUTOR = ThreadPoolExecutor()ctx = contextvars.copy_context() - try: - scope_stack = nemo_flow.get_scope_stack() - - def _run_with_scope_stack() -> Any: - nemo_flow.set_thread_scope_stack(scope_stack) - try: - return asyncio.run(coro) - finally: - nemo_flow.set_thread_scope_stack(None) - - return _RUN_SYNC_EXECUTOR.submit(ctx.run, _run_with_scope_stack).result() - - except Exception: - pass # Fall through to vanilla path - - return _RUN_SYNC_EXECUTOR.submit(asyncio.run, coro).result() + try: + scope_stack = nemo_flow.get_scope_stack() + except Exception: + _LOG.debug("Scope stack propagation failed, falling back to plain executor", exc_info=True) + return _RUN_SYNC_EXECUTOR.submit(ctx.run, asyncio.run, coro).result() + + def _run_with_scope_stack() -> Any: + nemo_flow.set_thread_scope_stack(scope_stack) + try: + return asyncio.run(coro) + finally: + nemo_flow.set_thread_scope_stack(None) + + return _RUN_SYNC_EXECUTOR.submit(ctx.run, _run_with_scope_stack).result()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@python/nemo_flow/utils.py` around lines 34 - 49, The try block currently covers both nemo_flow.get_scope_stack() and the Future.result() call, causing application errors from asyncio.run(coro) to be swallowed and the coroutine to be re-used; narrow the try to only wrap nemo_flow.get_scope_stack() so exceptions from the submitted task propagate, keep using _run_with_scope_stack (which calls nemo_flow.set_thread_scope_stack before asyncio.run and clears it after) and submit it via _RUN_SYNC_EXECUTOR.submit(ctx.run, _run_with_scope_stack).result(), and ensure the fallback path also submits via ctx.run (not plain asyncio.run) so ctx is preserved; remove the bare except Exception: pass and either let exceptions propagate or log them instead so you never attempt to await an already-exhausted coroutine.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@integrations/langchain/tests/test_middleware.py`:
- Line 13: The test fails because ModelResponse is not re-exported from
langchain.agents.middleware; update the middleware package to re-export it by
adding an import/export for ModelResponse (and ModelRequest if missing) from the
internal types module (e.g., add "from .types import ModelRequest,
ModelResponse" to the middleware package init) so the test import in
test_middleware.py succeeds, or alternatively change the test and any runtime
imports to import ModelResponse directly from langchain.agents.middleware.types.
---
Duplicate comments:
In `@python/nemo_flow/utils.py`:
- Around line 34-49: The try block currently covers both
nemo_flow.get_scope_stack() and the Future.result() call, causing application
errors from asyncio.run(coro) to be swallowed and the coroutine to be re-used;
narrow the try to only wrap nemo_flow.get_scope_stack() so exceptions from the
submitted task propagate, keep using _run_with_scope_stack (which calls
nemo_flow.set_thread_scope_stack before asyncio.run and clears it after) and
submit it via _RUN_SYNC_EXECUTOR.submit(ctx.run,
_run_with_scope_stack).result(), and ensure the fallback path also submits via
ctx.run (not plain asyncio.run) so ctx is preserved; remove the bare except
Exception: pass and either let exceptions propagate or log them instead so you
never attempt to await an already-exhausted coroutine.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Enterprise
Run ID: 6b19a65f-ea30-4a14-855b-979ceda56132
📒 Files selected for processing (2)
integrations/langchain/tests/test_middleware.pypython/nemo_flow/utils.py
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Check / Run
🧰 Additional context used
📓 Path-based instructions (14)
**/*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Format changed Python wrapper and test files with
uv run ruff format python
**/*.py: Use Ruff with rule sets E, F, W, I for Python linting
Use Ruff formatter for Python code with line length 120 and double quotes
Usetyfor Python type checking
Use snake_case naming convention for Python code
Include SPDX license headers in all Python source files using Python comment syntax
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{rs,py,go,js,ts,tsx}
📄 CodeRabbit inference engine (CONTRIBUTING.md)
Use
SONAR_IGNORE_START/SONAR_IGNORE_ENDmarkers only for documented false positives that cannot be resolved in code; keep ignored blocks small, add explanatory comments, and require reviewer sign-off
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
python/nemo_flow/**/*.py
📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)
Use
snake_casenaming convention for Python API implementations and wrappers
Files:
python/nemo_flow/utils.py
**/*.{md,markdown,py,sh,bash,js,ts,java,cpp,go,rust}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep package names, repo references, and build commands current in documentation
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{md,markdown,py,sh,bash}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{md,markdown,py,sh,bash,js,ts,example}
📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)
Example commands must match current package names and paths
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
{docs/**,examples/**,crates/adaptive/**,python/nemo_flow/**,go/nemo_flow/**,**/{example,component}.{ts,tsx,js,rs,py,go}}
📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)
Any new adaptive component kind must have documentation, examples, and binding coverage across all supported languages
Files:
python/nemo_flow/utils.py
**/*.{rs,py,go,js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
**/*.{rs,py,go,js,ts,jsx,tsx}: Format changed files with the language-native formatter before the final lint/test pass
Use language-native formatters:cargo fmt --allfor Rust,uv run ruff format pythonfor Python,go fmt ./...for Go,npm --prefix crates/node run formatfor Node.js, andnpm --prefix crates/node run precommit:format -- crates/wasm/wrappers crates/wasm/tests-js crates/wasm/scriptsfor WebAssembly
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{py,go,js,ts,jsx,tsx}
📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)
If a language surface changed, always run that language's test target even when Rust core did not change
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{js,ts,tsx,jsx,py,rs,go,java,c,cpp,h,cc,cxx,cs,rb,php,swift,kt}
📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)
Changed files must be formatted with the language-native formatter
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
**/*.{rs,py,js,ts,mjs,cjs,go,sh,yml,yaml,toml,lock}
📄 CodeRabbit inference engine (AGENTS.md)
Keep SPDX headers on source, docs, scripts, and configuration files. The project is Apache-2.0.
Files:
python/nemo_flow/utils.pyintegrations/langchain/tests/test_middleware.py
python/nemo_flow/**/*
⚙️ CodeRabbit configuration file
python/nemo_flow/**/*: Review Python wrapper changes for typed API consistency, contextvars-based scope isolation, async behavior, and parity with the native extension.
Stubs and runtime implementations should stay aligned.
Files:
python/nemo_flow/utils.py
**/test_*.py
📄 CodeRabbit inference engine (.agents/skills/test-python-binding/SKILL.md)
Run focused
pytestfirst when you know the affected area usingpytest -k "<pattern>"
Files:
integrations/langchain/tests/test_middleware.py
**/{integrations,integration,*-integration}/**
📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)
**/{integrations,integration,*-integration}/**: Keep NeMo Flow optional in framework integrations
Preserve the framework's original behavior when NeMo Flow is absent
Wrap tool and LLM paths at the correct framework boundary
Integration pattern must followdocs/integrate-frameworks/adding-scopes.md
Files:
integrations/langchain/tests/test_middleware.py
🧠 Learnings (1)
📓 Common learnings
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Follow binding naming conventions: Rust and Python `snake_case`, C FFI exports prefixed `nemo_flow_`, Go `PascalCase` for public APIs, Node.js `camelCase`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Preserve the shared runtime model across bindings. Do not add behavior to one primary binding without considering Rust, Python, and Node.js parity.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Do not hand-edit generated or packaged outputs unless the repository workflow expects them to be checked in. Regenerate through the documented recipe or script.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Scope stacks are hierarchical and always have a root scope. They establish parent-child event relationships, visibility for scope-local middleware and subscribers, cleanup boundaries, and concurrent request isolation.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Scope-local middleware and subscribers are owned by a scope and disappear when that scope closes. Global registrations stay process-wide until removed.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Middleware is priority-ordered after merging global and visible scope-local entries.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Intercepts change the real execution path. Request intercepts rewrite the request. Execution intercepts wrap or replace the callback. Stream execution intercepts handle streaming lifecycle behavior.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Guardrails either block execution or sanitize emitted observability payloads. Sanitize guardrails do not rewrite the real callback arguments or return value.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Managed execution order is conditional guardrails, request intercepts, sanitize-request guardrails for start events, execution intercepts, callback execution, then sanitize-response guardrails for end events.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Exporters can transform runtime events to ATIF trajectories, OpenTelemetry traces, or OpenInference-compatible output. Root scope identity is used to isolate concurrent agents.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Rust is the source of truth for runtime behavior. Binding APIs should mirror the Rust semantics unless a language-specific wrapper intentionally improves ergonomics.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Python wrapper modules live under `python/nemo_flow/`; the native extension is built from `crates/python` with `maturin`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Node.js public entry points include the main runtime package plus `nemo-flow-node/typed`, `nemo-flow-node/plugin`, and `nemo-flow-node/adaptive`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Go uses the C FFI and requires the FFI library build before tests; `just test-go` handles the library path setup.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: WebAssembly includes Rust wasm-bindgen tests plus JS wrapper/package tests; `just test-wasm` runs both paths.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Use `./scripts/bootstrap-third-party.sh`, `./scripts/apply-patches.sh`, and `./scripts/generate-patches.sh` for managing patch-based integrations.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: `apply-patches.sh` expects clean third-party checkouts. After editing an integration checkout, run `./scripts/generate-patches.sh` to regenerate patch files and verify they apply to a clean detached checkout.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Update `README.md`, `docs/`, package READMEs, and binding-support notes when public behavior, package names, examples, or supported bindings change.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Keep release-process details in maintainer docs such as `RELEASING.md`. Do not move release-history policy into user-facing docs or `CHANGELOG.md`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Use branch prefixes from the contributor docs: `feat/`, `fix/`, `docs/`, `test/`, or `refactor/`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Use signed-off commits for PR work: `git commit -s`.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Before creating, opening, publishing, or editing a pull request, read `.github/pull_request_template.md` and use it as the PR body skeleton. Preserve its visible headings, checklist items, and related-issue guidance; fill the sections instead of replacing them with a generic summary.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: If repo-local PR guidance such as the `prepare-pr` skill conflicts with generic GitHub connector or plugin guidance, follow the repo-local PR guidance for PR body format and review handoff details.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: PR descriptions should include what changed, why, how it was tested, and any breaking changes within the repository template format.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Run tests for every language affected by a change. If you touch the Rust core runtime, middleware semantics, event shape, scope behavior, typed codecs, plugins, or observability, expect to validate every affected binding because the bindings share the same runtime contract.
Learnt from: CR
Repo: NVIDIA/NeMo-Flow
Timestamp: 2026-05-07T23:57:42.795Z
Learning: Before review, prefer `uv run pre-commit run --all-files` when the change crosses languages or tooling. The hooks enforce SPDX headers, file hygiene, Ruff, `ty`, docs link checks, Cargo formatting/lints/audits, Go formatting/vet, Node formatting, and public docstring checks.
🪛 Ruff (0.15.12)
python/nemo_flow/utils.py
[warning] 17-17: Dynamically typed expressions (typing.Any) are disallowed in coro
(ANN401)
[warning] 17-17: Dynamically typed expressions (typing.Any) are disallowed in run_sync
(ANN401)
[warning] 37-37: Dynamically typed expressions (typing.Any) are disallowed in _run_with_scope_stack
(ANN401)
[error] 46-47: try-except-pass detected, consider logging the exception
(S110)
[warning] 46-46: Do not catch blind exception: Exception
(BLE001)
🔇 Additional comments (1)
integrations/langchain/tests/test_middleware.py (1)
25-53:_llm_executeoverride correctly uses the private name — prior regression fixed.
RecordingMiddlewarenow overrides_llm_execute(with underscore) matching the base class's dispatch path, soself.callsis populated and assertions onmiddleware.calls[0]no longer throwIndexError. The intercept logic (appending metadata, reconstructing theLLMRequestwithtemperature: 0.25, thenawait func(intercepted)) is clean.
|
|
||
| import nemo_flow | ||
| import pytest | ||
| from langchain.agents.middleware import ModelRequest, ModelResponse |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify whether the PR's LangChain patches add ModelResponse to
# langchain/agents/middleware/__init__.py.
# Expect: a line matching `ModelResponse` in the patched __init__.py,
# or a patch file that adds it.
# Check patch files for the relevant export fix
rg -n "ModelResponse" --type=patch
# Also check any third-party/langchain patch files
fd -e patch -e diff | xargs grep -l "ModelResponse" 2>/dev/nullRepository: NVIDIA/NeMo-Flow
Length of output: 92
🏁 Script executed:
#!/bin/bash
# Find patch files and LangChain-related configuration
find . -type f \( -name "*.patch" -o -name "*.diff" \) | head -20
# Look for common patch directories
fd -d 2 -type d -e patch -e patches -e diffs
# Check for poetry.lock, requirements, setup.py to see LangChain version/patches
find . -maxdepth 3 -type f \( -name "poetry.lock" -o -name "pyproject.toml" -o -name "requirements*.txt" \)
# Search for any file mentioning langchain patches or monkey-patching
rg -l "langchain.*patch" --type=py -iRepository: NVIDIA/NeMo-Flow
Length of output: 696
🏁 Script executed:
#!/bin/bash
# Inspect the langchain patch file
cat "./patches/langchain/0001-add-nemo-flow-integration.patch" | head -500Repository: NVIDIA/NeMo-Flow
Length of output: 21771
🏁 Script executed:
#!/bin/bash
# Search the langchain patch for ModelResponse and agents.middleware references
rg "ModelResponse" ./patches/langchain/0001-add-nemo-flow-integration.patch
# Also check for agents/middleware changes
rg "agents/middleware|agents.middleware" ./patches/langchain/0001-add-nemo-flow-integration.patchRepository: NVIDIA/NeMo-Flow
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check the test file to see if ModelResponse is actually used
cat -n ./integrations/langchain/tests/test_middleware.py | head -150
# Also search for ModelResponse usage
rg "ModelResponse" ./integrations/langchain/tests/test_middleware.pyRepository: NVIDIA/NeMo-Flow
Length of output: 5452
🏁 Script executed:
#!/bin/bash
# Check LangChain version requirements
rg "langchain" ./pyproject.toml ./integrations/langchain/pyproject.toml -A 2
# Also check if there are any constraint specifications
rg "langchain.*=" ./pyproject.toml ./integrations/langchain/pyproject.tomlRepository: NVIDIA/NeMo-Flow
Length of output: 1328
🏁 Script executed:
#!/bin/bash
# Look for any documentation or comments about LangChain agents.middleware
rg -i "modelresponse|agents.middleware" --type=md --type=txt
# Check if there are any GitHub issue references in code or docs
rg "33453|33501"
# Look at the test file more carefully for any skip markers or version checks
rg "skip|xfail|version" ./integrations/langchain/tests/test_middleware.py -iRepository: NVIDIA/NeMo-Flow
Length of output: 42
🏁 Script executed:
#!/bin/bash
# Check if ModelResponse is used elsewhere in the codebase
rg "ModelResponse" --type=py
# Check the NemoFlowMiddleware to understand what it imports
find ./integrations -name "middleware.py" -o -name "*middleware*" -type f | grep -v __pycache__
# Look at the middleware imports
cat ./integrations/langchain/src/langchain_nemo_flow/middleware.py 2>/dev/null | head -50Repository: NVIDIA/NeMo-Flow
Length of output: 4211
🏁 Script executed:
#!/bin/bash
# Check if the patch modifies _serialization.py
rg "_serialization|_model_response" ./patches/langchain/0001-add-nemo-flow-integration.patch | head -20
# Look at the full _serialization.py to understand the imports
cat ./integrations/langchain/src/langchain_nemo_flow/_serialization.py | head -80Repository: NVIDIA/NeMo-Flow
Length of output: 2740
🏁 Script executed:
#!/bin/bash
# Check if ModelResponse import in _serialization.py is guarded
cat ./integrations/langchain/src/langchain_nemo_flow/_serialization.py | grep -A5 -B5 "from langchain.agents.middleware import ModelResponse"
# Check if the imports are TYPE_CHECKING guarded
head -30 ./integrations/langchain/src/langchain_nemo_flow/_serialization.py | grep -E "(TYPE_CHECKING|^if|^from langchain.agents)"Repository: NVIDIA/NeMo-Flow
Length of output: 405
Verify that ModelResponse is importable from langchain.agents.middleware in the patched LangChain version.
There is a documented bug in LangChain (ModelResponse is defined in langchain/agents/middleware/types.py but was omitted from langchain/agents/middleware/__init__.py's re-exports), causing ImportError when importing from langchain.agents.middleware. This test file and the middleware implementation import ModelResponse at runtime (not guarded by TYPE_CHECKING), so test collection will fail if the bug is not fixed.
The PR's LangChain patch does not include changes to agents/middleware/__init__.py to add the missing re-export. Unless langchain >= 1.2.9 has already fixed this upstream, the tests will fail at import time. Confirm either:
- That the target langchain version has already fixed this re-export, OR
- Add the missing re-export to the patch
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@integrations/langchain/tests/test_middleware.py` at line 13, The test fails
because ModelResponse is not re-exported from langchain.agents.middleware;
update the middleware package to re-export it by adding an import/export for
ModelResponse (and ModelRequest if missing) from the internal types module
(e.g., add "from .types import ModelRequest, ModelResponse" to the middleware
package init) so the test import in test_middleware.py succeeds, or
alternatively change the test and any runtime imports to import ModelResponse
directly from langchain.agents.middleware.types.
willkill07
left a comment
willkill07
left a comment
There was a problem hiding this comment.
Thanks for the PR. The overall direction is good: this uses LangChain’s public AgentMiddleware boundaries instead of patching LangChain for the new package, it keeps the integration observability-focused, and the callback tests do a nice job covering basic scope lifecycle and parent/child scope propagation.
I do think a few changes are needed before merge:
-
Use the root lockfile / uv workspace model for the integration dependencies. This PR adds
integrations/langchain/uv.lockand updates CI to cache both that file and the rootuv.lock. For this repo, please keep dependency resolution centralized: addintegrations/langchainas a uv workspace member, make the package resolve through the root workspace/rootuv.lock, remove the nested lockfile, and update thejust/CI recipes accordingly. uv’s workspace docs call out this exact model: each member can have its ownpyproject.toml, while the workspace shares a single lockfile. Reference: https://docs.astral.sh/uv/concepts/projects/workspaces/ -
Fix
run_syncbefore relying on it from the middleware.uv run pytest python/tests/test_utils.pyfails locally withRuntimeError: cannot reuse already awaited coroutine. The broadexcept Exception: passaround the scope-stack path can swallow a failure after the coroutine has already been submitted/awaited, then the fallback resubmits the same consumed coroutine. Please restructure this so only expected scope-stack lookup/setup failures fall back before the coroutine is executed, and worker execution failures propagate normally. The fallback should also run under the captured context rather than droppingcontextvars. -
Add focused coverage for the tool-call wrappers and the context/scope bridge. The integration implements sync and async model wrappers plus sync and async tool wrappers, but tests currently cover only model wrapping and callback lifecycle. LangChain documents
wrap_model_callandwrap_tool_callas the wrap-style boundaries around model and tool calls, so the tool path should be tested too: args rewrite,ToolMessage/Commandpassthrough, async path, and behavior when NeMo Flow changes tool args. Reference: https://docs.langchain.com/oss/python/langchain/middleware/custom -
Keep the PR scope tight.
_llm_stream_executeis currently unused and there is no streaming hook/test path for it, so I would remove it until it is wired through real public LangChain behavior. Separately, this PR updates the patch-based LangChain integration refs/patches and addsthird_party/deepagentswhile introducing a public API-based integration. If those patch refreshes are necessary, please call out why in the PR; otherwise they should be split out so this stays focused on the public LangChain observability package. -
Small hygiene items worth fixing while you are here. Make the internal
langchain_run_idmetadata key win over caller-supplied metadata, make the serialized-id fallback safe for an emptyidlist, fix the nonstandard# ty: ignore[...]comments in tests, and remove trailing whitespace frompatches/langchain/0001-add-nemo-flow-integration.patch(git diff --checkcurrently fails on that file).
Validation I ran:
just test-python-integrationpasses: 14 passed, 1 upstream LangGraph deprecation warning.uv run pytest python/tests/test_utils.pyfails with therun_synccoroutine reuse issue above.uv run ruff check python/nemo_flow/utils.py python/tests/test_utils.py integrations/langchain/src/langchain_nemo_flow integrations/langchain/testspasses.uv lock --checkpasses in both the root andintegrations/langchain, but the nested lockfile structure is still the concern.git diff --check c02d24cc31c26b656c5934daa9596a69124eeb8b...HEADfails due trailing whitespace in the LangChain patch file.
I also cross-checked the LangChain public middleware references: the use of wrap_model_call/awrap_model_call and wrap_tool_call/awrap_tool_call is the right integration boundary for this style of observability middleware; the main remaining issues are dependency/workspace shape, the sync bridge bug, and missing coverage for the tool path.
2 participants
Overview
integrations/langchainnemo_flow.utils.run_syncmethod, I figured other integrations will likely need this.Details
Where should the reviewer start?
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)
Summary by CodeRabbit
New Features
Tests
Documentation
Chores