Skip to content

Propagate cancellation to model and EP downloads#674

Open
bmehta001 wants to merge 11 commits into
mainfrom
bhamehta/sdk-download-cancellation
Open

Propagate cancellation to model and EP downloads#674
bmehta001 wants to merge 11 commits into
mainfrom
bhamehta/sdk-download-cancellation

Conversation

@bmehta001
Copy link
Copy Markdown
Contributor

@bmehta001 bmehta001 commented Apr 24, 2026

Adds cancellation support for SDK model and execution provider downloads across C#, Python, JavaScript/TypeScript, Rust, and C++.

  • Cancellation is wired through the existing native streaming callback path so downloads can be interrupted at the next native progress/callback boundary while preserving the existing non-streaming fast path when no progress callback or cancellation token/signal is supplied.

Changes

  • Added cancellable model download APIs across SDKs:
    • C#: CancellationToken
    • Python: cancellation event
    • JS/TS: AbortSignal, including model.download(signal)
    • Rust: Arc<AtomicBool>
    • C++: CancellationCallback
  • Preserved existing progress callback behavior and added cancellation support alongside progress callbacks.
  • Kept fast paths for downloads that do not request progress or cancellation.
  • Hardened callback handling:
    • Avoids converting successful operations into cancellation when cancellation is requested after the final observed callback.
    • Ensures native response buffers are cleaned up before surfacing cancellation/errors.
    • Avoids flushing buffered partial streaming data after cancellation.
  • Improved mixed progress parsing so status text and numeric progress chunks are handled consistently.
  • Updated SDK docs and samples

Validation

  • Added cancellation/progress parsing tests for C#, Python, JS, Rust

Copilot AI review requested due to automatic review settings April 24, 2026 17:02
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 24, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
foundry-local Ready Ready Preview, Comment May 14, 2026 4:03am

Request Review

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR adds download cancellation support across SDKs by routing model and EP downloads through streaming/callback interop when a cancellation signal/token/flag is supplied, and documenting the new behavior.

Changes:

  • Rust: add cancellable variants of model/EP download APIs and cancellable streaming interop in CoreInterop.
  • Python/JS/C#: add cancellation parameters (Event/AbortSignal/CancellationToken) and ensure downloads use callback/streaming paths to observe cancellation.
  • Docs/tests: document cancellation usage and add unit tests verifying the callback/streaming path is used when cancellation is provided.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
sdk/rust/src/foundry_local_manager.rs Adds cancellable EP download APIs and routes to cancellable streaming interop.
sdk/rust/src/detail/model_variant.rs Adds cancellable model variant download path via cancellable streaming interop.
sdk/rust/src/detail/model.rs Exposes cancellable model download API.
sdk/rust/src/detail/core_interop.rs Adds cancellable streaming interop and cancellation checks in the FFI callback trampoline.
sdk/rust/README.md Documents Rust cancellation usage for downloads.
sdk/python/test/test_model.py Adds a test ensuring model download uses callback path when cancel_event is provided.
sdk/python/test/test_foundry_local_manager.py Adds a test ensuring EP download uses callback path when cancel_event is provided.
sdk/python/src/imodel.py Extends download() interface with optional cancel_event.
sdk/python/src/foundry_local_manager.py Adds cancel_event support and routes through callback interop when provided.
sdk/python/src/detail/model_variant.py Adds cancel_event support and routes through callback interop when provided.
sdk/python/src/detail/model.py Plumbs cancel_event through Model.download() to the selected variant.
sdk/python/src/detail/core_interop.py Adds cancellation support to callback helper and raises on cancellation.
sdk/python/README.md Documents Python cancellation usage for downloads.
sdk/js/test/model.test.ts Adds a test ensuring model download uses streaming interop when only AbortSignal is provided.
sdk/js/test/foundryLocalManager.test.ts Adds a test ensuring EP download passes AbortSignal through to streaming interop.
sdk/js/src/imodel.ts Extends download() interface with optional AbortSignal.
sdk/js/src/foundryLocalManager.ts Adds AbortSignal overloads and passes signal to streaming interop.
sdk/js/src/detail/modelVariant.ts Routes model download to streaming when progress or cancellation is needed.
sdk/js/src/detail/model.ts Plumbs AbortSignal through Model.download() to the selected variant.
sdk/js/src/detail/coreInterop.ts Adds AbortSignal-aware streaming wrapper that rejects with AbortError on cancellation.
sdk/js/README.md Documents JS cancellation usage for downloads.
sdk/cs/test/FoundryLocal.Tests/Utils.cs Improves repo-root discovery to handle .git files (worktrees/submodules).
sdk/cs/test/FoundryLocal.Tests/DownloadCancellationTests.cs Adds a test covering cancellation propagation via callback path in model download.
sdk/cs/src/FoundryLocalManager.cs Routes EP download through callback path when cancellation is possible; checks token in callback.
sdk/cs/src/Detail/ModelVariant.cs Routes model download through callback path when cancellation is possible; checks token in callback.
sdk/cs/src/Detail/CoreInterop.cs Preserves OperationCanceledException propagation from callback execution.
sdk/cs/README.md Documents C# cancellation usage for downloads.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sdk/python/src/detail/core_interop.py
Comment thread sdk/python/src/detail/model_variant.py Outdated
Comment thread sdk/js/src/foundryLocalManager.ts Outdated
Comment thread sdk/rust/src/detail/core_interop.rs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 41 out of 41 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sdk/python/src/detail/core_interop.py Outdated
Comment thread sdk/cs/src/Detail/CoreInterop.cs Outdated
Comment thread sdk/cpp/src/core_helpers.h Outdated
Comment thread sdk/cs/src/Detail/ModelVariant.cs Outdated
@bmehta001 bmehta001 changed the title [Draft] Propagate cancellation to model and EP downloads Propagate cancellation to model and EP downloads May 4, 2026
@bmehta001 bmehta001 force-pushed the bhamehta/sdk-download-cancellation branch from d681c4e to b6d73f1 Compare May 4, 2026 17:49
@bmehta001 bmehta001 force-pushed the bhamehta/sdk-download-cancellation branch from b6d73f1 to 6968e52 Compare May 4, 2026 23:05
@bmehta001 bmehta001 force-pushed the bhamehta/sdk-download-cancellation branch from 6968e52 to 1fe95bd Compare May 7, 2026 20:45
@bmehta001 bmehta001 force-pushed the bhamehta/sdk-download-cancellation branch from 1fe95bd to 649ced6 Compare May 9, 2026 03:34
Comment thread sdk/python/requirements.txt Outdated
bmehta001 and others added 7 commits May 12, 2026 15:40
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Keep existing model download signatures source-compatible while adding cancellable alternatives and overload coverage across SDKs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Expose cancellation as additional parameters on existing C++ and Python download methods instead of adding Cancellable method names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use defaulted cancellation parameters on existing C++ download APIs instead of separate overloads now that the SDK has not officially shipped.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Use a defaulted cancellation token parameter on IModel.DownloadAsync now that custom implementers do not need to be preserved.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add AborSignal parameter to chat completions in ChatClient (JS) CancellationToken Not Honored in IModel.DownloadAsync Method

2 participants