Skip to content

feat(cache-provider): add on_set_prompt lifecycle hook (CORE-246)#14018

Draft
deepme987 wants to merge 1 commit into
masterfrom
deepme987/cache-provider-compute-timing
Draft

feat(cache-provider): add on_set_prompt lifecycle hook (CORE-246)#14018
deepme987 wants to merge 1 commit into
masterfrom
deepme987/cache-provider-compute-timing

Conversation

@deepme987
Copy link
Copy Markdown
Contributor

@deepme987 deepme987 commented May 21, 2026

Summary

Adds a new on_set_prompt() lifecycle hook on the CacheProvider interface that fires after the cache key set is prepared for a new prompt. Dispatched as an async task with errors swallowed (same fail-safe pattern as on_store / on_lookup / on_prompt_start / on_prompt_end).

Why

BasicCache's lifecycle notifications to external providers are incomplete. set_prompt is a key per-prompt event — the moment the cache key set is prepared and the cache is ready to serve lookups — and providers currently get no signal for it. There's a clean completeness argument for surfacing this.

The immediate motivating use case: cost-aware caching policies. A provider can set t = perf_counter() in on_set_prompt, then measure elapsed at each on_store to estimate the compute that a future hit on this entry would save. That gives the provider a simple, accurate per-prompt clock without needing per-node timing exposed in the API.

Implementation

comfy_api/latest/_caching.py
  CacheProvider gains async def on_set_prompt() with default no-op

comfy_execution/caching.py
  BasicCache.set_prompt(...) now awaits self._notify_providers_set_prompt()
  _notify_providers_set_prompt → asyncio.create_task per provider
  _safe_provider_set_prompt → swallows exceptions, same as _safe_provider_store

Only 31 lines, two files, fully additive.

Backward compatibility

  • New on_set_prompt() method has a pass default — existing providers compile and run unchanged.
  • New _notify_providers_set_prompt is gated on self.enable_providers + _has_cache_providers(), so no-op when no providers are registered.
  • Errors swallowed per existing pattern; provider faults never break execution.

Lifecycle position

on_prompt_start(prompt_id)            — already exists, fires at exec start
   ↓
caches.set_prompt(...) initializes cache_key_set
   ↓
on_set_prompt()                       — NEW, fires here
   ↓
execution loop → on_lookup / on_store per node
   ↓
on_prompt_end(prompt_id)              — already exists, fires at exec end

Test plan

  • Existing CacheProvider tests still pass (on_set_prompt defaults to no-op preserves behavior)
  • Manual: register a provider that overrides on_set_prompt and confirm it fires once per prompt, after cache keys are prepared

@deepme987 deepme987 force-pushed the deepme987/cache-provider-compute-timing branch from 035c98e to 615f551 Compare May 21, 2026 00:30
Adds a new on_set_prompt() lifecycle hook on CacheProvider that fires
after the cache key set is prepared for a new prompt. Dispatched via
asyncio.create_task with errors swallowed (same fail-safe pattern as
on_store / on_lookup).

Why: BasicCache's lifecycle notifications to external providers were
incomplete. set_prompt is a key per-prompt event that providers need
visibility into — for example, to reset per-prompt timing/state used
for cost-aware caching policies (a provider can set t=0 here, then
measure elapsed at each on_store to estimate compute saved by a hit).

Backward-compatible: default implementation is a no-op, existing
providers compile and run unchanged. Providers that need the per-prompt
boundary override on_set_prompt().
@deepme987 deepme987 force-pushed the deepme987/cache-provider-compute-timing branch from 615f551 to fcbe7db Compare May 21, 2026 04:15
@deepme987 deepme987 changed the title feat(cache-provider): expose per-node compute time to providers feat(cache-provider): add on_set_prompt lifecycle hook May 21, 2026
@deepme987 deepme987 marked this pull request as ready for review May 21, 2026 04:40
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 21, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR introduces a new provider lifecycle hook to the cache system. A new on_set_prompt() async method is added to the CacheProvider interface, allowing providers to react when a prompt is set. When BasicCache.set_prompt() is called, it now schedules provider notifications as background asyncio tasks after cache key initialization. A static helper method safely awaits each provider's hook and logs warnings if the provider raises an exception. Pending tasks are tracked in the cache for cleanup.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 22.22% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description check ✅ Passed The description is comprehensive and directly related to the changeset, covering motivation, implementation details, backward compatibility, and lifecycle positioning.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely describes the main change: adding a new lifecycle hook method to the CacheProvider interface.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.


Comment @coderabbitai help to get the list of available commands and usage tips.

@deepme987 deepme987 marked this pull request as draft May 21, 2026 23:27
@deepme987 deepme987 assigned deepme987 and unassigned guill and rattus128 May 21, 2026
@alexisrolland alexisrolland changed the title feat(cache-provider): add on_set_prompt lifecycle hook feat(cache-provider): add on_set_prompt lifecycle hook (CORE-246) May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants