Skip to content

Cancel Temporal timer when workflow.sleep() task is cancelled#1352

Open
carlosa54 wants to merge 1 commit intotemporalio:mainfrom
carlosa54:fix-workflow-sleep-timer-cancellation
Open

Cancel Temporal timer when workflow.sleep() task is cancelled#1352
carlosa54 wants to merge 1 commit intotemporalio:mainfrom
carlosa54:fix-workflow-sleep-timer-cancellation

Conversation

@carlosa54
Copy link

What was changed

When workflow.sleep() is wrapped in an asyncio.Task and that task is cancelled, two things now happen correctly:

  1. The timer callback guards against calling set_result() on an already-cancelled future, preventing asyncio.InvalidStateError
  2. A done_callback on the future propagates cancellation to the underlying Temporal timer via timer_handle.cancel(), which sends a cancel_timer command to the server

Previously, workflow.sleep() discarded the _TimerHandle returned by _timer_impl(), so there was no way to cancel the Temporal timer when the awaiting task was cancelled. This is in contrast to asyncio.sleep(), which goes through the event loop's call_later() path where Python's asyncio internals call handle.cancel() automatically

A new test (test_workflow_sleep_task_cancellation) mirrors the existing CancelSignalAndTimerFiredInSameTaskWorkflow test but uses workflow.sleep() instead of asyncio.sleep().

Why?

workflow.sleep() and asyncio.sleep() should behave the same way when cancelled.

Without this fix:

  • An asyncio.InvalidStateError is logged when the timer fires after the task is already cancelled.
  • The Temporal timer is never cancelled on the server (no TimerCanceled event in history), leaking server-side resources.

See Issues #782 and #1351 for the bug reports

Checklist

  1. Closes [Bug] cancelled timer callback causes asyncio.exceptions.InvalidStateError #782

  2. Closes [Bug] workflow.sleep() timer not canceled when wrapping asyncio.Task is canceled #1351

  3. How was this tested:

  • Added test_workflow_sleep_task_cancellation which starts a workflow.sleep(60 * 60) in a task, cancels it via signal, and asserts the workflow completes with "timer_cancelled" (no InvalidStateError)
  • Verified existing timer cancellation tests still pass: test_workflow_cancel_multi, test_workflow_cancel_signal_and_timer_fired_in_same_task, test_workflow_sleep, test_concurrent_sleeps_use_proper_options

Also tested locally:

Before
sleep-cancel-bug

After
sleep-cancel-success

  1. Any docs updates needed? No

@carlosa54 carlosa54 requested a review from a team as a code owner March 7, 2026 20:01
@CLAassistant
Copy link

CLAassistant commented Mar 7, 2026

CLA assistant check
All committers have signed the CLA.

@carlosa54 carlosa54 changed the title fix: cancel Temporal timer when workflow.sleep() task is cancelled Cancel Temporal timer when workflow.sleep() task is cancelled Mar 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants