[WIP]fix: preserve soft-delete flags in workflow task ACK writes.#4620
Open
leozhang2018 wants to merge 1 commit into
Open
[WIP]fix: preserve soft-delete flags in workflow task ACK writes.#4620leozhang2018 wants to merge 1 commit into
leozhang2018 wants to merge 1 commit into
Conversation
Signed-off-by: leozhang2018 <leozhang2018@gmail.com>
PetrusZ
approved these changes
Apr 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / Why we need it:
The workflow controller loads a WorkflowTask document into memory at startup and holds a reference (c.workflowTask) for the entire lifetime of the run. When a workflow is deleted while a task is still executing, DeleteWorkflowV4 soft-deletes all associated task documents by setting is_deleted: true and is_archived: true in MongoDB.
However, updateWorkflowTask (the ACK callback invoked after every job/stage transition) writes the full in-memory task object back to MongoDB using $set. Because the in-memory object was loaded before the workflow was deleted, its IsDeleted and IsArchived fields are both false. Every ACK call therefore silently undoes the soft-delete, resetting is_deleted to false and making the task reappear in the UI as "running" — even after the workflow has been removed.
This also meant that a soft-deleted task could never be cleared by InitQueue on service restart (which queries InCompletedTasks filtered by is_deleted: false), nor by any automatic cleanup path, leaving a permanent "zombie" running indicator for users.
What is changed and how it works?
Before writing the in-memory task back to MongoDB, sync IsDeleted and IsArchived from the freshly-read database copy (taskInColl) that is already fetched at the top of updateWorkflowTask. This ensures that a soft-delete performed concurrently (e.g., by DeleteWorkflowV4) is never overwritten by a stale ACK.
Does this PR introduce a user-facing change?
This change is