Skip to content

Commit 673cc30

Browse files
ssteele110claude
andcommitted
Fix race condition: set initial heartbeat in acquire_master_role
Previously, there was a window between acquiring master role and entering with_master_setup_heartbeat where no heartbeat existed. Other workers could see status='setup' with no heartbeat and incorrectly attempt takeover. Now the initial heartbeat is set immediately after acquiring master role, closing this window. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1 parent 087ed66 commit 673cc30

1 file changed

Lines changed: 7 additions & 1 deletion

File tree

ruby/lib/ci/queue/redis/worker.rb

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -512,9 +512,15 @@ def acquire_master_role?
512512
begin
513513
redis.set(key('master-worker-id'), worker_id)
514514
redis.expire(key('master-worker-id'), config.redis_ttl)
515+
516+
# Set initial heartbeat immediately to prevent premature takeover
517+
# This closes the window where status='setup' but no heartbeat exists
518+
redis.set(key('master-setup-heartbeat'), CI::Queue.time_now.to_f)
519+
redis.expire(key('master-setup-heartbeat'), config.redis_ttl)
520+
515521
warn "Worker #{worker_id} elected as master"
516522
rescue *CONNECTION_ERRORS
517-
# If setting master-worker-id fails, we still have master status
523+
# If setting master-worker-id/heartbeat fails, we still have master status
518524
# Log but don't lose master role
519525
warn("Failed to set master-worker-id: #{$!.message}")
520526
end

0 commit comments

Comments
 (0)