Skip to content

Improve logging when Hangfire operation canceled#874

Open
pmachapman wants to merge 1 commit intomainfrom
log_operation_canceled
Open

Improve logging when Hangfire operation canceled#874
pmachapman wants to merge 1 commit intomainfrom
log_operation_canceled

Conversation

@pmachapman
Copy link
Collaborator

@pmachapman pmachapman commented Feb 15, 2026

I had a really weird issue on ext-QA where my jobs were getting marked as Pending/Restarting:

{
  "_id": {
    "$oid": "69922109390b03b7aebfaa94"
  },
  "revision": 2,
  "engineRef": {
    "$oid": "67846017466a1867b5a21cd3"
  },
  "owner": "E4i07fEZraBMVQdCbDKR949GaMf4V8Jk@clients",
  "step": 0,
  "state": "Pending",
  "options": {
    "tags": "sf-dev"
  },
  "deploymentVersion": "1.15.0-rc.3",
  "executionData": {},
  "dateCreated": {
    "$date": "2026-02-15T19:39:53.463Z"
  },
  "message": "Restarting",
  "progress": 0
}

And it did not restart. I could not recreate it locally, nor diagnose the error on QA, apart from determine that it was coming from an OperationCanceled exception thrown in DoWork. Worse, it was only happening for some builds - a specific build from my local instance of Scripture Forge (changing translation engines did not help).

Redeploying the docker pod fixed the build, but I am still unable to determine the cause (it happened again later in the day).

To help diagnose the cause of any errors like this in future, this PR improves logging when this scenario occurs, so if it happens again, we will at least have a bit more info to go on.


This change is Reviewable

@codecov-commenter
Copy link

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 66.96%. Comparing base (ea081c6) to head (af27adc).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #874   +/-   ##
=======================================
  Coverage   66.96%   66.96%           
=======================================
  Files         384      384           
  Lines       20886    20889    +3     
  Branches     2700     2700           
=======================================
+ Hits        13986    13989    +3     
  Misses       5936     5936           
  Partials      964      964           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Collaborator

@Enkidu93 Enkidu93 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@Enkidu93 reviewed 1 file and all commit messages, and made 1 comment.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @ddaspit).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants