Skip to content

refactor: replace HTTP status code checks with semantic error types#1342

Merged
pranaygp merged 11 commits intomainfrom
pgp/semantic-world-errors
Mar 18, 2026
Merged

refactor: replace HTTP status code checks with semantic error types#1342
pranaygp merged 11 commits intomainfrom
pgp/semantic-world-errors

Conversation

@pranaygp
Copy link
Collaborator

@pranaygp pranaygp commented Mar 12, 2026

Summary

Replaces all patterns of catching WorkflowAPIError with HTTP status codes in the runtime and eliminates fake HTTP status code construction in world-local and world-postgres. The runtime now uses semantic error types exclusively, and world implementations throw semantic errors instead of simulating HTTP responses.

Breaking change: WorkflowAPIError has been renamed to WorkflowWorldError to better reflect that this error represents world (storage backend) failures, not HTTP API errors specifically.

Problem

The runtime had ~18+ places doing this:

if (WorkflowAPIError.is(err) && err.status === 409) {
  // entity already in terminal state, skip
}

And world-local/world-postgres were faking HTTP status codes:

throw new WorkflowAPIError('...', { status: 409 });

This is wrong because:

  • HTTP status codes are a world-vercel implementation detail leaking into the runtime
  • world-local and world-postgres had to fake HTTP status codes to match
  • The runtime shouldn't know about HTTP — it should handle semantic states

Solution

Semantic error types (@workflow/errors)

Error Replaces Meaning
EntityConflictError status === 409 Entity conflict (terminal state, already exists, creation race)
RunExpiredError status === 410 Run no longer available (expired, completed, cleaned up)
ThrottleError status === 429 Rate limited, carries retryAfter
TooEarlyError status === 425 RetryAfter timestamp not yet reached

Existing types also leveraged:

  • HookNotFoundError replaces status === 404 for hooks
  • WorkflowRunNotFoundError replaces status === 404 for runs

WorkflowAPIErrorWorkflowWorldError

Renamed to better reflect its role as the catch-all for world (storage backend) failures. It remains for:

  1. Genuinely unexpected HTTP errors from world-vercel (the only real HTTP boundary)
  2. Internal validation errors (step/wait/event not found) — but without fake status codes
  3. The runtime's err.status >= 500 check for infrastructure errors surfacing through user code

World implementations updated

  • world-vercel: makeRequest() maps HTTP 409/410/425/429 → semantic errors at the throw site. createWorkflowRunEvent() translates 404 → HookNotFoundError for hook events.
  • world-local: All fake HTTP status codes removed, semantic errors used throughout
  • world-postgres: Same pattern — all fake HTTP status codes removed

Runtime updated

All catch sites now use semantic checks:

// Before
if (WorkflowAPIError.is(err) && err.status === 409) { ... }

// After
if (EntityConflictError.is(err)) { ... }

Test plan

  • Build passes (all 27 packages)
  • Typecheck passes
  • All 472 core unit tests pass
  • All 248 world-local tests pass
  • Updated test files: helpers.test.ts, step-handler.test.ts, runs.test.ts, classify-error.test.ts, storage.test.ts (world-local), storage.test.ts (world-postgres)
  • E2E tests (to be run on CI)

🤖 Generated with Claude Code

@vercel
Copy link
Contributor

vercel bot commented Mar 12, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Mar 18, 2026 9:39pm
example-nextjs-workflow-webpack Ready Ready Preview, Comment Mar 18, 2026 9:39pm
example-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-astro-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-express-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-fastify-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-hono-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-nitro-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-nuxt-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-sveltekit-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workbench-vite-workflow Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workflow-docs Ready Ready Preview, Comment, Open in v0 Mar 18, 2026 9:39pm
workflow-nest Ready Ready Preview, Comment Mar 18, 2026 9:39pm
workflow-swc-playground Ready Ready Preview, Comment Mar 18, 2026 9:39pm

@github-actions
Copy link
Contributor

github-actions bot commented Mar 12, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 758 0 67 825
✅ 💻 Local Development 782 0 118 900
✅ 📦 Local Production 782 0 118 900
✅ 🐘 Local Postgres 782 0 118 900
✅ 🪟 Windows 72 0 3 75
❌ 🌍 Community Worlds 118 56 15 189
✅ 📋 Other 198 0 27 225
Total 3492 56 466 4014

❌ Failed Tests

🌍 Community Worlds (56 failed)

mongodb (3 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KM1E937MQ2CP3W6WCNSXM1MW
  • webhookWorkflow | wrun_01KM1E9C5C8CWMVDSMPBWZS7NW
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KM1EET3XX3Y1221HZBP95GHN

redis (2 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KM1E937MQ2CP3W6WCNSXM1MW
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KM1EET3XX3Y1221HZBP95GHN

turso (51 failed):

  • addTenWorkflow | wrun_01KM1E7Y9WRMDPN3Z89BRS8ESP
  • addTenWorkflow | wrun_01KM1E7Y9WRMDPN3Z89BRS8ESP
  • wellKnownAgentWorkflow (.well-known/agent) | wrun_01KM1E8Z02T7NRKBK7EA18DJSB
  • should work with react rendering in step
  • promiseAllWorkflow | wrun_01KM1E84XBPPF6X8AARKDS3VZV
  • promiseRaceWorkflow | wrun_01KM1E8AJ36B93D0P5691PX9TS
  • promiseAnyWorkflow | wrun_01KM1E8CN2G3351A7976TYHARW
  • importedStepOnlyWorkflow | wrun_01KM1E98BCAWB8Q87MDK9821HD
  • hookWorkflow | wrun_01KM1E8RRZHD4ZXNA1QF4FRMRX
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KM1E937MQ2CP3W6WCNSXM1MW
  • webhookWorkflow | wrun_01KM1E9C5C8CWMVDSMPBWZS7NW
  • sleepingWorkflow | wrun_01KM1E9NWWW8M2D0K0B4DEW7VE
  • parallelSleepWorkflow | wrun_01KM1EA1JEG4QB658RJECSAKZH
  • nullByteWorkflow | wrun_01KM1EA4X0RW1AAHQSRGZCSC9F
  • workflowAndStepMetadataWorkflow | wrun_01KM1EA727YFZ72A8TEVV29ASR
  • fetchWorkflow | wrun_01KM1EB65FYGNNCZJAXP9DSHF7
  • promiseRaceStressTestWorkflow | wrun_01KM1EB9JVRNB7RXY5HVA2N1ZE
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01KM1EE613M9A5Q9ARMQ6JRBAW
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KM1EET3XX3Y1221HZBP95GHN
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running | wrun_01KM1EFFCQJPTDBTSCHRKRD8VY
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars) | wrun_01KM1EG2W0YQETW9H78BZTW1X8
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument | wrun_01KM1EGBVEB0YVE1KSC4REFP0Y
  • closureVariableWorkflow - nested step functions with closure variables | wrun_01KM1EGH7YHV2F9TFCR8K7DH3V
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KM1EGKD7RGGRR3CFWZD6N3D5
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly | wrun_01KM1EH2S0M8D8P5P44EBH804J
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01KM1EH8VJ7Y2QM1RWY9T8PTB3
  • AllInOneService.processNumber - static workflow method using sibling static step methods | wrun_01KM1EHFDVSR00YKJ9HDSR61CM
  • ChainableService.processWithThis - static step methods using this to reference the class | wrun_01KM1EHP0DE82H5CBCJB00QG03
  • thisSerializationWorkflow - step function invoked with .call() and .apply() | wrun_01KM1EHWGSVP2BJ9MEFGMM15PK
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KM1EJ34XN0G3FF4SD9R80NS9
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KM1EJB3ZEKTZ7YK888F7ZDV4
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KM1EJMP746S52Z245C8B41PB
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01KM1EJWSXY1YD8BGWWRDC0HDS
  • cancelRun - cancelling a running workflow | wrun_01KM1EK38P8NMX3B3MBM33W0N7
  • cancelRun via CLI - cancelling a running workflow | wrun_01KM1EKCFN2AJK5QZ6GABDPSYJ
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router
  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01KM1EKRFVVF72DXDS0V07XCX8
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KM1EMCAXGJEYKWSG7C4SH4K7
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01KM1EMPMN6FSQTJND7ZX96767

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 68 0 7
✅ example 68 0 7
✅ express 68 0 7
✅ fastify 68 0 7
✅ hono 68 0 7
✅ nextjs-turbopack 73 0 2
✅ nextjs-webpack 73 0 2
✅ nitro 68 0 7
✅ nuxt 68 0 7
✅ sveltekit 68 0 7
✅ vite 68 0 7
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 66 0 9
✅ express-stable 66 0 9
✅ fastify-stable 66 0 9
✅ hono-stable 66 0 9
✅ nextjs-turbopack-canary 55 0 20
✅ nextjs-turbopack-stable 72 0 3
✅ nextjs-webpack-canary 55 0 20
✅ nextjs-webpack-stable 72 0 3
✅ nitro-stable 66 0 9
✅ nuxt-stable 66 0 9
✅ sveltekit-stable 66 0 9
✅ vite-stable 66 0 9
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 66 0 9
✅ express-stable 66 0 9
✅ fastify-stable 66 0 9
✅ hono-stable 66 0 9
✅ nextjs-turbopack-canary 55 0 20
✅ nextjs-turbopack-stable 72 0 3
✅ nextjs-webpack-canary 55 0 20
✅ nextjs-webpack-stable 72 0 3
✅ nitro-stable 66 0 9
✅ nuxt-stable 66 0 9
✅ sveltekit-stable 66 0 9
✅ vite-stable 66 0 9
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 66 0 9
✅ express-stable 66 0 9
✅ fastify-stable 66 0 9
✅ hono-stable 66 0 9
✅ nextjs-turbopack-canary 55 0 20
✅ nextjs-turbopack-stable 72 0 3
✅ nextjs-webpack-canary 55 0 20
✅ nextjs-webpack-stable 72 0 3
✅ nitro-stable 66 0 9
✅ nuxt-stable 66 0 9
✅ sveltekit-stable 66 0 9
✅ vite-stable 66 0 9
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 72 0 3
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 3 0 2
❌ mongodb 52 3 3
✅ redis-dev 3 0 2
❌ redis 53 2 3
✅ turso-dev 3 0 2
❌ turso 4 51 3
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 66 0 9
✅ e2e-local-postgres-nest-stable 66 0 9
✅ e2e-local-prod-nest-stable 66 0 9

📋 View full workflow run

Copy link
Collaborator Author

pranaygp commented Mar 12, 2026

@changeset-bot
Copy link

changeset-bot bot commented Mar 12, 2026

🦋 Changeset detected

Latest commit: 8ba3db5

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 20 packages
Name Type
@workflow/errors Patch
@workflow/core Patch
@workflow/world-local Patch
@workflow/world-vercel Patch
@workflow/world-postgres Patch
workflow Patch
@workflow/builders Patch
@workflow/cli Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/world-testing Patch
@workflow/ai Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@changeset-bot
Copy link

changeset-bot bot commented Mar 12, 2026

🦋 Changeset detected

Latest commit: 2841381

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 20 packages
Name Type
@workflow/errors Patch
@workflow/core Patch
@workflow/world-local Patch
@workflow/world-vercel Patch
@workflow/world-postgres Patch
@workflow/builders Patch
@workflow/cli Patch
workflow Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/world-testing Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/ai Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pranaygp pranaygp force-pushed the pgp/run-failed-schema-vailidation-error branch 5 times, most recently from c463509 to 256bb01 Compare March 18, 2026 01:37
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pranaygp pranaygp force-pushed the pgp/run-failed-schema-vailidation-error branch from 256bb01 to 1f26e4e Compare March 18, 2026 01:59
…runtime

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors error handling across the core runtime and world implementations to replace runtime-side HTTP status code branching (409/410/429) with semantic error types emitted by worlds, reducing coupling to HTTP transport details.

Changes:

  • Add new semantic error classes in @workflow/errors: EntityConflictError, RunExpiredError, ThrottleError.
  • Update world-vercel to map HTTP 409/410/429 to those semantic errors at the request boundary.
  • Update world-local, world-postgres, and @workflow/core runtime call sites to throw/catch semantic errors instead of WorkflowAPIError.status.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
packages/world-vercel/src/utils.ts Map HTTP 409/410/429 responses to semantic errors in makeRequest()
packages/world-postgres/src/storage.ts Replace storage-layer WorkflowAPIError({status:409/410}) with semantic errors
packages/world-local/src/storage/events-storage.ts Replace local storage WorkflowAPIError({status:409/410}) with semantic errors
packages/world-local/src/fs.ts Use EntityConflictError for write conflicts instead of HTTP-encoded errors
packages/errors/src/index.ts Introduce semantic error classes for conflict/expired/throttle cases
packages/core/src/runtime/suspension-handler.ts Replace runtime checks for 409/410 with semantic errors in suspension flow
packages/core/src/runtime/step-handler.ts Replace runtime checks for 409/410/429 with semantic errors in step processing
packages/core/src/runtime/step-handler.test.ts Update tests to throw EntityConflictError instead of WorkflowAPIError(409)
packages/core/src/runtime/runs.ts Use EntityConflictError for wake-up conflict handling
packages/core/src/runtime/runs.test.ts Update tests to use EntityConflictError
packages/core/src/runtime.ts Replace runtime status checks (409/410) with semantic error guards
.changeset/semantic-world-errors.md Publish patch bumps for affected packages

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +261 to +262
* Thrown when attempting to operate on a run that has been cleaned up or expired.
* Replaces WorkflowAPIError with status 410.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b45a71d — broadened the docstring to cover terminal state, expired, and completed/failed runs.

Comment on lines +188 to +191
// 425 Too Early: retryAfter timestamp not reached yet
// Return timeout to queue so it retries later
if (WorkflowAPIError.is(err) && err.status === 425) {
// Parse retryAfter from error response meta
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b45a71d — introduced TooEarlyError semantic type for the 425 case. All three world implementations (vercel, local, postgres) now throw TooEarlyError with a retryAfter: Date field, and the runtime checks TooEarlyError.is(err) instead of inspecting HTTP status.

Comment on lines +177 to +183
} else if (WorkflowAPIError.is(err) && err.status === 404) {
// Hook may have already been disposed or never created
runtimeLogger.info('Hook not found for disposal, continuing', {
workflowRunId: runId,
correlationId: queueItem.correlationId,
message: err.message,
});
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b45a71d — replaced WorkflowAPIError.status === 404 with HookNotFoundError.is(err), which is already thrown by all world implementations for missing hooks.

}

/**
* Thrown when attempting to modify an entity that is already in a terminal state.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in b45a71d — broadened the docstring to cover all 409-style conflicts (terminal state, already exists, creation conflicts).

pranaygp and others added 5 commits March 17, 2026 23:58
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Merge latest changes from pgp/run-failed-schema-vailidation-error
- Broaden EntityConflictError/RunExpiredError docstrings to match actual usage
- Add TooEarlyError semantic type to replace status === 425 checks
- Replace status === 404 with HookNotFoundError.is() in suspension-handler

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add RUN_ERROR_CODES (USER_ERROR, RUNTIME_ERROR) to @workflow/errors
- Populate errorCode in run_failed events via classifyRunError()
- Update web UI StatusBadge to show amber dot for infrastructure errors
- Improve world-local queue error logging (concise, no body dump)
- Improve schema validation error messages (concise, verbose behind DEBUG)
- Add e2e tests for error code flow and infrastructure error retry

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…on-error

Convert new 409/410 status checks from base branch to semantic error types
(EntityConflictError, RunExpiredError) in runtime.ts and suspension-handler.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Merge latest main (base PR was merged)
- Convert all remaining WorkflowAPIError({ status: 409 }) to EntityConflictError
  in world-local and world-postgres
- Convert run 404s to WorkflowRunNotFoundError, hook 404s to HookNotFoundError
- Strip fake HTTP status codes from remaining WorkflowAPIError calls
  (step/wait/event not-found, validation errors)
- Update world-local and world-postgres tests to assert semantic error types
- Only remaining status check in runtime is err.status >= 500 for
  genuine HTTP infrastructure errors from world-vercel

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 18, 2026

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Nitro 0.042s (-11.0% 🟢) 1.005s (~) 0.963s 10 1.00x
💻 Local Express 0.044s (+3.0%) 1.006s (~) 0.961s 10 1.05x
💻 Local Next.js (Turbopack) 0.049s 1.005s 0.957s 10 1.16x
🌐 Redis Next.js (Turbopack) 0.054s 1.006s 0.952s 10 1.30x
🐘 Postgres Nitro 0.057s (-7.1% 🟢) 1.011s (~) 0.954s 10 1.36x
🐘 Postgres Next.js (Turbopack) 0.058s 1.011s 0.954s 10 1.37x
🐘 Postgres Express 0.063s (+4.5%) 1.011s (~) 0.949s 10 1.49x
🌐 MongoDB Next.js (Turbopack) 0.100s 1.007s 0.907s 10 2.39x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 0.457s (-4.7%) 2.164s (-16.7% 🟢) 1.707s 10 1.00x
▲ Vercel Nitro 0.462s (+4.8%) 2.139s (-0.6%) 1.677s 10 1.01x
▲ Vercel Next.js (Turbopack) 0.479s (+11.9% 🔺) 2.133s (-8.2% 🟢) 1.654s 10 1.05x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.118s (-2.5%) 2.013s (~) 0.895s 10 1.00x
💻 Local Nitro 1.120s (-0.8%) 2.006s (~) 0.886s 10 1.00x
🌐 Redis Next.js (Turbopack) 1.121s 2.007s 0.886s 10 1.00x
💻 Local Express 1.124s (~) 2.006s (~) 0.882s 10 1.01x
💻 Local Next.js (Turbopack) 1.125s 2.006s 0.882s 10 1.01x
🐘 Postgres Next.js (Turbopack) 1.132s 2.011s 0.879s 10 1.01x
🐘 Postgres Express 1.145s (-1.4%) 2.012s (~) 0.867s 10 1.02x
🌐 MongoDB Next.js (Turbopack) 1.303s 2.008s 0.705s 10 1.17x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.007s (-4.8%) 3.347s (-2.5%) 1.339s 10 1.00x
▲ Vercel Express 2.037s (-4.0%) 3.306s (-9.3% 🟢) 1.270s 10 1.01x
▲ Vercel Next.js (Turbopack) 2.057s (-10.6% 🟢) 3.356s (-4.2%) 1.299s 10 1.02x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 10.694s (-2.8%) 11.040s (~) 0.346s 3 1.00x
🌐 Redis Next.js (Turbopack) 10.777s 11.023s 0.246s 3 1.01x
💻 Local Next.js (Turbopack) 10.779s 11.024s 0.244s 3 1.01x
🐘 Postgres Next.js (Turbopack) 10.895s 11.039s 0.144s 3 1.02x
💻 Local Nitro 10.909s (~) 11.023s (~) 0.114s 3 1.02x
🐘 Postgres Express 10.919s (-0.6%) 11.046s (~) 0.127s 3 1.02x
💻 Local Express 10.925s (~) 11.024s (~) 0.099s 3 1.02x
🌐 MongoDB Next.js (Turbopack) 12.236s 13.016s 0.780s 3 1.14x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 16.933s (-2.7%) 18.452s (~) 1.518s 2 1.00x
▲ Vercel Next.js (Turbopack) 16.995s (-8.3% 🟢) 18.510s (-5.6% 🟢) 1.515s 2 1.00x
▲ Vercel Express 17.224s (-1.7%) 18.384s (-5.9% 🟢) 1.160s 2 1.02x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

workflow with 25 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 26.639s (-2.2%) 27.060s (-3.6%) 0.421s 3 1.00x
🌐 Redis Next.js (Turbopack) 26.801s 27.050s 0.250s 3 1.01x
🐘 Postgres Next.js (Turbopack) 26.993s 27.391s 0.398s 3 1.01x
💻 Local Next.js (Turbopack) 27.113s 27.719s 0.606s 3 1.02x
🐘 Postgres Express 27.201s (~) 28.061s (~) 0.860s 3 1.02x
💻 Local Nitro 27.429s (~) 28.049s (~) 0.620s 3 1.03x
💻 Local Express 27.463s (~) 28.053s (~) 0.591s 3 1.03x
🌐 MongoDB Next.js (Turbopack) 30.399s 31.035s 0.636s 2 1.14x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 44.063s (-4.6%) 45.556s (-3.9%) 1.492s 2 1.00x
▲ Vercel Express 44.776s (-3.8%) 46.143s (-5.0%) 1.367s 2 1.02x
▲ Vercel Nitro 45.343s (+0.9%) 46.208s (~) 0.865s 2 1.03x

🔍 Observability: Next.js (Turbopack) | Express | Nitro

workflow with 50 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 53.241s (-2.0%) 54.103s (-1.8%) 0.862s 2 1.00x
🌐 Redis Next.js (Turbopack) 53.434s 54.098s 0.664s 2 1.00x
🐘 Postgres Next.js (Turbopack) 53.929s 54.089s 0.159s 2 1.01x
🐘 Postgres Express 54.166s (~) 55.095s (~) 0.928s 2 1.02x
💻 Local Next.js (Turbopack) 55.813s 56.102s 0.289s 2 1.05x
💻 Local Nitro 56.489s (~) 57.095s (~) 0.606s 2 1.06x
💻 Local Express 56.532s (~) 57.104s (~) 0.572s 2 1.06x
🌐 MongoDB Next.js (Turbopack) 60.591s 61.060s 0.469s 2 1.14x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 95.868s (-2.5%) 96.727s (-3.4%) 0.859s 1 1.00x
▲ Vercel Nitro 96.704s (-1.9%) 98.701s (-0.9%) 1.997s 1 1.01x
▲ Vercel Next.js (Turbopack) 97.449s (-6.3% 🟢) 98.680s (-5.9% 🟢) 1.231s 1 1.02x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.204s (-7.2% 🟢) 2.011s (~) 0.806s 15 1.00x
🐘 Postgres Next.js (Turbopack) 1.268s 2.010s 0.742s 15 1.05x
🐘 Postgres Express 1.271s (-0.5%) 2.011s (~) 0.740s 15 1.06x
🌐 Redis Next.js (Turbopack) 1.325s 2.006s 0.681s 15 1.10x
💻 Local Nitro 1.493s (-0.9%) 2.005s (~) 0.511s 15 1.24x
💻 Local Next.js (Turbopack) 1.519s 2.006s 0.487s 15 1.26x
💻 Local Express 1.531s (+1.4%) 2.005s (~) 0.474s 15 1.27x
🌐 MongoDB Next.js (Turbopack) 2.176s 3.008s 0.833s 10 1.81x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Next.js (Turbopack) 2.270s (-22.6% 🟢) 3.362s (-16.1% 🟢) 1.092s 9 1.00x
▲ Vercel Nitro 2.624s (+7.7% 🔺) 3.977s (+1.7%) 1.353s 8 1.16x
▲ Vercel Express 2.653s (+2.6%) 3.856s (-8.3% 🟢) 1.203s 8 1.17x

🔍 Observability: Next.js (Turbopack) | Nitro | Express

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.351s (-4.9%) 3.012s (~) 0.661s 10 1.00x
🐘 Postgres Express 2.462s (~) 3.013s (~) 0.552s 10 1.05x
🐘 Postgres Next.js (Turbopack) 2.491s 3.012s 0.521s 10 1.06x
🌐 Redis Next.js (Turbopack) 2.557s 3.008s 0.451s 10 1.09x
💻 Local Next.js (Turbopack) 2.861s 3.209s 0.348s 10 1.22x
💻 Local Nitro 2.895s (-3.2%) 3.108s (-7.0% 🟢) 0.213s 10 1.23x
💻 Local Express 2.957s (+3.6%) 3.341s (+11.1% 🔺) 0.384s 9 1.26x
🌐 MongoDB Next.js (Turbopack) 4.659s 5.176s 0.517s 6 1.98x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.523s (-1.7%) 3.617s (-3.8%) 1.095s 9 1.00x
▲ Vercel Express 2.615s (+4.7%) 3.717s (-6.3% 🟢) 1.103s 9 1.04x
▲ Vercel Next.js (Turbopack) 3.206s (+13.6% 🔺) 4.691s (+19.5% 🔺) 1.485s 7 1.27x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.all with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 3.565s (-1.8%) 4.014s (~) 0.449s 8 1.00x
🐘 Postgres Express 3.606s (~) 4.013s (~) 0.408s 8 1.01x
🐘 Postgres Next.js (Turbopack) 3.756s 4.013s 0.257s 8 1.05x
🌐 Redis Next.js (Turbopack) 4.178s 5.011s 0.833s 6 1.17x
💻 Local Next.js (Turbopack) 7.564s 8.014s 0.450s 4 2.12x
💻 Local Express 7.788s (-2.1%) 8.272s (~) 0.484s 4 2.18x
💻 Local Nitro 7.790s (-1.3%) 8.021s (-8.6% 🟢) 0.231s 4 2.19x
🌐 MongoDB Next.js (Turbopack) 9.738s 10.348s 0.610s 3 2.73x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.664s (-20.6% 🟢) 4.123s (-13.4% 🟢) 1.459s 8 1.00x
▲ Vercel Express 3.223s (+4.9%) 4.419s (+2.2%) 1.196s 7 1.21x
▲ Vercel Next.js (Turbopack) 3.679s (+7.0% 🔺) 5.061s (+7.2% 🔺) 1.382s 6 1.38x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 1.219s (-5.5% 🟢) 2.011s (~) 0.791s 15 1.00x
🐘 Postgres Express 1.260s (~) 2.011s (~) 0.751s 15 1.03x
🐘 Postgres Next.js (Turbopack) 1.262s 2.011s 0.749s 15 1.03x
🌐 Redis Next.js (Turbopack) 1.301s 2.006s 0.706s 15 1.07x
💻 Local Nitro 1.517s (-1.5%) 2.004s (~) 0.488s 15 1.24x
💻 Local Next.js (Turbopack) 1.535s 2.005s 0.470s 15 1.26x
💻 Local Express 1.539s (+1.4%) 2.006s (~) 0.466s 15 1.26x
🌐 MongoDB Next.js (Turbopack) 2.205s 3.008s 0.803s 10 1.81x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.177s (-1.4%) 3.612s (+2.9%) 1.435s 9 1.00x
▲ Vercel Next.js (Turbopack) 2.299s (-11.1% 🟢) 3.462s (-14.2% 🟢) 1.163s 9 1.06x
▲ Vercel Nitro 2.577s (+16.2% 🔺) 3.903s (+8.9% 🔺) 1.326s 8 1.18x

🔍 Observability: Express | Next.js (Turbopack) | Nitro

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 2.353s (-4.4%) 3.011s (~) 0.658s 10 1.00x
🐘 Postgres Express 2.461s (+0.8%) 3.012s (~) 0.551s 10 1.05x
🐘 Postgres Next.js (Turbopack) 2.479s 3.011s 0.532s 10 1.05x
🌐 Redis Next.js (Turbopack) 2.624s 3.008s 0.384s 10 1.12x
💻 Local Next.js (Turbopack) 2.935s 3.209s 0.274s 10 1.25x
💻 Local Nitro 2.993s (-4.0%) 3.564s (-8.3% 🟢) 0.572s 9 1.27x
💻 Local Express 3.039s (+3.2%) 3.760s (+13.7% 🔺) 0.722s 8 1.29x
🌐 MongoDB Next.js (Turbopack) 4.745s 5.176s 0.431s 6 2.02x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.497s (~) 3.582s (-2.3%) 1.084s 9 1.00x
▲ Vercel Express 2.686s (+7.1% 🔺) 3.805s (~) 1.119s 8 1.08x
▲ Vercel Next.js (Turbopack) 2.752s (+9.3% 🔺) 4.194s (+20.9% 🔺) 1.442s 8 1.10x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.race with 50 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 3.505s (-3.2%) 4.015s (~) 0.510s 8 1.00x
🐘 Postgres Express 3.606s (~) 4.013s (~) 0.407s 8 1.03x
🐘 Postgres Next.js (Turbopack) 3.707s 4.012s 0.305s 8 1.06x
🌐 Redis Next.js (Turbopack) 4.236s 4.868s 0.632s 7 1.21x
💻 Local Next.js (Turbopack) 8.021s 8.515s 0.495s 4 2.29x
💻 Local Nitro 8.349s (-3.2%) 9.020s (~) 0.671s 4 2.38x
💻 Local Express 8.587s (~) 9.021s (~) 0.435s 4 2.45x
🌐 MongoDB Next.js (Turbopack) 9.881s 10.349s 0.469s 3 2.82x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.877s (-2.1%) 4.111s (-4.8%) 1.234s 8 1.00x
▲ Vercel Nitro 3.492s (+15.7% 🔺) 5.047s (+15.2% 🔺) 1.556s 6 1.21x
▲ Vercel Next.js (Turbopack) 3.810s (-4.3%) 4.919s (-1.6%) 1.109s 7 1.32x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🐘 Postgres 🥇 Nitro 0.161s (-28.1% 🟢) 1.001s (+0.6%) 0.002s (+14.3% 🔺) 1.012s (~) 0.851s 10 1.00x
💻 Local Next.js (Turbopack) 0.168s 1.002s 0.011s 1.017s 0.850s 10 1.04x
🌐 Redis Next.js (Turbopack) 0.183s 1.000s 0.001s 1.008s 0.824s 10 1.14x
💻 Local Nitro 0.196s (-2.6%) 1.003s (~) 0.011s (-5.1% 🟢) 1.017s (~) 0.821s 10 1.21x
💻 Local Express 0.202s (+2.4%) 1.003s (~) 0.011s (~) 1.017s (~) 0.816s 10 1.25x
🐘 Postgres Next.js (Turbopack) 0.202s 1.002s 0.002s 1.012s 0.811s 10 1.25x
🐘 Postgres Express 0.224s (+2.5%) 0.997s (~) 0.002s (+25.0% 🔺) 1.013s (~) 0.789s 10 1.39x
🌐 MongoDB Next.js (Turbopack) 0.515s 0.943s 0.001s 1.009s 0.493s 10 3.20x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 1.550s (-10.5% 🟢) 2.449s (-15.1% 🟢) 0.005s (-8.3% 🟢) 2.929s (-17.2% 🟢) 1.379s 10 1.00x
▲ Vercel Nitro 1.561s (+1.5%) 2.608s (-5.7% 🟢) 0.007s (-50.0% 🟢) 3.045s (-10.1% 🟢) 1.484s 10 1.01x
▲ Vercel Next.js (Turbopack) 1.621s (-1.0%) 2.769s (-3.9%) 0.005s (-5.5% 🟢) 3.271s (-1.5%) 1.649s 10 1.05x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Next.js (Turbopack) 8/12
🐘 Postgres Nitro 12/12
▲ Vercel Express 5/12
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 🐘 Postgres 7/12
Next.js (Turbopack) 🐘 Postgres 5/12
Nitro 🐘 Postgres 9/12
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run

@@ -1,4 +1,4 @@
import { WorkflowAPIError } from '@workflow/errors';
import { EntityConflictError, WorkflowAPIError } from '@workflow/errors';
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which error paths in the runtime are still checking/handling WorkflowApiError - and what's happening when this is encountered? is it properly bubbling up unknown APIErrors to the queue so that queue can retry?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two places in the runtime still check/handle WorkflowAPIError:

  1. step-handler.ts:428 — checks err.status >= 500 on errors that surface through user code. This catches infrastructure errors (genuine HTTP 5xx from world-vercel) and re-throws them so they bubble up to the queue handler for retry instead of consuming a step attempt. Non-5xx WorkflowAPIErrors fall through and are treated as user code errors (recorded as step failure).

  2. suspension-handler.ts:184 — checks WorkflowAPIError.is(err) && err.status === 404 as a fallback for hook disposal. This is needed because world-vercel's makeRequest() doesn't map 404 to HookNotFoundError (only getHookByToken() in hooks.ts does that). See the thread on TooTallNate's comment for context.

Both paths properly bubble unknown errors to the queue for retry — any error that isn't explicitly caught gets re-thrown.

Copy link
Member

@TooTallNate TooTallNate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

Solid refactor. The motivation is sound — the runtime was coupling to HTTP status codes that only make sense for world-vercel, forcing world-local and world-postgres to fake HTTP responses. The new semantic error types (EntityConflictError, RunExpiredError, ThrottleError, TooEarlyError) are well-named, consistently applied, and make the catch sites significantly more readable.

What looks good

  • Comprehensive sweep: Every WorkflowAPIError.is(err) && err.status === N pattern in the runtime has been converted. The one remaining err.status >= 500 check is correctly justified — it only applies to genuine HTTP infrastructure errors from world-vercel.
  • TooEarlyError is a nice cleanup: The old (err as any).meta?.retryAfter hack is replaced with a typed retryAfter: Date property on the error class. Much cleaner.
  • World implementations are consistent: world-local and world-postgres mirror each other's conversions.
  • world-vercel mapping is correctly placed: The HTTP-to-semantic conversion happens at the makeRequest() boundary, which is exactly the right location.
  • Tests are updated: All test assertions now use semantic error types.
  • Changeset is correct: All 5 affected packages are listed.

Nits (non-blocking)

See inline comments.

}

/**
* Thrown when an operation cannot proceed because a required timestamp
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: meta is accepted in the constructor options but never stored on the instance. This is dead code — either remove it or assign it to a property if it is intended for future use.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e08a8fa — removed the dead meta option from the constructor.

}
if (response.status === 410) {
const error = new RunExpiredError(defaultMessage);
span?.setAttributes({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: The span attributes + recordException pattern is duplicated across the 409/410/425/429 branches. Consider extracting a small helper like:

function throwSemanticError(error: Error, span: Span | undefined, code: string): never {
  span?.setAttributes({ ...ErrorType(code) });
  span?.recordException?.(error);
  throw error;
}

This would reduce the ~20 lines of duplication down to single calls and make it easier to add new status mappings in the future.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e08a8fa — extracted a throwWithTrace helper that handles span attributes + recordException, replacing the duplicated pattern across all status branches.


console.error(
`[world-local] Queue message failed (attempt ${attempt + 1}/${maxAttempts}, status ${response.status}): ${text}`,
`[world-local] Queue message failed (attempt ${attempt + 1}/${attempt + 1 + defaultRetriesLeft}, status ${response.status}): ${text}`,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: This expression attempt + 1 + defaultRetriesLeft computes a different total on each iteration because defaultRetriesLeft is decremented each loop. On the first failure (attempt=0, defaultRetriesLeft=2) it shows 1/3, second (attempt=1, defaultRetriesLeft=1) shows 2/3, but third (attempt=2, defaultRetriesLeft=0) shows 3/3. So it happens to produce correct output by coincidence. But it's fragile — consider capturing const maxAttempts = defaultRetriesLeft before the loop (the line this PR removed) or just hardcoding 3.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in e08a8fa — restored const maxAttempts = 3 before the loop and use it in the log message.

Copy link
Member

@TooTallNate TooTallNate left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated Review: Behavioral Regression Found

After a deeper audit focused specifically on whether the same errors that previously bubbled up to the queue retry handler still do so (and vice versa), I found one confirmed behavioral regression and one observation.

BLOCKING: HTTP 404 on hook disposal no longer swallowed (world-vercel path)

See inline comment on suspension-handler.ts. In the world-vercel code path, when the workflow server returns HTTP 404 for a hook_disposed event, makeRequest() throws a generic WorkflowAPIError({ status: 404 }) — it does NOT throw HookNotFoundError. The old code caught this via WorkflowAPIError.is(err) && err.status === 404. The new code checks HookNotFoundError.is(err), which won't match.

This means a 404 on hook disposal that was previously gracefully logged-and-continued will now throw, causing the suspension handler to fail and likely trigger queue retries.

Observation: world-local/world-postgres error handling is actually IMPROVED

The old runtime code checked WorkflowAPIError.is(err) && err.status === 409. Since world-local and world-postgres now throw EntityConflictError (which is NOT a WorkflowAPIError and has no status property), the old catch predicates would have never matched errors from these backends. The new EntityConflictError.is(err) predicate correctly catches them. This is a correctness improvement, not a regression — but worth calling out as a non-trivial behavioral change for local/postgres backends.

All other catch sites: verified correct

  • runtime.ts: All 4 catch sites are purely semantic renames. Non-409/410 errors bubble up identically.
  • step-handler.ts: All catch sites are correct. The remaining WorkflowAPIError.is(err) && err.status >= 500 check for infrastructure errors in user code scope is unaffected and still works.
  • runs.ts: The single catch site is correct.

);
} else if (HookNotFoundError.is(err)) {
// Hook may have already been disposed or never created
runtimeLogger.info('Hook not found for disposal, continuing', {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: Behavioral regression for world-vercel on 404 during hook disposal.

In the old code, this catch block had:

if (WorkflowAPIError.is(err)) {
  if (err.status === 409 || err.status === 410) { /* log, continue */ }
  else if (err.status === 404) { /* log, continue */ }
  else { throw err; }
}

The new code replaces the err.status === 404 check with HookNotFoundError.is(err). However, in the world-vercel path, makeRequest() does not map HTTP 404 to HookNotFoundError — it throws a generic WorkflowAPIError({ status: 404 }). The only place 404 is converted to HookNotFoundError is in getHookByToken() (hooks.ts:115), not in the event creation path.

So when the workflow server returns 404 for a hook_disposed event (hook already disposed or never created):

  • Old behavior: Caught by err.status === 404, logged, and swallowed gracefully.
  • New behavior: HookNotFoundError.is(err) returns false (it's a WorkflowAPIError, not a HookNotFoundError), falls through to throw err, bubbles up to queue retry handler.

Suggested fixes (pick one):

  1. Add error translation in world-vercel/src/events.ts in createWorkflowRunEvent(): catch WorkflowAPIError({ status: 404 }) and re-throw as HookNotFoundError when the event type is hook_disposed or hook_received. This mirrors the existing pattern in getHookByToken() at hooks.ts:115.

  2. Add a fallback in this catch block: HookNotFoundError.is(err) || (WorkflowAPIError.is(err) && err.status === 404) — less clean but preserves behavior immediately.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in e08a8fa using option 2 (fallback). The catch block now checks:

HookNotFoundError.is(err) ||
(WorkflowAPIError.is(err) && err.status === 404)

with a comment explaining that world-vercel returns WorkflowAPIError({ status: 404 }) for missing hooks during event creation, while world-local and world-postgres throw HookNotFoundError directly.

Option 1 (translating in world-vercel/events.ts) would be cleaner long-term but would require knowing the event type context inside makeRequest() to distinguish hook 404s from other 404s — so the fallback is the safer fix for now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated in c15c613 — went with option 1 instead. The 404 → HookNotFoundError translation now lives in world-vercel/src/events.ts inside createWorkflowRunEvent. For hook-related event types (hook_created, hook_disposed, hook_received, hook_conflict), a 404 from the server is translated to HookNotFoundError.

The runtime's suspension-handler is back to just checking HookNotFoundError.is(err) with no WorkflowAPIError fallback.

pranaygp and others added 2 commits March 18, 2026 13:56
- Remove dead `meta` option from TooEarlyError constructor (TooTallNate)
- Extract `throwWithTrace` helper to deduplicate span recording in
  world-vercel makeRequest (TooTallNate)
- Restore `maxAttempts` const for stable retry count logging (TooTallNate)
- Fix behavioral regression: add WorkflowAPIError 404 fallback in
  suspension-handler hook disposal to handle world-vercel path where
  makeRequest doesn't map 404 to HookNotFoundError (TooTallNate)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move the 404 → HookNotFoundError translation into world-vercel's
createWorkflowRunEvent, where we know the event type context. For
hook-related events (hook_created, hook_disposed, hook_received,
hook_conflict), a 404 from the server means the hook was not found.

This removes the WorkflowAPIError 404 fallback from the runtime's
suspension-handler, keeping the runtime fully decoupled from HTTP
status codes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the runtime and world implementations to stop relying on HTTP status codes for control flow, replacing them with semantic error types from @workflow/errors (e.g., conflict/expired/throttle/too-early). This keeps HTTP concerns contained to the HTTP boundary (world-vercel) and simplifies runtime error handling.

Changes:

  • Add semantic error classes (EntityConflictError, RunExpiredError, ThrottleError, TooEarlyError) to @workflow/errors.
  • Update core runtime and world implementations (local/postgres/vercel) to throw/catch semantic errors instead of checking WorkflowAPIError.status.
  • Update affected unit/integration tests and add a changeset for patch releases.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/world-vercel/src/utils.ts Map specific HTTP status codes to semantic errors at the request boundary.
packages/world-vercel/src/events.ts Translate hook-related 404s into HookNotFoundError for event creation.
packages/world-postgres/test/storage.test.ts Update assertions to reflect semantic/not-status-based errors.
packages/world-postgres/src/storage.ts Replace fake HTTP-status WorkflowAPIErrors with semantic errors (and status-free WorkflowAPIError where appropriate).
packages/world-local/src/storage/events-storage.ts Replace status-based WorkflowAPIErrors with semantic errors for conflicts/expired/too-early/hook-not-found.
packages/world-local/src/storage.test.ts Update concurrency/conflict tests to assert semantic errors.
packages/world-local/src/queue.ts Minor retry loop refactor (no behavior change intended).
packages/world-local/src/fs.ts Replace “file already exists” 409-style errors with EntityConflictError.
packages/errors/src/index.ts Introduce new semantic error types with type guards.
packages/core/src/runtime/suspension-handler.ts Replace status checks with semantic error type checks.
packages/core/src/runtime/step-handler.ts Replace status checks with semantic error type checks; use TooEarlyError.retryAfter.
packages/core/src/runtime/step-handler.test.ts Update mocks/assertions for conflict handling to use EntityConflictError.
packages/core/src/runtime/runs.ts Replace 409 status check with EntityConflictError.
packages/core/src/runtime/runs.test.ts Update wake-up tests to use EntityConflictError.
packages/core/src/runtime.ts Replace 409/410 status checks with EntityConflictError / RunExpiredError.
.changeset/semantic-world-errors.md Publish patch changes for affected packages.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment on lines +351 to +355
if (response.status === 425) {
const retryAfterDate = retryAfter
? new Date(Date.now() + retryAfter * 1000)
: undefined;
throwWithTrace(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch — fixed in f529deb. The Retry-After header is now parsed unconditionally (not gated on response.status === 429), so both 425 and 429 responses get the server-provided delay.

Comment on lines +369 to +372
'hook_created',
'hook_disposed',
'hook_received',
'hook_conflict',
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in f529deb — narrowed to hook_disposed and hook_received only, matching world-local's hookEventsRequiringExistence set. hook_created and hook_conflict don't require the hook to already exist, so a 404 there would be a different kind of error.

Copy link
Member

@VaguelySerious VaguelySerious left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WorkflowAPIError remains for: Genuinely unexpected HTTP errors from world-vercel (the only real HTTP boundary)

Should we name this WorkflowWorldError? Might be easier for users to interpret

- Parse Retry-After header unconditionally so TooEarlyError gets
  the server-provided delay instead of always falling back to ~1s
- Narrow hookEventsRequiringExistence to only hook_disposed and
  hook_received (matching world-local's set), since hook_created
  and hook_conflict don't imply the hook must already exist

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pranaygp
Copy link
Collaborator Author

pranaygp commented Mar 18, 2026

Re: @VaguelySerious's suggestion to rename WorkflowAPIErrorWorkflowWorldError — that makes sense but feels like a separate PR since it's a public API rename that would touch many files and need a breaking change annotation. Happy to do it if you want it in this PR though.

human: actually maybe we should do this because we're already breaking the functionality of WorkflowAPIError - if people are relying on WorkflowAPIError checks right now in their own code (unlikely), then their catch handlers will start to fail when we're bubbling up semantic errors instead.

Maybe this is a good time to make the breaking error change (patch release is fine for beta though I think)

Breaking change: rename WorkflowAPIError → WorkflowWorldError to
better reflect that this error represents world (storage backend)
failures, not HTTP API errors specifically. Updated across all
packages: errors, core, world-local, world-vercel, world-postgres,
workflow, and web.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pranaygp
Copy link
Collaborator Author

Done in 8ba3db5 — renamed WorkflowAPIErrorWorkflowWorldError across all 16 source files. Marked as a breaking change in the changeset. Updated the docstring to reflect its new role as the catch-all for world (storage backend) failures.

@pranaygp
Copy link
Collaborator Author

I also started a new PR to export and document the new error types. Including documenting the pattern of starting a workflow in response to HookNotFound - #1447

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants