Skip to content

Validate cached workflows/steps and reduce build debounce #700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
ijjk wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Validate cached workflows/steps and reduce build debounce #700

ijjk wants to merge 1 commit into from
+50 −3

Conversation

@ijjk
Copy link
Member

@ijjk ijjk commented Dec 28, 2025

This ensures when we read step/workflows state from cache that we validate they are still valid before initial compile, this also tweaks the stubs build latency and adds signal when the stubs are waiting.

@changeset-bot
Copy link

changeset-bot bot commented Dec 28, 2025

🦋 Changeset detected

Latest commit: a734ad3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@workflow/next Patch
workflow Patch
@workflow/ai Patch
@workflow/world-testing Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Contributor

vercel bot commented Dec 28, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Dec 28, 2025 9:50pm
example-nextjs-workflow-webpack Ready Ready Preview, Comment Dec 28, 2025 9:50pm
example-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-astro-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-express-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-fastify-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-hono-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-nitro-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-nuxt-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-sveltekit-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm
workbench-vite-workflow Ready Ready Preview, Comment Dec 28, 2025 9:50pm

@github-actions
Copy link
Contributor

github-actions bot commented Dec 28, 2025 •
edited
Loading

📊 Benchmark Results

📈 Comparing against baseline from main branch. Green 🟢 = faster, Red 🔺 = slower.

workflow with no steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 0.039s (+1.6%) 1.017s (~) 0.978s 10 1.00x
🌐 Starter Next.js (Turbopack) 0.039s (+1.3%) 1.014s (~) 0.975s 10 1.01x
💻 Local Nitro 0.042s (-0.7%) 1.006s (~) 0.965s 10 1.06x
🌐 Redis Next.js (Turbopack) 0.043s (+10.1% 🔺) 1.019s (~) 0.976s 10 1.09x
💻 Local Express 0.044s (~) 1.007s (~) 0.963s 10 1.12x
🌐 MongoDB Next.js (Turbopack) 0.084s (+22.1% 🔺) 1.015s (~) 0.931s 10 2.14x
🌐 Turso Next.js (Turbopack) 0.093s (-13.1% 🟢) 1.013s (~) 0.920s 10 2.36x
🐘 Postgres Next.js (Turbopack) 0.217s (-26.5% 🟢) 1.020s (~) 0.803s 10 5.53x
🐘 Postgres Nitro 0.322s (+5.9% 🔺) 1.012s (~) 0.690s 10 8.21x
🐘 Postgres Express 0.344s (+41.6% 🔺) 1.013s (-1.3%) 0.668s 10 8.79x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 0.541s (-1.5%) 1.426s (~) 0.885s 10 1.00x
▲ Vercel Nitro 0.542s (-2.1%) 1.539s (+8.3% 🔺) 0.997s 10 1.00x
▲ Vercel Next.js (Turbopack) 0.731s (+23.1% 🔺) 1.675s (+7.3% 🔺) 0.944s 10 1.35x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

workflow with 1 step

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Starter 🥇 Next.js (Turbopack) 1.082s (-1.0%) 2.011s (~) 0.929s 10 1.00x
🌐 Redis Next.js (Turbopack) 1.094s (~) 2.013s (~) 0.920s 10 1.01x
💻 Local Next.js (Turbopack) 1.096s (~) 2.011s (~) 0.915s 10 1.01x
💻 Local Nitro 1.113s (~) 2.006s (~) 0.893s 10 1.03x
💻 Local Express 1.113s (~) 2.007s (~) 0.893s 10 1.03x
🌐 MongoDB Next.js (Turbopack) 1.296s (-1.4%) 2.013s (~) 0.718s 10 1.20x
🌐 Turso Next.js (Turbopack) 1.311s (~) 2.011s (~) 0.700s 10 1.21x
🐘 Postgres Nitro 2.164s (-8.6% 🟢) 3.015s (~) 0.852s 10 2.00x
🐘 Postgres Express 2.200s (-4.0%) 3.014s (~) 0.814s 10 2.03x
🐘 Postgres Next.js (Turbopack) 2.385s (+0.8%) 3.017s (~) 0.631s 10 2.20x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.517s (-1.6%) 3.489s (-0.8%) 0.973s 10 1.00x
▲ Vercel Next.js (Turbopack) 2.569s (-8.6% 🟢) 3.633s (-4.5%) 1.064s 10 1.02x
▲ Vercel Express 2.650s (+2.9%) 3.527s (-0.9%) 0.877s 10 1.05x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

workflow with 10 sequential steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Starter 🥇 Next.js (Turbopack) 10.590s (~) 11.012s (~) 0.422s 5 1.00x
💻 Local Next.js (Turbopack) 10.649s (~) 11.018s (~) 0.369s 5 1.01x
🌐 Redis Next.js (Turbopack) 10.671s (~) 11.017s (~) 0.346s 5 1.01x
💻 Local Nitro 10.789s (~) 11.013s (~) 0.224s 5 1.02x
💻 Local Express 10.797s (~) 11.013s (~) 0.216s 5 1.02x
🌐 Turso Next.js (Turbopack) 12.211s (~) 13.023s (~) 0.812s 5 1.15x
🌐 MongoDB Next.js (Turbopack) 12.251s (~) 13.026s (~) 0.775s 5 1.16x
🐘 Postgres Next.js (Turbopack) 15.418s (-23.6% 🟢) 16.239s (-22.8% 🟢) 0.821s 5 1.46x
🐘 Postgres Express 20.354s (+10.0% 🔺) 21.032s (+11.7% 🔺) 0.678s 5 1.92x
🐘 Postgres Nitro 20.422s (~) 21.032s (~) 0.610s 5 1.93x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 21.091s (-2.2%) 21.851s (-3.0%) 0.759s 5 1.00x
▲ Vercel Nitro 21.245s (-2.0%) 22.063s (-1.5%) 0.817s 5 1.01x
▲ Vercel Next.js (Turbopack) 21.475s (-4.1%) 21.973s (-4.2%) 0.497s 5 1.02x

🔍 Observability: Express | Nitro | Next.js (Turbopack)

Promise.all with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Starter 🥇 Next.js (Turbopack) 1.351s (~) 2.008s (~) 0.657s 15 1.00x
🌐 Redis Next.js (Turbopack) 1.359s (-0.5%) 2.011s (~) 0.652s 15 1.01x
💻 Local Next.js (Turbopack) 1.381s (-1.0%) 2.012s (~) 0.631s 15 1.02x
💻 Local Nitro 1.422s (~) 2.005s (~) 0.584s 15 1.05x
💻 Local Express 1.424s (+1.2%) 2.007s (~) 0.583s 15 1.05x
🐘 Postgres Next.js (Turbopack) 2.047s (-11.2% 🟢) 2.227s (-18.8% 🟢) 0.180s 14 1.52x
🌐 MongoDB Next.js (Turbopack) 2.171s (+1.9%) 3.015s (~) 0.844s 10 1.61x
🌐 Turso Next.js (Turbopack) 2.199s (-1.4%) 3.013s (~) 0.814s 10 1.63x
🐘 Postgres Nitro 2.431s (~) 3.012s (~) 0.581s 10 1.80x
🐘 Postgres Express 2.515s (+6.7% 🔺) 3.012s (~) 0.497s 10 1.86x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.779s (-4.5%) 3.710s (-2.1%) 0.931s 9 1.00x
▲ Vercel Next.js (Turbopack) 2.824s (+1.8%) 3.691s (+0.6%) 0.867s 9 1.02x
▲ Vercel Express 3.200s (+12.5% 🔺) 4.128s (+11.4% 🔺) 0.928s 8 1.15x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.all with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Next.js (Turbopack) 2.145s (-1.6%) 3.054s (-1.2%) 0.909s 10 1.00x
💻 Local Nitro 2.218s (~) 3.157s (~) 0.939s 10 1.03x
💻 Local Express 2.249s (+1.0%) 3.184s (~) 0.935s 10 1.05x
🌐 Starter Next.js (Turbopack) 2.459s (~) 3.010s (~) 0.551s 10 1.15x
🌐 Redis Next.js (Turbopack) 2.486s (~) 3.016s (~) 0.529s 10 1.16x
🐘 Postgres Next.js (Turbopack) 2.800s (+0.9%) 3.223s (+6.9% 🔺) 0.423s 10 1.31x
🐘 Postgres Nitro 2.881s (-6.0% 🟢) 3.114s (-13.0% 🟢) 0.233s 10 1.34x
🐘 Postgres Express 3.101s (~) 3.528s (+1.7%) 0.427s 9 1.45x
🌐 MongoDB Next.js (Turbopack) 4.568s (-3.2%) 5.187s (~) 0.619s 6 2.13x
🌐 Turso Next.js (Turbopack) 4.675s (-0.6%) 5.182s (~) 0.507s 6 2.18x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 3.285s (-12.4% 🟢) 3.869s (-13.3% 🟢) 0.585s 8 1.00x
▲ Vercel Next.js (Turbopack) 3.451s (-7.0% 🟢) 3.977s (-12.3% 🟢) 0.526s 8 1.05x
▲ Vercel Express 3.613s (-1.4%) 4.290s (-5.3% 🟢) 0.676s 7 1.10x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Promise.race with 10 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
🌐 Starter 🥇 Next.js (Turbopack) 1.349s (-1.1%) 2.008s (~) 0.658s 15 1.00x
🌐 Redis Next.js (Turbopack) 1.372s (+1.4%) 2.011s (~) 0.639s 15 1.02x
💻 Local Next.js (Turbopack) 1.405s (~) 2.014s (~) 0.609s 15 1.04x
💻 Local Nitro 1.410s (~) 2.005s (~) 0.596s 15 1.04x
💻 Local Express 1.431s (~) 2.006s (~) 0.575s 15 1.06x
🐘 Postgres Next.js (Turbopack) 1.860s (+10.9% 🔺) 2.013s (~) 0.153s 15 1.38x
🐘 Postgres Nitro 1.963s (+12.2% 🔺) 2.161s (+7.5% 🔺) 0.198s 14 1.45x
🐘 Postgres Express 2.080s (+18.9% 🔺) 2.595s (+29.0% 🔺) 0.514s 12 1.54x
🌐 MongoDB Next.js (Turbopack) 2.189s (+2.4%) 3.011s (~) 0.823s 10 1.62x
🌐 Turso Next.js (Turbopack) 2.218s (-0.7%) 3.011s (~) 0.792s 10 1.64x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.823s (+2.8%) 3.784s (+5.8% 🔺) 0.961s 8 1.00x
▲ Vercel Express 2.975s (+13.1% 🔺) 3.697s (+3.1%) 0.723s 9 1.05x
▲ Vercel Next.js (Turbopack) 3.220s (+15.4% 🔺) 4.153s (+12.0% 🔺) 0.932s 8 1.14x

🔍 Observability: Nitro | Express | Next.js (Turbopack)

Promise.race with 25 concurrent steps

💻 Local Development

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
💻 Local 🥇 Nitro 2.235s (+0.9%) 3.183s (+1.2%) 0.948s 10 1.00x
💻 Local Next.js (Turbopack) 2.242s (+2.3%) 3.150s (+2.0%) 0.909s 10 1.00x
💻 Local Express 2.310s (+2.5%) 3.237s (+1.6%) 0.927s 10 1.03x
🌐 Starter Next.js (Turbopack) 2.459s (-1.3%) 3.010s (-0.8%) 0.551s 10 1.10x
🌐 Redis Next.js (Turbopack) 2.512s (+1.2%) 3.011s (~) 0.499s 10 1.12x
🐘 Postgres Next.js (Turbopack) 2.716s (+4.6%) 3.020s (~) 0.304s 10 1.22x
🐘 Postgres Nitro 2.722s (+4.2%) 3.012s (~) 0.291s 10 1.22x
🐘 Postgres Express 2.784s (+7.7% 🔺) 3.015s (~) 0.231s 10 1.25x
🌐 MongoDB Next.js (Turbopack) 4.574s (-3.5%) 5.181s (~) 0.607s 6 2.05x
🌐 Turso Next.js (Turbopack) 4.758s (+0.7%) 5.185s (~) 0.427s 6 2.13x

▲ Production (Vercel)

World Framework Workflow Time Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Nitro 2.978s (-9.7% 🟢) 3.615s (-9.3% 🟢) 0.637s 9 1.00x
▲ Vercel Next.js (Turbopack) 3.100s (-8.9% 🟢) 3.708s (-9.1% 🟢) 0.608s 9 1.04x
▲ Vercel Express 3.578s (+16.7% 🔺) 4.132s (+16.4% 🔺) 0.555s 8 1.20x

🔍 Observability: Nitro | Next.js (Turbopack) | Express

Stream Benchmarks (includes TTFB metrics)
workflow with stream

💻 Local Development

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
🌐 Starter 🥇 Next.js (Turbopack) 0.128s (-5.2% 🟢) 1.005s (~) 0.000s (~) 1.014s (~) 0.886s 10 1.00x
💻 Local Next.js (Turbopack) 0.137s (-7.9% 🟢) 1.003s (~) 0.018s (+6.6% 🔺) 1.029s (~) 0.893s 10 1.07x
🌐 Redis Next.js (Turbopack) 0.143s (-1.1%) 1.004s (~) 0.000s (NaN%) 1.014s (~) 0.870s 10 1.12x
💻 Local Nitro 0.176s (~) 0.993s (~) 0.017s (-17.2% 🟢) 1.023s (~) 0.847s 10 1.38x
💻 Local Express 0.177s (-1.7%) 0.993s (~) 0.018s (+22.9% 🔺) 1.025s (~) 0.848s 10 1.38x
🌐 Turso Next.js (Turbopack) 0.474s (-5.8% 🟢) 0.981s (+3.4%) 0.000s (NaN%) 1.013s (~) 0.539s 10 3.71x
🌐 MongoDB Next.js (Turbopack) 0.523s (+6.4% 🔺) 0.929s (-3.5%) 0.000s (-100.0% 🟢) 1.013s (~) 0.490s 10 4.10x
🐘 Postgres Next.js (Turbopack) 1.211s (-50.4% 🟢) 1.835s (-29.5% 🟢) 0.000s (-100.0% 🟢) 2.019s (-33.1% 🟢) 0.808s 10 9.48x
🐘 Postgres Nitro 2.346s (+7.3% 🔺) 2.697s (-5.6% 🟢) 0.000s (-100.0% 🟢) 3.014s (~) 0.668s 10 18.37x
🐘 Postgres Express 2.410s (+99.3% 🔺) 2.635s (+43.9% 🔺) 0.000s (+Infinity% 🔺) 3.015s (+49.7% 🔺) 0.604s 10 18.88x

▲ Production (Vercel)

World Framework Workflow Time TTFB Slurp Wall Time Overhead Samples vs Fastest
▲ Vercel 🥇 Express 2.609s (+3.3%) 3.032s (-9.5% 🟢) 0.359s (-27.4% 🟢) 3.829s (-9.6% 🟢) 1.220s 10 1.00x
▲ Vercel Next.js (Turbopack) 2.701s (+2.1%) 3.072s (-5.3% 🟢) 0.633s (+80.0% 🔺) 4.135s (+3.2%) 1.434s 10 1.04x
▲ Vercel Nitro 3.075s (-5.3% 🟢) 3.606s (-3.4%) 0.221s (-66.3% 🟢) 4.245s (-11.9% 🟢) 1.170s 10 1.18x

🔍 Observability: Express | Next.js (Turbopack) | Nitro

Summary

Fastest Framework by World

Winner determined by most benchmark wins

World 🥇 Fastest Framework Wins
💻 Local Next.js (Turbopack) 7/8
🐘 Postgres Next.js (Turbopack) 7/8
▲ Vercel Nitro 5/8
Fastest World by Framework

Winner determined by most benchmark wins

Framework 🥇 Fastest World Wins
Express 💻 Local 8/8
Next.js (Turbopack) 🌐 Starter 5/8
Nitro 💻 Local 8/8
Column Definitions
  • Workflow Time: Runtime reported by workflow (completedAt - createdAt) - primary metric
  • TTFB: Time to First Byte - time from workflow start until first stream byte received (stream benchmarks only)
  • Slurp: Time from first byte to complete stream consumption (stream benchmarks only)
  • Wall Time: Total testbench time (trigger workflow + poll for result)
  • Overhead: Testbench overhead (Wall Time - Workflow Time)
  • Samples: Number of benchmark iterations run
  • vs Fastest: How much slower compared to the fastest configuration for this benchmark

Worlds:

  • 💻 Local: In-memory filesystem world (local development)
  • 🐘 Postgres: PostgreSQL database world (local development)
  • ▲ Vercel: Vercel production/preview deployment
  • 🌐 Starter: Community world (local development)
  • 🌐 Turso: Community world (local development)
  • 🌐 MongoDB: Community world (local development)
  • 🌐 Redis: Community world (local development)
  • 🌐 Jazz: Community world (local development)

📋 View full workflow run

@github-actions
Copy link
Contributor

github-actions bot commented Dec 28, 2025 •
edited
Loading

🧪 E2E Test Results

❌ Some tests failed

Summary

Passed Failed Skipped Total
✅ ▲ Vercel Production 286 0 11 297
✅ 💻 Local Development 262 0 8 270
✅ 📦 Local Production 262 0 8 270
✅ 🐘 Local Postgres 262 0 8 270
✅ 🪟 Windows 27 0 0 27
❌ 🌍 Community Worlds 109 11 0 120
Total 1208 11 35 1254

❌ Failed Tests

🌍 Community Worlds (11 failed)

mongodb (1 failed):

  • webhookWorkflow

redis (1 failed):

  • webhookWorkflow

starter (8 failed):

  • addTenWorkflow
  • addTenWorkflow
  • retryAttemptCounterWorkflow
  • crossFileErrorWorkflow - stack traces work across imported modules
  • hookCleanupTestWorkflow - hook token reuse after workflow completion
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars)
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step

turso (1 failed):

  • webhookWorkflow

Details by Category

✅ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 26 0 1
✅ example 26 0 1
✅ express 26 0 1
✅ fastify 26 0 1
✅ hono 26 0 1
✅ nextjs-turbopack 26 0 1
✅ nextjs-webpack 26 0 1
✅ nitro 26 0 1
✅ nuxt 26 0 1
✅ sveltekit 26 0 1
✅ vite 26 0 1
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 26 0 1
✅ express-stable 26 0 1
✅ fastify-stable 26 0 1
✅ hono-stable 26 0 1
✅ nextjs-turbopack-stable 27 0 0
✅ nextjs-webpack-stable 27 0 0
✅ nitro-stable 26 0 1
✅ nuxt-stable 26 0 1
✅ sveltekit-stable 26 0 1
✅ vite-stable 26 0 1
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 26 0 1
✅ express-stable 26 0 1
✅ fastify-stable 26 0 1
✅ hono-stable 26 0 1
✅ nextjs-turbopack-stable 27 0 0
✅ nextjs-webpack-stable 27 0 0
✅ nitro-stable 26 0 1
✅ nuxt-stable 26 0 1
✅ sveltekit-stable 26 0 1
✅ vite-stable 26 0 1
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 26 0 1
✅ express-stable 26 0 1
✅ fastify-stable 26 0 1
✅ hono-stable 26 0 1
✅ nextjs-turbopack-stable 27 0 0
✅ nextjs-webpack-stable 27 0 0
✅ nitro-stable 26 0 1
✅ nuxt-stable 26 0 1
✅ sveltekit-stable 26 0 1
✅ vite-stable 26 0 1
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 27 0 0
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 3 0 0
❌ mongodb 26 1 0
✅ redis-dev 3 0 0
❌ redis 26 1 0
✅ starter-dev 3 0 0
❌ starter 19 8 0
✅ turso-dev 3 0 0
❌ turso 26 1 0

📋 View full workflow run

@ijjk ijjk marked this pull request as ready for review December 28, 2025 22:02
VaguelySerious
VaguelySerious approved these changes Dec 29, 2025
View reviewed changes
Copy link
Member

@VaguelySerious VaguelySerious left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work great for me locally. If this is an immediate fix for some of the issues we're seeing, let's merge soon and ship

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@VaguelySerious VaguelySerious VaguelySerious approved these changes

@TooTallNate TooTallNate Awaiting requested review from TooTallNate

@pranaygp pranaygp Awaiting requested review from pranaygp

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ijjk @VaguelySerious