Self-hosted Inngest: excessive rapid retries causing duplicate DB inserts

[kingRayhan]

Summary

On our self-hosted Inngest deployment (inngest.techdiary.dev), function runs are retrying far more often than expected—many attempts fire in rapid succession (e.g. 21 attempts, Attempt 0–20, all clustered within about one second of the initial queue time). That behavior drives duplicate side effects in our app, including unnecessary database insert operations.

Environment

• Inngest: self-hosted (dashboard served from inngest.techdiary.dev)
• App: this repo (techdiary.dev) — functions include DB-writing steps (e.g. notification / queue-related handlers tied to Inngest)

Observed behavior

• A single run (example Run ID prefix 0737c22d3bfae…) shows Attempt 0 through Attempt 20 in the run sidebar.
• Queued and ended timestamps for the selected attempt are only ~1s apart for the overall burst, with attempts stacked at the start of the timeline rather than spaced by normal backoff.
• Each attempt can re-execute logic that inserts rows, so retries multiply DB writes beyond what we intend for one logical event.

Expected behavior

• One successful execution per event, or a small number of retries with clear backoff when the handler fails or returns a retryable error.
• Retries should not fan out into dozens of near-simultaneous attempts for a single run unless we explicitly configured that (we did not).

Hypothesis / areas to investigate

• Misconfiguration of the self-hosted executor or SDK (retry policy, retries, step idempotency).
• Idempotency: ensure handlers use stable idempotency keys / dedupe for inserts where Inngest may legitimately retry.
• Version skew between Inngest server and @inngest package, or known bugs in self-hosted retry scheduling.
• Whether the handler is throwing or returning a status that triggers immediate retry loops on the server side.

Acceptance criteria

• Root cause identified (config vs. app code vs. Inngest self-hosted bug).
• Either fix deployment/config so retries match expectations, or harden DB paths so retries are idempotent (no duplicate inserts for one logical event).
• Document any required env vars / version pins for self-hosted Inngest in our runbooks or CLAUDE.md if applicable.

References

• Inngest dashboard (self-hosted): https://inngest.techdiary.dev (internal)