Skip to content

ci(chromatic): skip Chromatic on Dependabot PRs#7425

Draft
talissoncosta wants to merge 1 commit intomainfrom
ci/skip-chromatic-on-dependabot
Draft

ci(chromatic): skip Chromatic on Dependabot PRs#7425
talissoncosta wants to merge 1 commit intomainfrom
ci/skip-chromatic-on-dependabot

Conversation

@talissoncosta
Copy link
Copy Markdown
Contributor

@talissoncosta talissoncosta commented May 5, 2026

Thanks for submitting a PR! Please check the boxes below:

  • I have read the Contributing Guide.
  • I have added information to docs/ if required so people know about the feature.
  • I have filled in the "Changes" section below.
  • I have filled in the "How did you test this code" section below.

Changes

Follow-up to a question raised on #7375 by @Zaimwa9 — Chromatic CI is currently failing on every Dependabot npm PR with ✖ Missing project token.

TL;DR. Dependabot PRs can't read CHROMATIC_PROJECT_TOKEN (separate secrets store), so Chromatic fails with "Missing project token". This skips Chromatic on Dependabot PRs only. main's post-merge push still runs Chromatic with full secrets, so coverage isn't lost.

Cause

GitHub runs Dependabot PRs with a separate secrets store (Settings → Secrets and variables → Dependabot) that doesn't include the CHROMATIC_PROJECT_TOKEN we have in the regular Actions secrets. The Chromatic action then aborts with Missing project token.

This isn't a misconfiguration — it's the platform protecting us. If Actions secrets were exposed to Dependabot PRs, a malicious package-lock.json poisoned dep could exfiltrate them via postinstall scripts (see e.g. the Codecov 2021 incident for the canonical version of this attack).

Confirmed via the workflow logs of #7375's last run:

Secret source: Dependabot
✖ Missing project token

Fix

Skip the Chromatic job when github.actor == 'dependabot[bot]':

if: |
  github.actor != 'dependabot[bot]'
  && (github.event_name == 'push' || github.event.pull_request.draft == false)

When the Dependabot PR eventually merges into main, the push event runs Chromatic with the parent's secrets — so any visual regression a bump introduces is still caught, one merge later instead of pre-merge. Trade-off accepted given dep bumps rarely change visual output.

Alternatives considered

  • Add CHROMATIC_PROJECT_TOKEN to Dependabot secrets store. Would make Chromatic run on Dependabot PRs the same as any other. Rejected: widens the secret-access surface for the bot whose PRs are most likely to bring in arbitrary transitive deps.
  • Switch to pull_request_target event. Foot-gun. Gives PR code access to the base repo's secrets — exactly the attack vector the platform-level isolation is preventing. Hard pass.

How did you test this code?

@vercel
Copy link
Copy Markdown

vercel Bot commented May 5, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
docs Ignored Ignored Preview May 5, 2026 4:49pm
flagsmith-frontend-preview Ignored Ignored Preview May 5, 2026 4:49pm
flagsmith-frontend-staging Ignored Ignored Preview May 5, 2026 4:49pm

Request Review

@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 5, 2026
Dependabot PRs run with a separate secrets store (Settings → Secrets →
Dependabot) that doesn't include CHROMATIC_PROJECT_TOKEN, so the action
aborts with "Missing project token". Skip the job for Dependabot —
main's post-merge push runs Chromatic with full secrets, so any visual
regression a bump introduces is still caught one merge later.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 5, 2026
@talissoncosta talissoncosta force-pushed the ci/skip-chromatic-on-dependabot branch from 4b84781 to 349efa7 Compare May 5, 2026 16:49
@github-actions github-actions Bot added ci-cd Build, test and deployment related and removed ci-cd Build, test and deployment related labels May 5, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 5, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.44%. Comparing base (c771931) to head (349efa7).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #7425   +/-   ##
=======================================
  Coverage   98.44%   98.44%           
=======================================
  Files        1398     1398           
  Lines       52646    52646           
=======================================
  Hits        51826    51826           
  Misses        820      820           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cd Build, test and deployment related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant