Ecosystem ci

.github/workflows/ecosystem.yml

@@ -0,0 +1,63 @@
+name: ecosystem
+
+on:
+  pull_request:
+    branches: [main, master]
+    paths:
+      - '*.ts'
+      - 'lib/**'
+      - 'ecosystem/**'
+      - 'package.json'
+      - 'pnpm-lock.yaml'
+      - 'tsconfig.json'
+      - '.github/workflows/ecosystem.yml'
+  workflow_dispatch:
+
+# Cancel in-flight ecosystem runs when a PR force-pushes — clones are
+# expensive and the only result we care about is the latest one.
+concurrency:
+  group: ecosystem-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  ecosystem:
+    runs-on: ubuntu-latest
+    # Generous because targets get cloned + their deps installed for Glint
+    # type-aware extraction. First run is the slow path (cold caches);
+    # subsequent runs hit the pnpm/npm store cache and are much faster.
+    timeout-minutes: 60
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: pnpm/action-setup@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: '22'
+          cache: pnpm
+
+      # Targets pin their package manager via package.json#packageManager
+      # (e.g. cardstack-ui-components → yarn). Corepack routes the `yarn`
+      # / `pnpm` shims to the pinned version instead of the runner's
+      # global yarn 1.x — without it, `installDeps` falls back to
+      # no-Glint validation for those targets and the baseline silently
+      # loses Glint-resolved findings.
+      - run: corepack enable
+
+      - run: pnpm install --frozen-lockfile
+
+      # Cache target repo clones AND their installed node_modules across runs.
+      # Keyed on targets.json so any SHA bump invalidates the cache and forces
+      # a fresh fetch + reinstall (we want the baseline-vs-actual diff to
+      # reflect the pinned SHA, not whatever was cached from a previous run).
+      # The pnpm store itself is cached by setup-node's `cache: pnpm` (above)
+      # against the current repo's lockfile, so target installs reuse the same
+      # content-addressed store transparently.
+      - name: Cache target clones + node_modules
+        uses: actions/cache@v4
+        with:
+          path: ecosystem/.cache
+          key: ecosystem-clones-${{ hashFiles('ecosystem/targets.json') }}
+
+      - name: Run ecosystem checks
+        run: pnpm run ecosystem:check

.gitignore

@@ -2,4 +2,5 @@ node_modules/
 dist/
 *.log
 .DS_Store
+ecosystem/.cache/
 .claude/

README.md

@@ -121,6 +121,8 @@ templates/admin.gts:18: error [input-attributes] Attribute "readonly" not allowe
 | `eslint-plugin-ember` / `ember-template-lint` | Is this idiomatic Ember? Invocation style, reactivity, modifier API, built-in components, plus JS/TS-side rules from eslint-plugin-ember and some overlap of HTML spec and ARIA rules |
 | `html-validate-ember` (this plugin) | Is the rendered HTML spec-correct and accessible? — content model, ARIA / WCAG, form correctness, attribute validity, duplicate IDs, unique landmarks. |
 
+See [ECOSYSTEM-OVERLAP.md](https://github.com/johanrd/html-validate-ember/blob/main/ECOSYSTEM-OVERLAP.md)
+
 ---
 
 ## Inline errors in VS Code

ecosystem/README.md

@@ -0,0 +1,65 @@
+# Ecosystem CI
+
+Runs `html-validate-ember` against pinned snapshots of public Ember repos and diffs the findings against committed baselines. A non-empty diff fails the workflow.
+
+## What's here
+
+- `targets.json` — the list of pinned repos (owner/repo + SHA + globs + per-target options).
+- `run.ts` — the runner. Clones, installs deps, validates with Glint, diffs.
+- `baselines/<name>.json` — committed expected output per target. The runner refuses to diff if the baseline's pinned SHA disagrees with `targets.json`.
+
+Cached clones (with their installed `node_modules`) live in `.cache/` (gitignored).
+
+## Glint
+
+By default each target is validated with Glint on (`HVE_GLINT=1`). That requires the target's deps installed locally — Glint resolves `@glint/ember-tsc` and the target's TypeScript types from the target's own `node_modules`. The runner detects the package manager from the lockfile and installs with `--ignore-scripts`. If install fails the runner falls back to no-Glint validation for that target and prints a warning rather than failing the run.
+
+Per-target opt-out: set `"glint": false` on a target in `targets.json` (e.g., for repos where install doesn't work cleanly or isn't worth the time).
+
+## Local commands
+
+```bash
+npm run ecosystem:check                       # all targets, fail on diff
+npm run ecosystem:check -- --target=limber    # just one
+npm run ecosystem:check -- --target=a,b,c     # subset
+npm run ecosystem:check -- --no-clone         # skip clone/fetch (faster, uses .cache)
+npm run ecosystem:update                      # rewrite all baselines
+npm run ecosystem:update -- --target=foo      # rewrite one
+```
+
+## When to update a baseline
+
+You're working on the plugin and a real-world target's findings changed. That's the signal — the diff is the value statement of the change. Inspect the `added:` and `removed:` blocks in the runner output:
+
+- **Removed findings** (fewer reports) — usually a fix landed; a FP got eliminated. Good. Update the baseline in the same PR; reviewer sees what disappeared.
+- **Added findings** (new reports) — either the plugin now catches something it didn't before (good — explain in the PR), or it's a regression (don't update the baseline; fix it).
+
+The runner's diff output is the input to the PR description. Don't paraphrase it; paste the relevant lines.
+
+## Bumping target SHAs
+
+Manual for now. Re-fetch the default branch's HEAD, update `ref` in `targets.json`, run `npm run ecosystem:update -- --target=<name>`, commit baseline + targets.json together. The diff in the PR shows what changed in the target's templates between the old and new SHA — those are upstream-side changes, not plugin-side, but they're still worth reading: they're candidates for issue drafts to file with the upstream repo.
+
+If a baseline is committed against a different SHA than `targets.json`, the runner refuses to diff (apples-to-oranges) and asks you to re-baseline. This guards against forgetting the second commit.
+
+## Triage flow (separate from CI)
+
+The committed baselines contain a mix of real bugs, plugin FPs, and html-validate rules that fire on legitimate Ember patterns. Triaging them is *not* what CI does — CI only catches regressions. Triage is a manual, batched activity that produces:
+
+1. **Plugin fixes** for the FPs (which then show up as `removed:` in the next baseline update).
+2. **Issue drafts** for the real bugs in the target repos (one per repo, batched, you file).
+
+The expected workflow is to triage one target at a time, file an issue with the upstream repo if findings warrant it, and let the natural flow of plugin fixes update baselines over time.
+
+## Adding a target
+
+1. Pick a repo with non-trivial templates (`.gts`/`.gjs`/`.hbs`).
+2. Probe its layout: `git clone --depth=1 --filter=blob:none <url> /tmp/probe && find /tmp/probe -name '*.gts' -o -name '*.gjs' -o -name '*.hbs' | head`.
+3. Pin the current default-branch SHA: `git ls-remote <url> HEAD`.
+4. Add a target entry. Globs should cover *real* templates (addon source + docs/test apps), not test fixtures (those often contain intentionally broken HTML for integration tests).
+5. `npm run ecosystem:update -- --target=<name>` to seed the baseline.
+6. Commit `targets.json` and `baselines/<name>.json` together.
+
+## Why bake findings into the baseline rather than gate on zero findings
+
+These targets have hundreds of real-world findings — some genuine, some plugin FPs, some debatable a11y choices. Refusing to merge any plugin change that doesn't drive findings to zero is impractical (the targets are out of our control). Refusing changes that *introduce* new findings is tractable. So the baseline is "current state", and the CI signal is "did this PR change real-world output, and is the change intentional?".

ecosystem/baselines/cardstack-ui-components.json

@@ -0,0 +1,78 @@
+{
+  "ref": "589ad84bdf3afe4f35628f70497d720b9a89c193",
+  "fileCount": 31,
+  "findings": [
+    {
+      "file": "app/templates/components/cta.hbs",
+      "line": 1,
+      "column": 2,
+      "severity": "error",
+      "ruleId": "no-implicit-button-type",
+      "message": "<button> is missing recommended \"type\" attribute"
+    },
+    {
+      "file": "app/templates/components/emails.hbs",
+      "line": 5,
+      "column": 8,
+      "severity": "error",
+      "ruleId": "no-implicit-button-type",
+      "message": "<button> is missing recommended \"type\" attribute"
+    },
+    {
+      "file": "app/templates/components/file-upload-dialog.hbs",
+      "line": 20,
+      "column": 71,
+      "severity": "error",
+      "ruleId": "element-permitted-content",
+      "message": "<div> element is not permitted as content under <span>"
+    },
+    {
+      "file": "app/templates/components/file-upload-dialog.hbs",
+      "line": 21,
+      "column": 101,
+      "severity": "warning",
+      "ruleId": "prefer-native-element",
+      "message": "Prefer to use the native <button> element"
+    },
+    {
+      "file": "app/templates/components/password-field/visibility-toggle.hbs",
+      "line": 1,
+      "column": 2,
+      "severity": "error",
+      "ruleId": "no-implicit-button-type",
+      "message": "<button> is missing recommended \"type\" attribute"
+    },
+    {
+      "file": "app/templates/components/phone-number-fields.hbs",
+      "line": 10,
+      "column": 8,
+      "severity": "error",
+      "ruleId": "no-implicit-button-type",
+      "message": "<button> is missing recommended \"type\" attribute"
+    },
+    {
+      "file": "app/templates/components/street-addresses.hbs",
+      "line": 9,
+      "column": 8,
+      "severity": "error",
+      "ruleId": "no-implicit-button-type",
+      "message": "<button> is missing recommended \"type\" attribute"
+    },
+    {
+      "file": "tests/dummy/app/templates/examples/configurator.hbs",
+      "line": 6,
+      "column": 75,
+      "severity": "error",
+      "ruleId": "no-implicit-input-type",
+      "message": "<input> is missing recommended \"type\" attribute"
+    },
+    {
+      "file": "tests/dummy/app/templates/examples/configurator.hbs",
+      "line": 7,
+      "column": 75,
+      "severity": "error",
+      "ruleId": "no-implicit-input-type",
+      "message": "<input> is missing recommended \"type\" attribute"
+    }
+  ]
+}