Add Verification Stack use case group with QA Changes page

docs.json

@@ -185,7 +185,14 @@
             "group": "Use Cases",
             "pages": [
               "openhands/usage/use-cases/vulnerability-remediation",
-              "openhands/usage/use-cases/code-review",
+              {
+                "group": "Verification Stack",
+                "pages": [
+                  "openhands/usage/use-cases/verification-stack",
+                  "openhands/usage/use-cases/code-review",
+                  "openhands/usage/use-cases/qa-changes"
+                ]
+              },
               "openhands/usage/use-cases/incident-triage",
               "openhands/usage/use-cases/cobol-modernization",
               "openhands/usage/use-cases/dependency-upgrades",

openhands/usage/automations/event-automations.mdx

@@ -1,24 +1,24 @@
 ---
 title: Event-Based Automations
 description: Trigger automations from GitHub events or custom webhooks instead of cron schedules.
 ---
 
 <Info>
 **Beta Feature**: Event-based automations are in beta for **OpenHands Cloud** and **OpenHands Enterprise** users.
 </Info>
 
 Event-based automations run when something happens—a PR is opened, an issue is commented on, or a webhook fires—instead of on a schedule. This is ideal for responsive workflows like auto-reviewing PRs, triaging issues, or reacting to external service events.
 
 ## Built-In vs. Custom Integrations
 
 | Type | Setup | Best For |
 |------|-------|----------|
-| **Built-in (GitHub)** | None—just create the automation | PR reviews, issue triage, push-triggered tasks |
+| **Built-in (GitHub)** | [One-time org setup](#prerequisites-for-github-event-automations), then create automations | PR reviews, issue triage, push-triggered tasks |
 | **Custom Webhooks** | Register webhook first, then create automation | Linear, Stripe, Slack, and other services |
 
 ## GitHub Events (Built-In)
 
 GitHub is a built-in integration. Create automations that respond to GitHub events without any webhook setup.
 
 ### Example: Auto-Review PRs with a Specific Label
 
@@ -83,19 +83,98 @@
 
 ---
 
+## Prerequisites for GitHub Event Automations
+
+GitHub event automations require some one-time setup before events will flow. If any step is missing, automations will appear to work (manual triggers succeed) but GitHub events will silently never arrive.
+
+### 1. Install the OpenHands GitHub App
+
+The OpenHands GitHub App must be installed on the GitHub organization that owns the repositories you want to monitor. Install it from your [GitHub integration settings](/openhands/usage/cloud/github-installation). The app needs access to the repositories that will generate events.
+
+### 2. Create an OpenHands Team Organization
+
+If you're working with repositories owned by a GitHub organization (e.g., `myorg/my-repo`), you need an OpenHands **team organization** — not just a personal account. GitHub events for org repos are routed to team orgs, not personal orgs.
+
+If you don't already have one, create a team organization from the [OpenHands Cloud settings](https://app.all-hands.dev/settings).
+
+### 3. Claim Your GitHub Organization
+
+<Warning>
+**This is the most commonly missed step.** Without it, GitHub events have nowhere to be routed and will be silently dropped.
+</Warning>
+
+Your OpenHands team org must **claim** the GitHub organization to establish the link between GitHub webhooks and your OpenHands org. Claiming tells the event router: _"Events for repos in this GitHub org should go to this OpenHands team org."_
+
+To claim a GitHub org:
+
+1. Switch to your team org using the org switcher in the sidebar
+2. Go to **Organization Settings**
+3. In the **Git Conversation Routing** section, find your GitHub org
+4. Click **Claim**
+
+You must have admin access to the GitHub org to complete the claim. See [Claiming Git Organizations](/openhands/usage/cloud/organizations/settings#claiming-git-organizations) for full details.
+
+### 4. Create the Automation Under the Team Org
+
+Make sure you are switched to the **team org** (not your personal org) when creating the automation. The automation must live in the same org that claimed the GitHub organization — otherwise events won't match.
+
+### 5. (Optional) Add Service Accounts to the Team Org
+
+If you're using a service account (like a bot account) to create or own automations, that account must be a **member of the team org**. Invite them from the [Organization Members](/openhands/usage/cloud/organizations/managing-members) page.
+
+### Example: Setting Up a PR Review Bot
+
+Here's a complete walkthrough for setting up an event-driven PR review automation:
+
+1. **Install the GitHub App** on your GitHub org with access to the target repo
+2. **Switch to your team org** in OpenHands Cloud
+3. **Claim the GitHub org** in Organization Settings → Git Conversation Routing
+4. **Create the automation**:
+
+```
+Create an event-based automation called "PR Review Bot" that triggers
+when a pull request is labeled with "review" in the myorg/my-repo repository.
+
+It should review the PR for code quality, potential bugs, and best practices,
+then post the review as a PR comment.
+```
+
+5. **Test it**: Open a PR in the target repo and add the trigger label. Check your automation runs to verify it was triggered.
+
+### Troubleshooting
+
+If your automation doesn't trigger on GitHub events:
+
+<AccordionGroup>
+  <Accordion title="GitHub org not claimed">
+    The most common cause. Go to **Organization Settings → Git Conversation Routing** and check if your GitHub org shows as claimed. If not, click **Claim**. See [Claiming Git Organizations](/openhands/usage/cloud/organizations/settings#claiming-git-organizations).
+  </Accordion>
+  <Accordion title="Automation in personal org instead of team org">
+    GitHub events for org repos are routed to the **team org** that claimed the GitHub org. If you created the automation under your personal org, events will never reach it. Switch to the team org and recreate the automation.
+  </Accordion>
+  <Accordion title="Event type or filter mismatch">
+    Double-check that the event type (e.g., `pull_request.labeled`) and filter expression match the action you're testing. Use wildcards like `pull_request.*` to match all actions during debugging.
+  </Accordion>
+  <Accordion title="Automation is disabled">
+    Verify the automation is enabled. You can check via the automations list or by asking OpenHands to list your automations.
+  </Accordion>
+</AccordionGroup>
+
+---
+
 ## Custom Webhooks
 
 For services beyond GitHub—like Linear, Stripe, or Slack—register a custom webhook first, then create automations that use it.
 
 <Note>
 **Two-phase workflow for custom webhooks:**
 
 1. **Webhook registration (one-time setup)**: You execute the curl command yourself to register the webhook. This keeps your signing secrets secure—the agent provides the command but never handles your credentials directly.
 
 2. **Automation creation (repeatable)**: Once the webhook is registered, the agent can create, update, and manage automations for that webhook source conversationally—no manual curl commands needed.
 </Note>
 
 ### Walkthrough: Linear Integration
 
 This example walks through setting up a Linear webhook to auto-triage new issues.
 
@@ -138,7 +217,7 @@
 The response includes a `webhook_url` that you'll configure in Linear.
 
 <Accordion title="Understanding event_key_expr">
 The `event_key_expr` is a JMESPath expression that extracts the event type from incoming webhook payloads. This extracted value is what you match against in the automation's `on` field.
 
 For example, Linear sends payloads like:
 ```json
@@ -161,7 +240,7 @@
 
 #### Step 4: Create the Automation
 
 Now the webhook is registered, the agent can create automations for you end-to-end. Just describe what you want:
 
 ```
 Create an event-based automation called "Triage Linear Issues" that triggers
@@ -173,7 +252,7 @@
 
 The agent creates the automation with:
 - **Source**: `linear` (your registered webhook)
 - **Event**: `Issue` (Linear's event type)
 - **Filter**: `action == 'create'`
 
 ### Custom Webhook Parameters
@@ -184,7 +263,7 @@
 |-----------|----------|-------------|
 | `name` | Yes | Human-readable name |
 | `source` | Yes | Unique identifier (lowercase, alphanumeric with hyphens) |
 | `event_key_expr` | No | JMESPath to extract event type (default: `type`) |
 | `signature_header` | No | Header containing HMAC signature (default: `X-Signature-256`) |
 | `webhook_secret` | No | Signing secret—provide yours or let the system generate one |
 
@@ -197,13 +276,13 @@
 | Linear | `Linear-Signature` | `type` |
 | Stripe | `Stripe-Signature` | `type` |
 | Slack | `X-Slack-Signature` | `type` |
 | Twilio | `X-Twilio-Signature` | `type` |
 
 ---
 
 ## Next Steps
 
 New to automations? Start with the [Automations Overview](/openhands/usage/automations/overview) for the bigger picture, including cron-based scheduling and general concepts.
 
 - [Automations Overview](/openhands/usage/automations/overview) — Cron-based automations and general concepts
 - [Managing Automations](/openhands/usage/automations/managing-automations) — Update, disable, or delete automations

openhands/usage/use-cases/code-review.mdx

@@ -130,11 +130,11 @@
 | `use-sub-agents` | Enable sub-agent delegation for file-level reviews in `openhands` mode. Ignored in ACP mode. | No | `'false'` |
 | `extensions-repo` | Extensions repository (owner/repo) | No | `OpenHands/extensions` |
 | `extensions-version` | Git ref for extensions (tag, branch, or commit SHA) | No | `main` |
 | `openhands-sdk-package` | Package spec passed to `uv --with`; override only when pinning a specific SDK build for testing or rollout control | No | `openhands-sdk` |
 | `llm-api-key` | LLM API key. Required when `agent-kind` is `openhands`; ignored in ACP mode. | Yes for OpenHands mode | - |
 | `github-token` | GitHub token for API access | Yes | - |
 | `lmnr-api-key` | Laminar API key for observability | No | `''` |
 | `enable-uv-cache` | Enable setup-uv's GitHub Actions cache for Python deps. Default `false` for security. | No | `'false'` |
 
 <Note>
 Use `extensions-version` to pin to a specific version tag (e.g., `v1.0.0`) for production stability, or use `main` to always get the latest features. The extensions repository contains the PR review plugin scripts.
@@ -148,13 +148,13 @@
 authentication, and tool execution.
 
 Use ACP mode when your runner already has an authenticated ACP CLI available.
 The action does not install ACP CLIs for you; install and authenticate the ACP
 server in workflow steps before invoking the PR review action.
 
 <Warning>
 ACP mode is experimental. Use it on trusted self-hosted runners where you
 control the installed ACP command and the authentication material. Do not expose
 subscription credentials to workflows that run untrusted pull request code.
 </Warning>
 
 ### Codex ACP Example
@@ -326,7 +326,7 @@
 The workflow uses `pull_request_target` so the code review agent can work properly for PRs from forks. Only users with write access can trigger reviews via labels or reviewer requests.
 
 <Warning>
 **Potential Risk**: A malicious contributor could submit a PR from a fork containing code designed to exfiltrate your `LLM_API_KEY` when the review agent analyzes their code.
 
 To mitigate this, the PR review workflow passes API keys as [SDK secrets](/sdk/guides/secrets) rather than environment variables, which prevents the agent from directly accessing these credentials during code execution.
 </Warning>
@@ -340,7 +340,7 @@
 | [#1927](https://github.com/OpenHands/software-agent-sdk/pull/1927#pullrequestreview-3767493657) | Composite GitHub Action refactor | Comprehensive review with 🔴 Critical, 🟠 Important, and 🟡 Suggestion labels |
 | [#1916](https://github.com/OpenHands/software-agent-sdk/pull/1916#pullrequestreview-3758297071) | Add example for reconstructing messages | Critical issues flagged with clear explanations |
 | [#1904](https://github.com/OpenHands/software-agent-sdk/pull/1904#pullrequestreview-3751821740) | Update code-review skill guidelines | APPROVED review highlighting key strengths |
 | [#1889](https://github.com/OpenHands/software-agent-sdk/pull/1889#pullrequestreview-3747576245) | Fix tmux race condition | Technical review of concurrency fix with dual-lock strategy analysis |
 
 ## Troubleshooting
 
@@ -365,25 +365,33 @@
   </Accordion>
 </AccordionGroup>
 
-## Automate This
+## OpenHands Automations (Beta)
 
-You can schedule daily code reviews using [OpenHands Automations](/openhands/usage/automations/overview).
-Copy this prompt into a new conversation to set one up:
+<Warning>
+OpenHands Automations is currently an experimental beta feature. The API and configuration format may change.
+</Warning>
 
-```
-Create an automation called "Daily Code Review" that runs every weekday at 9 AM.
+The alternative to GitHub Actions is [OpenHands Automations](/openhands/usage/automations/overview), our event-triggered automation system. With Automations, you define the trigger once and it covers all repositories the bot account has access to — no per-repo workflow files needed.
 
-It should:
-1. Find all open PRs that have no reviews yet
-2. For each PR, review the diff for bugs, style issues, and security concerns
-3. Post a summary of findings as a comment on each PR
+**Trade-offs vs GitHub Actions:** Simpler to set up and maintain across repos. Also leverages the full OpenHands runtime (browser, tools, sandbox), which GitHub Actions cannot. GitHub Actions gives more control over the execution environment and integrates directly with your CI pipeline.
 
-Learn more at https://docs.openhands.dev/openhands/usage/use-cases/code-review
-```
+<Steps>
+  <Step title="Create a bot account on OpenHands Cloud">
+    Log in to [OpenHands Cloud](https://app.all-hands.dev) with your organization's bot GitHub account.
+  </Step>
 
-For inline review comments on every push, use the
-[pr-review plugin](https://github.com/OpenHands/extensions/tree/main/plugins/pr-review)
-as a GitHub Action instead.
+  <Step title="Connect GitHub">
+    Connect the bot account's GitHub to OpenHands Cloud via the [GitHub installation](/openhands/usage/cloud/github-installation) flow. Make sure to complete the [prerequisites](/openhands/usage/automations/event-automations#prerequisites-for-github-event-automations) — including installing the GitHub App and claiming your org.
+  </Step>
+
+  <Step title="Create the automation">
+    Log in as the bot account and instruct the agent to set up the automation:
+
+    ```
+    Create an OpenHands Cloud automation using the Plugin Preset that automatically reviews pull requests. Use the pr-review plugin from github:OpenHands/extensions (repo_path: plugins/pr-review). Trigger it on pull_request.ready_for_review and pull_request.review_requested events for the repository YOUR_ORG/YOUR_REPO. Filter to trusted contributors only (exclude first-time contributors and unknown users) and only fire on review_requested when the requested reviewer is YOUR_BOT_LOGIN. Set timeout to 3600 seconds. Use this exact prompt for the automation: "Review the pull request from the GitHub event payload using the pr-review plugin. Post a comprehensive code review on GitHub with inline comments on specific changed lines where appropriate, and a concise overall summary. Avoid duplicating existing unresolved review comments. Include a brief note that the review was generated by an AI agent (OpenHands) on behalf of the user through OpenHands Automation."
+    ```
+  </Step>
+</Steps>
 
 ## Related Resources
 

openhands/usage/use-cases/overview.mdx

@@ -8,6 +8,13 @@
 Each use case can be implemented in different ways—as a one-off conversation, a scheduled [automation](/openhands/usage/automations/overview), a [plugin](https://github.com/OpenHands/extensions), or through the [SDK](/sdk/index). Pick the approach that fits your workflow.
 
 <CardGroup cols={2}>
+  <Card
+    title="Verification Stack"
+    icon="layer-group"
+    href="/openhands/usage/use-cases/verification-stack"
+  >
+    A layered system of automated verifiers — trajectory scoring, code review, and QA — that helps coding agents fail fast and produce changes you can trust.
+  </Card>
   <Card
     title="Vulnerability Remediation"
     icon="shield-halved"
@@ -22,6 +29,13 @@
   >
     Set up automated PR reviews to maintain code quality and catch bugs early.
   </Card>
+  <Card
+    title="Automated QA Testing"
+    icon="vial"
+    href="/openhands/usage/use-cases/qa-changes"
+  >
+    Validate PR changes by actually running the software as a real user would.
+  </Card>
   <Card
     title="Incident Triage"
     icon="triangle-exclamation"
@@ -54,7 +68,7 @@
 
 ## Automate Any Use Case
 
 Many use cases work best as scheduled automations. Browse ready-to-use automation templates on the [Automations Overview](/openhands/usage/automations/overview) page—just copy a prompt and paste it into OpenHands.
 
 <CardGroup cols={3}>
   <Card title="View Automation Templates" icon="clock" href="/openhands/usage/automations/overview">

openhands/usage/use-cases/qa-changes.mdx

@@ -0,0 +1,195 @@
+---
+title: Automated QA Testing
+description: Validate pull request changes by actually running the software — not just reading code or running tests
+automation:
+  icon: vial
+  summary: >-
+    Functionally test PR changes by exercising the software as a real user would.
+---
+
+<Card
+  title="View QA Changes Plugin"
+  icon="github"
+  href="https://github.com/OpenHands/extensions/tree/main/plugins/qa-changes"
+>
+  Check out the complete QA changes plugin with ready-to-use code and configuration.
+</Card>
+
+Automated QA testing goes beyond code review and CI: instead of reading diffs or running the test suite, the QA agent actually **runs the software** and verifies that changes work as claimed. It sets up the environment, exercises changed behavior as a real user would (browser, CLI, API requests), and posts a structured report with evidence.
+
+This is Layer 2 of the [Verification Stack](/openhands/usage/use-cases/verification-stack), complementing the [code review agent](/openhands/usage/use-cases/code-review).
+
+## Overview
+
+The QA agent follows a four-phase methodology:
+
+1. **Understand** — Reads the PR diff, title, and description. Classifies changes (new feature, bug fix, refactor, config) and identifies entry points (CLI commands, API endpoints, UI pages).
+2. **Setup** — Bootstraps the repository: installs dependencies, builds the project, notes CI status.
+3. **Exercise** — The core phase: spins up servers, opens browsers, runs CLI commands, makes HTTP requests — testing the changed behavior as a real user would. For bug fixes, it reproduces the bug on the base branch and verifies the fix on the PR branch.
+4. **Report** — Posts a structured QA report as a PR comment, with evidence (commands run, outputs, screenshots) and a verdict (PASS / FAIL / PARTIAL).
+
+The QA agent knows when to give up: if an approach fails after three materially different attempts, it switches strategy. If two fundamentally different strategies fail, it reports what it tried and stops — rather than spinning endlessly.
+
+## What It Does (and Doesn't)
+
+<CardGroup cols={2}>
+  <Card title="QA Agent Does" icon="check">
+    - Run the actual application and interact with it
+    - Make real HTTP requests, run real CLI commands
+    - Open browsers and verify UI changes
+    - Reproduce bugs and verify fixes end-to-end
+    - Report with evidence (commands, outputs, screenshots)
+  </Card>
+  <Card title="QA Agent Does NOT" icon="xmark">
+    - Run the test suite (that's CI's job)
+    - Analyze code for style or structure (that's code review's job)
+    - Run linters, formatters, or type checkers
+    - Substitute `--help` or `--dry-run` for real execution
+  </Card>
+</CardGroup>
+
+## Quick Start
+
+### GitHub Actions
+
+Create `.github/workflows/qa-changes.yml` in your repository:
+
+```yaml
+name: QA Changes
+
+on:
+  pull_request:
+    types: [opened, ready_for_review, labeled]
+
+permissions:
+  contents: read
+  pull-requests: write
+  issues: write
+
+jobs:
+  qa:
+    if: |
+      (github.event.action == 'opened' && github.event.pull_request.draft == false) ||
+      github.event.action == 'ready_for_review' ||
+      github.event.label.name == 'qa-this'
+    runs-on: ubuntu-latest
+    steps:
+      - name: Run QA Changes
+        uses: OpenHands/extensions/plugins/qa-changes@main
+        with:
+          llm-model: anthropic/claude-sonnet-4-5-20250929
+          llm-api-key: ${{ secrets.LLM_API_KEY }}
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+```
+
+Add your `LLM_API_KEY` to your repository's **Settings → Secrets and variables → Actions**.
+
+### In a Conversation
+
+You can also trigger QA manually in any OpenHands conversation by invoking the skill:
+
+```
+/qa-changes
+```
+
+The agent will ask for the PR to test, or you can provide context directly:
+
+```
+/qa-changes — Please QA PR #42 on the my-org/my-repo repository.
+Focus on the new dashboard page and verify it renders correctly.
+```
+
+## QA Report Format
+
+The QA agent posts a structured report as a PR comment:
+
+```
+## QA Report
+
+**Status: PASS** ✅
+
+### Changes Tested
+- New `/api/health` endpoint returns 200 with version info
+- Dashboard page renders at `/dashboard` with correct data
+
+### Evidence
+1. Started server with `npm run dev`
+2. `curl http://localhost:3000/api/health` → 200 OK, body: {"status":"ok","version":"1.2.0"}
+3. Navigated to http://localhost:3000/dashboard — page renders correctly
+   [screenshot attached]
+
+### Edge Cases
+- Empty database state: dashboard shows "No data" placeholder ✅
+- Invalid auth token: returns 401 as expected ✅
+```
+
+## Customization
+
+### Change Types
+
+The QA agent adapts its approach based on the type of change:
+
+| Change Type | QA Approach |
+|-------------|-------------|
+| **Frontend / UI** | Starts dev server, opens browser, verifies visual changes, tests interactions |
+| **CLI** | Runs commands with realistic arguments, verifies output, tests edge cases |
+| **API / Backend** | Starts server, makes HTTP requests, verifies responses and side effects |
+| **Bug fix** | Reproduces bug on base branch, verifies fix on PR branch (before/after) |
+| **Library / SDK** | Writes and runs a short script that imports and calls changed functions |
+
+### Repository-Specific QA Guidelines
+
+Add repo-specific QA instructions by creating `.agents/skills/custom-qa-guide.md`:
+
+```markdown
+---
+name: custom-qa-guide
+description: Custom QA guidelines for this repository
+triggers:
+- /qa-changes
+---
+
+# QA Guidelines for [Your Project]
+
+## Environment Setup
+- Run `make setup` to initialize the development environment
+- The dev server runs on port 8080
+
+## Key Test Scenarios
+- Always verify the admin dashboard at /admin after backend changes
+- For API changes, test with both authenticated and unauthenticated requests
+
+## Known Limitations
+- The payment module requires a Stripe test key — skip payment flow testing
+```
+
+## Integration with the Verification Stack
+
+The QA agent is most powerful when used alongside the [code review agent](/openhands/usage/use-cases/code-review) and the [iterate skill](/openhands/usage/use-cases/verification-stack#closing-the-loop-the-iterate-skill) as part of the full [Verification Stack](/openhands/usage/use-cases/verification-stack):
+
+1. **Code review** catches issues by reading the diff (style, security, data structures)
+2. **QA** catches issues by running the software (behavioral regressions, UI bugs)
+3. **Iterate** orchestrates the loop — fixing issues flagged by either verifier and re-polling until the PR is clean
+
+## Troubleshooting
+
+<AccordionGroup>
+  <Accordion title="QA agent can't start the server">
+    Ensure your repository's setup instructions are documented in `README.md` or `AGENTS.md`. The agent follows these to bootstrap the environment. If setup requires special steps, add them to a custom QA guide.
+  </Accordion>
+
+  <Accordion title="QA report says PARTIAL">
+    PARTIAL means some scenarios passed and others failed or couldn't be tested. Read the report details — it will explain what worked and what didn't. Common causes: missing environment variables, external service dependencies, or insufficient permissions.
+  </Accordion>
+
+  <Accordion title="QA takes too long">
+    For large PRs with many changed entry points, the agent may need more time. Consider splitting large PRs into smaller, focused changes. You can also add a custom QA guide that prioritizes the most important scenarios.
+  </Accordion>
+</AccordionGroup>
+
+## Related Resources
+
+- [QA Changes Plugin](https://github.com/OpenHands/extensions/tree/main/plugins/qa-changes) — GitHub Actions plugin
+- [QA Changes Skill](https://github.com/OpenHands/extensions/tree/main/skills/qa-changes) — Detailed skill methodology
+- [Verification Stack](/openhands/usage/use-cases/verification-stack) — How QA fits into the full verification pipeline
+- [Automated Code Review](/openhands/usage/use-cases/code-review) — The complementary code review agent