docs: add QA Changes use case and SDK workflow guide#431
docs: add QA Changes use case and SDK workflow guide#431
Conversation
Add documentation for the new QA Changes plugin: - openhands/usage/use-cases/qa-changes.mdx: Use case page with overview, quick start, customization, and troubleshooting - sdk/guides/github-workflows/qa-changes.mdx: SDK guide with reference workflow, action inputs, and skill customization - docs.json: Add both pages to navigation Co-authored-by: openhands <openhands@all-hands.dev>
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
all-hands-bot
left a comment
all-hands-bot
left a comment
There was a problem hiding this comment.
🔴 Needs improvement - Security vulnerability and broken auto-sync infrastructure. Core content is solid, but the Quick Start workflow is missing critical security controls.
| if: | | ||
| (github.event.action == 'opened' && github.event.pull_request.draft == false) || | ||
| github.event.action == 'ready_for_review' || | ||
| github.event.label.name == 'qa-this' || | ||
| github.event.requested_reviewer.login == 'openhands-agent' |
There was a problem hiding this comment.
🔴 Critical - Security: This workflow is missing author association checks, directly contradicting your security warning at line 197.
The actual source workflow (verified in extensions PR #135) includes:
if: |
(github.event.action == 'opened' && github.event.pull_request.draft == false && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||
(github.event.action == 'ready_for_review' && github.event.pull_request.author_association != 'FIRST_TIME_CONTRIBUTOR' && github.event.pull_request.author_association != 'NONE') ||
github.event.label.name == 'qa-this' ||
github.event.requested_reviewer.login == 'openhands-agent'Without these checks, first-time contributors can automatically trigger code execution, which you explicitly warn against. This is a real security issue, not theoretical.
| if: | | ||
| (github.event.action == 'opened' && github.event.pull_request.draft == false) || | ||
| github.event.action == 'ready_for_review' || | ||
| github.event.label.name == 'qa-this' || | ||
| github.event.requested_reviewer.login == 'openhands-agent' |
There was a problem hiding this comment.
🔴 Critical - Concurrency: Missing concurrency block from the source workflow.
The actual workflow includes:
concurrency:
group: qa-changes-${{ github.event.pull_request.number }}
cancel-in-progress: trueWithout this, multiple triggers (e.g., label + reviewer request) will spawn concurrent QA runs that interfere with each other and waste compute.
| github.event.action == 'ready_for_review' || | ||
| github.event.label.name == 'qa-this' || | ||
| github.event.requested_reviewer.login == 'openhands-agent' | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
🟡 Suggestion: Use ubuntu-24.04 to match the source workflow instead of ubuntu-latest. Pinning the Ubuntu version ensures consistent behavior and avoids surprises when GitHub changes what "latest" means.
| ```bash | ||
| # 1. Copy workflow to your repository | ||
| cp plugins/qa-changes/workflows/qa-changes-by-openhands.yml \ | ||
| .github/workflows/qa-changes-by-openhands.yml |
There was a problem hiding this comment.
🟠 Important - Broken Instructions: This assumes users already have the extensions repo cloned, but that's never explained.
Either:
- Change this to reference the GitHub URL directly (recommended):
curl -o .github/workflows/qa-changes-by-openhands.yml \ https://raw.githubusercontent.com/OpenHands/extensions/main/plugins/qa-changes/workflows/qa-changes-by-openhands.yml
- Add a step to clone the extensions repo first
The former is simpler and matches how users actually consume GitHub Actions workflows.
| This example is available on GitHub: [plugins/qa-changes/workflows/](https://github.com/OpenHands/extensions/tree/main/plugins/qa-changes/workflows) | ||
| </Note> | ||
|
|
||
| ```yaml icon="yaml" expandable plugins/qa-changes/workflows/qa-changes-by-openhands.yml |
There was a problem hiding this comment.
🟠 Important - Broken Auto-Sync: This uses the code block sync format (plugins/qa-changes/workflows/qa-changes-by-openhands.yml), but your sync infrastructure only works with the agent-sdk repo.
I verified .github/workflows/sync-docs-code-blocks.yml only checks out OpenHands/software-agent-sdk, not OpenHands/extensions. This means:
- The workflow YAML won't auto-sync when the extensions repo changes
- It will drift over time and become incorrect
- Users will copy broken workflows
Fix: Either extend the sync script to support multiple source repos, or remove the file reference and treat this as a manual code block (less ideal, but honest about the maintenance burden).
- CI-aware: check CI first, only run tests CI doesn't cover - High-bar exercise: browsers, CLI, HTTP requests - Graceful failure: give up and suggest AGENTS.md guidance - PARTIAL verdict for incomplete verification Co-authored-by: openhands <openhands@all-hands.dev>
3 participants
Summary
Add documentation for the new QA Changes plugin (
OpenHands/extensions#135).Changes
Use Case Page (
openhands/usage/use-cases/qa-changes.mdx)SDK Workflow Guide (
sdk/guides/github-workflows/qa-changes.mdx)Navigation (
docs.json)qa-changesto Use Cases (after code-review)qa-changesto SDK GitHub Workflows (after pr-review)Related
This PR was created by an AI assistant (OpenHands).