Add load-testing skill to all agent templates: Load Testing Template + Skill + Locust Load Testing Script + Dashboard Generation#184
Conversation
Adds a shared /load-testing Claude Code skill that guides users through load testing their Databricks Apps to find maximum QPS. Includes mock LLM setup, Locust test scripts, deployment matrix, and dashboard generation. - Add source skill at .claude/skills/load-testing/SKILL.md - Add "load-testing" to sync-skills.py shared skills list - Sync skill to all 5 registered agent templates - Whitelist skill in .gitignore for all 7 agent templates (including advanced) - Use `uv add` instead of `pip install` for dependency management - Pass --workers directly to start-server (no wrapper script needed) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
589c6dc to
fce48b0
Compare
… specific results - Make Step 2 (mocking) explicitly optional — users can test real agents E2E - Lower defaults to keep tests under 1 hour (max_users 300, step_duration 30s) - Change default workers from 4 to 2 - Switch all commands from `python` to `uv run` - Remove internal-specific reference results (too specific to mocked LLM case) - Re-sync skill to all templates Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
dhruv0811
left a comment
There was a problem hiding this comment.
Looks great!! It will be super useful for customers to be able to load test their agents easily now.
Left a few comments, also will need to rebase to use the new templates!
| copy_skill(SOURCE / "quickstart", dest / "quickstart", quickstart_subs) | ||
|
|
||
| # Shared skills (no substitution needed) | ||
| for skill in ["run-locally", "discover-tools", "migrate-from-model-serving"]: |
There was a problem hiding this comment.
Can we exclude agent-non-conversational from getting this skill maybe? I think the input format we are using in this skill doesn't work with the non-conv template's expectations
|
|
||
| ### Install Dependencies | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
nit: can we clarify to either cd load-test-scripts/ or use a separate pyproject.toml, otherwise we end up adding locust and requests to the template's own pyproject.toml which feels like we are polluting the agent's prod setup.
There was a problem hiding this comment.
Do we want to add load-test-runs/<run-name>/ to the gitignore out of the box as well?
| ### Install Dependencies | ||
|
|
||
| ```bash | ||
| uv add locust requests |
There was a problem hiding this comment.
Also seems that latest locust >=2.43 throws a RecursionError? The workaround was pinning locust>=2.32,<2.40 and urllib3<2.3. Maybe we can pin these in the instructions here too?
…gitignore runs - Exclude agent-non-conversational from load-testing skill (incompatible input format) - Use separate pyproject.toml in load-test-scripts/ to avoid polluting agent deps - Pin locust>=2.32,<2.40 and urllib3<2.3 to avoid RecursionError in locust>=2.43 - Add load-test-runs/ to .gitignore in all templates Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds a shared load-testing Claude Code skill that helps users benchmark their Databricks Apps QPS using Locust. The skill guides users through mocking LLM calls, setting up load test scripts, running ramp-to-saturation tests, and viewing interactive dashboards.
See what example resulting app template looks like from this repo:
#180
Users can use skill or run load testing script by running command like:
which will allow them to see progress while load tests run



generates HTML dashboard at end

https://agent-app-load-test-results-5763476629593792.aws.databricksapps.com/