ci: concurrent sampling ci test#4209
Conversation
Signed-off-by: kapish <upadhyaykapish@gmail.com>
There was a problem hiding this comment.
Pull request overview
Adds a dedicated GitHub Actions workflow to run an end-to-end concurrency sampling check against the samples-go/sampling-test app, validating that --enable-sampling gates recordings under concurrent load without breaking client responses.
Changes:
- Introduces a new CI workflow (
Sampling Test) to build Keploy, check outkeploy/samples-go, and execute the sampling E2E test. - Adds a new bash harness that records with
--enable-sampling, generates concurrent traffic, stops the recorder, and asserts captured test counts fall within expected bounds. - Uploads relevant artifacts (record log, recordings, curl results) on failure for debugging.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| .github/workflows/test_workflow_scripts/sampling/run-sampling-test.sh | Adds the end-to-end sampling test driver script (build app, run record with sampling, fire concurrent traffic, assert capture bounds). |
| .github/workflows/sampling-test.yml | Adds a new workflow to build Keploy, check out samples-go, run the sampling test script, and upload artifacts on failures. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
The previous count used `find -name 'test-*.yaml'`, but keploy names test cases by HTTP method + path (get-work-N.yaml), so the find matched zero files and the assertion failed even though sampling was working correctly (K=5 /work captures out of N=20 concurrent requests). Switch the find pattern to `get-work-*.yaml` so the readiness probe on /health that lands in the same test-set doesn't inflate the count above the sampling budget either. Also print the full file count separately for debugging visibility. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Signed-off-by: kapish <upadhyaykapish@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
Signed-off-by: kapish <upadhyaykapish@gmail.com>
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
🚀 Keploy Performance Test ResultsMulti-Run Validation: Tests run 3 times, pipeline fails only if 2+ runs show regression.
Thresholds: P50 < 5ms, P90 < 15ms, P99 < 70ms, RPS >= 100 (±1% tolerance), Error Rate < 1% ✅ Result: PASSED - Only 0 out of 3 runs failed (threshold: 2) P50, P90, and P99 percentiles naturally filter out outliers |
Describe the changes that are made
Adds an end-to-end GitHub Actions pipeline that exercises the
--enable-samplingflag ofkeploy recordand asserts the documented bypass behaviour.What runs on every PR and every push to
main:keployfrom the current branch.sampling-testsample app fromkeploy/samples-go.keploy record --enable-sampling=$K -c ./sampling-test.:8080, then runcurl.shto fire$NconcurrentGET /workrequests, each on its own TCP connection. The handler sleepsHANDLER_DELAY_MS(default 500 ms) so the burst overlaps inside the proxy and the slot semaphore is genuinely contended.SIGINTto keploy../keploy/test-set-0/tests/and assert:feedback_no_client_errors), and$K ≤ captured_test_cases < $N(sampling actually gated some traffic, but not all of it).Defaults:
K=5,N=20, handler delay 500 ms. All overridable viaworkflow_dispatchinputs (sampling_limit,total_requests,samples_go_ref).Files added:
.github/workflows/sampling-test.yml— workflow definition; triggers onpull_request,pushtomain, andworkflow_dispatch..github/workflows/test_workflow_scripts/sampling/run-sampling-test.sh— driver script (build sample, record, curl burst, SIGINT, assert).The companion sample app lives in keploy/samples-go#232. The workflow currently pins
samples_go_reftogo-samplingso the two PRs can be merged in either order; flip the default tomainonce the samples-go PR is in.Links & References
Closes: #4211
🔗 Related PRs
feat: app for testing concurrent sampling(the sample app this workflow drives)🐞 Related Issues
📄 Related Documents
What type of PR is this? (check all applicable)
Added e2e test pipeline?
Added comments for hard-to-understand areas?
get-work-*.yamlonlyAdded to documentation?
README.md(in samples-go) explains how to reproduce locallyAre there any sample code or steps to test the changes?
Reproduce locally:
Self Review done?
Any relevant screenshots, recordings or logs?
Latest CI run on this branch — successful assertion (post-fix in commit 843d753):
curl-side, every client got a 2xx response:
🧠 Semantics for PR Title & Branch Name
ci: concurrent sampling ci testci-samplingAdditional checklist: