-
Notifications
You must be signed in to change notification settings - Fork 5
Benchmarks v2 #118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Benchmarks v2 #118
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
3 issues found across 7 files
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="benchmarks2/app.ts">
<violation number="1" location="benchmarks2/app.ts:31">
P3: Use nullish coalescing (or an explicit numeric check) so a valid `0` delay isn't replaced by the default.</violation>
<violation number="2" location="benchmarks2/app.ts:32">
P2: Global `fetch` is only available in Node 18+, but the project supports Node 16. Calling `fetch` here will crash on Node 16. Import a fetch implementation (e.g., undici/node-fetch) or guard for unsupported runtimes.</violation>
</file>
<file name="benchmarks2/run_benchmark.sh">
<violation number="1" location="benchmarks2/run_benchmark.sh:6">
P2: The benchmark command is piped into `tee` without `pipefail`, so failures in benchmark.py can be ignored and the script will continue with a bad baseline. Add `set -o pipefail` to ensure pipeline failures stop the script.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| # | ||
| # Usage: ./run_benchmark.sh [duration_seconds] | ||
|
|
||
| set -e |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: The benchmark command is piped into tee without pipefail, so failures in benchmark.py can be ignored and the script will continue with a bad baseline. Add set -o pipefail to ensure pipeline failures stop the script.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At benchmarks2/run_benchmark.sh, line 6:
<comment>The benchmark command is piped into `tee` without `pipefail`, so failures in benchmark.py can be ignored and the script will continue with a bad baseline. Add `set -o pipefail` to ensure pipeline failures stop the script.</comment>
<file context>
@@ -0,0 +1,53 @@
+#
+# Usage: ./run_benchmark.sh [duration_seconds]
+
+set -e
+
+DURATION=${1:-5}
</file context>
| }); | ||
|
|
||
| app.post('/api/downstream', async (req, res) => { | ||
| const delayMs = req.body.delay_ms || 10; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3: Use nullish coalescing (or an explicit numeric check) so a valid 0 delay isn't replaced by the default.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At benchmarks2/app.ts, line 31:
<comment>Use nullish coalescing (or an explicit numeric check) so a valid `0` delay isn't replaced by the default.</comment>
<file context>
@@ -0,0 +1,37 @@
+});
+
+app.post('/api/downstream', async (req, res) => {
+ const delayMs = req.body.delay_ms || 10;
+ await fetch(`${DELAY_SERVER}/delay?ms=${delayMs}`);
+ res.json({ status: 'ok' });
</file context>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
2 issues found across 5 files (changes from recent commits).
Prompt for AI agents (all issues)
Check if these issues are valid — if so, understand the root cause of each and fix them.
<file name="benchmarks2/benchmark.py">
<violation number="1" location="benchmarks2/benchmark.py:75">
P2: The health check treats any HTTP response as ready. If /health returns 4xx/5xx, the loop still breaks and benchmarks run against a failing server. Capture the response and call raise_for_status (or check status_code) before breaking.</violation>
</file>
<file name="benchmarks2/app.ts">
<violation number="1" location="benchmarks2/app.ts:36">
P2: http.get only supports http:// URLs, so a https DELAY_SERVER will now fail. Consider using a protocol-aware client (http/https switch) or revert to fetch to keep HTTPS support.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| print(f"Connecting to {base_url}...") | ||
| for _ in range(50): | ||
| try: | ||
| session.get(f'{base_url}/health', timeout=0.5) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: The health check treats any HTTP response as ready. If /health returns 4xx/5xx, the loop still breaks and benchmarks run against a failing server. Capture the response and call raise_for_status (or check status_code) before breaking.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At benchmarks2/benchmark.py, line 75:
<comment>The health check treats any HTTP response as ready. If /health returns 4xx/5xx, the loop still breaks and benchmarks run against a failing server. Capture the response and call raise_for_status (or check status_code) before breaking.</comment>
<file context>
@@ -0,0 +1,114 @@
+ print(f"Connecting to {base_url}...")
+ for _ in range(50):
+ try:
+ session.get(f'{base_url}/health', timeout=0.5)
+ break
+ except Exception:
</file context>
| app.post("/api/downstream", async (req, res) => { | ||
| const delayMs = req.body.delay_ms || 10; | ||
| await new Promise<void>((resolve, reject) => { | ||
| http.get(`${DELAY_SERVER}/delay?ms=${delayMs}`, (response) => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2: http.get only supports http:// URLs, so a https DELAY_SERVER will now fail. Consider using a protocol-aware client (http/https switch) or revert to fetch to keep HTTPS support.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At benchmarks2/app.ts, line 36:
<comment>http.get only supports http:// URLs, so a https DELAY_SERVER will now fail. Consider using a protocol-aware client (http/https switch) or revert to fetch to keep HTTPS support.</comment>
<file context>
@@ -18,20 +21,26 @@ await initSDK();
- await fetch(`${DELAY_SERVER}/delay?ms=${delayMs}`);
- res.json({ status: 'ok' });
+ await new Promise<void>((resolve, reject) => {
+ http.get(`${DELAY_SERVER}/delay?ms=${delayMs}`, (response) => {
+ response.resume();
+ response.on("end", resolve);
</file context>
No description provided.