Skip to content

Effinitive v2.0.0#747

Draft
HBartosch wants to merge 9 commits into
MDA2AV:mainfrom
HBartosch:main
Draft

Effinitive v2.0.0#747
HBartosch wants to merge 9 commits into
MDA2AV:mainfrom
HBartosch:main

Conversation

@HBartosch
Copy link
Copy Markdown

Description

Adding The Effinitive Framework

PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@HBartosch
Copy link
Copy Markdown
Author

/benchmark -f effinitive --save

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@HBartosch HBartosch marked this pull request as ready for review May 20, 2026 13:07
Copilot AI review requested due to automatic review settings May 20, 2026 13:07
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new effinitive framework implementation to HttpArena, including a .NET 10 container build and a set of benchmark endpoints intended to participate in the standard profile suite.

Changes:

  • Added frameworks/effinitive metadata (meta.json) and documentation (README.md) for the new framework entry.
  • Added a .NET 10 Docker build (SDK build stage + runtime stage with libmsquic) and a build helper script.
  • Implemented baseline/json/upload/compression/db/ws/static endpoints and DTOs/source-generated JSON context under frameworks/effinitive/app.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
frameworks/effinitive/README.md Framework description, endpoint list, and operational notes.
frameworks/effinitive/meta.json Leaderboard/tooling metadata and subscribed benchmark profiles.
frameworks/effinitive/Dockerfile Builds and packages the Effinitive .NET app into a runtime image (incl. libmsquic).
frameworks/effinitive/build.sh Local build helper for the framework image.
frameworks/effinitive/app/Program.cs Effinitive app bootstrap: ports, TLS, static files, WebSockets, compression, endpoint mapping.
frameworks/effinitive/app/Models.cs DTOs and JsonSerializerContext source-gen declarations.
frameworks/effinitive/app/effinitive-arena.csproj .NET 10 project configuration + package references.
frameworks/effinitive/app/Tests/Baseline.cs /pipeline, /baseline11 (GET/POST), /baseline2 handlers.
frameworks/effinitive/app/Tests/Upload.cs /upload streaming body reader + byte count response.
frameworks/effinitive/app/Tests/Json.cs /json/{count} dataset preload + response projection.
frameworks/effinitive/app/Tests/Compression.cs /compression pre-serialized JSON payload endpoint.
frameworks/effinitive/app/Tests/Database.cs /db SQLite range query endpoint with a simple connection pool.
frameworks/effinitive/app/Tests/AsyncDatabase.cs /async-db Postgres range query endpoint using NpgsqlDataSource.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread frameworks/effinitive/README.md
Comment thread frameworks/effinitive/README.md Outdated
Comment thread frameworks/effinitive/meta.json Outdated
Comment thread frameworks/effinitive/meta.json
Comment thread frameworks/effinitive/Dockerfile
Comment thread frameworks/effinitive/app/Tests/Json.cs
Comment thread frameworks/effinitive/app/Tests/AsyncDatabase.cs
@HBartosch HBartosch marked this pull request as draft May 20, 2026 13:27
@HBartosch HBartosch marked this pull request as ready for review May 20, 2026 13:54
@HBartosch HBartosch marked this pull request as draft May 20, 2026 14:01
@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: effinitive | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,385,487 5618.1% 431MiB NEW NEW
baseline 4096 1,435,736 5790.0% 1.6GiB NEW NEW
pipelined 512 4,892,880 5568.9% 398MiB NEW NEW
pipelined 4096 5,467,452 6208.9% 1.5GiB NEW NEW
limited-conn 512 374,230 2714.7% 182MiB NEW NEW
limited-conn 4096 362,707 2667.2% 205MiB NEW NEW
json 4096 376,957 4131.8% 371MiB NEW NEW
upload 32 1,166 3008.1% 200MiB NEW NEW
upload 256 1,450 4858.8% 480MiB NEW NEW
api-4 256 46,030 394.3% 280MiB NEW NEW
api-16 1024 78,773 1545.2% 314MiB NEW NEW
static 1024 150,585 5198.6% 1.4GiB NEW NEW
static 4096 159,063 5312.1% 1.7GiB NEW NEW
static 6800 161,470 5396.5% 2.0GiB NEW NEW
async-db 1024 166,493 3522.6% 568MiB NEW NEW
baseline-h2 256 1,876,072 6266.4% 2.5GiB NEW NEW
baseline-h2 1024 2,021,859 6136.2% 12.9GiB NEW NEW
static-h2 256 101 87.5% 157MiB NEW NEW
static-h2 1024 404 286.7% 327MiB NEW NEW
baseline-h3 64 100,377 1950.7% 569MiB NEW NEW
static-h3 64 59,773 4509.9% 936MiB NEW NEW
echo-ws 512 414,850 3818.7% 510MiB NEW NEW
echo-ws 4096 383,556 2505.8% 1.6GiB NEW NEW
echo-ws 16384 404,118 2517.1% 3.2GiB NEW NEW
echo-ws-pipeline 512 757,013 2405.0% 674MiB NEW NEW
echo-ws-pipeline 4096 874,276 1589.9% 1.4GiB NEW NEW
echo-ws-pipeline 16384 725,773 1418.1% 1.9GiB NEW NEW
Full log
  3788456 frames sent in 5.00s, 3785065 frames received
  Throughput: 756.74K req/s
  Bandwidth:  5.06MB/s
  WS upgrades: 512
  WS frames:   3785065
  Latency samples: 3785057 / 3785065 responses (100.0%)
[info] CPU 2405.0% | Mem 674MiB

=== Best: 757013 req/s (CPU: 2405.0%, Mem: 674MiB) ===
[info] saved results/echo-ws-pipeline/512/effinitive.json
httparena-bench-effinitive
httparena-bench-effinitive

==============================================
=== effinitive / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   31.02ms   19.40ms   72.20ms   180.70ms   246.30ms

  3804222 frames sent in 5.00s, 3741038 frames received
  Throughput: 747.94K req/s
  Bandwidth:  5.09MB/s
  WS upgrades: 4091
  WS frames:   3741038
  Latency samples: 3741038 / 3741038 responses (100.0%)
[info] CPU 2074.3% | Mem 1012MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   39.66ms   25.50ms   97.50ms   182.30ms   249.90ms

  4397640 frames sent in 5.00s, 4336888 frames received
  Throughput: 867.05K req/s
  Bandwidth:  5.88MB/s
  WS upgrades: 4096
  WS frames:   4336888
  Latency samples: 4336883 / 4336888 responses (100.0%)
[info] CPU 1472.9% | Mem 1.3GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   47.53ms   35.20ms   109.60ms   159.10ms   195.60ms

  4432313 frames sent in 5.00s, 4371384 frames received
  Throughput: 873.97K req/s
  Bandwidth:  5.93MB/s
  WS upgrades: 4096
  WS frames:   4371384
  Latency samples: 4371371 / 4371384 responses (100.0%)
[info] CPU 1589.9% | Mem 1.4GiB

=== Best: 874276 req/s (CPU: 1589.9%, Mem: 1.4GiB) ===
[info] saved results/echo-ws-pipeline/4096/effinitive.json
httparena-bench-effinitive
httparena-bench-effinitive

==============================================
=== effinitive / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   34.27ms   24.70ms   81.20ms   135.00ms   178.20ms

  3270572 frames sent in 5.00s, 3208412 frames received
  Throughput: 641.45K req/s
  Bandwidth:  4.38MB/s
  WS upgrades: 3885
  WS frames:   3208412
  Latency samples: 3208412 / 3208412 responses (100.0%)
[info] CPU 1926.2% | Mem 1.0GiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   79.91ms   53.20ms   171.60ms   413.40ms   548.50ms

  3600165 frames sent in 5.00s, 3481749 frames received
  Throughput: 696.10K req/s
  Bandwidth:  4.83MB/s
  WS upgrades: 7657
  WS frames:   3481749
  Latency samples: 3481486 / 3481749 responses (100.0%)
[info] CPU 1293.7% | Mem 1.6GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   62.28ms   40.50ms   155.90ms   248.60ms   294.70ms

  3738293 frames sent in 5.00s, 3628869 frames received
  Throughput: 725.52K req/s
  Bandwidth:  5.01MB/s
  WS upgrades: 6839
  WS frames:   3628869
  Latency samples: 3628869 / 3628869 responses (100.0%)
[info] CPU 1418.1% | Mem 1.9GiB

=== Best: 725773 req/s (CPU: 1418.1%, Mem: 1.9GiB) ===
[info] saved results/echo-ws-pipeline/16384/effinitive.json
httparena-bench-effinitive
httparena-bench-effinitive
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h2-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-h3-64.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants