Skip to content

Commit bb1b2ed

Browse files
brunoborgesCopilot
andcommitted
Add generator benchmark results (fat JAR vs JBang vs Python)
Fat JAR is fastest warm (0.46s), ~40% faster than JBang (0.77s) and ~3x faster than Python (1.37s). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent b6fc5b5 commit bb1b2ed

File tree

1 file changed

+36
-0
lines changed

1 file changed

+36
-0
lines changed

html-generators/BENCHMARK.md

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Generator Benchmarks
2+
3+
Performance comparison of the three ways to run the HTML generator, measured on 85 snippets across 10 categories.
4+
5+
## Results
6+
7+
| Method | Cold Start | Warm Average | Notes |
8+
|--------|-----------|-------------|-------|
9+
| **Fat JAR** (`java -jar`) | 1.72s | **0.46s** | Fastest warm; no JBang needed |
10+
| **JBang** (`jbang generate.java`) | 0.96s | 0.77s | Includes JBang overhead |
11+
| **Python** (`python3 generate.py`) | 0.17s | 1.37s | Fastest cold start; slowest warm |
12+
13+
- **Cold start**: First run after clearing caches / fresh process
14+
- **Warm average**: Mean of 4 subsequent runs
15+
16+
## Key Takeaways
17+
18+
- The **fat JAR** is the fastest option for repeated builds — **~40% faster** than JBang and **~3× faster** than Python at warm steady-state
19+
- **Python** has the fastest cold start (no JVM boot) but is the slowest overall
20+
- **JBang** adds ~0.3s overhead vs the fat JAR due to its launcher and cache lookup
21+
- For CI/CD, the fat JAR is ideal — no JBang installation or dependency caching required
22+
23+
## Environment
24+
25+
| | |
26+
|---|---|
27+
| **CPU** | Apple M1 Max |
28+
| **RAM** | 32 GB |
29+
| **Java** | OpenJDK 25.0.1 (Temurin) |
30+
| **JBang** | 0.136.0 |
31+
| **Python** | 3.14.3 |
32+
| **OS** | macOS (Darwin) |
33+
34+
## Methodology
35+
36+
Each method was warmed up once, then timed 5 times using `/usr/bin/time -p`. The first run is reported as "cold start" and the remaining 4 runs are averaged for "warm average". Between each run, `site/index.html` was reset via `git checkout` to ensure the `{{snippetCount}}` patch runs each time.

0 commit comments

Comments
 (0)