Skip to content

Update simplew-tuned#768

Open
stratdev3 wants to merge 2 commits into
MDA2AV:mainfrom
stratdev3:update-simplew-tuned
Open

Update simplew-tuned#768
stratdev3 wants to merge 2 commits into
MDA2AV:mainfrom
stratdev3:update-simplew-tuned

Conversation

@stratdev3
Copy link
Copy Markdown
Contributor

Description

update simplew-tuned to 26.0.1-alpha.20260530-1860
this fixes a stackoverflow under heavy load !!


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run all benchmark tests
/benchmark -f <framework> -t <test> Run a specific test
/benchmark -f <framework> --save Run and save results (updates leaderboard on merge)

Always specify -f <framework>. Results are automatically compared against the current leaderboard.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

update to 26.0.1-alpha.20260530-1860
this fixes a stackoverflow under heavy load !!
@stratdev3
Copy link
Copy Markdown
Contributor Author

/benchmark -f simplew-tuned

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew-tuned | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,845,367 5808.1% 213MiB -10.2% +13.9%
baseline 4096 1,802,233 6055.9% 717MiB -7.9% +9.1%
pipelined 512 2,986,346 5988.1% 236MiB -13.0% -0.8%
pipelined 4096 2,814,208 6036.5% 271MiB -16.1% -22.8%
limited-conn 512 870,534 4514.1% 123MiB +1.2% -30.9%
limited-conn 4096 869,125 4208.8% 264MiB -0.4% +55.3%
json 4096 810,392 5660.4% 336MiB -4.5% +4.3%
json-comp 512 454,271 5740.2% 196MiB -4.2% +1.0%
json-comp 4096 457,201 5714.2% 247MiB -5.0% -22.1%
json-comp 16384 461,442 5647.9% 319MiB -3.5% +14.7%
json-tls 4096 633,868 6214.9% 1.0GiB +9.8% -9.1%
upload 32 1,836 3084.5% 2.2GiB +1.9% ~0%
upload 256 1,967 6108.1% 6.3GiB +2.3% +3.3%
api-4 256 51,167 388.5% 149MiB -4.8% -5.7%
api-16 1024 135,325 1509.7% 272MiB -4.2% -4.2%
static 1024 63,387 4143.6% 875MiB -13.8% -9.0%
static 4096 64,562 4428.9% 1.3GiB -16.7% -13.3%
static 6800 65,055 4418.0% 1.7GiB -12.2% -5.6%
async-db 1024 187,988 3699.1% 382MiB +4.6% +0.8%
crud 4096 431,804 3659.5% 1.2GiB -0.6% +9.1%
echo-ws 512 1,773,155 5855.7% 153MiB -10.0% -8.4%
echo-ws 4096 1,706,356 6232.5% 635MiB -8.7% -7.0%
echo-ws 16384 1,506,661 5629.0% 1.6GiB -4.6% +14.3%
echo-ws-pipeline 512 2,767,337 6137.7% 225MiB -3.7% +11.4%
echo-ws-pipeline 4096 2,568,166 5912.8% 430MiB -12.8% -8.9%
echo-ws-pipeline 16384 2,406,244 5972.3% 484MiB -13.7% -30.1%
Full log
  WS upgrades: 184
  WS frames:   13664736
  Latency samples: 13664704 / 13664736 responses (100.0%)
  Latency overflow (>5s): 2
[info] CPU 6153.6% | Mem 249MiB

=== Best: 2767337 req/s (CPU: 6137.7%, Mem: 225MiB) ===
[info] saved results/echo-ws-pipeline/512/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    697us    398us    547us   1.45ms   11.70ms

  11494015 frames sent in 5.00s, 11488742 frames received
  Throughput: 2.30M req/s
  Bandwidth:  15.34MB/s
  WS upgrades: 500
  WS frames:   11488742
  Latency samples: 11488742 / 11488742 responses (100.0%)
[info] CPU 5948.5% | Mem 159MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    562us    387us    447us   2.09ms   7.77ms

  11756707 frames sent in 5.00s, 11747095 frames received
  Throughput: 2.35M req/s
  Bandwidth:  15.70MB/s
  WS upgrades: 1109
  WS frames:   11747095
  Latency samples: 11747062 / 11747095 responses (100.0%)
  Latency overflow (>5s): 3
[info] CPU 5771.0% | Mem 265MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    632us    368us    430us   3.44ms   15.60ms

  12851721 frames sent in 5.00s, 12840834 frames received
  Throughput: 2.57M req/s
  Bandwidth:  17.16MB/s
  WS upgrades: 1088
  WS frames:   12840834
  Latency samples: 12840828 / 12840834 responses (100.0%)
  Latency overflow (>5s): 10
[info] CPU 5912.8% | Mem 430MiB

=== Best: 2568166 req/s (CPU: 5912.8%, Mem: 430MiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.62ms    420us    588us   3.52ms   208.50ms

  10124733 frames sent in 5.00s, 10109747 frames received
  Throughput: 2.02M req/s
  Bandwidth:  13.52MB/s
  WS upgrades: 937
  WS frames:   10109747
  Latency samples: 10109747 / 10109747 responses (100.0%)
[info] CPU 5359.9% | Mem 261MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   2.42ms    383us    452us   3.50ms   839.70ms

  12071809 frames sent in 5.00s, 12031221 frames received
  Throughput: 2.41M req/s
  Bandwidth:  16.14MB/s
  WS upgrades: 2537
  WS frames:   12031221
  Latency samples: 12031221 / 12031221 responses (100.0%)
[info] CPU 5972.3% | Mem 484MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.41ms    387us    463us   4.33ms    1.04s

  11831167 frames sent in 5.00s, 11785266 frames received
  Throughput: 2.36M req/s
  Bandwidth:  15.82MB/s
  WS upgrades: 2869
  WS frames:   11785266
  Latency samples: 11785266 / 11785266 responses (100.0%)
[info] CPU 5450.7% | Mem 720MiB

=== Best: 2406244 req/s (CPU: 5972.3%, Mem: 484MiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

@stratdev3
Copy link
Copy Markdown
Contributor Author

/benchmark -f simplew-tuned

@github-actions
Copy link
Copy Markdown
Contributor

👋 /benchmark request received. A collaborator will review and approve the run.

@github-actions
Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew-tuned | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 1,798,910 5766.9% 183MiB -12.5% -2.1%
baseline 4096 1,764,876 5670.0% 575MiB -9.9% -12.5%
pipelined 512 2,978,323 6028.0% 245MiB -13.3% +2.9%
pipelined 4096 2,718,964 5935.5% 301MiB -18.9% -14.2%
limited-conn 512 859,772 4479.7% 132MiB ~0% -25.8%
limited-conn 4096 868,806 4404.1% 234MiB -0.5% +37.6%
json 4096 781,256 5710.1% 303MiB -7.9% -5.9%
json-comp 512 455,302 5677.1% 195MiB -4.0% +0.5%
json-comp 4096 459,864 5579.0% 370MiB -4.5% +16.7%
json-comp 16384 460,770 5482.4% 421MiB -3.7% +51.4%
json-tls 4096 640,925 6167.1% 1.2GiB +11.0% +9.1%
upload 32 1,832 2919.1% 2.3GiB +1.7% +4.5%
upload 256 1,973 6018.1% 6.5GiB +2.7% +6.6%
api-4 256 51,748 389.8% 150MiB -3.7% -5.1%
api-16 1024 134,945 1506.0% 300MiB -4.5% +5.6%
static 1024 67,912 4357.0% 918MiB -7.7% -4.6%
static 4096 69,082 4594.9% 1.4GiB -10.8% -6.7%
static 6800 65,222 4333.3% 1.6GiB -12.0% -11.1%
async-db 1024 178,661 3363.6% 388MiB -0.5% +2.4%
crud 4096 429,111 3665.9% 956MiB -1.2% -15.1%
echo-ws 512 1,802,015 5938.0% 223MiB -8.6% +33.5%
echo-ws 4096 1,707,829 6066.1% 537MiB -8.6% -21.4%
echo-ws 16384 1,445,541 5709.4% 1.9GiB -8.4% +35.7%
echo-ws-pipeline 512 2,751,608 6170.4% 211MiB -4.2% +4.5%
echo-ws-pipeline 4096 2,717,531 5983.7% 425MiB -7.7% -10.0%
echo-ws-pipeline 16384 2,570,606 5968.5% 625MiB -7.8% -9.7%
Full log
  WS upgrades: 171
  WS frames:   13758042
  Latency samples: 13757954 / 13758042 responses (100.0%)
  Latency overflow (>5s): 1
[info] CPU 6170.4% | Mem 211MiB

=== Best: 2751608 req/s (CPU: 6170.4%, Mem: 211MiB) ===
[info] saved results/echo-ws-pipeline/512/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    972us    408us    580us   3.26ms   51.30ms

  11152012 frames sent in 5.00s, 11149233 frames received
  Throughput: 2.23M req/s
  Bandwidth:  14.89MB/s
  WS upgrades: 345
  WS frames:   11149233
  Latency samples: 11149233 / 11149233 responses (100.0%)
[info] CPU 5922.5% | Mem 142MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    897us    391us    463us   4.58ms   18.40ms

  11811486 frames sent in 5.00s, 11800648 frames received
  Throughput: 2.36M req/s
  Bandwidth:  15.77MB/s
  WS upgrades: 1187
  WS frames:   11800648
  Latency samples: 11800599 / 11800648 responses (100.0%)
  Latency overflow (>5s): 13
[info] CPU 5872.4% | Mem 321MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency    423us    352us    409us   3.35ms   10.10ms

  13595011 frames sent in 5.00s, 13587658 frames received
  Throughput: 2.72M req/s
  Bandwidth:  18.15MB/s
  WS upgrades: 1018
  WS frames:   13587658
  Latency samples: 13587618 / 13587658 responses (100.0%)
  Latency overflow (>5s): 24
[info] CPU 5983.7% | Mem 425MiB

=== Best: 2717531 req/s (CPU: 5983.7%, Mem: 425MiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.15ms    426us    599us   3.37ms   36.10ms

  10147283 frames sent in 5.00s, 10137369 frames received
  Throughput: 2.03M req/s
  Bandwidth:  13.55MB/s
  WS upgrades: 620
  WS frames:   10137369
  Latency samples: 10137369 / 10137369 responses (100.0%)
[info] CPU 5688.9% | Mem 250MiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   1.55ms    390us    461us   3.85ms   92.00ms

  11954758 frames sent in 5.00s, 11924315 frames received
  Throughput: 2.38M req/s
  Bandwidth:  15.98MB/s
  WS upgrades: 1903
  WS frames:   11924315
  Latency samples: 11924315 / 11924315 responses (100.0%)
[info] CPU 5924.2% | Mem 397MiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.02ms    376us    449us   3.40ms    1.06s

  12890020 frames sent in 5.00s, 12853030 frames received
  Throughput: 2.57M req/s
  Bandwidth:  17.23MB/s
  WS upgrades: 2312
  WS frames:   12853030
  Latency samples: 12853030 / 12853030 responses (100.0%)
[info] CPU 5968.5% | Mem 625MiB

=== Best: 2570606 req/s (CPU: 5968.5%, Mem: 625MiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-16-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/api-4-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/async-db-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/crud-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/echo-ws-pipeline-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-16384.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-comp-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-tls-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/limited-conn-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/pipelined-512.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/static-6800.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/upload-32.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant