[kawshikbuet17]

Adds a benchmark report for the OmniVoice code merged in #1208: how much voice-cloning throughput one RTX 4090 delivers with the C++ runtime vs the Python OmniVoice package, when you add batch size and more instances until the GPU saturates.

What was compared

• C++: perf_mode=off (default) and perf_mode=flash_attention
• Python: plain PyTorch, upstream apply_flashinfer, and FlashInfer + CUDA graphs
• f16 and bf16 for each

How

• One round = N instances (separate processes, one model each) start together; each makes one call with B different sentences, so N x B concurrent requests. Time = until the last output is ready.
• Batch size goes 1, 2, 3, ... until 3 sizes in a row give no new best throughput (or OOM); instances go 1, 2, 3, ... until one more gives no gain (or OOM).
• Bengali voice clone, 16 steps, sentences of 5–30 words and a mixed list. 5 warm-up rounds, then 1 timed round per row.

Results

• Single request: C++ flash_attention is the fastest at every length (106–270 ms for 5–30 words, f16).
• Peak throughput: Python + FlashInfer is the highest, but only with 5–6 processes and 19.9–22.5 GiB of GPU memory. One C++ process already saturates the GPU; a second instance lowers throughput by 5–7%.
• bf16: C++ is 3–12% slower than f16. Plain FlashInfer cannot run bf16 (the upstream patch plans attention as f16).

Files

• docs/reports/omnivoice_concurrency_benchmark.md: findings, setup, method, saturation point per variant, and every round (1511: 1496 measured, 15 failed with the reason)
• docs/reports/data/omnivoice_concurrency_rtx4090.csv: the same table as CSV
• docs/models/omnivoice.md: link to the report

Docs only; no code changes.