Fast transmission of large blocks at moderate IB rate?

[bwbush]

At 163840 bytes/IB the Haskell simulations indicate faster diffusion of blocks at moderate rates, in contrast to slower or higher rates. (This is a strong enough effect that it is statistically significant.) What explains this? Is there some aspect of TCP that causes this?

[Saizan]

When traffic is low enough the congestion window has time to shrink back down between messages and will have to grow again: e.g. we expect a 96kB message to take 3 fragments from cold. Then it's reasonable there's a sweet spot for diffusion where there's enough traffic that most bodies are sent in just one fragment, while the processing delays are still small enough that they don't cancel the benefit.

@bwbush If the effect disappear with tcp-congestion-control: false then that'd be a pretty direct confirmation.

It's quite interesting that at 10 IB/s there's no long tail. Can you share the topology used for these runs?

[bwbush]

It's quite interesting that at 10 IB/s there's no long tail. Can you share the topology used for these runs?

I'm using the 100-node default topology: https://github.com/input-output-hk/ouroboros-leios/blob/main/analysis/sims/2025w10/topo-default-100.yaml

[bwbush]

If the effect disappear with tcp-congestion-control: false then that'd be a pretty direct confirmation.

I'll try this and have added the retest tag to this ticket to remind me.

[will-break-it]

Would it be worth testing ib-body-avg-size-bytes: 196608 (2× the current size)?
It would perhaps tell us more about whether 163840 bytes/IB is truly optimal or just one point on a performance curve.
I'd like to know at what threshold processing delays outweigh transmission efficiency benefits?

cc @Saizan @bwbush

[bwbush]

I compared the cases with optimizations on vs off and, arguably, it looks like the optimizations cause this effect. (I'd have to do another set of runs with finer resolution to be 100% sure.) Also, we have a coherent explanation of this as a TCP effect. I propose we close this issue, but I'm fine doing the additional runs if we want.