Revisit question of whether frames need to be ordered on incoming

[alvestrand]

The example use case is when you want to do a jitter buffer in Javascript; you want to have maximum time available for out-of-order frames to arrive, but you want to have the option to decide to display a frame while skipping the late one.

The opposite argument is that out of order is hard to handle, and most devs won't do a good job of handling it.

Possibly we need configuration variables for controlling this?

[englishm-mux]

My sense is that providing unordered frames would provide developers with the most flexibility to develop applications with the lowest possible latency. If I understand correctly, application code will already need to tolerate lost frames, so tolerating out of order frames doesn't seem like it would impose too much more burden.

Offering a configurable buffer as an alternative interface could make things simpler for developers who don't wish to have as much control or don't require such low latency, but that could also be done in Javascript.

Besides the question of the usability and affordances of the interface, are there relevant tradeoffs with regard to overall memory footprint whether a jitter buffer is owned by the browser or by Javascript code?

The one reason I could see this as a "footgun" is that for some (many?) use cases, out of order frames may not occur often enough to be noticeable and therefore developers may mistakenly assume the interface guarantees ordered frames. This could be mitigated by being explicit in the specification and API documentation.