Description
Describe the bug
The quality score reported to the user through server events is too low in some cases.
To Reproduce
Steps to reproduce the behavior:
- Configure video capture to be 960x720 (4:3 720 preset)
- Connect to a room with certain Android devices that does not support natively capturing that resolution (e.g. Fairphone 5)
- Observe connection quality score go down to
Goodafter a few seconds
This even happens on a local dev environment with low ping times/RTT.
Expected behavior
Excellent score due to local network being fast enough
Platform information
- Flutter version: 3.22.2
- Plugin version: 2.1.4
- Flutter target OS: Android
- Flutter target OS version: 13
Additional context
This is tightly related to webrtc-sdk/webrtc#133.
However, even with the partially fixing PR applied, the quality score is still too low.
The reason for this issue is that WebRTC Simulcast layer counts (https://github.com/webrtc-sdk/webrtc/blob/7ddfc43fcd2e4ba0548258a76fe88d49b34588c5/video/config/simulcast.cc#L82-L106) do not always match Livekit Flutter SDK layer counts https://github.com/livekit/client-sdk-flutter/blob/main/lib/src/utils.dart#L473-L477. Livekit server then assumes that bad network is the cause for the missing layers
Actual reasons for the mismatch:
- this SDK only considers max dimension, WebRTC considers pixel count (16:9 vs 4:3 comes into play too), and
- WebRTC potentially resizes video streams before calculating simulcast layers (see issue mentioned above)