UI Freezes on rtcVideoTrack.add(VideoRendererAdapter(target: videoRenderer)), when entering from background

[patryk-sredzinski]

We were observing issues with videoViews stopping rendering when entered background for at least 15 seconds and returning. Camera was on and delivering to server, although the preview was not working.
All VideoViews stops rendering, both local and remote, although the stream is received/published

So we started observing AppState like that:

import LiveKit
import UIKit

extension CallController {
    func startObservingAppState() {
        let center = NotificationCenter.default
        center.addObserver(self, selector: #selector(didEnterBackgroundNotification), name: UIApplication.didEnterBackgroundNotification, object: nil)
        center.addObserver(self, selector: #selector(willEnterForegroundNotification), name: UIApplication.willEnterForegroundNotification, object: nil)
    }
    
    func stopObservingAppState() {
        let center = NotificationCenter.default
        center.removeObserver(self, name: UIApplication.didEnterBackgroundNotification, object: nil)
        center.removeObserver(self, name: UIApplication.willEnterForegroundNotification, object: nil)
    }
    
    @objc private func didEnterBackgroundNotification() {
        updateVideoViews(areEnabled: false)
    }
    
    @objc private func willEnterForegroundNotification() {
        updateVideoViews(areEnabled: true)
    }
    
    private func updateVideoViews(areEnabled: Bool) {
        var videoViews = [PersonVideoView]()
        videoViews.append(localPipManager.localParticipantView.videoView)
        videoViews.append(localPipManager.currentParticipantView.videoView)
        videoViews.append(contentsOf: collectionManager.participantViews.values.compactMap { $0.videoView })
        videoViews.forEach { $0.videoView.isEnabled = areEnabled }
    }
}

This sometimes ends up in UI Freeze which leads to a crash. When we pause the debugger it points to:

rtcVideoTrack.add(VideoRendererAdapter(target: videoRenderer))

Should we observe app state?

🔷 EDIT:
We are rendering in SampleBuffer mode, not Metal one
We picked up SampleBuffer, because the camera looks very pixelated on small views.
If we switch to Metal, background does not freeze, but the previews are having really bad quality

[pblazej]

Hi @patryk-sredzinski! Unfortunately, I was unable to reproduce this one.

Should we observe app state?

Yes, technically it should be possible to pause rendering - looks like the main thread is being blocked for too long.

• Can you provide a minimal reproducible example?
• Can you check didBecomeActiveNotification as well (bad timing)?
• Is delaying the update (via dispatch main) changing anything here?