Picture in Picture feature

[haroldh17]

Describe the bug
I'm struggling to enable Picture-in-Picture mode for videos when the user navigates away from my application. Is this functionality feasible, and if so, how might it be implemented?

To Reproduce
Steps to reproduce the behavior:

1. Start a session
2. navigate outside of the app

Expected behavior
A minimized Video of the callee should show in a corner, if the user navigates away from the app to another. And then it should be clickable to open back up to the app. This feature is on iOS FaceTime and WhatsApp: Multitask with Picture in Picture on iPhone

Please help.

[dineshiOSDev]

PIP is not supported yet. I was facing the same and Chime not supporting as of now is what their support team replied.
Instead try adding floating view so that we can use pip like effect inside the application.

[hvsw]

You can implement PiP with AVPictureInPictureController, but so far I found a bunch of rough edges.

---

Using AVPictureInPictureVideoCallViewController as described in Adopting Picture in Picture for video calls
works, however

The PiP window doesn’t receive touch events when you use AVPictureInPictureVideoCallViewController, so you can’t customize the window’s user interface by adding buttons.

so you have no customization options like having a close button to leave the call.

---

You can instead use AVPictureInPictureController.ContentSource(sampleBufferDisplayLayer:playbackDelegate:). See UIPiPView for some tips on how to make a sample buffer. CPU usage is 85%(out of 600%) fairly high on iPhone 15 Pro using this sampling method - I think creating a view class conforming to VideoRenderView and calling AmazonChimeSDK.AudioVideoFacade.bindVideoView(videoView:tileId) directly to it instead of sampling should improve performance but I was not able to test this (for some reason PiP is not starting).

Consider a call with multiple video tiles and you want to show just the active speaker in your PiP window. Considering you're using a collection view to show a grid of videos, sampling a single cell is complex to manage because of cell reuse. The easy way here is to render the entire collection view configured as you want, but as I mentioned the CPU usage is too high.

---

What do other competitors do?

Other video service providers offer direct integration and helper classes so I'll share them here for reference:

Twilio provides a VideoTrackStoringSampleBufferVideoView to simplify PiP implementation:
https://www.twilio.com/docs/video/changelog-twilio-video-ios-latest#580-february-28-2024

100ms provides guidance and an example project with a sample buffer view:
https://www.100ms.live/docs/ios/v2/how-to-guides/set-up-video-conferencing/render-video/pip-mode
https://github.com/100mslive/100ms-ios-sdk/tree/main/Example

How to display participant's video in PiP

AVPictureInPictureController requires source content to use AVSampleBufferDisplayLayer on it's subview. You need to use HMSSampleBufferDisplayView instead. HMSSampleBufferDisplayView is an UIImageView that uses AVSampleBufferDisplayLayer for drawing.

Zoom provides seamless integration:
https://developers.zoom.us/blog/meeting-sdk-ios-picture-in-picture/