[Enhancement]Implement static frame pipeline for PiP #724

ipavlidakis · 2025-03-21T15:05:17Z

🔗 Issue Links

Resolves https://linear.app/stream/issue/IOS-802/picture-in-picture-improvements
Resolves https://linear.app/stream/issue/IOS-701/[blink]picture-in-picture-black-window-when-no-one-has-video
Resolves https://linear.app/stream/issue/IOS-502/[video][reconnection]-pip-should-not-disappear-on-rejoin

🎯 Goal

Improve Picture-in-Picture by:

Simplifying desing
Enriched UI
Improved state management

🧪 Manual Testing Notes

Join a call with a few participants
Enter PiP
Toggle media between participants and observe PiP updating
Start/stop screensharing

☑️ Contributor Checklist

I have signed the Stream CLA (required)
This change follows zero ⚠️ policy (required)
This change should receive manual QA
Changelog is updated with client-facing changes
New code is covered by unit tests
Comparison screenshots added for visual changes
Affected documentation updated (tutorial, CMS)

github-actions · 2025-03-21T15:06:53Z

Public Interface

+ public struct PictureInPictureParticipantImageView: View  
+ 
+   public var body: some View
+   
+ 
+   public init(imageURL: URL? = nil,@ViewBuilder contentProvider: () -> some View)



 public final class SerialActorQueue: Sendable  
-   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func cancelAll()
-   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws
+   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws

github-actions · 2025-03-21T15:06:56Z

	3 Errors
🚫	Please start subject with capital letter. `b7250dd`
🚫	Please start subject with capital letter. `0054ff8`
🚫	Please start subject with capital letter. `28a76c3`

	1 Warning
⚠️	Big PR

Generated by 🚫 Danger

Stream-SDK-Bot · 2025-03-21T15:11:47Z

SDK Size

`title`	`develop`	`branch`	`diff`	`status`
StreamVideo	7.62 MB	7.62 MB	0 KB	🟢
StreamVideoSwiftUI	2.13 MB	2.25 MB	+115 KB	🟢
StreamVideoUIKit	2.27 MB	2.38 MB	+114 KB	🟢
StreamWebRTC	9.85 MB	9.85 MB	0 KB	🟢

github-actions · 2025-03-26T17:10:31Z

Public Interface

+ public struct PictureInPictureParticipantImageView: View  
+ 
+   public var body: some View
+   
+ 
+   public init(imageURL: URL? = nil,@ViewBuilder contentProvider: () -> some View)



 public final class SerialActorQueue: Sendable  
-   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func cancelAll()
-   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws
+   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws

github-actions · 2025-03-27T10:34:55Z

Public Interface

+ public struct PictureInPictureParticipantImageView: View  
+ 
+   public var body: some View
+   
+ 
+   public init(imageURL: URL? = nil,@ViewBuilder contentProvider: () -> some View)



 public final class SerialActorQueue: Sendable  
-   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func cancelAll()
-   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws
+   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws

github-actions · 2025-03-27T16:58:33Z

Public Interface

+ public struct PictureInPictureParticipantImageView: View  
+ 
+   public var body: some View
+   
+ 
+   public init(imageURL: URL? = nil,@ViewBuilder contentProvider: () -> some View)



 public final class SerialActorQueue: Sendable  
-   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func cancelAll()
-   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws
+   public func async(file: StaticString = #file,functionName: StaticString = #function,line: UInt = #line,_ block: @Sendable @escaping () async throws -> Void)
+   public func sync(_ block: @Sendable @escaping () async throws -> Void)async throws

github-actions · 2025-04-02T08:29:40Z

Public Interface

+ public struct PictureInPictureParticipantImageView: View  
+ 
+   public var body: some View
+   
+ 
+   public init(imageURL: URL? = nil,@ViewBuilder contentProvider: () -> some View)

github-actions · 2025-04-08T19:59:16Z

Public Interface

🚀 No changes affecting the public interface.

github-actions · 2025-04-09T09:31:33Z

Public Interface

🚀 No changes affecting the public interface.

github-actions · 2025-04-09T12:13:26Z

Public Interface

🚀 No changes affecting the public interface.

github-actions · 2025-04-09T14:48:06Z

Public Interface

🚀 No changes affecting the public interface.

github-actions · 2025-04-16T09:12:35Z

Public Interface

+ public final class AnyViewFactory: ViewFactory  
+ 
+   public init(_ factory: T)
+   
+ 
+   public func makeCallControlsView(viewModel: CallViewModel)-> some View
+   public func makeOutgoingCallView(viewModel: CallViewModel)-> some View
+   public func makeJoiningCallView(viewModel: CallViewModel)-> some View
+   public func makeIncomingCallView(viewModel: CallViewModel,callInfo: IncomingCall)-> some View
+   public func makeWaitingLocalUserView(viewModel: CallViewModel)-> some View
+   public func makeVideoParticipantsView(viewModel: CallViewModel,availableFrame: CGRect,onChangeTrackVisibility: @escaping @MainActor(CallParticipant, Bool) -> Void)-> some View
+   public func makeVideoParticipantView(participant: CallParticipant,id: String,availableFrame: CGRect,contentMode: UIView.ContentMode,customData: [String: RawJSON],call: Call?)-> some View
+   public func makeVideoCallParticipantModifier(participant: CallParticipant,call: Call?,availableFrame: CGRect,ratio: CGFloat,showAllInfo: Bool)-> any ViewModifier
+   public func makeCallView(viewModel: CallViewModel)-> some View
+   public func makeMinimizedCallView(viewModel: CallViewModel)-> some View
+   public func makeCallTopView(viewModel: CallViewModel)-> some View
+   public func makeParticipantsListView(viewModel: CallViewModel)-> some View
+   public func makeScreenSharingView(viewModel: CallViewModel,screensharingSession: ScreenSharingSession,availableFrame: CGRect)-> some View
+   public func makeLobbyView(viewModel: CallViewModel,lobbyInfo: LobbyInfo,callSettings: Binding<CallSettings>)-> some View
+   public func makeReconnectionView(viewModel: CallViewModel)-> some View
+   public func makeLocalParticipantViewModifier(localParticipant: CallParticipant,callSettings: Binding<CallSettings>,call: Call?)-> any ViewModifier
+   public func makeUserAvatar(_ user: User,with options: UserAvatarViewOptions)-> some View

github-actions · 2025-04-16T09:25:14Z

Public Interface

+ public final class AnyViewFactory: ViewFactory  
+ 
+   public init(_ factory: T)
+   
+ 
+   public func makeCallControlsView(viewModel: CallViewModel)-> some View
+   public func makeOutgoingCallView(viewModel: CallViewModel)-> some View
+   public func makeJoiningCallView(viewModel: CallViewModel)-> some View
+   public func makeIncomingCallView(viewModel: CallViewModel,callInfo: IncomingCall)-> some View
+   public func makeWaitingLocalUserView(viewModel: CallViewModel)-> some View
+   public func makeVideoParticipantsView(viewModel: CallViewModel,availableFrame: CGRect,onChangeTrackVisibility: @escaping @MainActor(CallParticipant, Bool) -> Void)-> some View
+   public func makeVideoParticipantView(participant: CallParticipant,id: String,availableFrame: CGRect,contentMode: UIView.ContentMode,customData: [String: RawJSON],call: Call?)-> some View
+   public func makeVideoCallParticipantModifier(participant: CallParticipant,call: Call?,availableFrame: CGRect,ratio: CGFloat,showAllInfo: Bool)-> any ViewModifier
+   public func makeCallView(viewModel: CallViewModel)-> some View
+   public func makeMinimizedCallView(viewModel: CallViewModel)-> some View
+   public func makeCallTopView(viewModel: CallViewModel)-> some View
+   public func makeParticipantsListView(viewModel: CallViewModel)-> some View
+   public func makeScreenSharingView(viewModel: CallViewModel,screensharingSession: ScreenSharingSession,availableFrame: CGRect)-> some View
+   public func makeLobbyView(viewModel: CallViewModel,lobbyInfo: LobbyInfo,callSettings: Binding<CallSettings>)-> some View
+   public func makeReconnectionView(viewModel: CallViewModel)-> some View
+   public func makeLocalParticipantViewModifier(localParticipant: CallParticipant,callSettings: Binding<CallSettings>,call: Call?)-> any ViewModifier
+   public func makeUserAvatar(_ user: User,with options: UserAvatarViewOptions)-> some View

+ public protocol InternetConnectionProtocol

+ public enum Status: Equatable  
+ 
+   case unknown
+   case available(Quality)
+   case unavailable

+ public enum Quality: Equatable  
+ 
+   case great
+   case expensive
+   case constrained

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

+ extension InternetConnection.Status  
+ 
+   public var isAvailable: Bool

+ public final class InternetConnection: @unchecked Sendable

github-actions · 2025-04-16T10:56:19Z

Public Interface

+ public final class InternetConnection: @unchecked Sendable

+ public enum Quality: Equatable  
+ 
+   case great
+   case expensive
+   case constrained

+ public final class AnyViewFactory: ViewFactory  
+ 
+   public init(_ factory: T)
+   
+ 
+   public func makeCallControlsView(viewModel: CallViewModel)-> some View
+   public func makeOutgoingCallView(viewModel: CallViewModel)-> some View
+   public func makeJoiningCallView(viewModel: CallViewModel)-> some View
+   public func makeIncomingCallView(viewModel: CallViewModel,callInfo: IncomingCall)-> some View
+   public func makeWaitingLocalUserView(viewModel: CallViewModel)-> some View
+   public func makeVideoParticipantsView(viewModel: CallViewModel,availableFrame: CGRect,onChangeTrackVisibility: @escaping @MainActor(CallParticipant, Bool) -> Void)-> some View
+   public func makeVideoParticipantView(participant: CallParticipant,id: String,availableFrame: CGRect,contentMode: UIView.ContentMode,customData: [String: RawJSON],call: Call?)-> some View
+   public func makeVideoCallParticipantModifier(participant: CallParticipant,call: Call?,availableFrame: CGRect,ratio: CGFloat,showAllInfo: Bool)-> any ViewModifier
+   public func makeCallView(viewModel: CallViewModel)-> some View
+   public func makeMinimizedCallView(viewModel: CallViewModel)-> some View
+   public func makeCallTopView(viewModel: CallViewModel)-> some View
+   public func makeParticipantsListView(viewModel: CallViewModel)-> some View
+   public func makeScreenSharingView(viewModel: CallViewModel,screensharingSession: ScreenSharingSession,availableFrame: CGRect)-> some View
+   public func makeLobbyView(viewModel: CallViewModel,lobbyInfo: LobbyInfo,callSettings: Binding<CallSettings>)-> some View
+   public func makeReconnectionView(viewModel: CallViewModel)-> some View
+   public func makeLocalParticipantViewModifier(localParticipant: CallParticipant,callSettings: Binding<CallSettings>,call: Call?)-> any ViewModifier
+   public func makeUserAvatar(_ user: User,with options: UserAvatarViewOptions)-> some View

+ public protocol InternetConnectionProtocol

+ public enum Status: Equatable  
+ 
+   case unknown
+   case available(Quality)
+   case unavailable

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

+ extension InternetConnection.Status  
+ 
+   public var isAvailable: Bool

github-actions · 2025-04-16T14:58:20Z

Public Interface

+ public enum Status: Equatable  
+ 
+   case unknown
+   case available(Quality)
+   case unavailable

+ public protocol InternetConnectionProtocol

+ public enum Quality: Equatable  
+ 
+   case great
+   case expensive
+   case constrained

+ public final class AnyViewFactory: ViewFactory  
+ 
+   public init(_ factory: T)
+   
+ 
+   public func makeCallControlsView(viewModel: CallViewModel)-> some View
+   public func makeOutgoingCallView(viewModel: CallViewModel)-> some View
+   public func makeJoiningCallView(viewModel: CallViewModel)-> some View
+   public func makeIncomingCallView(viewModel: CallViewModel,callInfo: IncomingCall)-> some View
+   public func makeWaitingLocalUserView(viewModel: CallViewModel)-> some View
+   public func makeVideoParticipantsView(viewModel: CallViewModel,availableFrame: CGRect,onChangeTrackVisibility: @escaping @MainActor(CallParticipant, Bool) -> Void)-> some View
+   public func makeVideoParticipantView(participant: CallParticipant,id: String,availableFrame: CGRect,contentMode: UIView.ContentMode,customData: [String: RawJSON],call: Call?)-> some View
+   public func makeVideoCallParticipantModifier(participant: CallParticipant,call: Call?,availableFrame: CGRect,ratio: CGFloat,showAllInfo: Bool)-> any ViewModifier
+   public func makeCallView(viewModel: CallViewModel)-> some View
+   public func makeMinimizedCallView(viewModel: CallViewModel)-> some View
+   public func makeCallTopView(viewModel: CallViewModel)-> some View
+   public func makeParticipantsListView(viewModel: CallViewModel)-> some View
+   public func makeScreenSharingView(viewModel: CallViewModel,screensharingSession: ScreenSharingSession,availableFrame: CGRect)-> some View
+   public func makeLobbyView(viewModel: CallViewModel,lobbyInfo: LobbyInfo,callSettings: Binding<CallSettings>)-> some View
+   public func makeReconnectionView(viewModel: CallViewModel)-> some View
+   public func makeLocalParticipantViewModifier(localParticipant: CallParticipant,callSettings: Binding<CallSettings>,call: Call?)-> any ViewModifier
+   public func makeUserAvatar(_ user: User,with options: UserAvatarViewOptions)-> some View

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

+ public final class InternetConnection: @unchecked Sendable

+ extension InternetConnection.Status  
+ 
+   public var isAvailable: Bool

ipavlidakis · 2025-04-16T15:08:18Z

Sources/StreamVideo/HTTPClient/InternetConnection.swift

@@ -26,7 +26,7 @@ extension Notification {
 ///
 /// Basically, it's a wrapper over legacy monitor based on `Reachability` (iOS 11 only)
 /// and default monitor based on `Network`.`NWPathMonitor` (iOS 12+).
-class InternetConnection: @unchecked Sendable {
+public final class InternetConnection: @unchecked Sendable {


Making Internet connection public so it can be use from SwiftUI SDK. Let me know your thoughts

github-actions · 2025-04-22T09:14:15Z

Public Interface

+ public enum InternetConnectionQuality: Equatable, Sendable  
+ 
+   case great
+   case expensive
+   case constrained

+ extension InternetConnectionStatus  
+ 
+   public var isAvailable: Bool

+ public enum InternetConnectionStatus: Equatable, Sendable  
+ 
+   case unknown
+   case available(InternetConnectionQuality)
+   case unavailable

+ public protocol InternetConnectionProtocol

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

github-actions · 2025-04-22T09:48:08Z

Public Interface

+ extension InternetConnectionStatus  
+ 
+   public var isAvailable: Bool

+ public protocol InternetConnectionProtocol

+ public enum InternetConnectionQuality: Equatable, Sendable  
+ 
+   case great
+   case expensive
+   case constrained

+ public enum InternetConnectionStatus: Equatable, Sendable  
+ 
+   case unknown
+   case available(InternetConnectionQuality)
+   case unavailable

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

github-actions · 2025-04-22T12:08:54Z

Public Interface

+ public protocol InternetConnectionProtocol

+ public enum InternetConnectionStatus: Equatable, Sendable  
+ 
+   case unknown
+   case available(InternetConnectionQuality)
+   case unavailable

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

+ extension InternetConnectionStatus  
+ 
+   public var isAvailable: Bool

+ public enum InternetConnectionQuality: Equatable, Sendable  
+ 
+   case great
+   case expensive
+   case constrained

sonarqubecloud · 2025-04-22T13:00:10Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2025-04-22T13:15:58Z

Public Interface

+ public enum InternetConnectionStatus: Equatable, Sendable  
+ 
+   case unknown
+   case available(InternetConnectionQuality)
+   case unavailable

+ extension InternetConnectionStatus  
+ 
+   public var isAvailable: Bool

+ extension InternetConnection: InjectionKey  
+ 
+   nonisolated public static var currentValue: InternetConnectionProtocol

+ public protocol InternetConnectionProtocol

+ public enum InternetConnectionQuality: Equatable, Sendable  
+ 
+   case great
+   case expensive
+   case constrained

ipavlidakis added the enhancement New feature or request label Mar 21, 2025

ipavlidakis self-assigned this Mar 21, 2025

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from 02ce9a8 to fae5854 Compare March 26, 2025 17:00

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch 2 times, most recently from 1a1dc2d to febb279 Compare March 27, 2025 10:22

ipavlidakis force-pushed the develop branch from 0aa4cb2 to 3e815f5 Compare March 27, 2025 13:45

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from febb279 to 3a3058f Compare March 27, 2025 16:10

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from 8b9dd39 to 8fb61a0 Compare April 2, 2025 08:20

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from 8fb61a0 to c6881fd Compare April 8, 2025 19:57

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from a1a26ec to e6f762c Compare April 9, 2025 12:04

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from f12c77a to ae5dcdd Compare April 16, 2025 09:10

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from ae5dcdd to f78066b Compare April 16, 2025 09:23

ipavlidakis marked this pull request as ready for review April 16, 2025 14:57

ipavlidakis requested a review from a team as a code owner April 16, 2025 14:57

ipavlidakis commented Apr 16, 2025

View reviewed changes

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from e60fdda to 433f2b0 Compare April 22, 2025 09:39

martinmitrevski approved these changes Apr 22, 2025

View reviewed changes

ipavlidakis added 18 commits April 22, 2025 15:04

[Enhancement]Picture-in-Picture improvements

dcab57f

Fix black track when coming back from PiP

2692411

Added comments

781d6cd

Add tests

ac464ab

Update Changelog

896ed74

Resolve concurrency warning

723d809

Fix compilation errors

852b2fb

Fix compilation errors

be94cb1

fix failing tests

28a76c3

Fix xcode 15 issues

c8dadf5

Fix documentation tests

a762e6d

Update and reenabled tests for TrackStateAdapter

028bced

Remove AnyViewFactory

4c285cf

Fix issue where if call was set before the adapter has been created

662c72b

Fix tests

f8e8729

Fix flakey tests

3a06ad3

Fix warnings

95eed3d

update changelog

0054ff8

ipavlidakis force-pushed the enhancement/picture-in-picture-static-frame branch from a95c43a to 0054ff8 Compare April 22, 2025 12:06

fix xcode 15 compilation issues

b7250dd

ipavlidakis merged commit bb694ae into develop Apr 22, 2025
9 of 11 checks passed

ipavlidakis deleted the enhancement/picture-in-picture-static-frame branch April 22, 2025 14:32

Stream-SDK-Bot mentioned this pull request Apr 22, 2025

1.21.0 Release #767

Merged

[Enhancement]Implement static frame pipeline for PiP #724

[Enhancement]Implement static frame pipeline for PiP #724

Uh oh!

Conversation

ipavlidakis commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Issue Links

🎯 Goal

🧪 Manual Testing Notes

☑️ Contributor Checklist

Uh oh!

github-actions bot commented Mar 21, 2025

Public Interface

Uh oh!

github-actions bot commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Stream-SDK-Bot commented Mar 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

SDK Size

Uh oh!

github-actions bot commented Mar 26, 2025

Public Interface

Uh oh!

github-actions bot commented Mar 27, 2025

Public Interface

Uh oh!

github-actions bot commented Mar 27, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 2, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 8, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 9, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 9, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 9, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 16, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 16, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 16, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 16, 2025

Public Interface

Uh oh!

ipavlidakis Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Apr 22, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 22, 2025

Public Interface

Uh oh!

github-actions bot commented Apr 22, 2025

Public Interface

Uh oh!

sonarqubecloud bot commented Apr 22, 2025

Quality Gate passed

Uh oh!

github-actions bot commented Apr 22, 2025

Public Interface

Uh oh!

Uh oh!

Uh oh!

ipavlidakis commented Mar 21, 2025 •

edited

Loading

github-actions bot commented Mar 21, 2025 •

edited

Loading

Stream-SDK-Bot commented Mar 21, 2025 •

edited

Loading