reland: audio & video player: migrate to common playbin implementation #483
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When uploading memory with size X into a dmabuf, create a buffer that is square (e.g. sqrt(x) by sqrt(x)) instead of one that is X pixels wide and one pixel high. Drivers have limitations on buffer dimensions, and e.g. the intel (iris) driver ran into this limitation.
Use playbin3 for playback instead of uridecodebin. This also enables audio support. Furthermore, separate out the appsink (which uploads buffers into a flutter texture) into a separate "pseudo" gstreamer-element, which can be created using `flutter_gl_texture_sink_new`. Synchronization is reworked as well, it is now assumed the player is only accessed from a single thread, the platform thread. The callbacks called by gstreamer (which are called on internal threads) will rethread, if necessary, by posting messages to the GstBus. The buffering query is also now properly fixed. Previously it queried the playbin instead of the element that does the actual queueing (a `multiqueue`). Also, a one-shot debug logging message is added to the frame uploader, to inform the user when slower, manual dmabuf uploads are used instead of zero-copy.
Only supported since gstreamer 1.24.0 and requires some more work to implement correctly.
Also don't use conditional compilation in places that don't absolutely depend on it.
There's an issue in the gstreamer video4linux2 plugin where seeking will lead to buffer management issues, causing streaming to stop, or even memory leaks. See: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4465
- add `raw_std_method_call_from_buffer` to create a "raw_std_value" from a byte buffer & size, and check whether its encoding is valid - add `platch_respond_malformed_message_std` to send a pre-determined response if the encoding was invalid
Used by Zed to store clangd cache.
- add audio balance setter/getter - support releasing & prerolling a playback source - add separate seeking info, duration and end of stream notifiers - make gstreamer_video_player.h not depend on gstreamer headers (if possible)
Finishing unifying playback element used by audioplayers and video player plugins.
Did cause a message to be logged on every atomic commit.
On Raspberry Pi, the fix is now backported to 1.22: RPi-Distro/repo#391 This reverts commit dc0ff2a.
Was added in gstreamer 1.20, debian bullseye is still on 1.18.
Uses gstreamer 1.20.0-only API and is only used for 1.20.0 anyway.
Change notifiers should only trigger when notifier_notify is called, not on listen.
gstplayer: - allow registering callbacks for async seeks/state changes - implement gapless looping using playbin about-to-finish or segments - listen to GST_MESSAGE_EOS instead of appsink eos signal, since appsink is only used as a video sink for video playback. - query duration in GST_MESSAGE_DURATION_CHANGED message audioplayers: - actually send prepared, completed, seek completed and error events - handle incorrect file source uris by audioplayers - fix incorrect non-null terminated string printing gstreamer_video_player: - enable gapless looping by default - allow overriding it in set_looping call
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Expand the
gstplayer(previously used to implement video playback for the video player):playbin/playbin3, to enable audio supportgstplayer_get_duration)gstplayer_set_audio_balance,gstplayer_get_audio_balance)Migrate the audio player to
gstplayer.