feat: add Ojin avatar plugin - #7162
Open
php-workx wants to merge 3 commits into
Open
Conversation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Note
This report is out of date. Scroll down for Devin Review's latest report on this PR.
Devin Review found 8 potential issues.
2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
php-workx
force-pushed
the
feat/ojin-stv-avatar
branch
from
September 8, 2026 04:46
3e654ed to
167ac0f
Compare
Contributor
There was a problem hiding this comment.
Note
This report is out of date. Scroll down for Devin Review's latest report on this PR.
Devin Review found 4 new potential issues.
3 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
Adds livekit-plugins-ojin, driving Ojin's Speech-To-Video models from an AgentSession. Ojin renders in-process through its SDK rather than joining the room as its own participant, so the plugin runs on the local-runner path (AvatarSession -> AvatarRunner -> VideoGenerator) alongside bithuman, and owns the media path itself rather than being a thin API client. The subtle part is segment accounting. Ojin's playback clock emits an audio frame on every tick - zero-filled while idle - whereas AvatarRunner reports playback-finished only when the generator yields an AudioSegmentEnd, and AgentSession blocks on that after every utterance. A missing marker wedges the conversation permanently, so the decision about when an utterance is actually over lives in its own module with its own tests. Ships 92 hermetic tests driven by a fake STV client - no network, no credentials - covering the frame conversions, the frame sink, the turn state machine, barge-in, the frame watchdog and the degradation path, plus an end-to-end suite over the real QueueAudioOutput. Registered as the `ojin` extra alongside the other avatar plugins.
php-workx
force-pushed
the
feat/ojin-stv-avatar
branch
from
September 8, 2026 05:02
167ac0f to
3099740
Compare
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
owes_segment_end only asked the tracker, which lets go of a segment the moment the marker is queued. Turning that marker into a playback report is the runner's job, so a fatal landing between the two discarded it and left the session waiting in wait_for_playout(). Treat a marker still sitting in the queue as owed.
_on_fatal schedules the degrade only once a runner exists to tear down. A fatal between the first-frame check and that assignment therefore scheduled nothing, while its first-wins guard stopped any later event or the watchdog from scheduling one - and startup went on to install an avatar that could never report playback. Re-check after the runner starts and fail the start, which rolls back through the existing path.
php-workx
added a commit
to ojinai/livekit-demo
that referenced
this pull request
Sep 9, 2026
The branch pin resolved to whatever feat/ojin-stv-avatar pointed at, and upstream review keeps rebasing that branch - which orphaned the pinned commit twice, leaving `uv sync` unable to resolve the plugin at all. The tag cannot move. Bugfixes arrive as a new preview tag, and PyPI takes over once livekit/agents#7162 merges.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ojin · Docs
This PR adds the Ojin avatar plugin, driving Ojin's Speech-To-Video models from an
AgentSession.