fix(deepgram): include word confidence for stt v1 alternatives - #7181
Open
Snowfly0709 wants to merge 1 commit into
Open
fix(deepgram): include word confidence for stt v1 alternatives#7181Snowfly0709 wants to merge 1 commit into
Snowfly0709 wants to merge 1 commit into
Conversation
stt_v2.py::_parse_transcription forwards Deepgram's per-word confidence into TimedString (livekit#5034). stt.py builds the same TimedString at two sites without it, so every word from the v1 STT reports confidence as NOT_GIVEN even though the response carries a value for it. Both sites now forward it, defaulting to NOT_GIVEN so an absent value stays distinguishable from a genuine zero.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #7180 .
What
Forward Deepgram's per-word
confidenceintoTimedStringat the twoplaces the plugin builds them.
Why
This is #5034's fix applied to the other module.
stt_v2.pyforwardsDeepgram's per-word confidence into
TimedString;stt.pybuilds the sameTimedStringat two sites without it, so words from the v1STTalwaysreport
NOT_GIVEN. As #5034 put it for v2, the confidence "wasn't set onthe word level, even though it is used to set the overall combined
confidence" — v1 likewise reads
alt["confidence"]for the utterance anddrops the per-word values.
The field is declared:
and Deepgram returns
confidenceon every entry ofwords[].livekit-plugins-assemblyaiandlivekit-plugins-googlepopulate it too,so v1 is now the outlier.
The raw JSON is discarded inside the plugin, so downstream code cannot
recover the value:
Agent.stt_nodereceives already-parsedTimedStrings,and
UserInputTranscribedEventcarries onlytranscript: str.Per-word confidence lets an agent tell which part of a spoken value it is
unsure about, rather than only that the utterance might be wrong — which
is what matters for structured values like email addresses and reference
numbers.
Default value
word.get("confidence", NOT_GIVEN)rather than0, matching the field'sdeclared
NotGivenOr[float]contract: a consumer ranking words byconfidence needs "unknown" to stay distinguishable from "zero confidence",
since those call for opposite handling.
Changes
Two lines, no new imports (
NOT_GIVENis already imported at the top ofthe file):
live_transcription_to_speech_dataprerecorded_transcription_to_speech_eventVerification
ruff checkandruff formatclean under the repo's own config. Bothfunctions are pure
dict -> SpeechData, so this is checkable without a network call:I did not add a test file —
tests/has no unit tests for this plugin'sparsing helpers, so I did not want to establish a pattern uninvited. Happy
to add one wherever you'd like it.
Behaviour change
Additive, and the same shape as #5034. Words gain a value where they
previously had
NOT_GIVEN; nothing that already worked changes. Consumersgating on
is_given()keep working either way.