Skip to content

fix(deepgram): include word confidence for stt v1 alternatives - #7181

Open
Snowfly0709 wants to merge 1 commit into
livekit:mainfrom
Snowfly0709:fix/deepgram-stt-v1-word-confidence
Open

fix(deepgram): include word confidence for stt v1 alternatives#7181
Snowfly0709 wants to merge 1 commit into
livekit:mainfrom
Snowfly0709:fix/deepgram-stt-v1-word-confidence

Conversation

@Snowfly0709

@Snowfly0709 Snowfly0709 commented Sep 9, 2026

Copy link
Copy Markdown

Closes #7180 .

What

Forward Deepgram's per-word confidence into TimedString at the two
places the plugin builds them.

Why

This is #5034's fix applied to the other module. stt_v2.py forwards
Deepgram's per-word confidence into TimedString; stt.py builds the same
TimedString at two sites without it, so words from the v1 STT always
report NOT_GIVEN. As #5034 put it for v2, the confidence "wasn't set on
the word level, even though it is used to set the overall combined
confidence" — v1 likewise reads alt["confidence"] for the utterance and
drops the per-word values.

The field is declared:

class TimedString(str):
    confidence: NotGivenOr[float]   # NOT_GIVEN when unavailable

and Deepgram returns confidence on every entry of words[].
livekit-plugins-assemblyai and livekit-plugins-google populate it too,
so v1 is now the outlier.

The raw JSON is discarded inside the plugin, so downstream code cannot
recover the value: Agent.stt_node receives already-parsed TimedStrings,
and UserInputTranscribedEvent carries only transcript: str.

Per-word confidence lets an agent tell which part of a spoken value it is
unsure about, rather than only that the utterance might be wrong — which
is what matters for structured values like email addresses and reference
numbers.

Default value

word.get("confidence", NOT_GIVEN) rather than 0, matching the field's
declared NotGivenOr[float] contract: a consumer ranking words by
confidence needs "unknown" to stay distinguishable from "zero confidence",
since those call for opposite handling.

Changes

Two lines, no new imports (NOT_GIVEN is already imported at the top of
the file):

  • live_transcription_to_speech_data
  • prerecorded_transcription_to_speech_event

Verification

ruff check and ruff format clean under the repo's own config. Both
functions are pure
dict -> SpeechData, so this is checkable without a network call:

from livekit.agents.utils import is_given
from livekit.plugins.deepgram.stt import prerecorded_transcription_to_speech_event

event = prerecorded_transcription_to_speech_event(
    "en-US",
    {
        "metadata": {"request_id": "r"},
        "results": {
            "channels": [
                {
                    "alternatives": [
                        {
                            "transcript": "hello world",
                            "confidence": 0.98,
                            "words": [
                                {"word": "hello", "start": 0.0, "end": 0.4,
                                 "confidence": 0.99},
                                {"word": "world", "start": 0.4, "end": 0.9,
                                 "confidence": 0.42},
                            ],
                        }
                    ]
                }
            ]
        },
    },
)

words = event.alternatives[0].words
assert [w.confidence for w in words] == [0.99, 0.42]   # before: [NOT_GIVEN, NOT_GIVEN]
assert all(is_given(w.confidence) for w in words)

I did not add a test file — tests/ has no unit tests for this plugin's
parsing helpers, so I did not want to establish a pattern uninvited. Happy
to add one wherever you'd like it.

Behaviour change

Additive, and the same shape as #5034. Words gain a value where they
previously had NOT_GIVEN; nothing that already worked changes. Consumers
gating on is_given() keep working either way.

stt_v2.py::_parse_transcription forwards Deepgram's per-word confidence
into TimedString (livekit#5034). stt.py builds the same TimedString at two sites
without it, so every word from the v1 STT reports confidence as
NOT_GIVEN even though the response carries a value for it.

Both sites now forward it, defaulting to NOT_GIVEN so an absent value
stays distinguishable from a genuine zero.
@Snowfly0709
Snowfly0709 requested a review from a team as a code owner September 9, 2026 03:08
@CLAassistant

CLAassistant commented Sep 9, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Deepgram STT v1 word confidence isn't set on the word level

2 participants