Skip to content

feat(maya): add Maya Research voice models TTS plugin - #7175

Open
dheemutech wants to merge 3 commits into
livekit:mainfrom
MayaResearch:maya-research-tts
Open

feat(maya): add Maya Research voice models TTS plugin#7175
dheemutech wants to merge 3 commits into
livekit:mainfrom
MayaResearch:maya-research-tts

Conversation

@dheemutech

@dheemutech dheemutech commented Sep 8, 2026

Copy link
Copy Markdown

Summary

Add a native Maya Research text-to-speech provider, built directly from upstream main (e9a3422f47f070fbecfb5e8580d9803f48e1b6d6). This is a self-contained contribution with its own implementation, tests and branch, not a dependency on another pending contribution.

Provider/package names are model-independent. The current public model is Maya Calyx, with Aarav as the explicit default voice. Model and voice strings are not restricted to an enum at runtime. See the Maya public API contract and Maya Research Cookbook.

Implementation

  • Native TTS, ChunkedStream, SynthesizeStream, ConnectionPool, AudioEmitter, plugin registration, typed namespace package and maya extra.
  • Public WebSocket v2: bearer header, validated startup metadata, 24 kHz mono PCM, one context per turn, and one final closer after incremental sentences.
  • Strict base64 decoding, sample-byte carry across payloads, retained final partial frames and filtering of stale/unscoped audio.
  • Cancellation terminates the context and discards its connection. Completed turns reuse pooled connections; concurrent turns never share an active connection.
  • Changing settings does not interrupt an active turn; the next acquired connection must match the new model/voice/language.
  • Sanitized failures and no automatic replay after audio receipt. Current LiveKit semantics: a new stream per segment, incremental push_text, then end_input. No core implementation changes.

Review fixes in 10cfe4d

All three initial review concerns were reproduced and addressed, with replies in their existing threads:

  • Shutdown race: open-state guards before/after asynchronous acquisition phases and before returning sockets; directly constructed streams register for shutdown too. Late handshakes cannot reopen a closed provider.
  • Input pauses: progress-aware response deadlines permit waiting for LLM input after audio arrives, re-arm on new text/final closer, and bound text writes. Repeated unanswered input cannot keep extending the first-response timer. Deadline changes preserve the pending receive.
  • Transport security: custom URLs accept HTTPS/WSS only, including environment configuration. HTTP/WS fails before a connection; no plaintext exception was added. The default endpoint was already encrypted.

The v2 protocol has no per-sentence completion acknowledgement. Audio progress is not proof every sentence finished; the README makes the idle-input tradeoff explicit and requires applications to bound the overall LLM/turn and end or cancel abandoned input.

Follow-up a2333554 fixes the fourth finding: update_options(language=None) restores automatic/mixed-language mode for the next turn, while omitted arguments leave settings unchanged. Active turns keep their snapshot. Constructor language=None also omits the startup language field. Regression coverage includes active-turn isolation and invalid-update atomicity. A real Hindi turn followed by a cleared-language mixed Hindi/English turn completed, producing 130,560 and 192,000 PCM bytes respectively, with the second startup omitting language.

Verification of the revised candidate

  • 121 hermetic Maya tests passed on Python 3.10.21, 3.12.14 and 3.13.15. In-memory WebSocket, pytest.mark.unit, no service keys/network.
  • 147 combined tests passed on Python 3.12: Maya, core audio emitter, connection pool. Ruff and format checks passed repository-wide; focused strict mypy passed on all three Python versions. Wheel and source distribution built successfully.
  • Eight actual Maya checks reran successfully: one-shot and streamed Hindi/Telugu/English, completed-turn connection reuse, active cancellation and a fresh turn after cancellation. Seven completed non-silent WAVs remain private.
  • One additional real pause regression passed: two sentences in the same context, 2.201-second input pause with a 1-second response timeout, normal completion and 183,840 PCM bytes. This controlled-emission check uses an explicit BasicSentenceTokenizer; the eight-check suite uses the default tokenizer.
  • Full synthetic AgentSession reran successfully: released LiveKit 1.8.0, Soniox, GPT-4o-mini through OpenRouter and the revised Maya source. Final STT transcript received; 184,320 PCM bytes reached the sink; zero agent errors.
  • Public files scanned for private recordings, credential patterns and exact matches to the three test credentials. No credentials, customer recordings or private run evidence are included.
  • A fresh source-only cookbook extraction using the revised immutable plugin pin passed all 20 install/check steps, including 183 Python tests, 33 TypeScript tests and each recipe's separate installation/import.

The current GitHub checks are attached to this PR. Reproduction commands are in the plugin README. Live tests used explicitly authorized synthetic inputs and are not CI requirements. No physical microphone, speaker listening assessment or LiveKit room transport was tested. No latency SLA or universal voice quality is claimed.

Final revision a2333554 passed the upstream Linux unit job: 2,745 passed, 5 skipped, including tests/test_maya_tts.py. Repository lint, both type-check jobs and Devin Review passed. Devin confirmed all four reported findings resolved. The cookbook pins this revision at e166feed; its six required CI jobs passed (scheduled drift job skipped; public-source checks were also run manually and passed).

No unrelated provider implementation or existing package version was changed. Initial new-package metadata and workspace registration enable installation; release/version policy is left to maintainers. The CLA is signed. Maintainer review/merge remains pending.

@dheemutech
dheemutech requested a review from a team as a code owner September 8, 2026 18:38
@CLAassistant

CLAassistant commented Sep 8, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants