Conversation
Signed-off-by: Valentin Mendelev <vmendelev@nvidia.com>
Signed-off-by: Valentin Mendelev <vmendelev@nvidia.com>
Signed-off-by: Valentin Mendelev <vmendelev@nvidia.com>
Create a small dummy context wav for requests without context_audio_filepath to prevent dataloader failures (missing d*.wav) and 500s from the unified server.
Avoid KV-cache shape mismatches when batch sizes vary between requests in the unified server.
Route HuggingFace resolve URLs used by NeMo audio codec checkpoints through huggingface_hub download so multi-rank server startup avoids repeated downloads and 429s.
Longform decoding with the transformer cache path can produce sequence-length mismatches; disable cache per request batch to prevent 500s in serve_unified.
Correct HuggingFace resolve URL matching so downloads go through hf_hub_download() and avoid multi-rank 429s.
Stop setting srun --wait by default; allow opt-in via cluster_config.srun_wait_seconds.
Add a large srun --wait for multi-instance runs to override nemo_run's default --wait=60, preventing premature termination when some ranks finish earlier.
Lower Magpie inference runner batch size to reduce memory/latency spikes under multi-instance load.
Use a 1-hour default srun --wait for multi-instance runs to avoid premature task termination when chunk runtimes differ.
Introduce the emergent_tts dataset package with prepare/generate/score helpers and default configs to run EmergentTTS evaluation via NeMo-Skills. Co-authored-by: Cursor <cursoragent@cursor.com>
Install google-genai for EmergentTTS-Eval, run scoring from the dataset base dir so relative paths resolve, and avoid shipping large local caches/data. Document EmergentTTS-Eval usage in nv_tts guide. Co-authored-by: Cursor <cursoragent@cursor.com>
Document dataset preparation (HF_TOKEN) and evaluation workflow, including cloning and patching EmergentTTS-Eval for NVIDIA Inference API judging. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Creating this PR just to easily view diffs.