fix(nano): correct punctuation timestamps across VAD merges

[Bruce-Yii]

Closes #3702.

Problem

With Fun-ASR-Nano and merge_vad enabled, a sentence-final punctuation mark's timestamp lands on the next sentence's start time.

Root cause

merge_vad intentionally merges neighboring VAD regions (including the silence gaps between them) into one ASR chunk, so a single decode window can span several sentences. Nano emits punctuation natively and its per-token times come from CTC forced alignment (forced_align), which must place every target token — including punctuation tokens with zero acoustic realization — on at least one frame. The alignment puts such a token at the next sentence's onset frame. That span is then propagated 1:1 to the user-visible timestamp list (frame→ms scaling plus chunk-offset restoration), with no correction anywhere on the Nano path (the separate punctuation model is skipped by design for Nano). The same shared forced_align feeds the offline path, the vLLM path, and the vLLM pipeline path.

Fix and invariant

Punctuation carries no acoustic extent. anchor_punctuation_timestamps (new, in funasr/models/fun_asr_nano/tools/utils.py) pins a punctuation span that is followed by more speech to the preceding spoken token's end (zero-width). It uses one explicit spoken/punctuation/special classifier so <sil>-style special tokens neither trigger anchoring nor serve as anchors. Leading/trailing punctuation and special tokens are left untouched, preserving single-segment behavior. Wired into all three Nano timestamp surfaces: FunASRNano.inference_llm, FunASRNanoVLLM._compute_timestamps, and FunASRNanoVLLMPipeline._compute_all_timestamps — the pipeline applies it per independently aligned VAD segment, so speech in a later segment can never rewrite a prior segment's trailing punctuation. ctc_timestamps is intentionally untouched.

Validation performed

• New tests/test_fun_asr_nano_punctuation_timestamps.py: 8/8 pass, including a stubbed-engine behavior test of the pipeline path (no weights/GPU) proving intra-segment anchoring plus segment-boundary preservation.
• Neighboring offline tests (test_timestamp_tools.py, test_fun_asr_nano_missing_ctc_weights.py): 15/15 pass (23/23 total with the new file).
• git diff --check clean; added lines follow repo conventions (black hunks verified to avoid them).
• Deterministic forced-alignment reproduction with synthetic peaky emissions through the repo's exact forced_align: merged-window punctuation lands at the silence end (next-sentence onset) while the single-sentence control keeps it at the window end; after anchoring it sits at the sentence end.
• Full test-suite validation was not run, and real-weight Nano inference was not run; no such claims are made.

[LauraGPT]

I ran the three reported test files on exact head 4552e01: 23 passed. I then checked the token-classification assumption against Nano's actual SenseVoiceTokenizer using the official multilingual.tiktoken at fixed HF revision 272c57b82523ada6fd87095e955f8e29100979ab (SHA256 747979631e813193436aabcff7c1c235d37de8097b71c563ec8b63b7a515c718, verified against the public artifact). This uncovered a spoken-token timestamp regression, detailed inline.

The additional probe uses real tokenization and the repository's actual forced_align with deterministic synthetic log probabilities. It has 3 failing preservation assertions (helper, actual vLLM timestamp method, actual pipeline timestamp method) and 2 passing controls (tokenizer/alignment roundtrip and genuine sentence punctuation anchoring). The neural encoder/decoder/frontend are synthetic stand-ins; no weights, GPU, acoustic accuracy, full offline inference or vLLM-engine acceptance is claimed. Please add a real-tokenizer byte-fragment regression and address classification before merging. Issue #3702 should remain open until the revised behavior is verified.

[LauraGPT]

[P2] Do not classify undecodable spoken byte fragments as punctuation. Nano's real CTC tokenizer is byte-based, and all three new call sites classify decode([token_id]) independently. For the ordinary surname in 我叫郗明。你好。, the official vocabulary encodes 郗 as IDs [10958, 245]: decoding them together gives 郗, but each separately gives U+FFFD (�). This regex classifies both as punctuation. With actual forced_align on deterministic emissions, their valid spans [0.60, 0.72] and [0.84, 0.96] seconds are both changed to [0.48, 0.48], the preceding 叫's end. The actual vLLM method reproduces this, and the pipeline reproduces it after its +1s VAD offset (both become [1.48, 1.48]). Thus this patch deletes spoken timing information for byte-split characters, not merely punctuation. Please base classification on valid decoded text/token provenance (at minimum, undecodable fragments must not be treated as punctuation), and add a regression using the official tokenizer plus multi-ID spoken characters across the timestamp methods. The current tests use a one-ID-to-one-character stub and cannot catch this. Full-sequence decoding works; the pre-existing per-ID display replacement is not itself introduced by this PR, but collapsing its valid acoustic spans is.