fix: prevent KV cache corruption on SWA/ISWA models + hot-path perf

[avion23]

Problem

Gemma-4 and any model with interleaved sliding window attention (ISWA) crashes on the second call to create_completion:

llm = Llama(model_path="gemma4-q4.gguf", n_gpu_layers=-1)

llm.create_chat_completion(messages=[{"role": "user", "content": "What is 2+2?"}])
# → OK

llm.create_chat_completion(messages=[{"role": "user", "content": "Write a hello world"}])
# → RuntimeError: error during generation: [end of text]

Root cause

ISWA KV caches store position tracking in global maps (g_iswa_pos_max / g_iswa_pos_min) that are cleared by llama_memory_clear() but not by llama_memory_seq_rm(). The generate() method detects a prefix match between consecutive prompts (shared BOS token), calls kv_cache_seq_rm() to remove the divergent tail, sees it return True, and skips the full reset. The stale position maps then cause batch allocator inconsistency → decode failure.

Additionally, reset() was a no-op on KV cache state (only reset n_tokens), so even explicitly calling llm.reset() between prompts didn't help.

Fix

1. reset() now calls llama_memory_clear() unconditionally
2. generate() bypasses prefix-match for SWA models, forcing full reset (same path as recurrent models)
3. eval() only calls kv_cache_seq_rm() when actually rewinding (len(tokens) < n_tokens), skipping the FFI call on the common append path

llm = Llama(model_path="gemma4-q4.gguf", n_gpu_layers=-1)

llm.create_chat_completion(messages=[{"role": "user", "content": "What is 2+2?"}])
# → "2 + 2 = 4"

llm.create_chat_completion(messages=[{"role": "user", "content": "Write a hello world"}])
# → "print('Hello, World!')"

Performance

Location	Change
set_batch()	numpy bulk writes replace Python loop (512 tokens × 5 assignments → 5 numpy ops)
_create_completion	incremental token_to_piece() accumulation replaces O(n) re-detokenization per token
_create_completion	in-place logit_bias removes 128K-element np.copy() per token
_create_completion	tokens[:-1] → tokens fixes broken prefix detection (last token was excluded)
sample()	np.argpartition O(V) replaces sorted() O(V log V) for top-k logprobs
_internals.token_to_piece	returns actual byte length instead of padded 32-byte buffer

Files

• llama_cpp/llama.py — reset(), eval(), generate(), _create_completion(), sample()
• llama_cpp/_internals.py — set_batch(), token_to_piece()