Skip to content

Harden GGUF role gating and fold projector size into the memory gate#338

Merged
quiet-node merged 2 commits into
mainfrom
fix/gguf-acg-harden
Jul 18, 2026
Merged

Harden GGUF role gating and fold projector size into the memory gate#338
quiet-node merged 2 commits into
mainfrom
fix/gguf-acg-harden

Conversation

@quiet-node

Copy link
Copy Markdown
Owner

Description

Thuki re-reads a GGUF's header at install and load so non-chat models can never become a Ready primary chat model, and folds a vision model's projector size into the pre-load memory gate so vision loads are sized accurately. This closes the gap where a renamed companion, an embedding model, or a mis-sized projector could slip past the browse-time filename heuristics.

Key changes

  • Architecture gating at finalize/load. The local blob header is re-read before a model can become Ready or load as -m. Known non-chat families (embedding/encoder arches such as bert, t5, and clip, plus audio tokenizers) and projector/adapter roles are rejected with a clear "not a chat model" message. Missing architecture stays soft (filename heuristics) so an incomplete or list-time header never bricks an install.
  • Projector size in the memory gate. A vision model's projector (mmproj) size, taken from the on-disk blob length when present or the curated registry size otherwise, folds into the pre-load memory gate and the Library/Discover fit hint through one shared helper, so the gate and the fit estimate never size a projector differently.
  • Projector companion ranking. When a repo ships several projectors, Thuki ranks them like llama.cpp: shared directory depth with the chat weights first, then the closest quant bit-width.
  • Browse-all listing. Only primary text weights with complete LFS metadata are listed; projectors, embedders, draft/MTP/dspark helpers, and LoRA adapters never appear as standalone chat downloads.
  • Clearer engine-start errors. A wrong projector paired with text weights surfaces a fixable "Vision projector mismatch" message. The classifier requires the failure to actually name a projector, so a text-only load's error keeps its raw reason instead of being mislabeled a projector problem.

Testing

cargo test + cargo llvm-cov (100% line coverage), cargo clippy -D warnings, and the full frontend + backend build gate.

Notes

  • DENIED_PRIMARY_ARCHES is a baked-in, pin-scoped denylist: an engine bump may need it extended when new non-chat families appear. It lives in config/defaults.rs with the other baked-in constants and is documented in docs/configurations.md.
  • The Library fit badge for a Browse-installed vision model with no registry entry counts weights only, while the load gate folds the projector blob. This is an under-estimate on the badge (never a false block), kept deliberately because the badge builder has no model-store handle.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
@quiet-node
quiet-node merged commit 71809e1 into main Jul 18, 2026
6 checks passed
@quiet-node
quiet-node deleted the fix/gguf-acg-harden branch July 18, 2026 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant