feat(embedding): make vector dimension configurable#324
Conversation
- Add dim field to EmbeddingSettings - Use settings.dim in build_embedding_provider - Read dimension from settings in LanceDB table schemas - Add default dim = 1024 in default.toml Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Hi EverOS team, thanks for the great work on this project. We're building CodeKeeper Advance (which will release to Github after we fullfill all features 😊) on top of EverOS. In our setup, we use embedding models like This PR makes the embedding dimension configurable via Happy to adjust naming or implementation based on your feedback. Thanks for reviewing! |
Motivation
Different embedding models produce vectors with different dimensions (e.g.,
intfloat/multilingual-e5-smalluses 384, whileintfloat/multilingual-e5-largeuses 1024). Currently EverOS hard-codes_DIM = 1024in LanceDB table schemas andbuild_embedding_provider, which prevents using smaller/larger embedding models without patching source code.Changes
dimfield toEmbeddingSettingswith env bindingEVEROS_EMBEDDING__DIM.dim = 1024default indefault.toml.build_embedding_providerto usesettings.dimwhen no explicitdimis passed.agent_case,agent_skill,atomic_fact,episode,foresight,knowledge_topic) to read_DIMfromload_settings().embedding.diminstead of hard-coding 1024.Backward compatibility
The default remains
1024, so existing configurations continue to work without changes.Testing
python -m py_compilepasses for all modified Python files.EVEROS_EMBEDDING__DIMor[embedding].dimis explicitly set.