Add RAG support for man page retrieval #33
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 6 commits
January 31, 2025 00:32
- Add LanceDB vector store for man pages - Implement sentence chunking and embedding - Add search functionality using cosine similarity
- Add optional RAG dependencies (lancedb, chonkie, sentence_transformers) - Create RAG functionality to search and retrieve relevant man pages - Add --retrieve flag to CLI for enabling RAG features - Move database to ~/.cache/shell_sage/db - Add search threshold and limit parameters - Clean up code and remove unused cells
- Lower default similarity threshold for better results - Add package name to retrieved docs - Add ssage_index CLI command - Clean up notebook organization
… switch to gte-small model
- Add hybrid search combining text and vector similarity - Implement LinearCombinationReranker for better results - Rename retrieve flag to use_retrieval for consistency - Clean up code and improve progress reporting
|
Found 3 changed notebooks. Review the changes at https://app.gitnotebooks.com/AnswerDotAI/shell_sage/pull/33 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds Retrieval Augmented Generation (RAG) support to enhance ShellSage's responses by retrieving relevant man pages.
Key changes:
To use RAG:
The retrieved man pages are added to the prompt context to help the LLM provide more accurate command-line assistance.