This repository was archived by the owner on May 10, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 110
VoyageAI embedding function #226
Open
fzowl
wants to merge
21
commits into
chroma-core:main
Choose a base branch
from
voyage-ai:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
8b3dea1
VoyageAI doc
fodizoltan b5a83c5
VoyageAI doc
fodizoltan 6c974c1
voyage-02 to voyage-2
fodizoltan b10d06b
voyage-02 to voyage-large-2
fodizoltan 991970b
Merge pull request #1 from voyage-ai/voyageai_embeddings
Liuhong99 cd5ccfe
Reverting the js_reference changes
fodizoltan 6ac1cdb
Merge pull request #2 from voyage-ai/voyageai_embeddings
fzowl 47025a8
Spaces?!
fodizoltan ef9d109
Merge pull request #3 from voyage-ai/voyageai_embeddings
fzowl b74359c
Wo spaces?!(again)
fodizoltan 9b0b111
Merge pull request #4 from voyage-ai/voyageai_embeddings
fzowl c4b912d
Adding InputType documentation
fzowl 6cf8fcb
Merge pull request #5 from voyage-ai/voyageai_embeddings
fzowl 5d86696
Merge branch 'chroma-core:main' into main
fzowl d67ffd0
Correcting due to comments, use voyage-law-2 in sample
fzowl d195c2e
Merge pull request #6 from voyage-ai/voyageai_embeddings
fzowl b5f8e0b
Revert yarn.locak changes
fzowl 44cb5b9
Merge pull request #7 from voyage-ai/voyageai_embeddings
fzowl 78ee227
Merge branch 'main' into main
fzowl 48f74ac
Merge branch 'main' into main
fzowl 09cd40b
Merge branch 'chroma-core:main' into main
fzowl File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
--- | ||
--- | ||
|
||
# VoyageAI | ||
|
||
import Tabs from '@theme/Tabs'; | ||
import TabItem from '@theme/TabItem'; | ||
|
||
<div class="select-language">Select a language</div> | ||
|
||
<Tabs queryString groupId="lang"> | ||
<TabItem value="py" label="Python"></TabItem> | ||
<TabItem value="js" label="JavaScript"></TabItem> | ||
</Tabs> | ||
|
||
Chroma also provides a convenient wrapper around VoyageAI's embedding API. This embedding function runs remotely on VoyageAI’s servers, and requires an API key. You can get an API key by signing up for an account at [VoyageAI](https://dash.voyageai.com/api-keys). | ||
|
||
<Tabs queryString groupId="lang" className="hideTabSwitcher"> | ||
<TabItem value="py" label="Python"> | ||
|
||
This embedding function relies on the `voyageai` python package, which you can install with `pip install voyageai`. | ||
|
||
```python | ||
from chromadb.utils.embedding_functions import VoyageAIEmbeddingFunction | ||
voyageai_ef = VoyageAIEmbeddingFunction(api_key="YOUR_API_KEY", model_name="voyage-law-2", input_type=VoyageAIEmbeddingFunction.InputType.DOCUMENT) | ||
result = voyageai_ef(input=["document1","document2"]) | ||
``` | ||
|
||
</TabItem> | ||
<TabItem value="js" label="JavaScript"> | ||
|
||
```javascript | ||
const {VoyageAIEmbeddingFunction, InputType} = require('chromadb'); | ||
// const {VoyageAIEmbeddingFunction, InputType} from "chromadb"; // ESM import | ||
const embedder = new VoyageAIEmbeddingFunction("apiKey", "voyage-law-2", InputType.DOCUMENT) | ||
|
||
// use directly | ||
const embeddings = embedder.generate(["document1","document2"]) | ||
|
||
// pass documents to query for .add and .query | ||
const collection = await client.createCollection({name: "name", embeddingFunction: embedder}) | ||
const collectionGet = await client.getCollection({name:"name", embeddingFunction: embedder}) | ||
``` | ||
|
||
</TabItem> | ||
|
||
</Tabs> | ||
|
||
|
||
|
||
You should pass in the `model_name` argument, which lets you choose which VoyageAI embeddings model to use. You can see the available models [here](https://docs.voyageai.com/docs/embeddings). | ||
|
||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.