Skip to content

Commit 429e7cf

Browse files
authored
Add guide on Semantic search with gemini (#3405)
1 parent f0529ca commit 429e7cf

File tree

2 files changed

+100
-0
lines changed

2 files changed

+100
-0
lines changed

docs.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -410,6 +410,7 @@
410410
"guides/embedders/cohere",
411411
"guides/embedders/mistral",
412412
"guides/embedders/voyage",
413+
"guides/embedders/gemini",
413414
"guides/computing_hugging_face_embeddings_gpu"
414415
]
415416
},

guides/embedders/gemini.mdx

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
---
2+
title: Semantic Search with Gemini Embeddings
3+
description: This guide will walk you through the process of setting up Meilisearch with Gemini embeddings to enable semantic search capabilities.
4+
---
5+
6+
## Requirements
7+
8+
To follow this guide, you'll need:
9+
10+
- A [Meilisearch Cloud](https://www.meilisearch.com/cloud) project running version >=1.13
11+
- A Google account with an API key for embedding generation. You can sign up for a Google account at [Google](https://google.com/)
12+
13+
## Setting up Meilisearch
14+
15+
To set up an embedder in Meilisearch, you need to configure it to your settings. You can refer to the [Meilisearch documentation](/reference/api/settings) for more details on updating the embedder settings.
16+
17+
While using Gemini to generate embeddings, you'll need to use the model `gemini-embedding-001`. Unlike some other services, Gemini currently offers only one embedding model.
18+
19+
Here's an example of embedder settings for Gemini:
20+
21+
```json
22+
{
23+
"gemini": {
24+
"source": "rest",
25+
"dimensions": 3072,
26+
"documentTemplate": "<Custom template (Optional, but recommended)>",
27+
"headers": {
28+
"Content-Type": "application/json",
29+
"x-goog-api-key": "<Google API Key>"
30+
},
31+
"url": "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:batchEmbedContents",
32+
"request": {
33+
"requests": [
34+
{
35+
"model": "models/gemini-embedding-001",
36+
"content": {
37+
"parts": [
38+
{ "text": "{{text}}" }
39+
]
40+
}
41+
},
42+
"{{..}}"
43+
]
44+
},
45+
"response": {
46+
"embeddings": [
47+
{ "values": "{{embedding}}" },
48+
"{{..}}"
49+
]
50+
}
51+
}
52+
}
53+
```
54+
55+
In this configuration:
56+
57+
- `source`: Specifies the source of the embedder, which is set to "rest" for using a REST API.
58+
- `headers`: Replace `<Google API Key>` with your actual Google API key.
59+
- `dimensions`: Specifies the dimensions of the embeddings, set to 3072 for the `gemini-embedding-001` model.
60+
- `documentTemplate`: Optionally, you can provide a [custom template](/learn/ai_powered_search/getting_started_with_ai_search) for generating embeddings from your documents.
61+
- `url`: Specifies the URL of the Gemini API endpoint.
62+
- `request`: Defines the request structure for the Gemini API, including the model name and input parameters.
63+
- `response`: Defines the expected response structure from the Gemini API, including the embedding data.
64+
65+
Once you've configured the embedder settings, Meilisearch will automatically generate embeddings for your documents and store them in the vector store.
66+
67+
Please note that most third-party tools have rate limiting, which is managed by Meilisearch. If you have a free account, the indexation process may take some time, but Meilisearch will handle it with a retry strategy.
68+
69+
It's recommended to monitor the tasks queue to ensure everything is running smoothly. You can access the tasks queue using the Cloud UI or the [Meilisearch API](https://www.meilisearch.com/docs/reference/api/tasks).
70+
71+
## Testing semantic search
72+
73+
With the embedder set up, you can now perform semantic searches using Meilisearch. When you send a search query, Meilisearch will generate an embedding for the query using the configured embedder and then use it to find the most semantically similar documents in the vector store.
74+
To perform a semantic search, you simply need to make a normal search request but include the hybrid parameter:
75+
76+
```json
77+
{
78+
"q": "<Query made by the user>",
79+
"hybrid": {
80+
"semanticRatio": 1,
81+
"embedder": "gemini"
82+
}
83+
}
84+
```
85+
86+
In this request:
87+
88+
- `q`: Represents the user's search query.
89+
- `hybrid`: Specifies the configuration for the hybrid search.
90+
- `semanticRatio`: Allows you to control the balance between semantic search and traditional search. A value of 1 indicates pure semantic search, while a value of 0 represents full-text search. You can adjust this parameter to achieve a hybrid search experience.
91+
- `embedder`: The name of the embedder used for generating embeddings. Make sure to use the same name as specified in the embedder configuration, which in this case is "gemini".
92+
93+
You can use the Meilisearch API or client libraries to perform searches and retrieve the relevant documents based on semantic similarity.
94+
95+
## Conclusion
96+
97+
By following this guide, you should now have Meilisearch set up with Gemini embedding, enabling you to leverage semantic search capabilities in your application. Meilisearch's auto-batching and efficient handling of embeddings make it a powerful choice for integrating semantic search into your project.
98+
99+
To explore further configuration options for embedders, consult the [detailed documentation about the embedder setting possibilities](/reference/api/settings).

0 commit comments

Comments
 (0)