|
| 1 | +--- |
| 2 | +title: Semantic Search with Gemini Embeddings |
| 3 | +description: This guide will walk you through the process of setting up Meilisearch with Gemini embeddings to enable semantic search capabilities. |
| 4 | +--- |
| 5 | + |
| 6 | +## Requirements |
| 7 | + |
| 8 | +To follow this guide, you'll need: |
| 9 | + |
| 10 | +- A [Meilisearch Cloud](https://www.meilisearch.com/cloud) project running version >=1.13 |
| 11 | +- A Google account with an API key for embedding generation. You can sign up for a Google account at [Google](https://google.com/) |
| 12 | + |
| 13 | +## Setting up Meilisearch |
| 14 | + |
| 15 | +To set up an embedder in Meilisearch, you need to configure it to your settings. You can refer to the [Meilisearch documentation](/reference/api/settings) for more details on updating the embedder settings. |
| 16 | + |
| 17 | +While using Gemini to generate embeddings, you'll need to use the model `gemini-embedding-001`. Unlike some other services, Gemini currently offers only one embedding model. |
| 18 | + |
| 19 | +Here's an example of embedder settings for Gemini: |
| 20 | + |
| 21 | +```json |
| 22 | +{ |
| 23 | + "gemini": { |
| 24 | + "source": "rest", |
| 25 | + "dimensions": 3072, |
| 26 | + "documentTemplate": "<Custom template (Optional, but recommended)>", |
| 27 | + "headers": { |
| 28 | + "Content-Type": "application/json", |
| 29 | + "x-goog-api-key": "<Google API Key>" |
| 30 | + }, |
| 31 | + "url": "https://generativelanguage.googleapis.com/v1beta/models/gemini-embedding-001:batchEmbedContents", |
| 32 | + "request": { |
| 33 | + "requests": [ |
| 34 | + { |
| 35 | + "model": "models/gemini-embedding-001", |
| 36 | + "content": { |
| 37 | + "parts": [ |
| 38 | + { "text": "{{text}}" } |
| 39 | + ] |
| 40 | + } |
| 41 | + }, |
| 42 | + "{{..}}" |
| 43 | + ] |
| 44 | + }, |
| 45 | + "response": { |
| 46 | + "embeddings": [ |
| 47 | + { "values": "{{embedding}}" }, |
| 48 | + "{{..}}" |
| 49 | + ] |
| 50 | + } |
| 51 | + } |
| 52 | +} |
| 53 | + ``` |
| 54 | + |
| 55 | +In this configuration: |
| 56 | + |
| 57 | +- `source`: Specifies the source of the embedder, which is set to "rest" for using a REST API. |
| 58 | +- `headers`: Replace `<Google API Key>` with your actual Google API key. |
| 59 | +- `dimensions`: Specifies the dimensions of the embeddings, set to 3072 for the `gemini-embedding-001` model. |
| 60 | +- `documentTemplate`: Optionally, you can provide a [custom template](/learn/ai_powered_search/getting_started_with_ai_search) for generating embeddings from your documents. |
| 61 | +- `url`: Specifies the URL of the Gemini API endpoint. |
| 62 | +- `request`: Defines the request structure for the Gemini API, including the model name and input parameters. |
| 63 | +- `response`: Defines the expected response structure from the Gemini API, including the embedding data. |
| 64 | + |
| 65 | +Once you've configured the embedder settings, Meilisearch will automatically generate embeddings for your documents and store them in the vector store. |
| 66 | + |
| 67 | +Please note that most third-party tools have rate limiting, which is managed by Meilisearch. If you have a free account, the indexation process may take some time, but Meilisearch will handle it with a retry strategy. |
| 68 | + |
| 69 | +It's recommended to monitor the tasks queue to ensure everything is running smoothly. You can access the tasks queue using the Cloud UI or the [Meilisearch API](https://www.meilisearch.com/docs/reference/api/tasks). |
| 70 | + |
| 71 | +## Testing semantic search |
| 72 | + |
| 73 | +With the embedder set up, you can now perform semantic searches using Meilisearch. When you send a search query, Meilisearch will generate an embedding for the query using the configured embedder and then use it to find the most semantically similar documents in the vector store. |
| 74 | +To perform a semantic search, you simply need to make a normal search request but include the hybrid parameter: |
| 75 | + |
| 76 | +```json |
| 77 | +{ |
| 78 | + "q": "<Query made by the user>", |
| 79 | + "hybrid": { |
| 80 | + "semanticRatio": 1, |
| 81 | + "embedder": "gemini" |
| 82 | + } |
| 83 | +} |
| 84 | +``` |
| 85 | + |
| 86 | +In this request: |
| 87 | + |
| 88 | +- `q`: Represents the user's search query. |
| 89 | +- `hybrid`: Specifies the configuration for the hybrid search. |
| 90 | + - `semanticRatio`: Allows you to control the balance between semantic search and traditional search. A value of 1 indicates pure semantic search, while a value of 0 represents full-text search. You can adjust this parameter to achieve a hybrid search experience. |
| 91 | + - `embedder`: The name of the embedder used for generating embeddings. Make sure to use the same name as specified in the embedder configuration, which in this case is "gemini". |
| 92 | + |
| 93 | +You can use the Meilisearch API or client libraries to perform searches and retrieve the relevant documents based on semantic similarity. |
| 94 | + |
| 95 | +## Conclusion |
| 96 | + |
| 97 | +By following this guide, you should now have Meilisearch set up with Gemini embedding, enabling you to leverage semantic search capabilities in your application. Meilisearch's auto-batching and efficient handling of embeddings make it a powerful choice for integrating semantic search into your project. |
| 98 | + |
| 99 | +To explore further configuration options for embedders, consult the [detailed documentation about the embedder setting possibilities](/reference/api/settings). |
0 commit comments