You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: content/develop/ai/search-and-query/vectors.md
+63-3Lines changed: 63 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -25,7 +25,7 @@ To quickly get started, check out the [Redis vector quickstart guide]({{< relref
25
25
26
26
## Overview
27
27
28
-
1.[**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}) and [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) vector index types.
28
+
1.[**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}), [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) and [`SVS-VAMANA`]({{< relref "develop/ai/search-and-query/vectors#svs-vamana-index" >}}) vector index types.
29
29
1.[**Store and update vectors**]({{< relref "develop/ai/search-and-query/vectors#store-and-update-vectors" >}}): Redis stores vectors and metadata in hashes or JSON objects.
30
30
1.[**Search with vectors**]({{< relref "develop/ai/search-and-query/vectors#search-with-vectors" >}}): Redis supports several advanced querying strategies with vector fields including k-nearest neighbor ([KNN]({{< relref "develop/ai/search-and-query/vectors#knn-vector-search" >}})), [vector range queries]({{< relref "develop/ai/search-and-query/vectors#vector-range-queries" >}}), and [metadata filters]({{< relref "develop/ai/search-and-query/vectors#filters" >}}).
31
31
1.[**Configure vector queries at runtime**]({{< relref "develop/ai/search-and-query/vectors#runtime-query-params" >}}).
@@ -129,6 +129,55 @@ FT.CREATE documents
129
129
130
130
In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `HNSW` vector field named `doc_embedding` with five index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `M`, and `EF_CONSTRUCTION`.
131
131
132
+
### SVS-VAMANA index
133
+
134
+
Scalable Vector Search (SVS) is an Intel project in which a new vector search library, VAMANA graph index, was created. SVS-VAMANA supports highly accurate compressed vector indexes. You can read more about the project [here](https://intel.github.io/ScalableVectorSearch/intro.html). Support for `SVS-VAMANA` indexing was added in Redis 8.2.
135
+
136
+
Choose the `SYS-VAMANA` index type when you need vector search
|`TYPE`| Vector type (`FLOAT16` or `FLOAT32`). Note: `COMPRESSION` is supported with both types. |
147
+
|`DIM`| The width, or number of dimensions, of the vector embeddings stored in this field. In other words, the number of floating point elements comprising the vector. `DIM` must be a positive integer. The vector used to query this field must have the exact dimensions as the field itself. |
|`COMPRESSION`| Compression algorithm (`LVQ8`, `LVQ8`, `LVQ4`, `LVQ4x4`, `LVQ4x8`, `LeanVec4x8`, or `LeanVec8x8`). Vectors will be compressed during indexing. Note: On non-Intel platforms, `SVS-VAMANA` with `COMPRESSION` will fall back to Intel’s basic quantization implementation. |
157
+
|`CONSTRUCTION_WINDOW_SIZE`| The search window size (the default is 200) to use during graph construction. A higher search window size will yield a higher quality graph since more overall vertexes are considered, but will increase construction time. |
158
+
|`GRAPH_MAX_DEGREE`| The maximum node degree in the graph (the default is 32). A higher max degree may yield a higher quality graph in terms of recall for performance, but the memory footprint of the graph is directly proportional to the maximum degree. |
159
+
|`SEARCH_WINDOW_SIZE`| The size of the search window size (the default is 10). Increasing the search window size and capacity generally yields more accurate but slower search results. |
160
+
|`EPSILON`| The range search approximation factor (default is 0.01). |
161
+
|`TRAINING_THRESHOLD`| The number of vectors after which training is triggered. Applicable only when used with `COMPRESSION`. The default value is `10 * DEFAULT_BLOCK_SIZE` or, if provided, the value is limited to `100 * DEFAULT_BLOCK_SIZE`, where `DEFAULT_BLOCK_SIZE` is 1024. |
162
+
|`LEANVEC_DIM`| The dimension used when using `LeanVec4x8` or `LeanVec8x8` compression for dimensionality reduction. The default value is `DIM / 2`. If provided, the value should be less than `DIM`. |
163
+
164
+
**Example**
165
+
166
+
```
167
+
FT.CREATE documents
168
+
ON HASH
169
+
PREFIX 1 docs:
170
+
SCHEMA doc_embedding VECTOR SVS-VAMANA 12
171
+
TYPE FLOAT32
172
+
DIM 1536
173
+
DISTANCE_METRIC COSINE
174
+
GRAPH_MAX_DEGREE 40
175
+
CONSTRUCTION_WINDOW_SIZE 250
176
+
COMPRESSION LVQ8
177
+
```
178
+
179
+
In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `SVS-VAMANA` vector field named `doc_embedding` with six index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `GRAPH_MAX_DEGREE`, `CONSTRUCTION_WINDOW_SIZE` and `COMPRESSION`.
180
+
132
181
### Distance metrics
133
182
134
183
Redis supports three popular distance metrics to measure the degree of similarity between two vectors $u$, $v$ $\in \mathbb{R}^n$, where $n$ is the length of the vectors:
|`EF_RUNTIME`| The maximum number of top candidates to hold during the KNN search. Higher values lead to more accurate results at the expense of a longer query runtime. | The value passed during index creation. The default is 10. |
379
428
|`EPSILON`| The relative factor that sets the boundaries for a vector range query. Vector candidates whose distance from the query vector is `radius * (1 + EPSILON)` are potentially scanned, allowing a more extensive search and more accurate results at the expense of runtime. | The value passed during index creation. The default is 0.01. |
380
429
430
+
**SVS-VAMANA**
431
+
432
+
Optional runtime parameters for SYS-VAMANA indexes are:
433
+
434
+
| Parameter | Description | Default value |
435
+
|:----------|:------------|:--------------|
436
+
|`SEARCH_WINDOW_SIZE`| The size of the search window size (applies only to KNN searches). | 10 or the value that was passed upon index creation. |
437
+
|`GRAPH_MAX_DEGREE`| The maximum node degree in the graph. | 32 or the value that was passed upon index creation. |
438
+
|`SEARCH_WINDOW_SIZE`| The size of the search window size. | 10 or the value that was passed upon index creation. |
439
+
|`EPSILON`| The range search approximation factor. | 0.01 or the value that was passed upon index creation. |
440
+
381
441
382
442
### Important notes
383
443
@@ -526,6 +586,6 @@ Here are some additional resources that apply vector search for different use ca
526
586
-[Retrieval augmented generation from scratch](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/01_redisvl.ipynb)
0 commit comments