Skip to content

Commit 2440a42

Browse files
committed
DOC-5457: add the new SVS-VAMANA vector algorithm type
1 parent b35f6d1 commit 2440a42

File tree

2 files changed

+64
-3
lines changed

2 files changed

+64
-3
lines changed

content/develop/ai/search-and-query/indexing/field-and-type-options.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,7 @@ Where:
157157

158158
- `FLAT`: brute force algorithm.
159159
- `HNSW`: hierarchical, navigable, small world algorithm.
160+
- `SVS-VAMANA`: a graph-based nearest neighbor search algorithm.
160161

161162
The `{algorithm}` attribute specifies the algorithm to use when searching `k` most similar vectors in the index or filtering vectors by range.
162163

content/develop/ai/search-and-query/vectors.md

Lines changed: 63 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ To quickly get started, check out the [Redis vector quickstart guide]({{< relref
2525

2626
## Overview
2727

28-
1. [**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}) and [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) vector index types.
28+
1. [**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}), [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) and [`SVS-VAMANA`]({{< relref "develop/ai/search-and-query/vectors#svs-vamana-index" >}}) vector index types.
2929
1. [**Store and update vectors**]({{< relref "develop/ai/search-and-query/vectors#store-and-update-vectors" >}}): Redis stores vectors and metadata in hashes or JSON objects.
3030
1. [**Search with vectors**]({{< relref "develop/ai/search-and-query/vectors#search-with-vectors" >}}): Redis supports several advanced querying strategies with vector fields including k-nearest neighbor ([KNN]({{< relref "develop/ai/search-and-query/vectors#knn-vector-search" >}})), [vector range queries]({{< relref "develop/ai/search-and-query/vectors#vector-range-queries" >}}), and [metadata filters]({{< relref "develop/ai/search-and-query/vectors#filters" >}}).
3131
1. [**Configure vector queries at runtime**]({{< relref "develop/ai/search-and-query/vectors#runtime-query-params" >}}).
@@ -129,6 +129,55 @@ FT.CREATE documents
129129

130130
In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `HNSW` vector field named `doc_embedding` with five index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `M`, and `EF_CONSTRUCTION`.
131131

132+
### SVS-VAMANA index
133+
134+
Scalable Vector Search (SVS) is an Intel project in which a new vector search library, VAMANA graph index, was created. SVS-VAMANA supports highly accurate compressed vector indexes. You can read more about the project [here](https://intel.github.io/ScalableVectorSearch/intro.html). Support for `SVS-VAMANA` indexing was added in Redis 8.2.
135+
136+
Choose the `SYS-VAMANA` index type when you need vector search
137+
138+
- on billions of high-dimensional vectors,
139+
- at high accuracy and state-of-the-art speed,
140+
- using less memory than alternatives.
141+
142+
**Required attributes**
143+
144+
| Attribute | Description |
145+
|:-------------------|:-----------------------------------------|
146+
| `TYPE` | Vector type (`FLOAT16` or `FLOAT32`). Note: `COMPRESSION` is supported with both types. |
147+
| `DIM` | The width, or number of dimensions, of the vector embeddings stored in this field. In other words, the number of floating point elements comprising the vector. `DIM` must be a positive integer. The vector used to query this field must have the exact dimensions as the field itself. |
148+
| `DISTANCE_METRIC` | Distance metric (`L2`, `IP`, `COSINE`). |
149+
150+
**Optional attributes**
151+
152+
`SVS-VAMANA` supports a number of additional parameters to tune the accuracy of the queries.
153+
154+
| Attribute | Description |
155+
|:---------------------------|:-----------------------------------------|
156+
| `COMPRESSION` | Compression algorithm (`LVQ8`, `LVQ8`, `LVQ4`, `LVQ4x4`, `LVQ4x8`, `LeanVec4x8`, or `LeanVec8x8`). Vectors will be compressed during indexing. Note: On non-Intel platforms, `SVS-VAMANA` with `COMPRESSION` will fall back to Intel’s basic quantization implementation. |
157+
| `CONSTRUCTION_WINDOW_SIZE` | The search window size (the default is 200) to use during graph construction. A higher search window size will yield a higher quality graph since more overall vertexes are considered, but will increase construction time. |
158+
| `GRAPH_MAX_DEGREE` | The maximum node degree in the graph (the default is 32). A higher max degree may yield a higher quality graph in terms of recall for performance, but the memory footprint of the graph is directly proportional to the maximum degree. |
159+
| `SEARCH_WINDOW_SIZE` | The size of the search window size (the default is 10). Increasing the search window size and capacity generally yields more accurate but slower search results. |
160+
| `EPSILON` | The range search approximation factor (default is 0.01). |
161+
| `TRAINING_THRESHOLD` | The number of vectors after which training is triggered. Applicable only when used with `COMPRESSION`. The default value is `10 * DEFAULT_BLOCK_SIZE` or, if provided, the value is limited to `100 * DEFAULT_BLOCK_SIZE`, where `DEFAULT_BLOCK_SIZE` is 1024. |
162+
| `LEANVEC_DIM` | The dimension used when using `LeanVec4x8` or `LeanVec8x8` compression for dimensionality reduction. The default value is `DIM / 2`. If provided, the value should be less than `DIM`. |
163+
164+
**Example**
165+
166+
```
167+
FT.CREATE documents
168+
ON HASH
169+
PREFIX 1 docs:
170+
SCHEMA doc_embedding VECTOR SVS-VAMANA 12
171+
TYPE FLOAT32
172+
DIM 1536
173+
DISTANCE_METRIC COSINE
174+
GRAPH_MAX_DEGREE 40
175+
CONSTRUCTION_WINDOW_SIZE 250
176+
COMPRESSION LVQ8
177+
```
178+
179+
In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `SVS-VAMANA` vector field named `doc_embedding` with six index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `GRAPH_MAX_DEGREE`, `CONSTRUCTION_WINDOW_SIZE` and `COMPRESSION`.
180+
132181
### Distance metrics
133182

134183
Redis supports three popular distance metrics to measure the degree of similarity between two vectors $u$, $v$ $\in \mathbb{R}^n$, where $n$ is the length of the vectors:
@@ -378,6 +427,17 @@ Optional runtime parameters for HNSW indexes are:
378427
| `EF_RUNTIME` | The maximum number of top candidates to hold during the KNN search. Higher values lead to more accurate results at the expense of a longer query runtime. | The value passed during index creation. The default is 10. |
379428
| `EPSILON` | The relative factor that sets the boundaries for a vector range query. Vector candidates whose distance from the query vector is `radius * (1 + EPSILON)` are potentially scanned, allowing a more extensive search and more accurate results at the expense of runtime. | The value passed during index creation. The default is 0.01. |
380429

430+
**SVS-VAMANA**
431+
432+
Optional runtime parameters for SYS-VAMANA indexes are:
433+
434+
| Parameter | Description | Default value |
435+
|:----------|:------------|:--------------|
436+
| `SEARCH_WINDOW_SIZE` | The size of the search window size (applies only to KNN searches). | 10 or the value that was passed upon index creation. |
437+
| `GRAPH_MAX_DEGREE` | The maximum node degree in the graph. | 32 or the value that was passed upon index creation. |
438+
| `SEARCH_WINDOW_SIZE` | The size of the search window size. | 10 or the value that was passed upon index creation. |
439+
| `EPSILON` | The range search approximation factor. | 0.01 or the value that was passed upon index creation. |
440+
381441

382442
### Important notes
383443

@@ -526,6 +586,6 @@ Here are some additional resources that apply vector search for different use ca
526586
- [Retrieval augmented generation from scratch](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/01_redisvl.ipynb)
527587
- [Semantic caching](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/semantic_caching_gemini.ipynb)
528588

529-
## Continue learning with Redis University
589+
<!-- ## Continue learning with Redis University
530590
531-
{{< university-links >}}
591+
{{< university-links >}}-->

0 commit comments

Comments
 (0)