Skip to content

DOC-5457: add the new SVS-VAMANA vector algorithm type #1822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jul 9, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,7 @@ Where:

- `FLAT`: brute force algorithm.
- `HNSW`: hierarchical, navigable, small world algorithm.
- `SVS-VAMANA`: a graph-based nearest neighbor search algorithm, which is optimized for use with compression methods to reduce its memory footprint.

The `{algorithm}` attribute specifies the algorithm to use when searching `k` most similar vectors in the index or filtering vectors by range.

Expand Down
77 changes: 70 additions & 7 deletions content/develop/ai/search-and-query/vectors.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ To quickly get started, check out the [Redis vector quickstart guide]({{< relref

## Overview

1. [**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}) and [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) vector index types.
1. [**Create a vector index**]({{< relref "develop/ai/search-and-query/vectors#create-a-vector-index" >}}): Redis maintains a secondary index over your data with a defined schema (including vector fields and metadata). Redis supports [`FLAT`]({{< relref "develop/ai/search-and-query/vectors#flat-index" >}}), [`HNSW`]({{< relref "develop/ai/search-and-query/vectors#hnsw-index" >}}) and [`SVS-VAMANA`]({{< relref "develop/ai/search-and-query/vectors#svs-vamana-index" >}}) vector index types.
1. [**Store and update vectors**]({{< relref "develop/ai/search-and-query/vectors#store-and-update-vectors" >}}): Redis stores vectors and metadata in hashes or JSON objects.
1. [**Search with vectors**]({{< relref "develop/ai/search-and-query/vectors#search-with-vectors" >}}): Redis supports several advanced querying strategies with vector fields including k-nearest neighbor ([KNN]({{< relref "develop/ai/search-and-query/vectors#knn-vector-search" >}})), [vector range queries]({{< relref "develop/ai/search-and-query/vectors#vector-range-queries" >}}), and [metadata filters]({{< relref "develop/ai/search-and-query/vectors#filters" >}}).
1. [**Configure vector queries at runtime**]({{< relref "develop/ai/search-and-query/vectors#runtime-query-params" >}}).
Expand Down Expand Up @@ -106,12 +106,12 @@ Choose the `HNSW` index type when you have larger datasets (> 1M documents) or w
[`HNSW`](https://arxiv.org/ftp/arxiv/papers/1603/1603.09320.pdf) supports a number of additional parameters to tune
the accuracy of the queries, while trading off performance.

| Attribute | Description |
|:-------------------|:--------------------------------------------------------------------------------------------|
| `M` | Max number of outgoing edges (connections) for each node in a graph layer. On layer zero, the max number of connections will be `2 * M`. Higher values increase accuracy, but also increase memory usage and index build time. The default is 16. |
| `EF_CONSTRUCTION` | Max number of connected neighbors to consider during graph building. Higher values increase accuracy, but also increase index build time. The default is 200. |
| `EF_RUNTIME` | Max top candidates during KNN search. Higher values increase accuracy, but also increase search latency. The default is 10. |
| `EPSILON` | Relative factor that sets the boundaries in which a range query may search for candidates. That is, vector candidates whose distance from the query vector is `radius * (1 + EPSILON)` are potentially scanned, allowing more extensive search and more accurate results, at the expense of run time. The default is 0.01. |
| Attribute | Description | Default value |
|:-------------------|:-------------------------------------------------------------------|:-------------:|
| `M` | Max number of outgoing edges (connections) for each node in a graph layer. On layer zero, the max number of connections will be `2 * M`. Higher values increase accuracy, but also increase memory usage and index build time. | 16 |
| `EF_CONSTRUCTION` | Max number of connected neighbors to consider during graph building. Higher values increase accuracy, but also increase index build time. | 200 |
| `EF_RUNTIME` | Max top candidates during KNN search. Higher values increase accuracy, but also increase search latency. | 10 |
| `EPSILON` | Relative factor that sets the boundaries in which a range query may search for candidates. That is, vector candidates whose distance from the query vector is `radius * (1 + EPSILON)` are potentially scanned, allowing more extensive search and more accurate results, at the expense of run time. | 0.01 |

**Example**

Expand All @@ -129,6 +129,58 @@ FT.CREATE documents

In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `HNSW` vector field named `doc_embedding` with five index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `M`, and `EF_CONSTRUCTION`.

### SVS-VAMANA index

Scalable Vector Search (SVS) is an Intel project featuring a graph-based vector search algorithm that is optimized to work with compression methods to reduce memory usage. You can read more about the project [here](https://intel.github.io/ScalableVectorSearch/intro.html). Support for SVS-VAMANA indexing was added in Redis 8.2.

Choose the `SVS-VAMANA` index type when all of the following requirements apply:
- High search performance and scalability are more important than exact search accuracy (similar to HNSW)
- Reduced memory usage
- Performance optimizations for Intel hardware

**Required attributes**

| Attribute | Description |
|:-------------------|:-----------------------------------------|
| `TYPE` | Vector type (`FLOAT16` or `FLOAT32`). Note: `COMPRESSION` is supported with both types. |
| `DIM` | The width, or number of dimensions, of the vector embeddings stored in this field. In other words, the number of floating point elements comprising the vector. `DIM` must be a positive integer. The vector used to query this field must have the exact same dimensions as the field itself. |
| `DISTANCE_METRIC` | Distance metric (`L2`, `IP`, `COSINE`). |

**Optional attributes**

`SVS-VAMANA` supports a number of additional parameters to tune the accuracy of the queries.

| Attribute | Description | Default value |
|:---------------------------|:-----------------------------------------|:-------------:|
| `COMPRESSION` | Compression algorithm (`LVQ8`, `LVQ4`, `LVQ4x4`, `LVQ4x8`, `LeanVec4x8`, or `LeanVec8x8`). Vectors will be compressed during indexing. See these Intel pages for best practices on using these algorithms: [`COMPRESSION` settings](https://intel.github.io/ScalableVectorSearch/howtos.html#compression-setting) and [`LeanVec`](https://intel.github.io/ScalableVectorSearch/python/experimental/leanvec.html). | None |
| `CONSTRUCTION_WINDOW_SIZE` | The search window size to use during graph construction. A higher search window size will yield a higher quality graph since more overall vertexes are considered, but will increase construction time. | 200 |
| `GRAPH_MAX_DEGREE` | The maximum node degree in the graph. A higher max degree may yield a higher quality graph in terms of recall for performance, but the memory footprint of the graph is directly proportional to the maximum degree. | 32 |
| `SEARCH_WINDOW_SIZE` | The size of the search window. Increasing the search window size and capacity generally yields more accurate but slower search results. | 10 |
| `EPSILON` | The range search approximation factor. | 0.01 |
| `TRAINING_THRESHOLD` | The number of vectors after which training is triggered. Applicable only when used with `COMPRESSION`. If a value is provided, it be less than `100 * DEFAULT_BLOCK_SIZE`, where `DEFAULT_BLOCK_SIZE` is 1024. | `10 * DEFAULT_BLOCK_SIZE` |
| `LEANVEC_DIM` | The dimension used when using `LeanVec4x8` or `LeanVec8x8` compression for dimensionality reduction. If a value is provided, it should be less than `DIM`. | `DIM / 2` |

{{< warning >}}
On non-Intel platforms, `SVS-VAMANA` with `COMPRESSION` will fall back to Intel’s basic scalar quantization implementation.
{{< /warning >}}

**Example**

```
FT.CREATE documents
ON HASH
PREFIX 1 docs:
SCHEMA doc_embedding VECTOR SVS-VAMANA 12
TYPE FLOAT32
DIM 1536
DISTANCE_METRIC COSINE
GRAPH_MAX_DEGREE 40
CONSTRUCTION_WINDOW_SIZE 250
COMPRESSION LVQ8
```

In the example above, an index named `documents` is created over hashes with the key prefix `docs:` and an `SVS-VAMANA` vector field named `doc_embedding` with six index attributes: `TYPE`, `DIM`, `DISTANCE_METRIC`, `GRAPH_MAX_DEGREE`, `CONSTRUCTION_WINDOW_SIZE` and `COMPRESSION`.

### Distance metrics

Redis supports three popular distance metrics to measure the degree of similarity between two vectors $u$, $v$ $\in \mathbb{R}^n$, where $n$ is the length of the vectors:
Expand Down Expand Up @@ -378,6 +430,17 @@ Optional runtime parameters for HNSW indexes are:
| `EF_RUNTIME` | The maximum number of top candidates to hold during the KNN search. Higher values lead to more accurate results at the expense of a longer query runtime. | The value passed during index creation. The default is 10. |
| `EPSILON` | The relative factor that sets the boundaries for a vector range query. Vector candidates whose distance from the query vector is `radius * (1 + EPSILON)` are potentially scanned, allowing a more extensive search and more accurate results at the expense of runtime. | The value passed during index creation. The default is 0.01. |

**SVS-VAMANA**

Optional runtime parameters for SVS-VAMANA indexes are:

| Parameter | Description | Default value |
|:----------|:------------|:--------------|
| `SEARCH_WINDOW_SIZE` | The size of the search window (applies only to KNN searches). | 10 or the value that was passed upon index creation. |
| `EPSILON` | The range search approximation factor. | 0.01 or the value that was passed upon index creation. |
| `USE_SEARCH_HISTORY` | When building an index, either the contents of the search buffer is used or the entire search history is used. The latter case may yield a slightly better graph at the cost of more search time. Boolean options are `OFF`, `ON`, and `AUTO`. `AUTO` is always evaluated internally as `ON`. | `AUTO` |
| `SEARCH_BUFFER_CAPACITY` | A tuning parameter used in compressed `SVS-VAMANA` indexes, which are using a two-level compression type (`LVQ<X>x<Y>` or one of the `LeanVec` types), that determines the number of vector candidates to collect in the first level of the search, before the re-ranking level (which is the second level). | `SEARCH_WINDOW_SIZE` |


### Important notes

Expand Down
4 changes: 2 additions & 2 deletions static/resources/redisuniversity.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,6 @@ links:
text: See the [Get started with Redis Software learning path](https://university.redis.io/learningpath/an0mgw5bjpjfbe) for courses.

# Redis Develop links
- page: /develop/interact/search-and-query/advanced-concepts/vectors
- page: /develop/ai/search-and-query/vectors
text: See the [Introduction to vector search course](https://university.redis.io/course/yz1lretjfpdlew?tab=details) to learn more.