Merge branch 'main' into preview-rc-rdi

Author: cmilesb

config.toml

@@ -55,7 +55,7 @@ rdi_redis_gears_version = "1.2.6"
 rdi_debezium_server_version = "2.3.0.Final"
 rdi_db_types = "cassandra|mysql|oracle|postgresql|sqlserver"
 rdi_cli_latest = "latest"
-rdi_current_version = "v1.6.5"
+rdi_current_version = "v1.6.6"
 
 [params.clientsConfig]
 "Python"={quickstartSlug="redis-py"}

content/commands/json.arrappend/index.md

@@ -67,7 +67,7 @@ is JSONPath to specify. Default is root `$`.
 
 ## Return value 
 
-`JSON.ARRAPEND` returns an [array]({{< relref "develop/reference/protocol-spec#resp-arrays" >}}) of integer replies for each path, the array's new size, or `nil`, if the matching JSON value is not an array. 
+`JSON.ARRAPPEND` returns an [array]({{< relref "develop/reference/protocol-spec#resp-arrays" >}}) of integer replies for each path, the array's new size, or `nil`, if the matching JSON value is not an array. 
 For more information about replies, see [Redis serialization protocol specification]({{< relref "/develop/reference/protocol-spec" >}}). 
 
 ## Examples
@@ -82,7 +82,7 @@ redis> JSON.SET item:1 $ '{"name":"Noise-cancelling Bluetooth headphones","descr
 OK
 {{< / highlight >}}
 
-Add color `blue` to the end of the `colors` array. `JSON.ARRAPEND` returns the array's new size.
+Add color `blue` to the end of the `colors` array. `JSON.ARRAPPEND` returns the array's new size.
 
 {{< highlight bash >}}
 redis> JSON.ARRAPPEND item:1 $.colors '"blue"'

content/commands/json.debug-memory/index.md

@@ -72,7 +72,7 @@ Get the values' memory usage in bytes.
 
 {{< highlight bash >}}
 redis> JSON.DEBUG MEMORY item:2
-(integer) 253
+(integer) 573
 {{< / highlight >}}
 </details>
 
@@ -84,4 +84,3 @@ redis> JSON.DEBUG MEMORY item:2
 
 * [RedisJSON]({{< relref "/develop/data-types/json/" >}})
 * [Index and search JSON documents]({{< relref "/develop/interact/search-and-query/indexing/" >}})
-

content/develop/ai/index.md

@@ -34,6 +34,13 @@ This page organized into a few sections depending on what you’re trying to do:
 1. [**Search with vectors**]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#search-with-vectors" >}}): Redis supports several advanced querying strategies with vector fields including k-nearest neighbor ([KNN]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#knn-vector-search" >}})), [vector range queries]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#vector-range-queries" >}}), and [metadata filters]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#filters" >}}).
 1. [**Configure vector queries at runtime**]({{< relref "develop/interact/search-and-query/advanced-concepts/vectors#runtime-query-parameters" >}}). Select the best filter mode to optimize query execution.
 
+#### Learn how to index and query vector embeddings
+* [redis-py (Python)]({{< relref "/develop/clients/redis-py/vecsearch" >}})
+* [NRedisStack (C#/.NET)]({{< relref "/develop/clients/dotnet/vecsearch" >}})
+* [node-redis (JavaScript)]({{< relref "/develop/clients/nodejs/vecsearch" >}}) 
+* [Jedis (Java)]({{< relref "/develop/clients/jedis/vecsearch" >}})
+* [go-redis (Go)]({{< relref "/develop/clients/go/vecsearch" >}}) 
+
 ## Concepts 
 
 Learn to perform vector search and use gateways and semantic caching in your AI/ML projects.
@@ -44,7 +51,7 @@ Learn to perform vector search and use gateways and semantic caching in your AI/
 
 ## Quickstarts
 
-Quickstarts or recipes are useful when you are trying to build specific functionality. For example, you might want to do RAG with LangChain or set up LLM memory for you AI agent. Get started with the following Redis Python notebooks.
+Quickstarts or recipes are useful when you are trying to build specific functionality. For example, you might want to do RAG with LangChain or set up LLM memory for your AI agent. Get started with the following Redis Python notebooks.
 
 * [The place to start if you are brand new to Redis](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/redis-intro/00_redis_intro.ipynb)
 
@@ -53,6 +60,7 @@ Vector search retrieves results based on the similarity of high-dimensional nume
 * [Implementing hybrid search with Redis](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/02_hybrid_search.ipynb)
 * [Vector search with Redis Python client](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/00_redispy.ipynb) 
 * [Vector search with Redis Vector Library](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/01_redisvl.ipynb)
+* [Shows how to convert a float 32 index to float16 or integer data types](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/03_dtype_support.ipynb)
 
 #### RAG  
 Retrieval Augmented Generation (aka RAG) is a technique to enhance the ability of an LLM to respond to user queries. The retrieval part of RAG is supported by a vector database, which can return semantically relevant results to a user’s query, serving as contextual information to augment the generative capabilities of an LLM.
@@ -65,12 +73,15 @@ Retrieval Augmented Generation (aka RAG) is a technique to enhance the ability o
 * [Vector search with Azure](https://techcommunity.microsoft.com/blog/azuredevcommunityblog/vector-similarity-search-with-azure-cache-for-redis-enterprise/3822059)
 * [RAG with Spring AI](https://redis.io/blog/building-a-rag-application-with-redis-and-spring-ai/)
 * [RAG with Vertex AI](https://github.com/redis-developer/gcp-redis-llm-stack/tree/main)
-* [Notebook for additional tips and techniques to improve RAG quality](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/04_advanced_redisvl.ipynb) 
+* [Notebook for additional tips and techniques to improve RAG quality](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/04_advanced_redisvl.ipynb)
+* [Implement a simple RBAC policy with vector search using Redis](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/RAG/07_user_role_based_rag.ipynb) 
 
 #### Agents
 AI agents can act autonomously to plan and execute tasks for the user.
+* [Redis Notebooks for LangGraph](https://github.com/redis-developer/langgraph-redis/tree/main/examples)
 * [Notebook to get started with LangGraph and agents](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/00_langgraph_redis_agentic_rag.ipynb) 
 * [Build a collaborative movie recommendation system using Redis for data storage, CrewAI for agent-based task execution, and LangGraph for workflow management.](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/01_crewai_langgraph_redis.ipynb)
+* [Full-Featured Agent Architecture](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/agents/02_full_featured_agent.ipynb)
 
 #### LLM memory
 LLMs are stateless. To maintain context within a conversation chat sessions must be stored and resent to the LLM. Redis manages the storage and retrieval of chat sessions to maintain context and conversational relevance.
@@ -81,14 +92,24 @@ LLMs are stateless. To maintain context within a conversation chat sessions must
 An estimated 31% of LLM queries are potentially redundant. Redis enables semantic caching to help cut down on LLM costs quickly.
 * [Build a semantic cache using the Doc2Cache framework and Llama3.1](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/doc2cache_llama3_1.ipynb) 
 * [Build a semantic cache with Redis and Google Gemini](https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/semantic_caching_gemini.ipynb)
+* [Optimize semantic cache threshold with RedisVL](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/02_semantic_cache_optimization.ipynb)
+
+#### Semantic routing  
+Routing is a simple and effective way of preventing misuses with your AI application or for creating branching logic between data sources etc.
+* [Simple examples of how to build an allow/block list router in addition to a multi-topic router](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-router/00_semantic_routing.ipynb) 
+* [Use RouterThresholdOptimizer from redisvl to setup best router config](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-router/01_routing_optimization.ipynb)
 
 #### Computer vision
 Build a facial recognition system using the Facenet embedding model and RedisVL.
 * [Facial recognition](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/computer-vision/00_facial_recognition_facenet.ipynb)
 
 #### Recommendation systems
 * [Intro content filtering example with redisvl](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/00_content_filtering.ipynb) 
-* [Intro collaborative filtering example with redisvl](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/01_collaborative_filtering.ipynb) 
+* [Intro collaborative filtering example with redisvl](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/01_collaborative_filtering.ipynb)
+* [Intro deep learning two tower example with redisvl](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/recommendation-systems/02_two_towers.ipynb)
+
+#### Feature store
+* [Credit scoring system using Feast with Redis as the online store](https://github.com/redis-developer/redis-ai-resources/blob/main/python-recipes/feature-store/00_feast_credit_score.ipynb)
 
 ## Tutorials
 Need a deeper-dive through different use cases and topics?
@@ -97,13 +118,16 @@ Need a deeper-dive through different use cases and topics?
 * [Agentic RAG](https://github.com/redis-developer/agentic-rag) - A tutorial focused on agentic RAG with LlamaIndex and Amazon Bedrock
 * [RAG on Vertex AI](https://github.com/redis-developer/gcp-redis-llm-stack/tree/main) - A RAG tutorial featuring Redis with Vertex AI
 * [RAG workbench](https://github.com/redis-developer/redis-rag-workbench) - A development playground for exploring RAG techniques with Redis
+* [ArXiv Chat](https://github.com/redis-developer/ArxivChatGuru) - Streamlit demo of RAG over ArXiv documents with Redis & OpenAI
 
-#### Recommendation system
+#### Recommendations and search
 * [Recommendation systems w/ NVIDIA Merlin & Redis](https://github.com/redis-developer/redis-nvidia-recsys) - Three examples, each escalating in complexity, showcasing the process of building a realtime recsys with NVIDIA and Redis
 * [Redis product search](https://github.com/redis-developer/redis-product-search) - Build a real-time product search engine using features like full-text search, vector similarity, and real-time data updates
+* [ArXiv Search](https://github.com/redis-developer/redis-arxiv-search) - Full stack implementation of Redis with React FE
 
 ## Ecosystem integrations
 
+* [LangGraph & Redis: Build smarter AI agents with memory & persistence](https://redis.io/blog/langgraph-redis-build-smarter-ai-agents-with-memory-persistence/)
 * [Amazon Bedrock setup guide]({{< relref "/integrate/amazon-bedrock/set-up-redis" >}})
 * [LangChain Redis Package: Smarter AI apps with advanced vector storage and faster caching](https://redis.io/blog/langchain-redis-partner-package/)
 * [LlamaIndex integration for Redis as a vector store](https://gpt-index.readthedocs.io/en/latest/examples/vector_stores/RedisIndexDemo.html)
@@ -113,6 +137,8 @@ Need a deeper-dive through different use cases and topics?
 * [Deploy GenAI apps faster with Redis and NVIDIA NIM](https://redis.io/blog/use-redis-with-nvidia-nim-to-deploy-genai-apps-faster/)
 * [Building LLM Applications with Kernel Memory and Redis](https://redis.io/blog/building-llm-applications-with-kernel-memory-and-redis/)
 * [DocArray integration of Redis as a vector database by Jina AI](https://docs.docarray.org/user_guide/storing/index_redis/)
+* [Semantic Kernel: A popular library by Microsoft to integrate LLMs with plugins](https://learn.microsoft.com/en-us/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/redis-connector?pivots=programming-language-csharp)
+* [LiteLLM integration](https://docs.litellm.ai/docs/caching/all_caches#initialize-cache---in-memory-redis-s3-bucket-redis-semantic-disk-cache-qdrant-semantic)
 
 ## Benchmarks
 See how we stack up against the competition.

content/develop/clients/dotnet/vecsearch.md

@@ -32,7 +32,9 @@ In the example below, we use [Microsoft.ML](https://dotnet.microsoft.com/en-us/a
 to generate the vector embeddings to store and index with Redis Query Engine.
 We also show how to adapt the code to use
 [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=csharp)
-for the embeddings.
+for the embeddings. The code is first demonstrated for hash documents with a
+separate section to explain the
+[differences with JSON documents](#differences-with-json-documents).
 
 ## Initialize
 
@@ -89,7 +91,6 @@ using Azure;
 using Azure.AI.OpenAI;
 ```
 
-
 ## Define a function to obtain the embedding model
 
 {{< note >}}Ignore this step if you are using an Azure OpenAI
@@ -154,7 +155,9 @@ array as a `byte` string. To simplify this, we declare a
 then encodes the returned `float` array as a `byte` string. If you are
 storing your documents as JSON objects instead of hashes, then you should
 use the `float` array for the embedding directly, without first converting
-it to a `byte` string.
+it to a `byte` string (see [Differences with JSON documents](#differences-with-json-documents)
+below).
+
 
 ```csharp
 static byte[] GetEmbedding(
@@ -414,6 +417,156 @@ As you would expect, the result for `doc:1` with the content text
 is the result that is most similar in meaning to the query text
 *"That is a happy person"*.
 
+## Differences with JSON documents
+
+Indexing JSON documents is similar to hash indexing, but there are some
+important differences. JSON allows much richer data modeling with nested fields, so
+you must supply a [path]({{< relref "/develop/data-types/json/path" >}}) in the schema
+to identify each field you want to index. However, you can declare a short alias for each
+of these paths to avoid typing it in full for
+every query. Also, you must specify `IndexType.JSON` with the `On()` option when you
+create the index.
+
+The code below shows these differences, but the index is otherwise very similar to
+the one created previously for hashes:
+
+```cs
+var jsonSchema = new Schema()
+    .AddTextField(new FieldName("$.content", "content"))
+    .AddTagField(new FieldName("$.genre", "genre"))
+    .AddVectorField(
+        new FieldName("$.embedding", "embedding"),
+        VectorField.VectorAlgo.HNSW,
+        new Dictionary<string, object>()
+        {
+            ["TYPE"] = "FLOAT32",
+            ["DIM"] = "150",
+            ["DISTANCE_METRIC"] = "L2"
+        }
+    );
+
+
+db.FT().Create(
+    "vector_json_idx",
+    new FTCreateParams()
+        .On(IndexDataType.JSON)
+        .Prefix("jdoc:"),
+    jsonSchema
+);
+```
+
+An important difference with JSON indexing is that the vectors are
+specified using arrays of `float` instead of binary strings. This requires a modification
+to the `GetEmbedding()` function declared in
+[Define a function to generate an embedding](#define-a-function-to-generate-an-embedding)
+above:
+
+```cs
+static float[] GetFloatEmbedding(
+    PredictionEngine<TextData, TransformedTextData> model, string sentence
+)
+{
+    // Call the prediction API to convert the text into embedding vector.
+    var data = new TextData()
+    {
+        Text = sentence
+    };
+
+    var prediction = model.Predict(data);
+
+    float[] floatArray = Array.ConvertAll(prediction.Features, x => (float)x);
+    return floatArray;
+}
+```
+
+You should make a similar modification to the `GetEmbeddingFromAzure()` function
+if you are using Azure OpenAI with JSON.
+
+Use [`JSON().set()`]({{< relref "/commands/json.set" >}}) to add the data
+instead of [`HashSet()`]({{< relref "/commands/hset" >}}):
+
+```cs
+var jSentence1 = "That is a very happy person";
+
+var jdoc1 = new {
+    content = jSentence1,
+    genre = "persons",
+    embedding = GetFloatEmbedding(predEngine, jSentence1),
+};
+
+db.JSON().Set("jdoc:1", "$", jdoc1);
+
+var jSentence2 = "That is a happy dog";
+
+var jdoc2 = new {
+    content = jSentence2,
+    genre = "pets",
+    embedding = GetFloatEmbedding(predEngine, jSentence2),
+};
+
+db.JSON().Set("jdoc:2", "$", jdoc2);
+
+var jSentence3 = "Today is a sunny day";
+
+var jdoc3 = new {
+    content = jSentence3,
+    genre = "weather",
+    embedding = GetFloatEmbedding(predEngine, jSentence3),
+};
+
+db.JSON().Set("jdoc:3", "$", jdoc3);
+```
+
+The query is almost identical to the one for the hash documents. This
+demonstrates how the right choice of aliases for the JSON paths can
+save you having to write complex queries. The only significant difference is
+that the `FieldName` objects created for the `ReturnFields()` option must
+include the JSON path for the field.
+
+An important thing to notice
+is that the vector parameter for the query is still specified as a
+binary string (using the `GetEmbedding()` method), even though the data for
+the `embedding` field of the JSON was specified as a `float` array.
+
+```cs
+var jRes = db.FT().Search("vector_json_idx",
+    new Query("*=>[KNN 3 @embedding $query_vec AS score]")
+    .AddParam("query_vec", GetEmbedding(predEngine, "That is a happy person"))
+    .ReturnFields(
+        new FieldName("$.content", "content"),
+        new FieldName("$.score", "score")
+    )
+    .SetSortBy("score")
+    .Dialect(2));
+
+foreach (var doc in jRes.Documents) {
+    var props = doc.GetProperties();
+    var propText = string.Join(
+        ", ",
+        props.Select(p => $"{p.Key}: '{p.Value}'")
+    );
+
+    Console.WriteLine(
+        $"ID: {doc.Id}, Properties: [\n  {propText}\n]"
+    );
+}
+```
+
+Apart from the `jdoc:` prefixes for the keys, the result from the JSON
+query is the same as for hash:
+
+```
+ID: jdoc:1, Properties: [
+  score: '4.30777168274', content: 'That is a very happy person'
+]
+ID: jdoc:2, Properties: [
+  score: '25.9752807617', content: 'That is a happy dog'
+]
+ID: jdoc:3, Properties: [
+  score: '68.8638000488', content: 'Today is a sunny day'
+]
+```
+
 ## Learn more
 
 See