File tree 2 files changed +5
-4
lines changed
2 files changed +5
-4
lines changed Original file line number Diff line number Diff line change @@ -501,15 +501,15 @@ Perform keyword search
501
501
502
502
``` ruby
503
503
query = " growling bear"
504
- keyword_results = Document .keyword_search(query).first (20 )
504
+ keyword_results = Document .keyword_search(query).limit (20 ).load_async
505
505
```
506
506
507
507
And semantic search (the query prefix is specific to the [ embedding model] ( https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1#mxbai-embed-large-v1 ) )
508
508
509
509
``` ruby
510
510
query_prefix = " Represent this sentence for searching relevant passages: "
511
511
query_embedding = embed.(query_prefix + query)
512
- semantic_results = Document .nearest_neighbors(:embedding , query_embedding, distance: " cosine" ).first (20 )
512
+ semantic_results = Document .nearest_neighbors(:embedding , query_embedding, distance: " cosine" ).limit (20 ).load_async
513
513
```
514
514
515
515
And rerank the results
Original file line number Diff line number Diff line change 2
2
require "informers"
3
3
require "neighbor"
4
4
5
+ ActiveRecord . async_query_executor = :global_thread_pool
5
6
ActiveRecord ::Base . establish_connection adapter : "postgresql" , database : "neighbor_test"
6
7
ActiveRecord ::Schema . verbose = false
7
8
ActiveRecord ::Schema . define do
@@ -39,12 +40,12 @@ class Document < ActiveRecord::Base
39
40
Document . insert_all! ( documents )
40
41
41
42
query = "growling bear"
42
- keyword_results = Document . keyword_search ( query ) . first ( 20 )
43
+ keyword_results = Document . keyword_search ( query ) . limit ( 20 ) . load_async
43
44
44
45
# the query prefix is specific to the embedding model (https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1)
45
46
query_prefix = "Represent this sentence for searching relevant passages: "
46
47
query_embedding = embed . ( query_prefix + query )
47
- semantic_results = Document . nearest_neighbors ( :embedding , query_embedding , distance : "cosine" ) . first ( 20 )
48
+ semantic_results = Document . nearest_neighbors ( :embedding , query_embedding , distance : "cosine" ) . limit ( 20 ) . load_async
48
49
49
50
results = ( keyword_results + semantic_results ) . uniq ( &:id )
50
51
p rerank . ( query , results . map ( &:content ) , top_k : 5 ) . map { |v | results [ v [ :doc_id ] ] } . map ( &:content )
You can’t perform that action at this time.
0 commit comments