Skip to content

Commit c41573f

Browse files
q-andykolchfa-awsnatebower
authored
Add neural search stats API docs (#9624)
* Add neural search stats API docs Signed-off-by: Andy Qin <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * Apply suggestions from code review Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> --------- Signed-off-by: Andy Qin <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: kolchfa-aws <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: kolchfa-aws <[email protected]> Co-authored-by: Nathan Bower <[email protected]>
1 parent e4b1ca5 commit c41573f

File tree

1 file changed

+218
-0
lines changed

1 file changed

+218
-0
lines changed

‎_vector-search/api/neural.md

+218
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,218 @@
1+
---
2+
layout: default
3+
title: Neural Search API
4+
parent: Vector search API
5+
nav_order: 20
6+
has_children: false
7+
---
8+
9+
# Neural Search API
10+
11+
The Neural Search plugin provides several APIs for monitoring semantic and hybrid search features.
12+
13+
## Stats
14+
15+
The Neural Search Stats API provides information about the current status of the Neural Search plugin. This includes both cluster-level and node-level statistics. Cluster-level statistics have a single value for the entire cluster. Node-level statistics have a single value for each node in the cluster.
16+
17+
By default, the Neural Search Stats API is disabled through a cluster setting. To enable statistics collection, use the following command:
18+
19+
```json
20+
PUT /_cluster/settings
21+
{
22+
"persistent": {
23+
"plugins.neural_search.stats_enabled": "true"
24+
}
25+
}
26+
```
27+
{% include copy-curl.html %}
28+
29+
To disable statistics collection, set the cluster setting to `false`. When disabled, all values are reset and new statistics are not collected.
30+
31+
### Endpoints
32+
33+
```json
34+
GET /_plugins/_neural/stats
35+
GET /_plugins/_neural/stats/<stats>
36+
GET /_plugins/_neural/<nodes>/stats
37+
GET /_plugins/_neural/<nodes>/stats/<stats>
38+
```
39+
40+
### Path parameters
41+
42+
The following table lists the available path parameters. All path parameters are optional.
43+
44+
| Parameter | Data type | Description |
45+
| :--- | :--- | :--- |
46+
| `nodes` | String | A node or a list of nodes (comma-separated) to filter statistics by. Default is all nodes. |
47+
| `stats` | String | A statistic name or names (comma-separated) to return. Default is all statistics. |
48+
49+
### Query parameters
50+
51+
The following table lists the available query parameters. All query parameters are optional.
52+
53+
| Parameter | Data type | Description |
54+
| :--- | :--- | :--- |
55+
| `include_metadata` | Boolean | When `true`, includes additional metadata fields for each statistic (see [Available metadata](#available-metadata)). Default is `false`. |
56+
| `flat_stat_paths` | Boolean | When `true`, flattens the JSON response structure for easier parsing. Default is `false`. |
57+
58+
#### Example request
59+
60+
```json
61+
GET /_plugins/_neural/node1,node2/stats/stat1,stat2?include_metadata=true,flat_stat_paths=true
62+
```
63+
{% include copy-curl.html %}
64+
65+
#### Example response
66+
67+
If `include_metadata` is `false`, the response appears as follows:
68+
69+
```json
70+
{
71+
"_nodes": {
72+
"total": 1,
73+
"successful": 1,
74+
"failed": 0
75+
},
76+
"cluster_name": "integTest",
77+
"info": {
78+
"cluster_version": "3.0.0",
79+
"processors": {
80+
"ingest": {
81+
"text_embedding_processors_in_pipelines": 0
82+
}
83+
}
84+
},
85+
"all_nodes": {
86+
"processors": {
87+
"ingest": {
88+
"text_embedding_executions": 0
89+
}
90+
}
91+
},
92+
"nodes": {
93+
"896MIkjCSnWBHy-SxB62zQ": {
94+
"processors": {
95+
"ingest": {
96+
"text_embedding_executions": 0
97+
}
98+
}
99+
}
100+
}
101+
}
102+
```
103+
104+
If `include_metadata` is `true`, the response appears as follows:
105+
106+
```json
107+
{
108+
"_nodes": {
109+
"total": 1,
110+
"successful": 1,
111+
"failed": 0
112+
},
113+
"cluster_name": "integTest",
114+
"info": {
115+
"cluster_version": {
116+
"value": "3.0.0",
117+
"stat_type": "info_string"
118+
},
119+
"processors": {
120+
"ingest": {
121+
"text_embedding_processors_in_pipelines": {
122+
"value": 0,
123+
"stat_type": "info_counter"
124+
}
125+
}
126+
}
127+
},
128+
"all_nodes": {
129+
"processors": {
130+
"ingest": {
131+
"text_embedding_executions": {
132+
"value": 0,
133+
"stat_type": "timestamped_event_counter",
134+
"trailing_interval_value": 0,
135+
"minutes_since_last_event": 29061801
136+
}
137+
}
138+
}
139+
},
140+
"nodes": {
141+
"896MIkjCSnWBHy-SxB62zQ": {
142+
"processors": {
143+
"ingest": {
144+
"text_embedding_executions": {
145+
"value": 0,
146+
"stat_type": "timestamped_event_counter",
147+
"trailing_interval_value": 0,
148+
"minutes_since_last_event": 29061801
149+
}
150+
}
151+
}
152+
}
153+
}
154+
}
155+
```
156+
157+
### Response body fields
158+
159+
The following sections describe response body fields.
160+
161+
#### Categories of statistics
162+
163+
The following table lists all categories of statistics.
164+
165+
| Category | Data type | Description |
166+
| :--- | :--- | :--- |
167+
| `info` | Object | Contains cluster-wide information and statistics that are not specific to individual nodes. |
168+
| `all_nodes` | Object | Provides aggregated statistics across all nodes in the cluster. |
169+
| `nodes` | Object | Contains node-specific statistics, with each node identified by its unique node ID. |
170+
171+
#### Available statistics
172+
173+
The following table lists the available statistics. For statistics with paths prefixed with `nodes.<node_id>`, aggregate cluster-level statistics are also available at the same path prefixed with `all_nodes`.
174+
175+
| Statistic name | Category | Statistic path within category | Description |
176+
| :--- | :--- | :--- | :--- |
177+
| `cluster_version` | `info` | `cluster_version` | The version of the cluster. |
178+
| `text_embedding_processors_in_pipelines` | `info` | `processors.ingest.text_embedding_processors_in_pipelines` | The number of text embedding processors configured in ingest pipelines. |
179+
| `text_embedding_executions` | `nodes`, `all_nodes` | `processors.ingest.text_embedding_executions` | The number of text embedding processor executions per node. |
180+
181+
#### Available metadata
182+
183+
When `include_metadata` is `true`, the field values in the response are replaced by their respective metadata objects, which include additional information about the statistic types, as described in the following table.
184+
185+
| Statistic type | Description |
186+
| :--- | :--- |
187+
| `info_string` | A basic string value that provides informational content, such as versions or names. See [`info_string`](#info-string).|
188+
| `info_counter` | A numerical counter that represents static or slowly changing values. See [`info_counter`](#info-counter).|
189+
| `timestamped_event_counter` | A counter that tracks events over time, including information about recent activity. See [`timestamped_event_counter`](#timestamped-event-counter).|
190+
191+
<p id="info-string"></p>
192+
193+
The `info_string` object contains the following metadata fields.
194+
195+
| Metadata field | Data type | Description |
196+
| :--- | :--- | :--- |
197+
| `value` | String | The actual string value of the statistic. |
198+
| `stat_type` | String | Always set to `info_string`. |
199+
200+
<p id="info-counter"></p>
201+
202+
The `info_counter` object contains the following metadata fields.
203+
204+
| Metadata field | Data type | Description |
205+
| :--- | :--- | :--- |
206+
| `value` | Integer | The current count value. |
207+
| `stat_type` | String | Always set to `info_counter`. |
208+
209+
<p id="timestamped-event-counter"></p>
210+
211+
The `timestamped_event_counter` object contains the following metadata fields.
212+
213+
| Metadata field | Data type | Description |
214+
| :--- | :--- | :--- |
215+
| `value` | Integer | The total number of events that occurred since the node started. |
216+
| `stat_type` | String | Always set to `timestamped_event_counter`. |
217+
| `trailing_interval_value` | Integer | The number of events that occurred in the past 5 minutes. |
218+
| `minutes_since_last_event` | Integer | The amount of time (in minutes) since the last recorded event. |

0 commit comments

Comments
 (0)