You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
XADD key [NOMKSTREAM] [<MAXLEN | MINID> [= | ~] threshold [LIMIT count]] <* | id> field value [field value ...]
13
+
In Dragonfly, as well as in Redis and Valkey, the `XADD` command is used to append a new entry to a stream.
14
+
Streams are data structures that enable storing and processing ordered logs of events, and `XADD` is essential for adding data to these streams.
14
15
15
-
**Time complexity:** O(1) when adding a new entry, O(N) when trimming where N being the number of entries evicted
16
+
## Syntax
16
17
17
-
**ACL categories:**@write, @stream, @fast
18
+
```shell
19
+
XADD key [<MAXLEN | MINID> [~ |=] threshold] <*| id> field value [field value ...]
20
+
```
18
21
19
-
**XADD** command appends new entry to the specified key i.e. stream.
20
-
The command returns the *ID* of the new entry. User can either explicitly
21
-
specify the ID of the newly created entry or the command automatically
22
-
generates an ID.
22
+
## Parameter Explanations
23
23
24
-
Each entry can store multiple field-value pairs. The field-value pairs
25
-
are stored in the same order as the client specified. Commands that read
26
-
the stream, such as **XRANGE** or **XREAD**, are guaranteed to return the
27
-
fields and values exactly in the same order they were added by **XADD**.
24
+
-`key`: The key of the stream where the entry will be appended.
25
+
-`MAXLEN` (optional): A flag to cap the length of the stream to `threshold` items.
26
+
-`MINID` (optional): A flag to trim the stream to keep only entries with IDs greater than `threshold`.
27
+
- For `MAXLEN` or `MINID`, one of the following operators can be used:
28
+
- The `~` operator is used to trim approximately, which can be more efficient but may not be exact.
29
+
- The `=` operator is used to trim exactly to the specified threshold.
30
+
- In order to specify a unique ID for the entry in the stream, use one of the following options.
31
+
-`*`: Automatically generate an ID that includes a timestamp and a sequence number.
32
+
-`id`: A unique incremental ID for the entry specified by your application.
33
+
- Either way, the stored ID is a **string representing two 64-bit integers separated by a hyphen (`-`)**.
34
+
-`field value`: Pairs of field names and their corresponding values, forming the data within the entry.
28
35
29
-
Note that **XADD** is the only command that appends entries into a
30
-
stream. No other command has the ability to add new entries.
36
+
## Return Values
31
37
32
-
### Key
38
+
- The command returns the unique ID of the added stream entry.
39
+
- If the ID is automatically generated, the first part is a Unix timestamp in milliseconds, and the second part is an incrementing sequence number distinguishing entries with the same timestamp.
33
40
34
-
Key is the stream name to which the entry needs to be appended. If the
35
-
stream is not found, the command creates a new stream with the key and
36
-
appends the entry to it.
41
+
## Code Examples
37
42
38
-
### Specifying ID
43
+
### Basic Example
39
44
40
-
It is required to specify ID or a special character "**\***" before starting
41
-
the field-value pair list. An ID has two parts. These are -
42
-
***Unix time in milliseconds** - It is a 64 bit number that stores
43
-
unix time in milliseconds. When the entry is auto-generated (i.e. client
44
-
didn't specify an ID explicitly), the command uses the time of generation
45
-
to built the unique ID.
46
-
***Sequence** - 64 bit number. The sequence part of the ID helps distinguishing
47
-
two entries that were created at the same time.
45
+
Add an entry to a stream:
48
46
49
-
These two part are joined by the "**-**" character. So a complete ID has the following
50
-
pattern - *<time-in-ms\>-<sequence\>*. Below is an example -
51
47
```shell
52
-
1623910120014-1
48
+
dragonfly$> XADD mystream * sensor-id 1234 temperature 20.1
49
+
"1609459200001-0"
53
50
```
54
51
55
-
Clients can also specify incomplete IDs where the sequence part is
56
-
replaced with the **\*** character. In that case, the command generates
57
-
the id with greater sequence number than the previous entry having
58
-
the same unix time. For example, if the previously generated entry has
59
-
the id `1623910120014-1` and if the client specified `1623910120014-*`,
60
-
the new entry will have the id `1623910120014-2`.
52
+
### Limit Stream Length
53
+
54
+
Add an entry and trim the stream to keep only the latest 5 entries:
61
55
62
56
```shell
63
-
dragonfly> XADD mystream 1623910120014-1 name bob
64
-
"1623910120014-1"
65
-
dragonfly> XADD mystream 1623910120014-* name alice
The command returns the ID of the added entry. The ID is the
117
-
one auto-generated if * is passed as ID argument, otherwise
118
-
the command just returns the same ID specified by the user
119
-
during insertion.
114
+
Yes, you can specify your own unique ID instead of using `*`, but you must ensure the new ID is unique and is greater than the target stream top item's ID.
In Dragonfly, as well as in Redis and Valkey, the `XAUTOCLAIM` command is used within the context of streams for effectively managing pending messages.
14
+
The command transfers ownership of pending stream messages from one consumer to another,
15
+
which is particularly useful in scenarios where message processing needs to be resilient and messages shouldn't remain stuck with unavailable consumers.
16
+
`XAUTOCLAIM` works like using [`XPENDING`](xpending.md) followed by [`XCLAIM`](xclaim.md) but offers a simpler way to handle message delivery failures with [`SCAN`](../generic/scan.md)-like behavior.
17
+
11
18
## Syntax
12
19
13
-
XAUTOCLAIM key group consumer min-idle-time start [COUNT count] [JUSTID]
20
+
```shell
21
+
XAUTOCLAIM key group consumer min-idle-time start [COUNT count] [JUSTID]
22
+
```
14
23
15
-
**Time Complexity:** O(1) if COUNT is small.
24
+
## Parameter Explanations
16
25
17
-
**ACL categories:**@write, @stream, @fast
26
+
-`key`: The key of the stream.
27
+
-`group`: The consumer group name.
28
+
-`consumer`: The new owner consumer for the pending messages.
29
+
-`min-idle-time`: Minimum idle time (in milliseconds) for a message to be claimed.
30
+
-`start`: The ID from which to start scanning for potential messages to claim.
31
+
-`COUNT count` (optional): Limit the number of messages to claim in a single call.
32
+
-`JUSTID` (optional): If specified, only message IDs are returned, not the messages themselves.
18
33
19
-
This command transfers ownership of pending stream entries that match the specified criteria. Conceptually, `XAUTOCLAIM` is equivalent to calling `XPENDING` and then `XCLAIM`, but provides a more straightforward way to deal with message delivery failures via SCAN-like semantics.
34
+
## Return Values
20
35
21
-
Like `XCLAIM`, the command operates on the stream entries at `<key>` and in the context of the provided `<group>`. It transfers ownership to `<consumer>` of messages pending for more than `<min-idle-time>` milliseconds and having an equal or greater ID than `<start>`.
36
+
The command returns a three-element array:
22
37
23
-
The optional `<count>` argument, which defaults to 100, is the upper limit of the number of entries that the command attempts to claim. Internally, the command begins scanning the consumer group's Pending Entries List (PEL) from `<start>` and filters out entries having an idle time less than or equal to `<min-idle-time>`. The maximum number of pending entries that the command scans is the product of multiplying `<count>`'s value by 10 (hard-coded). It is possible, therefore, that the number of entries claimed will be less than the specified value.
38
+
- A stream entry ID for the next `XAUTOCLAIM` call. **Note that a returned ID of `0-0` means the entire stream was scanned.**
39
+
- An array of successfully claimed messages, formatted the same as the response of [`XRANGE`](xrange.md).
40
+
- An array of message IDs that no longer exist in the stream, and were deleted from the PEL in which they were found.
24
41
25
-
The optional `JUSTID` argument changes the reply to return just an array of IDs of messages successfully claimed, without returning the actual message. Using this option means the retry counter is not incremented.
42
+
## Code Examples
26
43
27
-
The command returns the claimed entries as an array. It also returns a stream ID intended for cursor-like use as the `<start>` argument for its subsequent call. When there are no remaining PEL entries, the command returns the special `0-0` ID to signal completion. However, note that you may want to continue calling `XAUTOCLAIM` even after the scan is complete with the `0-0` as `<start>` ID, because enough time passed, so older pending entries may now be eligible for claiming.
44
+
### Basic Example
28
45
29
-
Note that only messages that are idle longer than `<min-idle-time>` are claimed, and claiming a message resets its idle time. This ensures that only a single consumer can successfully claim a given pending message at a specific instant of time and trivially reduces the probability of processing the same message multiple times.
46
+
Claim messages that have been pending for over 5 seconds:
1) "0-0"# Next ID to scan from. Note that '0-0' means the entire stream was scanned.
64
+
2) 1) 1) "1736363819856-0"
65
+
2) 1) "name"
66
+
2) "Alice"
67
+
3) (empty array)
68
+
```
30
69
31
-
While iterating the PEL, if `XAUTOCLAIM`stumbles upon a message which doesn't exist in the stream anymore (either trimmed or deleted by `XDEL`) it does not claim it, and deletes it from the PEL in which it was found. These message IDs are returned to the caller as a part of `XAUTOCLAIM`'s reply.
70
+
### Using `XAUTOCLAIM`with `COUNT` Option
32
71
33
-
Lastly, claiming a message with `XAUTOCLAIM` also increments the attempted deliveries count for that message, unless the `JUSTID` option has been specified (which only delivers the message ID, not the message itself). Messages that cannot be processed for some reason - for example, because consumers systematically crash when processing them - will exhibit high attempted delivery counts that can be detected by monitoring.
- Regularly use `XAUTOCLAIM` to prevent pending messages from being stuck with unavailable consumers.
107
+
- Utilize the `JUSTID` option when only message identifiers are needed, reducing network load.
108
+
- Setting a sensible `COUNT` can optimize performance by not overloading the consumer with too many messages at once.
39
109
40
-
An array with three elements:
110
+
## Common Mistakes
41
111
42
-
1. A stream ID to be used as the `<start>` argument for the next call to `XAUTOCLAIM.
112
+
- Not correctly setting the `min-idle-time`, leading to unclaimed or incorrectly claimed messages.
113
+
- Forgetting to create the consumer group beforehand, as `XAUTOCLAIM` requires an existing consumer group.
43
114
44
-
2. An array containing all the successfully claimed messages in the same format as `XRANGE`.
115
+
## FAQs
45
116
46
-
3. An array containing message IDs that no longer exist in the stream, and were deleted from the PEL in which they were found.
117
+
### What happens if there are no messages to claim?
47
118
48
-
## Examples
119
+
If there are no messages pending or meeting the criteria, an empty array is returned.
49
120
50
-
```shell
51
-
dragonfly> XAUTOCLAIM mystream mygroup Alice 3600000 0-0 COUNT 25
52
-
1) "0-0"
53
-
2) 1) 1) "1609338752495-0"
54
-
2) 1) "field"
55
-
2) "value"
56
-
3) (empty array)
57
-
```
121
+
### Can I claim messages with a `min-idle-time` of zero?
58
122
59
-
In the above example, we attempt to claim for consumer `Alice` up to 25 entries that are pending and idle (not having been acknowledged or claimed) for at least an hour, starting at the stream's beginning. The consumer `Alice` from the `mygroup` group acquires ownership of these messages. Note that the stream ID returned in the example is `0-0`, indicating that the entire stream was scanned. We can also see that `XAUTOCLAIM did not stumble upon any deleted messages (the third reply element is an empty array).
123
+
Yes, but an idle time of zero means messages are immediately available for claiming, often unsuitable for normal operations where consumers might be briefly unavailable.
0 commit comments