Skip to content

Commit 9ab79e3

Browse files
authored
new tunnel-based RESP logging and validation (StackExchange#2660)
* Provide new LoggingTunnel API; this * words * fix PR number * fix file location * save the sln * identify smessage as out-of-band * add .ForAwait() throughout LoggingTunnel * clarify meaning of path parameter
1 parent ae72fb7 commit 9ab79e3

File tree

8 files changed

+787
-4
lines changed

8 files changed

+787
-4
lines changed

StackExchange.Redis.sln

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,7 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "docs", "docs", "{153A10E4-E
120120
docs\PubSubOrder.md = docs\PubSubOrder.md
121121
docs\ReleaseNotes.md = docs\ReleaseNotes.md
122122
docs\Resp3.md = docs\Resp3.md
123+
docs\RespLogging.md = docs\RespLogging.md
123124
docs\Scripting.md = docs\Scripting.md
124125
docs\Server.md = docs\Server.md
125126
docs\Testing.md = docs\Testing.md

docs/ReleaseNotes.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,12 @@ Current package versions:
88

99
## Unreleased
1010

11+
- Add new `LoggingTunnel` API; see https://stackexchange.github.io/StackExchange.Redis/Logging [#2660 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2660)
12+
1113
## 2.7.27
1214

1315
- Support `HeartbeatConsistencyChecks` and `HeartbeatInterval` in `Clone()` ([#2658 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2658))
14-
- Add `AddLibraryNameSuffix` to multiplexer; allows usage-specific tokens to be appended *after connect*
16+
- Add `AddLibraryNameSuffix` to multiplexer; allows usage-specific tokens to be appended *after connect* [#2659 by mgravell](https://github.com/StackExchange/StackExchange.Redis/pull/2659)
1517

1618
## 2.7.23
1719

docs/RespLogging.md

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
Logging and validating the underlying RESP stream
2+
===
3+
4+
Sometimes (rarely) there is a question over the validity of the RESP stream from a server (especially when using proxies
5+
or a "redis-like-but-not-actually-redis" server), and it is hard to know whether the *data sent* was bad, vs
6+
the client library tripped over the data.
7+
8+
To help with this, an experimental API exists to help log and validate RESP streams. This API is not intended
9+
for routine use (and may change at any time), but can be useful for diagnosing problems.
10+
11+
For example, consider we have the following load test which (on some setup) causes a failure with some
12+
degree of reliability (even if you need to run it 6 times to see a failure):
13+
14+
``` c#
15+
// connect
16+
Console.WriteLine("Connecting...");
17+
var options = ConfigurationOptions.Parse(ConnectionString);
18+
await using var muxer = await ConnectionMultiplexer.ConnectAsync(options);
19+
var db = muxer.GetDatabase();
20+
21+
// load
22+
RedisKey testKey = "marc_abc";
23+
await db.KeyDeleteAsync(testKey);
24+
Console.WriteLine("Writing...");
25+
for (int i = 0; i < 100; i++)
26+
{
27+
// sync every 50 iterations (pipeline the rest)
28+
var flags = (i % 50) == 0 ? CommandFlags.None : CommandFlags.FireAndForget;
29+
await db.SetAddAsync(testKey, Guid.NewGuid().ToString(), flags);
30+
}
31+
32+
// fetch
33+
Console.WriteLine("Reading...");
34+
int count = 0;
35+
for (int i = 0; i < 10; i++)
36+
{
37+
// this is deliberately not using SCARD
38+
// (to put load on the inbound)
39+
count += (await db.SetMembersAsync(testKey)).Length;
40+
}
41+
Console.WriteLine("all done");
42+
```
43+
44+
## Logging RESP streams
45+
46+
When this fails, it will not be obvious exactly who is to blame. However, we can ask for the data streams
47+
to be logged to the local file-system.
48+
49+
**Obviously, this may leave data on disk, so this may present security concerns if used with production data; use
50+
this feature sparingly, and clean up after yourself!**
51+
52+
``` c#
53+
// connect
54+
Console.WriteLine("Connecting...");
55+
var options = ConfigurationOptions.Parse(ConnectionString);
56+
LoggingTunnel.LogToDirectory(options, @"C:\Code\RedisLog"); // <=== added!
57+
await using var muxer = await ConnectionMultiplexer.ConnectAsync(options);
58+
...
59+
```
60+
61+
This API is marked `[Obsolete]` simply to discourage usage, but you can ignore this warning once you
62+
understand what it is saying (using `#pragma warning disable CS0618` if necessary).
63+
64+
This will update the `ConfigurationOptions` with a custom `Tunnel` that performs file-based mirroring
65+
of the RESP streams. If `Ssl` is enabled on the `ConfigurationOptions`, the `Tunnel` will *take over that responsibility*
66+
(so that the unencrypted data can be logged), and will *disable* `Ssl` on the `ConfigurationOptions` - but TLS
67+
will still be used correctly.
68+
69+
If we run our code, we will see that 2 files are written per connection ("in" and "out"); if you are using RESP2 (the default),
70+
then 2 connections are usually established (one for regular "interactive" commands, and one for pub/sub messages), so this will
71+
typically create 4 files.
72+
73+
## Validating RESP streams
74+
75+
RESP is *mostly* text, so a quick eyeball can be achieved using any text tool; an "out" file will typically start:
76+
77+
``` txt
78+
$6
79+
CLIENT
80+
$7
81+
SETNAME
82+
...
83+
```
84+
85+
and an "in" file will typically start:
86+
87+
``` txt
88+
+OK
89+
+OK
90+
+OK
91+
...
92+
```
93+
94+
This is the start of the handshakes for identifying the client to the redis server, and the server acknowledging this (if
95+
you have authentication enabled, there will be a `AUTH` command first, or `HELLO` on RESP3).
96+
97+
If there is a failure, you obviously don't want to manually check these files. Instead, an API exists to validate RESP streams:
98+
99+
``` c#
100+
var messages = await LoggingTunnel.ValidateAsync(@"C:\Code\RedisLog");
101+
Console.WriteLine($"{messages} RESP fragments validated");
102+
```
103+
104+
If the RESP streams are *not* valid, an exception will provide further details.
105+
106+
**An exception here is strong evidence that there is a fault either in the redis server, or an intermediate proxy**.
107+
108+
Conversely, if the library reported a protocol failure but the validation step here *does not* report an error, then
109+
that is strong evidence of a library error; [**please report this**](https://github.com/StackExchange/StackExchange.Redis/issues/new) (with details).
110+
111+
You can also *replay* the conversation locally, seeing the individual requests and responses:
112+
113+
``` c#
114+
var messages = await LoggingTunnel.ReplayAsync(@"C:\Code\RedisLog", (cmd, resp) =>
115+
{
116+
if (cmd.IsNull)
117+
{
118+
// out-of-band/"push" response
119+
Console.WriteLine("<< " + LoggingTunnel.DefaultFormatResponse(resp));
120+
}
121+
else
122+
{
123+
Console.WriteLine(" > " + LoggingTunnel.DefaultFormatCommand(cmd));
124+
Console.WriteLine(" < " + LoggingTunnel.DefaultFormatResponse(resp));
125+
}
126+
});
127+
Console.WriteLine($"{messages} RESP commands validated");
128+
```
129+
130+
The `DefaultFormatCommand` and `DefaultFormatResponse` methods are provided for convenience, but you
131+
can perform your own formatting logic if required. If a RESP erorr is encountered in the response to
132+
a particular message, the callback will still be invoked to indicate that error. For example, after deliberately
133+
introducing an error into the captured file, we might see:
134+
135+
``` txt
136+
> CLUSTER NODES
137+
< -ERR This instance has cluster support disabled
138+
> GET __Booksleeve_TieBreak
139+
< (null)
140+
> ECHO ...
141+
< -Invalid bulk string terminator
142+
Unhandled exception. StackExchange.Redis.RedisConnectionException: Invalid bulk string terminator
143+
```
144+
145+
The `-ERR` message is not a problem - that's normal and simply indicates that this is not a redis cluster; however, the
146+
final pair is an `ECHO` request, for which the corresponding response was invalid. This information is useful for finding
147+
out what happened.
148+
149+
Emphasis: this API is not intended for common/frequent usage; it is intended only to assist validating the underlying
150+
RESP stream.

docs/index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ Documentation
4848
- [Testing](Testing) - running the `StackExchange.Redis.Tests` suite to validate changes
4949
- [Timeouts](Timeouts) - guidance on dealing with timeout problems
5050
- [Thread Theft](ThreadTheft) - guidance on avoiding TPL threading problems
51+
- [RESP Logging](RespLogging) - capturing and validating RESP streams
5152

5253
Questions and Contributions
5354
---

src/StackExchange.Redis/BufferReader.cs

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ internal ref struct BufferReader
1717
public int OffsetThisSpan { get; private set; }
1818
public int RemainingThisSpan { get; private set; }
1919

20+
public long TotalConsumed => _totalConsumed;
21+
2022
private ReadOnlySequence<byte>.Enumerator _iterator;
2123
private ReadOnlySpan<byte> _current;
2224

0 commit comments

Comments
 (0)