Skip to content

Commit d8bd962

Browse files
authored
Merge pull request #66 from matrix-org/kegan/mitmdump-introspection
Replace tcpdump with mitmdump
2 parents 6b55d02 + d8d5994 commit d8bd962

File tree

8 files changed

+56
-43
lines changed

8 files changed

+56
-43
lines changed

ENVIRONMENT.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,16 @@
44
Complement-Crypto is configured exclusively through the use of environment variables. These variables are described below. Additional environment variables can be used, and are outlined at https://github.com/matrix-org/complement/blob/main/ENVIRONMENT.md
55
Complement-Crypto always runs in dirty mode (homeservers exist for the entire duration of the test suite) for performance reasons.
66

7+
#### `COMPLEMENT_CRYPTO_MITMDUMP`
8+
The path to dump the output from `mitmdump`. This file can then be used with mitmweb to view all the HTTP flows in the test.
9+
- Type: `string`
10+
- Default: ""
11+
712
#### `COMPLEMENT_CRYPTO_RPC_BINARY`
813
The absolute path to the pre-built rpc binary file. This binary is generated via `go build -tags=jssdk,rust ./cmd/rpc`. This binary is used when running multiprocess tests. If this environment variable is not supplied, tests which try to use multiprocess clients will be skipped, making this environment variable optional.
914
- Type: `string`
1015
- Default: ""
1116

12-
#### `COMPLEMENT_CRYPTO_TCPDUMP`
13-
If 1, automatically attempts to run `tcpdump` when the containers are running. Stops dumping when tests complete. This will probably require you to run `go test` with `sudo -E`. The `.pcap` file is written to `tests/test.pcap`.
14-
- Type: `bool`
15-
- Default: 0
16-
1717
#### `COMPLEMENT_CRYPTO_TEST_CLIENT_MATRIX`
1818
The client test matrix to run. Every test is run for each given permutation. The default matrix tests all JS/Rust permutations _ignoring federation_.
1919
```

FAQ.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ Now you can look around those log lines for any warnings/errors or unexpected be
2323

2424
Sometimes the bug cannot be found via log files alone. You may want to see server logs. To do this, [enable writing container logs](https://github.com/matrix-org/complement-crypto/blob/main/ENVIRONMENT.md#complement_crypto_write_container_logs) then re-run the test.
2525

26-
Sometimes, even that isn't enough. Perhaps server logs aren't giving enough information. In that case, [enable tcpdump](https://github.com/matrix-org/complement-crypto/blob/main/ENVIRONMENT.md#complement_crypto_tcpdump) and open the `.pcap` file in Wireshark to see the raw HTTP request/responses made by all clients.
26+
Sometimes, even that isn't enough. Perhaps server logs aren't giving enough information. In that case, [enable mitmdump](https://github.com/matrix-org/complement-crypto/blob/main/ENVIRONMENT.md#complement_crypto_mitmdump) and open the dump file in mitmweb to see the raw HTTP request/responses made by all clients. If you don't have mitmweb, run `./open_mitmweb.sh` which will use the mitmproxy image.
2727

2828
If you need to add console logging to clients, see below.
2929

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ go test -v -count=1 -tags=jssdk -timeout 15m ./tests
7777
`COMPLEMENT_CRYPTO_TEST_CLIENT_MATRIX` controls which SDK is used to create test clients, and the `-tags` option
7878
controls conditional compilation so other SDKs don't need to be compiled for the tests to run.
7979

80-
To test interoperability between the SDKs, `tcpdump` the traffic, run extra multiprocess tests and more,
80+
To test interoperability between the SDKs, `mitmdump` the traffic, run extra multiprocess tests and more,
8181
see [ENVIRONMENT.md](ENVIRONMENT.md) for the full configuration options.
8282

8383
*See [FAQ.md](FAQ.md) for more information around debugging.*

internal/config/config.go

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,12 +35,11 @@ type ComplementCrypto struct {
3535
// Derived from TestClientMatrix
3636
clientLangs map[api.ClientTypeLang]bool
3737

38-
// Name: COMPLEMENT_CRYPTO_TCPDUMP
39-
// Default: 0
40-
// Description: If 1, automatically attempts to run `tcpdump` when the containers are running. Stops dumping when
41-
// tests complete. This will probably require you to run `go test` with `sudo -E`. The `.pcap` file is written to
42-
// `tests/test.pcap`.
43-
TCPDump bool
38+
// Name: COMPLEMENT_CRYPTO_MITMDUMP
39+
// Default: ""
40+
// Description: The path to dump the output from `mitmdump`. This file can then be used with mitmweb to view
41+
// all the HTTP flows in the test.
42+
MITMDump string
4443

4544
// Name: COMPLEMENT_CRYPTO_RPC_BINARY
4645
// Default: ""
@@ -123,7 +122,7 @@ func NewComplementCryptoConfigFromEnvVars() *ComplementCrypto {
123122
}
124123
}
125124
return &ComplementCrypto{
126-
TCPDump: os.Getenv("COMPLEMENT_CRYPTO_TCPDUMP") == "1",
125+
MITMDump: os.Getenv("COMPLEMENT_CRYPTO_MITMDUMP"),
127126
RPCBinaryPath: rpcBinaryPath,
128127
TestClientMatrix: testClientMatrix,
129128
clientLangs: clientLangs,

internal/deploy/deploy.go

Lines changed: 28 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,6 @@ import (
1111
"net/http"
1212
"net/url"
1313
"os"
14-
"os/exec"
1514
"path/filepath"
1615
"runtime"
1716
"strings"
@@ -35,14 +34,16 @@ import (
3534
// must match the value in tests/addons/__init__.py
3635
const magicMITMURL = "http://mitm.code"
3736

37+
const mitmDumpFilePathOnContainer = "/tmp/mitm.dump"
38+
3839
type SlidingSyncDeployment struct {
3940
complement.Deployment
4041
extraContainers map[string]testcontainers.Container
4142
mitmClient *http.Client
4243
ControllerURL string
4344
dnsToReverseProxyURL map[string]string
4445
mu sync.RWMutex
45-
tcpdump *exec.Cmd
46+
mitmDumpFile string
4647
}
4748

4849
func (d *SlidingSyncDeployment) WithSniffedEndpoint(t *testing.T, partialPath string, onSniff func(CallbackData), inner func()) {
@@ -142,7 +143,29 @@ func (d *SlidingSyncDeployment) withReverseProxyURL(hsName string, c *client.CSA
142143
return c
143144
}
144145

146+
func (d *SlidingSyncDeployment) writeMITMDump() {
147+
if d.mitmDumpFile == "" {
148+
return
149+
}
150+
log.Printf("dumping mitmdump to '%s'\n", d.mitmDumpFile)
151+
fileContents, err := d.extraContainers["mitmproxy"].CopyFileFromContainer(context.Background(), mitmDumpFilePathOnContainer)
152+
if err != nil {
153+
log.Printf("failed to copy mitmdump from container: %s", err)
154+
return
155+
}
156+
contents, err := io.ReadAll(fileContents)
157+
if err != nil {
158+
log.Printf("failed to read mitmdump: %s", err)
159+
return
160+
}
161+
if err = os.WriteFile(d.mitmDumpFile, contents, os.ModePerm); err != nil {
162+
log.Printf("failed to write mitmdump to %s: %s", d.mitmDumpFile, err)
163+
return
164+
}
165+
}
166+
145167
func (d *SlidingSyncDeployment) Teardown() {
168+
d.writeMITMDump()
146169
for name, c := range d.extraContainers {
147170
filename := fmt.Sprintf("container-%s.log", name)
148171
logs, err := c.Logs(context.Background())
@@ -186,13 +209,9 @@ func (d *SlidingSyncDeployment) Teardown() {
186209
log.Fatalf("failed to stop %s: %s", name, err)
187210
}
188211
}
189-
if d.tcpdump != nil {
190-
fmt.Println("Sent SIGINT to tcpdump, waiting for it to exit, err=", d.tcpdump.Process.Signal(os.Interrupt))
191-
fmt.Println("tcpdump finished, err=", d.tcpdump.Wait())
192-
}
193212
}
194213

195-
func RunNewDeployment(t *testing.T, mitmProxyAddonsDir string, shouldTCPDump bool) *SlidingSyncDeployment {
214+
func RunNewDeployment(t *testing.T, mitmProxyAddonsDir string, mitmDumpFile string) *SlidingSyncDeployment {
196215
// allow time for everything to deploy
197216
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
198217
defer cancel()
@@ -261,6 +280,7 @@ func RunNewDeployment(t *testing.T, mitmProxyAddonsDir string, shouldTCPDump boo
261280
"--mode", "reverse:http://ssproxy1:6789@3002",
262281
"--mode", "reverse:http://ssproxy2:6789@3003",
263282
"--mode", "regular",
283+
"-w", mitmDumpFilePathOnContainer,
264284
},
265285
Networks: []string{networkName},
266286
NetworkAliases: map[string][]string{
@@ -353,25 +373,6 @@ func RunNewDeployment(t *testing.T, mitmProxyAddonsDir string, shouldTCPDump boo
353373
t.Logf(" synapse: hs2 %s (rp=%s)", csapi2.BaseURL, rpHS2URL)
354374
t.Logf(" postgres: postgres")
355375
t.Logf(" mitmproxy: mitmproxy controller=%s", controllerURL)
356-
var cmd *exec.Cmd
357-
if shouldTCPDump {
358-
t.Log("Running tcpdump...")
359-
urlsToTCPDump := []string{
360-
rpSS1URL, rpSS2URL, csapi1.BaseURL, csapi2.BaseURL, rpHS1URL, rpHS2URL, controllerURL,
361-
}
362-
tcpdumpFilter := []string{}
363-
for _, u := range urlsToTCPDump {
364-
parsedURL, _ := url.Parse(u)
365-
tcpdumpFilter = append(tcpdumpFilter, fmt.Sprintf("port %s", parsedURL.Port()))
366-
}
367-
filter := fmt.Sprintf("tcp " + strings.Join(tcpdumpFilter, " or "))
368-
cmd = exec.Command("tcpdump", "-i", "any", "-s", "0", filter, "-w", "test.pcap")
369-
t.Log(cmd.String())
370-
if err := cmd.Start(); err != nil {
371-
t.Fatalf("tcpdump failed: %v", err)
372-
}
373-
t.Logf("Started tcpdumping (requires sudo): PID %d", cmd.Process.Pid)
374-
}
375376
// without this, GHA will fail when trying to hit the controller with "Post "http://mitm.code/options/lock": EOF"
376377
// suspected IPv4 vs IPv6 problems in Docker as Flask is listening on v4/v6.
377378
controllerURL = strings.Replace(controllerURL, "localhost", "127.0.0.1", 1)
@@ -385,7 +386,6 @@ func RunNewDeployment(t *testing.T, mitmProxyAddonsDir string, shouldTCPDump boo
385386
"postgres": postgresContainer,
386387
"mitmproxy": mitmproxyContainer,
387388
},
388-
tcpdump: cmd,
389389
ControllerURL: controllerURL,
390390
mitmClient: &http.Client{
391391
Timeout: 5 * time.Second,
@@ -399,6 +399,7 @@ func RunNewDeployment(t *testing.T, mitmProxyAddonsDir string, shouldTCPDump boo
399399
"ssproxy1": rpSS1URL,
400400
"ssproxy2": rpSS2URL,
401401
},
402+
mitmDumpFile: mitmDumpFile,
402403
}
403404
}
404405

internal/tests/client_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ func Deploy(t *testing.T) *deploy.SlidingSyncDeployment {
3636
if ssDeployment != nil {
3737
return ssDeployment
3838
}
39-
ssDeployment = deploy.RunNewDeployment(t, "", false)
39+
ssDeployment = deploy.RunNewDeployment(t, "", "")
4040
return ssDeployment
4141
}
4242

open_mitmweb.sh

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
#!/bin/bash -e
2+
3+
if [ "$1" = "-h" ] || [ "$1" = "--help" ];
4+
then
5+
echo "Opens a browser with mitmweb. Then you can open a dump file made via COMPLEMENT_CRYPTO_MITMDUMP. (requires on PATH: docker)"
6+
exit 1
7+
fi
8+
9+
# - use python3 instead of xdg-open because it's more portable (xdg-open doesn't work on MacOS). Sleep 1s and do it in the background.
10+
(sleep 1 && python3 -m webbrowser http://localhost:1445) &
11+
12+
# - use same version as tests so we don't need to pull any new image. When the user CTRL+Cs this, the container quits.
13+
docker run --rm -p 1445:8081 mitmproxy/mitmproxy:10.1.5 mitmweb --web-host 0.0.0.0

tests/main_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ func Deploy(t *testing.T) *deploy.SlidingSyncDeployment {
5959
t.Fatalf("failed to find working directory: %s", err)
6060
}
6161
mitmProxyAddonsDir := filepath.Join(workingDir, "mitmproxy_addons")
62-
ssDeployment = deploy.RunNewDeployment(t, mitmProxyAddonsDir, complementCryptoConfig.TCPDump)
62+
ssDeployment = deploy.RunNewDeployment(t, mitmProxyAddonsDir, complementCryptoConfig.MITMDump)
6363
return ssDeployment
6464
}
6565

0 commit comments

Comments
 (0)