Skip to content

Daemon peak RSS scales ~linearly with query concurrency (29 MB idle -> 3.07 GB at 20 concurrent queries) and releases slowly #1362

Description

@artaommahe

Version

codebase-memory-mcp 0.9.1-rc.1

Platform

macOS (Apple Silicon)

Install channel

GitHub release archive / install.sh / install.ps1

Binary variant

standard

What happened, and what did you expect?

Daemon peak working set scales very steeply with query concurrency, and releases only slowly afterwards. On a 63,840-node / 120,805-edge TypeScript graph, 20 concurrent query_graph aggregates took the daemon from a 399 MB resident baseline to 3.07 GB — and it was still at 3.07 GB ~30 s after every query had returned, decaying back over roughly 100 s.

Filing this because the RC notes specifically ask for multi-agent concurrency reports, and "several agents hitting the same projects concurrently" is exactly the shape that produces it. Nothing here is a leak — memory does come back — so this is a peak-working-set / admission-control question rather than the #581 class.

Sequential load shows nothing: 30 sequential runs of the same query left RSS drifting down (256 MB → 214 MB), and the daemon returns to single-digit MB when idle. Only concurrency produces the spike.

Measured decay after the 20-concurrent burst, with no further queries issued:

time after burst daemon RSS
all queries returned 3.07 GB
+20 s 3.07 GB
+40 s 2.13 GB
+60 s 2.13 GB
+80 s 1.36 GB
+100 s (after one trivial list_projects) 1.28 GB

The concern is the ceiling, not the retention: 3.07 GB is 36% of the memory budget the daemon reports for itself on this box (budget_mb=8601, 24 GB RAM). Two or three agents each firing a handful of concurrent graph queries — the workflow the daemon exists to support — would plausibly exhaust that budget or the machine on a 16 GB laptop. I did not observe an OOM here, so I can't say what happens at the limit.

Expected: either bounded per-query working set, or admission control that queues concurrent heavy queries rather than letting each allocate its own peak simultaneously. The daemon already has "physical-job admission control for parallel indexing" per the release notes — the same idea applied to query execution would cap this.

Reproduction

Needs a reasonably large graph; I used a private 11,788-file Angular monorepo (63,840 nodes / 120,805 edges, --mode full). Any similarly-sized graph should do.

codebase-memory-mcp daemon start
DPID=$(codebase-memory-mcp daemon status | awk '/pid:/{print $2}')
Q="MATCH (a)-[r:CALLS]->(b:Method) RETURN b.qualified_name AS q, count(r) AS c ORDER BY c DESC LIMIT 50"

ps -o rss= -p $DPID                      # baseline

for i in $(seq 1 20); do
  codebase-memory-mcp cli query_graph --project <p> --query "$Q" < /dev/null > /dev/null 2>&1 &
done

# sample while they run and after they finish
for s in $(seq 1 15); do sleep 2; ps -o rss= -p $DPID; done

(Note the < /dev/null — without it the CLI can block on stdin, see #1359.)

Diagnostics trajectory (memory / performance / leak issues)

Scaling by concurrency, same query, same graph, daemon allowed to settle between runs:

concurrent query_graph calls daemon peak RSS
0 (idle baseline) 29 MB
1 229 MB
5 1,044 MB
10 2,275 MB
20 3,070 MB

Same query, same graph, daemon allowed to settle to idle between rounds, peak taken by
sampling ps -o rss= every 2 s for 16 s. It is close to linear at ~220 MB per
concurrent query
up to 10, then sublinear at 20 — each concurrent query appears to
allocate its own working set with no sharing between them. Sixty seconds after the
10-query round RSS was still 1,877 MB.

(The 3.07 GB figure above came from a run whose baseline was 399 MB rather than 29 MB,
because it directly followed an index; the peak was the same order either way.)

Attempts to find a genuine leak, all negative. Sequential load shows no growth at all: 30 sequential runs of this same aggregate followed by 30 search_graph calls took RSS down from 256 MB to 214 MB, and the daemon sat at 6 MB when checked again an hour later. Idle RSS consistently returns to single-digit MB. So the allocator is releasing correctly — consistent with the "harmless on macOS/Linux" note about mem.allocator.not_owned in the RC's Known Issues. The spike is specific to concurrency.

Project scale (if relevant)

11,788 files indexed, 63,840 nodes / 120,805 edges, --mode full, 10,175 .ts files. 24 GB RAM, daemon self-reported budget_mb=8601. Permanent daemon (daemon start), build 0.9.1-rc.1 (e4fe1cf4f764).

Related

Confirmations

  • I searched existing issues and this is not a duplicate.
  • My reproduction uses shareable code (a dummy snippet or a public OSS repository), not proprietary code.

Posted by Claude Code on behalf of @artaommahe

Metadata

Metadata

Assignees

No one assigned

    Labels

    editor/integrationEditor compatibility and CLI integrationparsing/qualityGraph extraction bugs, false positives, missing edgesstability/performanceServer crashes, OOM, hangs, high CPU/memorywindowsWindows-specific issues

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions