Skip to content

Windows support: fix daemon boot (SIGHUP), IPC listener, and ambient hooks; plus reliability fixes - #108

Open
Kmmille2 wants to merge 15 commits into
CodeAbra:mainfrom
Kmmille2:windows-adapt
Open

Kmmille2 wants to merge 15 commits into
CodeAbra:mainfrom
Kmmille2:windows-adapt

Conversation

@Kmmille2

Copy link
Copy Markdown

Summary

The daemon doesn't currently start on Windows — two Python-portability bugs kill it before it can serve, so the store stays empty and MCP hosts can't reach it. This PR fixes those, makes the ambient-capture hooks work under Git Bash, and includes three reliability fixes I hit while getting it running.

Verified end-to-end on Windows 11 / Python 3.11 (from-source build, MSVC toolchain): the daemon boots, binds its IPC listener, and capture → recall returns records through the MCP path in both Claude Code and Codex. Backfilled ~480 records and confirmed recall + the iai brain dashboard.

Windows portability

  • Daemon boot crash — signal.SIGHUP is absent on Windows. _install_boot_signal_trace built the tuple (SIGTERM, SIGINT, SIGHUP); evaluating signal.SIGHUP raised AttributeError before the per-signal try/except could catch it, killing boot. Resolve each signal lazily via getattr — mirrors the run-loop shutdown handler's existing guard.
  • IPC listener never binds — asyncio.start_unix_server is absent on Windows. SocketServer.serve() read inspect.signature(asyncio.start_unix_server) before the IS_WINDOWS branch; on Windows that attribute doesn't exist, so serve() raised immediately and — as a create_task'd coroutine — the exception was never retrieved. The daemon ran but was unreachable (no .daemon.port; daemon status reported not running). Moved the POSIX-only signature read past the Windows early return.
  • Ambient-capture hooks hardcoded /usr/bin/python3, which doesn't exist under Git Bash, so capture silently no-opped (fail-safe exit 0). Resolve the interpreter portably: honor IAI_MCP_PYTHON, skip the Windows Store python3 App-Execution-Alias stub, fall back to the py launcher.
  • Pinned *.sh to LF via .gitattributes so the hooks run under bash on Windows (autocrlf otherwise materializes CRLF → bad interpreter).
  • Console CTRL_C survival. A Task-Scheduler daemon attached to the interactive console was terminated with STATUS_CONTROL_C_EXIT when the host CLI restarted. Install a Windows console-ctrl handler (only under IAI_MCP_LAUNCHD_MANAGED) that swallows CTRL_C/CTRL_BREAK; Task Scheduler still stops it via TerminateProcess.

Reliability (general, not Windows-specific)

  • Poison-pill capture drain. A record capture_turn rejects (e.g. an embedder failure) raised out of both drain loops, so the spool file never unlinked and every subsequent drain re-hit the same record — silently wedging all future capture. Quarantine-skip the failing record so the file still unlinks and the backlog drains (re-runs stay loss-free via the source_uuid idem-tag).
  • RO-pool slot leak in recycle(). The except queue.Full branch closed a slot without decrementing _filled (unlike _release); a race with a concurrent _release() leaks a slot, and over a daemon lifetime the pool drains until borrow()'s get() blocks forever. Decrement under _fill_lock, matching _release().
  • Corrupt-page host abort/panic in lillibrain. read_overflow_chain reserved Vec::with_capacity(total_len) from a caller-supplied on-disk length — a corrupted/forged varint hits the allocator abort path (uncatchable, kills the host). cursor.rs payload_of/read_full_cell sliced page[start..start+len] raw from the same unvalidated decode (panics across PyO3). Cap the reservation at the physical maximum; use checked_add + .get().ok_or(Integrity) so a damaged page yields a typed error, as raw_leaf_cell_bytes already does.

Happy to split this into two PRs (Windows portability vs reliability) if you'd prefer smaller reviews. Thanks for the project — it's a genuinely nice piece of engineering.

Kmmille2 and others added 8 commits August 16, 2026 13:07
The four Claude/Codex capture+recall hooks hardcoded /usr/bin/python3,
absent on Windows (Git Bash), so ambient capture silently no-opped via
its fail-safe exit 0. Add a self-contained resolver that honors
IAI_MCP_PYTHON, skips the Windows Store python3 App-Execution-Alias stub,
and falls back to the py launcher. Verified on Windows 11 / Python 3.11.9.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_install_boot_signal_trace built (SIGTERM, SIGINT, SIGHUP); accessing the
POSIX-only SIGHUP attribute raised AttributeError before the per-signal
try/except could catch it, killing daemon boot on Windows. Resolve each
signal lazily via getattr and drop absent ones, matching the run-loop
shutdown handler's existing guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SocketServer.serve() read inspect.signature(asyncio.start_unix_server)
before the IS_WINDOWS branch. That attribute does not exist on Windows, so
serve() raised AttributeError immediately; as a create_task'd coroutine the
exception was never retrieved, so the daemon ran but its IPC listener never
bound (no .daemon.port; 'daemon status' reported not running; MCP hosts
could not reach it). Move the POSIX-only signature read past the Windows
early return.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…'t kill it

A Task-Scheduler-launched daemon attached to the interactive session's console
was terminated with STATUS_CONTROL_C_EXIT when the host CLI restarted: the
console CTRL_C delivered SIGINT and tripped the shutdown path on a healthy
daemon. Install a Windows console-ctrl handler (only when IAI_MCP_LAUNCHD_MANAGED)
that swallows CTRL_C/CTRL_BREAK; Task Scheduler still stops it via
TerminateProcess. Guarded and wrapped so it can never affect boot elsewhere.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A record capture_turn rejects (e.g. an embedder failure) raised out of both
drain loops, so the spool file was never unlinked and every subsequent drain
re-hit the same poison record — silently wedging all future capture. Guard the
per-record capture_turn call: on any failure, count it skipped and continue so
the file still unlinks and the backlog drains. Re-runs stay loss-free via the
source_uuid idem-tag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
recycle() drains idle slots then re-puts them; a concurrent _release() can
refill the queue in between, so a re-put can hit queue.Full. That branch closed
the slot but never decremented _filled (unlike _release), so the pool believed
it held a live slot it didn't. Over a daemon lifetime the accumulated leak
drains the pool until borrow()'s get() blocks forever and recall hangs.
Decrement _filled under _fill_lock, matching _release().

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
read_overflow_chain reserved Vec::with_capacity(total_len) from a caller-
supplied on-disk length; a corrupted/forged varint could make that enormous and
hit the allocator abort path (uncatchable, kills the host). Cap the reservation
at the physical maximum the chain can provide. cursor.rs payload_of and
read_full_cell indexed page[start..start+len] raw from the same unvalidated
decode, panicking across the PyO3 boundary on a damaged inline cell; use
checked_add + .get().ok_or(Integrity) so a bad page yields a typed error, as
raw_leaf_cell_bytes already does.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@CodeAbra

Copy link
Copy Markdown
Owner

Really nice work, Kyle, and it cleared our security review. We'll take the Windows fixes and merge them under your name.

The three engine hardening commits are good too. They'll come in through our storage-core pipeline rather than this PR, since we keep a single writer on that code, with your authorship on them.

The drain poison-pill guard is the one we can't take as-is: skip-and-unlink turns a stuck backlog into silent data loss, and a systemic failure like a cold embedder or a full disk would quietly wipe the whole backlog. If you want to rework it, quarantine the failing event to a .permanent-failed-*.jsonl sidecar, count it under a new failed key instead of skipped, and stop the drain past a small failure threshold. We already have a recovery command and a doctor check for those, and I'd be glad to review it.

Two small asks so I can merge the Windows part: rebase on latest main (3.0.2 just shipped and fixed the snapshot path your commits touch), and drop the bench-CSV churn the .gitattributes pulled in. Keep the .gitattributes itself.

@CodeAbra

Copy link
Copy Markdown
Owner

Really appreciate how thorough this is — you got the daemon booting on Windows end-to-end. Two things now that 3.0.5 landed: (1) it reworked the daemon internals, so this now conflicts and needs a rebase; (2) it overlaps #126, which is more surgical and ships tests for the same boot path, so I'm planning to base the Windows daemon-boot fix on that. Would you be up for narrowing this to the unique reliability fixes #126 doesn't cover? Separately, the Rust engine changes (crates/lillibrain/src/btree/…) should go through a review from our engine side rather than riding in with the Windows fixes — happy to coordinate. Credit for the boot diagnosis stands regardless.

…che)

Three Windows-only failures kept every hook capture from ever reaching the
store, even though each hook exited 0 (fail-safe):

- Turn-capture hook and `capture-turn-deferred` both fsync the state
  directory after publishing the offset. Windows cannot open a directory
  with os.open() (PermissionError: [Errno 13]) and has no directory fsync,
  so the embedded script aborted right after the spool write. Guard the
  directory fsync with `os.name != "nt"`; same guard in lillibrain
  `fsync_directory`.
- The hook's spool lock used `fcntl.flock`; `fcntl` does not exist on
  Windows (ModuleNotFoundError). Fall back to `msvcrt.locking(LK_NBLCK)`,
  which has the same non-blocking semantics.
- The Stop hook resolves the CLI via `~/.iai-mcp/.cli-path` before a PATH
  scan; nothing seeded that cache outside `cowork install`, and the seeder
  only recognised `iai-mcp`, not `iai-mcp.exe`. A venv/pipx install whose
  Scripts dir is not on the hook's PATH logged "iai-mcp CLI not found" and
  exited before rotating the live spool, so nothing ever drained. Seed the
  cache from `capture-hooks install` for every host target and accept the
  `.exe` / interpreter-sibling forms.

Verified on Windows 11: turn hook rc=0 with no traceback, Stop hook rotates
`.live.jsonl`, daemon startup drain consumed the rotated spool (store
488 -> 721 records) and `memory_recall` returns the captured turn.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@CodeAbra

Copy link
Copy Markdown
Owner

Thank you so much! This is really valuable. Can you please rebase so I can merge?

Kmmille2 and others added 4 commits September 15, 2026 08:13
…nstall

`capture-hooks install` registered `bash <WindowsPath>` unquoted, e.g.
`bash C:\Users\First Last\.claude\hooks\iai-mcp-turn-capture.sh`. The host
hands that string to a POSIX shell, which eats the backslashes and splits
on the space, so Claude Code logged a non-blocking hook error on every
UserPromptSubmit/Stop (`bash: C:UsersKyle: No such file or directory`) and
Codex never ran the hook either. Any home directory with a space
(`/Users/First Last`) fails the same way on macOS.

Register `bash "<posix path>"` via a shared `_hookcmd.hook_command()` for
the Claude Code, Codex and Cursor installers, and make reinstall repair an
existing registration whose command differs (the marker check previously
left the broken form in place forever).

Verified: the registered string, run through `sh -c` as the hosts do,
exits 0 and logs the turn; both hosts' configs now carry the quoted form.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…in budget

Hook registration: a bare `bash` on Windows resolves, for any host that is
not itself running inside Git Bash (Codex CLI/Desktop, GUI apps), to
`C:\Windows\System32\bash.exe` — the WSL shim — because Git for Windows
only puts `Git\cmd` on the machine PATH. WSL cannot open a `C:/...` script,
so every hook died with `/bin/bash: ...: No such file or directory` (127)
and Codex captured nothing. `hook_command()` now pins the Git Bash
executable (`Git/bin/bash.exe`, x86 and per-user installs, else a PATH hit
that is not the System32/Store shim) and quotes it; POSIX hosts keep `bash`.
Verified with a machine+user-only PATH: before rc=127, after rc=0.

deferred-drain: the per-batch wall-clock budget (120 s + 0.2 s/event, cap
1 h) is now env-overridable (IAI_MCP_DRAIN_BATCH_TIMEOUT_{BASE,PER_EVENT,CAP}_S).
A real backlog on a CPU embedder runs at ~2-3 events/s, so the default
budget kills a healthy child as "wedged" and aborts the whole recovery run;
with the override and 2 MB batches an 88k-event backlog drains steadily.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Follow-up to the Git Bash pin: `"C:/Program Files/Git/bin/bash.exe" "<script>"`
still failed under Codex on Windows. A command line that starts with a quote
and contains further quotes loses its first and last quote when cmd.exe
processes it (the /C rule), so it ran as `C:/Program` and Codex reported
every iai-mcp hook as Failed while plugin hooks completed. Register the
interpreter and the script as 8.3 short names (GetShortPathNameW) instead —
no spaces, no quotes, so the line survives cmd, PowerShell and a direct
CreateProcess. Long-path fallback (quoted) when 8.3 names are unavailable.

Verified with `codex exec`: SessionStart, UserPromptSubmit and Stop hooks all
Completed; the session-recall, turn-capture and session-capture hooks logged
the Codex session id and a spool file was written.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
GetShortPathNameW shortens every component, including the hook script's
filename (IAI-MC~1.SH), which defeats the installers' marker match (the next
install appends a duplicate) and makes the Codex hook browser unreadable.
Replace only components that contain a space with their 8.3 name; keep the
rest long: `C:/PROGRA~1/Git/bin/bash.exe C:/Users/KYLEMI~1/.codex/hooks/iai-mcp-turn-capture.sh`.

Verified with `codex exec`: SessionStart, UserPromptSubmit and Stop hooks
Completed and logged the session id; reinstall is idempotent again.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@CodeAbra

Copy link
Copy Markdown
Owner

Hey Kyle — any luck with the rebase? The branch still shows conflicts against main, so I can't merge it yet. Happy to help if you're stuck. Once it's rebased on top of current main I'll get it in.

Kmmille2 and others added 2 commits September 16, 2026 19:45
…rain

Three reasons captures never reached the store on a busy Windows machine:

- idle_detector had no Windows backend, so os_idle_time_sec() returned
  nothing and the daemon judged idleness only by the MCP wrapper heartbeat,
  which stays fresh while any host is open. It never reached DROWSY, so it
  never drained the capture queue or consolidated. Add a GetLastInputInfo
  backend (interactive session input idle, the HIDIdleTime counterpart) and
  report it in status()/describe().
- The startup drain is one-shot: when the background gate is busy at boot
  (graph preload / community detection over a large store) it returned
  "deferring to the next cycle", but nothing re-scheduled it. Retry on a
  cadence (IAI_MCP_BOOT_DRAIN_RETRY_SEC, default 120 s, up to
  IAI_MCP_BOOT_DRAIN_MAX_ATTEMPTS).
- Same-day recall needs a drain while a host is still open. When OS input
  idle shows the human away for IAI_MCP_WAKE_IDLE_DRAIN_SEC (default 300 s)
  and the FSM is WAKE, run the drowsy-edge drain, at most once per
  IAI_MCP_WAKE_IDLE_DRAIN_INTERVAL_SEC (default 600 s). 0 disables.

Verified: doctor reports "GetLastInputInfo: 1s PASS"; after a restart with a
135-file queue the retrying boot drain emptied it and turns captured that
day were recallable.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
save_state() writes a temp file and os.replace()s it over
lifecycle_state.json. On Windows a concurrent reader (hook or CLI
load_state, backup copy) holding the target open makes the rename fail at
once with PermissionError (WinError 5); the sleep pipeline caught that as a
pipeline exception and aborted the whole consolidation cycle. Retry the
rename briefly (8 attempts, 50 ms linear backoff) on nt only.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants