macOS shared-memory attach/create race fails with OSError(22, 'Invalid argument')

[christiefhyang]

What happened?

Shared-memory concurrency/race handling fails reproducibly on macOS when a process attempts to attach/open shared memory during the creation window.

Two separate SHM tests reproduce the same underlying failure:

OSError(22, 'Invalid argument')

Both tests were automatically rerun twice by pytest and continued to fail.

Related shared-memory KeyError messages also appear during the failures.

What did you expect?

The shared-memory attach/open path should tolerate the creation race and successfully attach once the shared-memory segment is ready.

Concurrent shared-memory initialization should not fail with OSError(22, 'Invalid argument').

System

macOS, Apple Silicon / arm64

Robot / Sim / Hardware

None — shared-memory / IPC unit tests.

Steps to reproduce

Test 1:

uv run pytest \
  'dimos/protocol/pubsub/shm/test_ipc_factory.py::test_concurrent_open_survives_the_creation_window' \
  -vv -s --tb=long

Test 2:

uv run pytest \
  'dimos/utils/test_shm.py::test_attach_shm_waits_out_the_race' \
  -vv -s --tb=long

Environment

Install method: uv

Python: 3.12.13

pytest: 8.3.5

Nix: 2.35.2

Cargo: 1.98.1

Rustc: 1.98.1

DimOS branch / commit:

git branch --show-current
git rev-parse --short HEAD

Logs / screenshots

test_concurrent_open_survives_the_creation_window:

KeyError: '/tq_529d9d5b03d9_data'
KeyError: '/tq_0683e5465929_data'
KeyError: '/tq_0e2f9cbf4b59_data'

AssertionError: concurrent open failed: [OSError(22, 'Invalid argument')]
assert not [OSError(22, 'Invalid argument')]

1 failed, 2 rerun

test_attach_shm_waits_out_the_race:

KeyError: '/dimos_test_dcbb21ec854b'
KeyError: '/dimos_test_41936f37a50f'
KeyError: '/dimos_test_16f9ccb17e16'

AssertionError: attacher did not survive the window: [OSError(22, 'Invalid argument')]
assert [OSError(22, 'Invalid argument')] == [65536]

1 failed, 2 rerun

[christiefhyang]

I reproduced the issue across four related shared-memory race tests:

dimos/protocol/pubsub/shm/test_ipc_factory.py::test_concurrent_open_survives_the_creation_window
dimos/utils/test_shm.py::test_attach_shm_waits_out_the_race
dimos/utils/test_shm.py::test_create_or_attach_splits_owner_and_reader
dimos/utils/test_shm.py::test_bare_attach_hits_the_race

The common failure is:

OSError: [Errno 22] Invalid argument

Examples:

AssertionError: concurrent open failed: [OSError(22, 'Invalid argument')]

AssertionError: attacher did not survive the window: [OSError(22, 'Invalid argument')]

test_create_or_attach_splits_owner_and_reader also shows the competing creation path:

FileExistsError: [Errno 17] File exists

During handling of the above exception, another exception occurred:

OSError: [Errno 22] Invalid argument

and test_bare_attach_hits_the_race raises the same OSError(22) from the worker thread.

The first two targeted tests were automatically rerun twice by pytest and continued to reproduce.

System: macOS, Apple Silicon / arm64
Python: 3.12.13
pytest: 8.3.5