[Bug]: `dstack preset stop` may leave the agent's detached helper processes running

[r4victor]

Steps to reproduce

1. dstack apply -f preset.dstack.yml --fleet <fleet>
2. Wait until a trial provisions a run and the agent attaches to it to benchmark.
3. dstack preset stop <id>
4. ps -Ao pid,ppid,pgid,command | grep dpe-

Actual behaviour

Step 4 may still list helpers the agent launched, for example:

  PID  PPID  PGID COMMAND
26529     1 26527 .../python3 /tmp/dpe-b388b239/w/bin/dstack attach qwen3-27b-agentic-18af21c8-4 -p 8002:8000

ppid 1, and a PGID and SID of its own rather than the agent's. It survives indefinitely (observed still running 40+ minutes after the stop, outliving both the agent and its own SSH tunnel), and neither preset stop nor preset delete ever reaps it. Whether a helper is left behind depends on whether the agent happened to background one during the session, so this does not reproduce on every run.

Cause: stop signals only the process group of the agent PID recorded in session.json.

dstack/src/dstack/_internal/cli/services/presets/agent.py

Lines 597 to 612 in 765fb31

	def terminate_agent_process(agent: Optional[PresetSessionProcess]) -> None:
	"""Twin of `_terminate_process` driven by pid, because the caller (`preset stop`) never owned the process."""
	if agent is None or not process_alive(agent):
	return
	agent_pid = agent.pid
	if IS_WINDOWS:
	_terminate_windows_process_tree(agent_pid)
	return
	with suppress(OSError):
	os.killpg(agent_pid, signal.SIGTERM) # pyright: ignore[reportAttributeAccessIssue]
	for _ in range(_TERMINATE_GRACE_SECONDS * 10):
	if not pid_running(agent_pid):
	return
	time.sleep(0.1)
	with suppress(OSError):
	os.killpg(agent_pid, signal.SIGKILL) # pyright: ignore[reportAttributeAccessIssue]

The agent is spawned as a session leader, so foreground tool commands share its group and are terminated correctly. Helpers the agent launches in the background get a new session and a new process group, and reparent to init when the agent dies, so os.killpg cannot reach them. Neither could SID matching, nor the psutil.children(recursive=True) walk the Windows branch uses, since the parent link is already gone.

The success path has the same gap, and additionally deletes the workspace while such a helper may still be running out of it:

dstack/src/dstack/_internal/cli/services/presets/create.py

Lines 756 to 758 in 765fb31

	def _close_agent_session(session: PresetSession, status: Literal["success", "failed"]) -> None:
	_finish_agent_session(session, status)
	remove_agent_workspace(session)

Expected behaviour

Stopping a session, or finishing one, terminates every process that session started.

dstack version

master at 765fb31 (source checkout; dstack --version reports 0.0.0). macOS 24.0.0.

Additional information