fix(hpc/sync): create remote project dir before parallel push rsyncs

[Jammy2211]

The bug

On the first push to a new remote project, push() launched the CODE_DIRS rsyncs in parallel before anything had created ${HPC_BASE}/${PROJECT_NAME}. rsync only creates the final path level, so every one of them died with mkdir failed: No such file or directory.

The later [root files] rsync then created the base dir as a side effect, so dataset/ synced fine and the whole command exited 0. The failure stayed silent until sbatch could not find hpc/batch_gpu.

Originally reported against autolens_assistant (job 330464 postmortem, 2026-07-16). This repo's hpc/sync is byte-identical to the file the bug was reported against (blob 1560e64), so it carries the same defect.

Reproduced against this repo's unmodified script with a local fake-HPC harness (real rsync, stubbed ssh): first push exits 0 with hpc/batch_gpu missing.

The fix

Both changes are confined to push().

1. Root cause — ssh mkdir -p the remote project dir before the parallel rsyncs, matching what pull, pull_logs and push_data_init already do. Skipped under --dry-run so status stays side-effect free.
2. Why it went unnoticed — plain wait returns its own status, so a failed backgrounded rsync was swallowed even under set -euo pipefail. The background PIDs are now collected and waited on individually; push() names the failing directory and returns 1, so push-submit aborts instead of submitting against an incomplete tree.

Verification

Case	Before	After
First push to fresh remote	exit 0, hpc/batch_gpu missing	exit 0, full tree transferred
Injected transfer failure	exit 0, silent	exit 1, names the dir
push-submit with a failure	reached sbatch	aborts before sbatch
Re-push / --no-data	exit 0	exit 0
status (dry-run)	exit 1, no ssh	exit 1, no ssh — 0 ssh calls, nothing created

status exiting 1 is pre-existing on both fresh and existing remotes; the baseline was checked and is unchanged by this diff. The file mode (100755) is preserved.

Related

Matching PRs: autolens_assistant (the original report) and autogalaxy_assistant. autocti_assistant has no hpc/sync and is unaffected.

---

Generated by Claude Code