Skip to content

arxiv inbox: fail loudly when the filing push is denied - #317

Merged
Jammy2211 merged 4 commits into
mainfrom
claude/arxiv-inbox-queue-issue-x16qll
Aug 25, 2026
Merged

arxiv inbox: fail loudly when the filing push is denied#317
Jammy2211 merged 4 commits into
mainfrom
claude/arxiv-inbox-queue-issue-x16qll

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

The first live run of the arXiv inbox filing step (PyAutoMemory#57, shipped
2026-08-24) failed on its first night and reported success.

Run 32803698132,
2026-08-25 03:03 UTC — every step green, and the inbox step's own log says:

inbox: appended:1, swept:0
[main 3798728] inbox: appended:1, swept:0 (arXiv digest 2026-08-25 (Tue))
 1 file changed, 1 insertion(+)
remote: Permission to PyAutoLabs/PyAutoMemory.git denied to Jammy2211.
fatal: unable to access 'https://github.com/PyAutoLabs/PyAutoMemory.git/': The requested URL returned error: 403

inbox_actions.py append worked, the commit was made, the push was denied
three times, and the step exited 0 with a ::warning::. The commit died with
the runner. The digest posted to #papers as normal and the knowledge board
read "nothing waiting", so nothing anywhere signalled that a paper was lost.

Root cause (already resolved out-of-band)

The token, not the code: PAT_PYAUTOLABS granted write on the *-template
repos that spawn_drift.yml writes, not on PyAutoMemory itself. The inbox
step was the first thing in the org to need Contents: read-and-write there.
The clone succeeds because read is unrestricted, so the first hint of trouble
is the push.

The scope has since been widened, and a workflow_dispatch on main with
lookback_hours=168
(run 32847233221)
filed appended:2, swept:0 and pushed clean. The inbox now holds both papers.

What this PR changes

arxiv_papers.yml's inbox step no longer exits 0 on a failed push. The step's
declared design was already "expected-absent preconditions warn and exit 0; a
genuinely broken script still fails the step"
— a denied push is the second
case and was wired as the first. The missing-survivors and unset-PAT
preconditions keep warn-and-exit-0.

This does not weaken "a cross-repo failure must never cost the morning's
post"
: the step runs after the Slack POST, so failing it cannot touch the
digest — it only turns the run red, which is the signal that was missing.

An auth failure also short-circuits the retry loop. That retry exists for a
concurrent-push race against the board and queue-action workflows; a 403 is not
a race and cannot clear on a rebase, so retrying only buried the real reason
three screens up.

Both error messages name the backfill recipe: a workflow_dispatch with
lookback_hours=168, not 24 — the lookback is submission-anchored, so a
paper announced today may have been submitted days earlier (PyAutoMind#79).

Also: a testing trap, recorded at the Claude step

The first backfill attempt was dispatched from this branch so the new
fail-loudly path would be exercised. It cannot be — claude-code-action@v1
refuses to run when the workflow file differs from the default branch's copy:

Skipping action due to workflow validation: Workflow validation failed. The
workflow file must exist and have identical content to the version on the
repository's default branch.

It then exits success, so the run continues with no slack_payload.json,
the POST fails on the missing file, and every later step — the inbox filing
included — is skipped. The error it prints blames an expired
CLAUDE_CODE_OAUTH_TOKEN, which is the wrong cause entirely. Noted in a
comment at the Claude step, where the next person editing this file will be
standing.

Verification, and its limit

  • lifecycle check: OK
  • 187 tests pass
  • YAML parses; the changed step passes bash -n

Being explicit about what is not proven: the 403 path this PR fixes cannot
be re-triggered now that the token works, and the trap above means a branch
dispatch cannot reach that step either. Those lines are shell-syntax-checked
and reasoned, not live-fired. The step is plain bash with no dependency on
claude-code-action, so merging is what proves it — the first real end-to-end
exercise is the next scheduled 02:00 UTC run on main.

Also filed

draft/bug/pyautomemory/arxiv_inbox_never_filled_pat_scope.md — the diagnosis,
the resolution, the trap, and one follow-up left open: the board's empty state
is still indistinguishable from a broken filing run, which is exactly what the
#papers heartbeat exists to prevent. That is a PyAutoMemory board change and
a separate task.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Sr4R6EDkgmAhm9tmvPiDUV


Generated by Claude Code

claude added 4 commits August 25, 2026 11:55
The first live run of the inbox filing step (PyAutoMemory#57) failed on its
first night and reported success. Run 32803698132: the digest posted to
#papers, `inbox_actions.py append` added the paper, the commit was made, and
the push was denied 403 three times — then the step exited 0 with a warning.
The knowledge board reads "nothing waiting" and the run is green, so nothing
anywhere signals that a paper was lost.

Root cause is the token, not the code: PAT_PYAUTOLABS grants write on the
*-template repos that spawn_drift.yml writes, not on PyAutoMemory itself. The
clone succeeds because read is unrestricted, so the first hint of trouble is
the push. That needs a human to widen the PAT scope; this commit fixes the
part that hid it.

The step's declared design was already "expected-absent preconditions warn and
exit 0; a genuinely broken script still fails the step". A denied push is the
second case and was wired as the first. Now it errors and exits 1. The
missing-survivors and unset-PAT preconditions keep warn-and-exit-0.

This does not weaken "a cross-repo failure must never cost the morning's post":
the step runs after the Slack POST, so failing it cannot touch the digest — it
only turns the run red, which is the signal that was missing.

An auth failure also short-circuits the retry loop. The retry is for a
concurrent-push race against the board and queue-action workflows; a 403 is not
a race and cannot clear on a rebase, so retrying only buried the real reason.

Both error messages name the backfill: a workflow_dispatch with
lookback_hours=168, not 24 — the lookback is submission-anchored, so a paper
announced today may have been submitted days earlier (PyAutoMind#79).

Also files draft/bug/pyautomemory/arxiv_inbox_never_filled_pat_scope.md with
the diagnosis, the two human-gated steps (grant the scope, backfill tonight's
lost paper) and a follow-up: the board's empty state is still indistinguishable
from a broken run, which is what the #papers heartbeat exists to prevent.

lifecycle check: OK
187 tests passed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4R6EDkgmAhm9tmvPiDUV
The PAT scope was widened, so the filing now works. A workflow_dispatch on
main with lookback_hours=168 (run 32847233221) filed appended:2, swept:0 and
pushed clean; PyAutoMemory's arxiv-inbox.md holds both papers and the board
renders them.

Also records a trap found while backfilling. The first attempt was dispatched
from this branch so the new fail-loudly path would be exercised — it cannot
be. claude-code-action refuses to run when the workflow file differs from the
default branch's copy, and then exits SUCCESS. The run continues with no
slack_payload.json, the POST fails on the missing file, and every later step
including the inbox filing is skipped. Worse, the failure it prints blames an
expired CLAUDE_CODE_OAUTH_TOKEN, which is the wrong cause entirely.

Noted at the Claude step, where the next person editing this file will be
standing, and in the prompt.

Consequence worth being explicit about: the 403 path the previous commit
fixes cannot be re-triggered now that the token works, so those lines are
shell-syntax-checked and reasoned, not live-fired. The step is plain bash with
no dependency on claude-code-action, so merging to main is what proves it.

lifecycle check: OK
187 tests passed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4R6EDkgmAhm9tmvPiDUV
Brings the branch up to date before regenerating dashboard.md, so the
generated page reflects the merged draft set rather than this branch's
older base.
The PR run's `refresh` check failed: "dashboard.md is stale — run
'pyauto-brain intake --apply dashboard' on this branch and commit the
result". Adding draft/bug/pyautomemory/arxiv_inbox_never_filled_pat_scope.md
changed the draft set and AGENTS.md requires regenerating the page after any
draft/ change; I missed it.

Regenerated after merging origin/main in, so the page reflects the merged
draft set (142 prompts) rather than this branch's older base — otherwise it
would have been stale again the moment it landed.

intake dashboard --check: current
lifecycle check: OK
lifecycle index: OK
197 tests passed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4R6EDkgmAhm9tmvPiDUV
@Jammy2211
Jammy2211 merged commit 308d4fa into main Aug 25, 2026
3 checks passed
Jammy2211 pushed a commit that referenced this pull request Aug 25, 2026
The filing fix shipped as PR #317 (main 308d4fa), so the draft that carried
the diagnosis is no longer a draft. Records it to complete/2026/08/ and folds
the prompt in under `## Original prompt` by hand — `record --prompt` folds an
active/ prompt, and this task never entered active/ (it was filed and shipped
in one session, never routed through /create_issue). `--prompt` is optional
and 299 existing records carry no issue line, so neither is a special case.

The record keeps five things worth not rediscovering:

- a green clone is NOT write access. The 403 lands on the push, several steps
  after a reader assumes auth was proven. #57's record checked that the secret
  existed, not that its scope covered the new target, and a public-repo clone
  succeeds either way.
- you cannot test arxiv_papers.yml from a branch that edits it.
  claude-code-action refuses when the file differs from the default branch's
  copy, then exits SUCCESS — so the POST fails on a missing payload, the inbox
  step never runs, and the error blames an expired OAuth token instead.
- the backfill window is 168h, not 24h: the lookback is submission-anchored,
  so a paper announced today may have been submitted days earlier (#79's trap,
  reachable again through the recovery path).
- adding a draft/ file makes dashboard.md stale and reddens PR CI; regenerate
  AFTER merging the base in, or it is stale again on landing.
- intake's --apply is a global flag before the subcommand while --check is a
  subcommand flag. They are not symmetric.

The one deliberately-open item is split out rather than buried: the board's
empty inbox still reads the same whether arXiv was quiet or the filing broke —
the ambiguity the #papers heartbeat already kills on the Slack side, with no
equivalent on the board. Filed as
draft/feature/pyautomemory/inbox_board_staleness_signal.md.

lifecycle check: OK
lifecycle index: OK
intake dashboard --check: current
197 tests passed

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Sr4R6EDkgmAhm9tmvPiDUV
Jammy2211 pushed a commit that referenced this pull request Aug 25, 2026
The board half of PyAutoMemory#58 reads a `last digest: <date>` line to tell a
quiet inbox from a broken one. That line is only worth anything if the digest
writes it on days with nothing to file — otherwise it goes stale on exactly the
days it exists to prove were quiet, and says "broken" about a working run.

So the inbox step loses its `count != 0` guard and runs every weekday. To keep
its preconditions honest, the empty-day heartbeat step now also writes
arxiv_survivors.json as {"papers": []} — the same file the Claude step already
writes on its own no-papers day. Every path produces one, so a MISSING
survivors file in the inbox step stays what it has always been: a real failure,
not an empty day.

The step's other invariants are untouched. It still runs after the Slack POST,
so a cross-repo auth or push failure cannot cost the morning's post; the
missing-survivors and unset-PAT preconditions still warn and exit 0; a denied
push still reddens the run (PR #317).

Two consequences worth naming:
  - a quiet day is now a one-line commit to PyAutoMemory rather than silence,
    which also re-renders the knowledge board daily. That is the point — it is
    the board's version of the #papers empty-day heartbeat.
  - the sweep now runs every weekday too, so a lapsed line no longer lingers
    through a quiet stretch waiting for the next paper day to evict it.

Re-running the same day is still a no-op: the stamp is unchanged, the porcelain
check sees nothing, and the step exits without a commit.

Validated: YAML parses, every run block passes bash -n, and all four paths were
rehearsed against a real PyAutoMemory checkout with the two shell blocks
extracted verbatim from this file — quiet day same date (no commit), quiet day
new date (heartbeat commit), paper day (paper lands below the stamp), and a
quiet day sweeping lapsed papers. NOT fired live: claude-code-action refuses to
run when this file differs from the default branch's copy and then exits
success, so no branch dispatch can reach these steps. The scheduled run on main
is the first genuine exercise.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ex9YARe5kDjsT3Ug7LKQ1r
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants