Skip to content

fix(grading): stop autograder from silently crashing under bash 5#6

Merged
lassebenni merged 1 commit into
mainfrom
fix/grader-arithmetic-crash
Jul 12, 2026
Merged

fix(grading): stop autograder from silently crashing under bash 5#6
lassebenni merged 1 commit into
mainfrom
fix/grader-arithmetic-crash

Conversation

@lassebenni

Copy link
Copy Markdown
Collaborator

Summary

  • ((var += expr)) reports a failing exit status whenever the resulting arithmetic value is 0 -- a well-known bash gotcha. Combined with set -euo pipefail, the grader silently dies mid-run the moment a student's running score hits exactly 0 at a checkpoint (e.g. a mostly-empty early submission).
  • This only shows up on GitHub Actions' Ubuntu runners (bash 5+); it's invisible when testing locally on macOS (bash 3.2 ships without this strictness), which is how it went unnoticed.
  • Also guards bare check_*() helper calls (which legitimately return 1 to signal warn/fail, not a script error) with || true so they don't trip set -e either.

Test plan

  • bash -n .hyf/test.sh
  • Reproduced against an empty scaffold repo with GNU bash 5: exit code 1 before the fix, exit code 0 after.
  • Re-ran against a known-good local solution: score/pass output unchanged (regression-free).
  • Confirm a real PR run in CI now completes the "Run tests" step for a low/zero-scoring submission.

🤖 Generated with Claude Code

`((score += l1))`-style compound assignment is a false command in bash
whenever the resulting value is 0 -- exit status 1 even though the
assignment succeeded. Under `set -e` (used by every autograder here),
that silently kills the script mid-run on GitHub Actions' Ubuntu
runners (bash 5), while working fine locally on macOS (bash 3.2, which
does not enforce this). Any student whose running score hits exactly 0
at a checkpoint -- most commonly an early, mostly-empty submission --
got a grading run that failed with a bare "exit code 1" and no
score.json update, not a real 0 score.

Reproduced locally with bash 5 against an empty scaffold; confirmed
exit 0 after the fix, and re-verified real solution runs still score
correctly. Fix: rewrite every `((var += expr))`/`((var -= expr))` as
`var=$((var + expr))`, and guard bare check_*() calls that can
legitimately return non-zero (a warn/fail signal, not a script error)
with `|| true` so `set -e` does not treat them as fatal.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📝 HackYourFuture auto grade

Assignment Score: 39 / 100 ❌

Status: ❌ Not passed
Minimum score to pass: 60
🧪 The auto grade is experimental and still being improved

Test Details

=== Week 5 Autograder ===
  PASS: Level 1: required files (15/15 pts)
  FAIL: Dockerfile does not use python:3.11-slim base image
  FAIL: Dockerfile does not copy requirements before source code
  PASS: Dockerfile has a CMD instruction
  PASS: Level 2: Dockerfile (3/15 pts)
  PASS: tests/test_pipeline.py has 10 test functions (≥2 required)
  PASS: tests/test_pipeline.py has no NotImplementedError stubs remaining
  PASS: Level 3: unit tests (10/10 pts)
  FAIL: requirements.txt has no pinned packages (use package==version)
  PASS: Level 4: pinned dependencies (0/10 pts)
  PASS: ci.yml triggers on pull_request
  PASS: ci.yml triggers on push to main
  FAIL: ci.yml missing ruff check step
  FAIL: ci.yml missing ruff format step
  FAIL: ci.yml missing pytest step
  FAIL: ci.yml missing docker build step
  PASS: Level 5: CI workflow (8/20 pts)
  FAIL: pipeline.py does not read from os.environ or os.getenv
  FAIL: pipeline.py still contains NotImplementedError
  PASS: Level 6: env-var config (0/15 pts)
  FAIL: assets/acr_push_week5.png missing (Task 7 deliverable)
  PASS: Level 7: ACR screenshot (0/10 pts)
  PASS: AI_ASSIST.md has all three required sections
  FAIL: AI_ASSIST.md still contains TODO placeholders or is too short (681 chars)
  PASS: Level 8: AI report (3/5 pts)
  PASS: .gitignore correctly excludes __pycache__/, *.pyc, and .env

Score: 39 / 100  (passing: 60)  pass=false

@lassebenni
lassebenni merged commit ca1669b into main Jul 12, 2026
1 of 2 checks passed
@lassebenni
lassebenni deleted the fix/grader-arithmetic-crash branch July 12, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant