Skip to content

fix(grading): stop autograder from silently crashing under bash 5#10

Merged
lassebenni merged 1 commit into
mainfrom
fix/grader-arithmetic-crash
Jul 12, 2026
Merged

fix(grading): stop autograder from silently crashing under bash 5#10
lassebenni merged 1 commit into
mainfrom
fix/grader-arithmetic-crash

Conversation

@lassebenni

Copy link
Copy Markdown
Collaborator

Summary

  • ((var += expr)) reports a failing exit status whenever the resulting arithmetic value is 0 -- a well-known bash gotcha. Combined with set -euo pipefail, the grader silently dies mid-run the moment a student's running score hits exactly 0 at a checkpoint (e.g. a mostly-empty early submission).
  • This only shows up on GitHub Actions' Ubuntu runners (bash 5+); it's invisible when testing locally on macOS (bash 3.2 ships without this strictness), which is how it went unnoticed.
  • Also guards bare check_*() helper calls (which legitimately return 1 to signal warn/fail, not a script error) with || true so they don't trip set -e either.

Test plan

  • bash -n .hyf/test.sh
  • Reproduced against an empty scaffold repo with GNU bash 5: exit code 1 before the fix, exit code 0 after.
  • Re-ran against a known-good local solution: score/pass output unchanged (regression-free).
  • Confirm a real PR run in CI now completes the "Run tests" step for a low/zero-scoring submission.

🤖 Generated with Claude Code

`((score += l1))`-style compound assignment is a false command in bash
whenever the resulting value is 0 -- exit status 1 even though the
assignment succeeded. Under `set -e` (used by every autograder here),
that silently kills the script mid-run on GitHub Actions' Ubuntu
runners (bash 5), while working fine locally on macOS (bash 3.2, which
does not enforce this). Any student whose running score hits exactly 0
at a checkpoint -- most commonly an early, mostly-empty submission --
got a grading run that failed with a bare "exit code 1" and no
score.json update, not a real 0 score.

Reproduced locally with bash 5 against an empty scaffold; confirmed
exit 0 after the fix, and re-verified real solution runs still score
correctly. Fix: rewrite every `((var += expr))`/`((var -= expr))` as
`var=$((var + expr))`, and guard bare check_*() calls that can
legitimately return non-zero (a warn/fail signal, not a script error)
with `|| true` so `set -e` does not treat them as fatal.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

📝 HackYourFuture auto grade

Assignment Score: 48 / 100 ❌

Status: ❌ Not passed
Minimum score to pass: 60
🧪 The auto grade is experimental and still being improved

Test Details

=== Week 10 Autograder ===
  ✓ PASS  found dbt_project.yml
  ✓ PASS  found packages.yml
  ✓ PASS  found profiles.yml.example
  ✓ PASS  found macros/safe_divide.sql
  ✓ PASS  found models/staging/_sources.yml
  ✓ PASS  found models/staging/stg_trips.sql
  ✓ PASS  found models/staging/stg_zones.sql
  ✓ PASS  found models/marts/fct_daily_borough_stats.sql
  ✓ PASS  found reports/answers.md
  ✓ PASS  found AI_ASSIST.md
  ✓ PASS  found singular test in tests/
  ✗ FAIL  docs/lineage.png missing -- run dbt docs generate + serve and screenshot the lineage graph
  ✓ PASS  Level 1: required files (0/10 pts)
  ✓ PASS  .gitignore: profiles.yml excluded
  ✓ PASS  profiles.yml.example: uses env_var() -- no hardcoded password
  ✓ PASS  profiles.yml not committed (correctly git-ignored)
  ✓ PASS  Level 2: secrets hygiene (15/15 pts)
  ✓ PASS  stg_trips.sql: uses {{ source() }} reference
  ✗ FAIL  stg_trips.sql: must filter WHERE pickup_location_id IS NOT NULL
  ✗ FAIL  stg_trips.sql: must filter WHERE fare_amount >= 0
  ✗ FAIL  stg_trips.sql: tip_pct column missing -- add {{ safe_divide('tip_amount', 'fare_amount') }} AS tip_pct
  ✓ PASS  stg_zones.sql: uses {{ source() }} and is filled
  ✓ PASS  Level 3: staging models (8/20 pts)
  ✓ PASS  fct_daily_borough_stats.sql: file filled
  ✓ PASS  mart: ref() to both stg_trips and stg_zones
  ✗ FAIL  mart: no JOIN -- must join stg_trips to stg_zones on pickup_location_id = location_id
  ✗ FAIL  mart: no GROUP BY -- must aggregate to (pickup_borough, pickup_date) grain
  ✓ PASS  mart: all 6 required output columns present
  ✓ PASS  Level 4: mart model (12/20 pts)
  ✗ FAIL  mart YAML: missing dbt_utils.unique_combination_of_columns on (pickup_borough, pickup_date)
  ✓ PASS  staging YAML: not_null tests present
  ✓ PASS  singular test: filled (does not reference fct_daily_borough_stats -- check)
  ✓ PASS  Level 5: tests (8/15 pts)
  ✗ FAIL  mart YAML: grain mentioned but description still has TODO -- fill it in
  ✗ FAIL  mart YAML: column descriptions are empty -- explain meaning and units for each column
  ✓ PASS  Level 6: documentation (0/10 pts)
  ✓ PASS  reports/answers.md: 4 SQL blocks present and filled
  ✗ FAIL  AI_ASSIST.md: empty or still contains TODO stubs
  ✓ PASS  Level 7: business answers + AI log (5/10 pts)

Score: 48 / 100  (passing: 60)  pass=false

@lassebenni
lassebenni merged commit b5b6c8b into main Jul 12, 2026
1 of 2 checks passed
@lassebenni
lassebenni deleted the fix/grader-arithmetic-crash branch July 12, 2026 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant