Skip to content

fix(skills): leave agent-enhanced files uncommitted for human review (fixes #738) - #743

Open
Daniel-park10 wants to merge 2 commits into
darnitdevorg:mainfrom
Daniel-park10:fix/comply-uncommitted-edits-report
Open

Daniel-park10 wants to merge 2 commits into
darnitdevorg:mainfrom
Daniel-park10:fix/comply-uncommitted-edits-report

Conversation

@Daniel-park10

Copy link
Copy Markdown
Contributor

Summary

/darnit-comply and /darnit-remediate previously asked the agent to make a new commit after enhancing generated templates or fixing syntax. In headless MCP setups without shell access, the agent has no git commit tool. On top of that, commit_remediation_changes intentionally commits only files that match what remediation wrote (framework-design 15.6 / 15.7), so any edited file gets flagged as a conflict and left uncommitted.

Following Option 1 from #738, this updates the skills so the agent leaves enhanced files uncommitted in the working tree and surfaces them in the final report for the person to review and commit. This preserves darnit's core rule that it never commits content it did not write. Also bumped the skill metadata versions, added regression tests in test_procedures.py, and updated CHANGELOG.md.

Fixes #738

Type of Change

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)

Framework Changes Checklist

If this PR modifies the darnit framework (packages/darnit/):

  • Updated framework spec (docs/architecture/framework-design.md) if behavior changed
  • Ran uv run python scripts/validate_sync.py --verbose and it passes

Control/TOML Changes Checklist

If this PR modifies controls or TOML configuration:

  • Control metadata defined in TOML (not Python code)
  • SARIF fields (description, severity, help_url) included where appropriate
  • Ran validation to confirm TOML schema compliance

Testing

  • Tests pass locally (uv run pytest tests/ -v)
  • Added tests for new functionality (if applicable)
  • Linting passes (uv run ruff check .)

AI assistance

  • No AI assistance was used
  • AI assistance was used

Additional Notes

@mlieberman85 mlieberman85 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Daniel-park10. This takes option 1 from #738, and the new text matches framework-design 15.7. One ordering problem remains: in both skills, the PR is opened while the agent's edits are still uncommitted, so the PR doesn't contain them. That matters most in /darnit-remediate, so I'm requesting changes.

Tested: tests/darnit/server/ on main (aae02e4) with this branch merged in: 249 passed. The added lines are ASCII. Neither the other skills nor the docs still tell the agent to make a commit. The ordering problem below comes from reading the skill text against create_remediation_pr and git_state.py; I didn't run an agent through it.

Blocking: the PR doesn't contain the edits. create_remediation_pr pushes the run's branch as committed, so files left uncommitted are not in the PR.

  • In /darnit-remediate, step 3b fixes broken syntax, such as malformed workflow YAML, and step 4 then offers the PR. The PR goes out with the broken file the agent just found.
  • In /darnit-comply, steps 7-9 judge, confirm and re-audit the enhanced working-tree files. The PR then carries the generated versions, which weren't the ones judged.

The person can't simply commit the files and then ask for the PR either. create_remediation_pr refuses a branch with a commit that lacks the Darnit-Remediation-Run trailer (git_operations.py:537, via foreign_commits in git_state.py:182), so a plain commit blocks the tool.

Asking the person to add the trailer to their own commit would get past the check, but the trailer is what marks content remediation wrote, so that would make the record wrong.

Before offering the PR, both skills should list the files left uncommitted and ask the person to review them. Then the agent should say plainly that the PR will not contain them: the person commits and pushes them to the PR branch after it is opened. In /darnit-comply, add that the judgments describe the uncommitted versions. For /darnit-remediate, a generated file that needed a syntax fix is a template bug, and the summary should say so rather than offer a PR that carries the broken file without a warning. If you think the tools should instead commit person-approved edits, raise it on #738, since that changes what the trailer means.

Smaller:

  • The darnit-comply text says commit_remediation_changes "commits only unchanged files from the run manifest". "Files whose content still matches what remediation wrote (framework-design 15.7)" would be clearer, because "unchanged" can be read as "files the run didn't change".
  • test_remediation_skills_instruct_uncommitted_edits_for_review checks only for two phrases. Once the ordering is fixed, a check that each skill names the uncommitted files before it offers the PR would guard what matters.

Drafted with AI assistance (Claude), reviewed and filed by me.

@Daniel-park10

Copy link
Copy Markdown
Contributor Author

@mlieberman85 thanks for pointing the ordering problem with create_remediation_pr.
i pushed an update fixing the flow in both skills. before offering to create a PR, each skill now lists the uncommitted working-tree files for review and states plainly that the PR will not contain them, so the maintainer commits and pushes them to the PR branch after it is opened.

In /darnit-comply, it explicitly notes that the judgments describe these uncommitted versions, and updated the wording for commit_remediation_changes to match framework-design 15.7. In /darnit-remediate, any syntax fix is now explicitly called out as a template bug before PR creation. Also added a test in test_procedures.py ensuring the uncommitted file warning precedes the PR offer.
PTAL when you have a moment!

@Daniel-park10
Daniel-park10 force-pushed the fix/comply-uncommitted-edits-report branch from e256a6f to b3a25f1 Compare November 13, 2026 20:12
mlieberman85
mlieberman85 previously approved these changes Nov 16, 2026

@mlieberman85 mlieberman85 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Daniel-park10. Since the change is in what the agent does, I tested it with an agent.

Tested on main (c39bb37), and on main with this branch merged in. Each side ran /darnit:darnit-comply headless in Claude Code on whiskeytastingfoundation/wtf-frontend at 939e838. Setup on both sides:

  • a checkout with no remote;
  • darnit in a throwaway HOME, with no platform token;
  • recorded answers, every file change approved, and no PR wanted.

Unlike our usual replay, the agent could edit files in the checkout, so step 6 ran.

main here
files the agent enhanced in step 6 6 5
commit_remediation_changes calls 2, both refused: "Conflict: files changed after remediation wrote them. Nothing was committed." none
enhanced files at the end uncommitted uncommitted
final report lists them as uncommitted and suggests git add -A && git commit lists them under "Uncommitted files for your review", says that a PR from the branch would not contain them until they are committed and pushed, and that the judgments describe the working-tree versions

Both sides had the same remediation outcomes, the same 20 changed files and the same final audit. I ran one run per side, and the agent's wording varies from run to run.

Full suite on the merged tree: 6000 passed, 11 skipped; ruff check and validate_sync.py pass.


Drafted with AI assistance (Claude), reviewed and filed by me. Measurements quoted were executed and verified.

@Marc-cn

Marc-cn commented Nov 16, 2026

Copy link
Copy Markdown
Member

Tested merged with current main on a VM: skill tests pass. Only CHANGELOG.md conflicts no please rebase and I will merge.

Assisted by Claude Code

…ixes darnitdevorg#738)

In /darnit-comply and /darnit-remediate, instruct the agent to leave
enhanced files and syntax fixes uncommitted in the working tree and report
them to the person for review and commit. This preserves darnit's rule
that it never commits content it did not write (framework-design 15.6 /
15.7) and eliminates instructions to 'make a new commit' where MCP agents
lack shell access and commit_remediation_changes deliberately commits
only unmodified manifest files.

Add regression test in test_procedures.py asserting no skill instructs
the agent to make a new commit.

Fixes darnitdevorg#738

Signed-off-by: Daniel-park10 <dev10.tennyson@gmail.com>
…itdevorg#738)

Instruct /darnit-comply and /darnit-remediate to list uncommitted working-tree
files for review before offering PR creation, and state plainly that the pull
request will not contain them. Because create_remediation_pr pushes only what
remediation committed, uncommitted files must be reviewed, committed, and
pushed to the PR branch by the maintainer after it is opened.

In /darnit-comply, clarify that step 7-8 judgments describe these uncommitted
versions and update the commit_remediation_changes wording to match framework
design 15.7. In /darnit-remediate, explicitly flag generated template syntax
fixes as template bugs before offering the PR. Add procedure tests ensuring
uncommitted file warnings precede the PR creation call.

Signed-off-by: Daniel-park10 <dev10.tennyson@gmail.com>
@Daniel-park10

Copy link
Copy Markdown
Contributor Author

@Marc-cn rebased and resolved the CHANGELOG.md conflict

@mlieberman85 mlieberman85 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Daniel-park10. The rebase dismissed my approval, so here it is again. Compared with b3a25f1, only the CHANGELOG context changed: the skill text and the tests are the ones I ran an agent through last time. I didn't rerun the agent. CI is green on this head, which is on current main (714b3e6).


Drafted with AI assistance (Claude), reviewed and filed by me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

darnit-comply asks the agent to commit its edits, but no tool can commit them

3 participants