Skip to content

docs: add doc-audit skill and document it in README and CLAUDE.md#138

Open
thephez wants to merge 4 commits intodashpay:3.1.0from
thephez:ai-audit-skill
Open

docs: add doc-audit skill and document it in README and CLAUDE.md#138
thephez wants to merge 4 commits intodashpay:3.1.0from
thephez:ai-audit-skill

Conversation

@thephez
Copy link
Copy Markdown
Collaborator

@thephez thephez commented Apr 1, 2026

Summary

Adds a Claude Code doc-audit skill for systematically checking documentation against the Platform codebase, with references in README and CLAUDE.md so doc team members are aware of it.

Preview build: https://dash-docs-platform--138.org.readthedocs.build/en/138/

Summary by CodeRabbit

  • New Features

    • Added doc-audit skill to run evidence-driven documentation audits with structured findings and severity classification.
    • Added doc-audit-update skill to apply approved audit findings as minimal, traceable edits with conflict handling and validation.
  • Documentation

    • Added "Auditing Documentation" usage section and a Claude Code Skills overview documenting both skills and expected audit outputs (timestamped reports and findings).

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 1, 2026

📝 Walkthrough

Walkthrough

Added two new Claude Code skill specs: doc-audit (discovers and reports documentation discrepancies) and doc-audit-update (applies structured, minimal-diff edits for audit findings). Updated CLAUDE.md and README.md to reference these skills.

Changes

Cohort / File(s) Summary
Claude Code Skills
/.claude/skills/doc-audit/SKILL.md, /.claude/skills/doc-audit-update/SKILL.md
Added doc-audit spec (discovery, per-page auditing, findings schema, taxonomy, output audit-results/{timestamp}/summary.md + findings.yaml) and doc-audit-update spec (deterministic apply flow: load findings, group by file, re-resolve locators after each edit, edit types: replace, insert_before/after, augment, delete, conflict handling, validation rules, reporting applied-updates.md and optional changes.diff).
Docs / Manifests
CLAUDE.md, README.md
Inserted a "Claude Code Skills" section listing the two new skills and added a README subsection instructing use of doc-audit (notes on cloning the platform repo and link to spec).

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant DocAuditAgent as DocAudit Agent
  participant Repo as Local Repo / Filesystem
  participant Findings as Findings (audit-results/*)
  participant DocUpdateAgent as DocUpdate Agent
  participant GitHub

  User->>DocAuditAgent: Request audit (scope, repos)
  DocAuditAgent->>Repo: Glob/Grep to discover docs
  Repo-->>DocAuditAgent: Doc file list & contents
  DocAuditAgent->>DocAuditAgent: Per-page audits -> generate `findings.yaml`
  DocAuditAgent->>Findings: Write `audit-results/{timestamp}/findings.yaml` & `summary.md`
  User->>DocUpdateAgent: Invoke apply (provide findings)
  DocUpdateAgent->>Findings: Load findings.yaml
  DocUpdateAgent->>Repo: For each `doc_file`, load content
  DocUpdateAgent->>Repo: Apply edits sequentially (re-resolve locators after each edit)
  Repo-->>DocUpdateAgent: Save edits, produce optional `changes.diff`
  DocUpdateAgent->>GitHub: Validate GitHub line-links & update refs if needed
  DocUpdateAgent->>Findings: Emit `applied-updates.md` with applied/skipped items
  DocUpdateAgent-->>User: Report complete
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 I hopped through docs with careful paws,
Found drifting lines and flagged their cause,
I nudged and patched with gentle art,
Kept tone and structure, played my part,
A little diff, a lighter heart.

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'docs: add doc-audit skill and document it in README and CLAUDE.md' is fully related to the changeset, accurately summarizing the primary changes: adding a doc-audit skill and documenting it in two key files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@thephez thephez marked this pull request as ready for review April 8, 2026 19:40
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
.claude/skills/doc-audit-update/SKILL.md (1)

205-233: Define {timestamp} format explicitly to avoid inconsistent output paths.

Line 205 and Line 233 use {timestamp} but this spec never defines its format. Please standardize it (ideally matching doc-audit) so downstream tooling can rely on stable paths.

Suggested patch
 ### 2. Change report (required)
@@
 Write:
@@
 audit-results/{timestamp}/applied-updates.md
+
+Where `{timestamp}` is the update start time formatted as `YYYYMMDD-HHmmss`.
@@
 ### 3. Optional: patch output
@@
 If requested:
@@
 audit-results/{timestamp}/changes.diff
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.claude/skills/doc-audit-update/SKILL.md around lines 205 - 233, The
SKILL.md uses the placeholder {timestamp} without defining its format; update
the document to explicitly specify the timestamp format (match the existing
doc-audit convention) and give one concrete example string so downstream tooling
can rely on stable paths—e.g., add a short section or sentence near the
audit-results examples that states the exact format to use (the canonical
doc-audit format) and ensure the two occurrences of {timestamp} in the examples
(the applied-updates.md and changes.diff references) are documented to use that
same format.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.claude/skills/doc-audit/SKILL.md:
- Line 297: Update the `audit_id` example to avoid a stale year by replacing
`audit-2024-001` with a year-agnostic or current-year pattern; e.g., use
`audit-YYYY-001` or `audit-<id>` (or substitute the current year such as
`audit-2026-001`) wherever `audit_id` is documented so the example remains
relevant; locate the `audit_id` entry in SKILL.md and edit the example string
accordingly.

---

Nitpick comments:
In @.claude/skills/doc-audit-update/SKILL.md:
- Around line 205-233: The SKILL.md uses the placeholder {timestamp} without
defining its format; update the document to explicitly specify the timestamp
format (match the existing doc-audit convention) and give one concrete example
string so downstream tooling can rely on stable paths—e.g., add a short section
or sentence near the audit-results examples that states the exact format to use
(the canonical doc-audit format) and ensure the two occurrences of {timestamp}
in the examples (the applied-updates.md and changes.diff references) are
documented to use that same format.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f0590320-7b5a-496f-8621-32a6d5b7d9fe

📥 Commits

Reviewing files that changed from the base of the PR and between 5bb2b69 and 09d4cb1.

📒 Files selected for processing (4)
  • .claude/skills/doc-audit-update/SKILL.md
  • .claude/skills/doc-audit/SKILL.md
  • CLAUDE.md
  • README.md


## Schema field notes

- `audit_id` — generate a short unique ID, e.g. `audit-2024-001`
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Update the audit_id example to a non-stale pattern.

Using audit-2024-001 can read as outdated. Prefer a year-agnostic template or current-year example.

Suggested patch
-- `audit_id` — generate a short unique ID, e.g. `audit-2024-001`
+- `audit_id` — generate a short unique ID, e.g. `audit-YYYY-001`
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- `audit_id` — generate a short unique ID, e.g. `audit-2024-001`
- `audit_id` — generate a short unique ID, e.g. `audit-YYYY-001`
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.claude/skills/doc-audit/SKILL.md at line 297, Update the `audit_id` example
to avoid a stale year by replacing `audit-2024-001` with a year-agnostic or
current-year pattern; e.g., use `audit-YYYY-001` or `audit-<id>` (or substitute
the current year such as `audit-2026-001`) wherever `audit_id` is documented so
the example remains relevant; locate the `audit_id` entry in SKILL.md and edit
the example string accordingly.

thephez and others added 4 commits April 8, 2026 15:57
Remove redundant confidence section, update skill_version to 2026-04-06,
add branch/commit fields to audit_run schema, and slim down the example
finding to reduce noise.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Introduces a new skill that applies structured audit findings to docs
with minimal diffs, preserving tone, markup, and style.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
.claude/skills/doc-audit/SKILL.md (1)

297-297: ⚠️ Potential issue | 🟡 Minor

Use a year-agnostic audit_id example.

At Line 297, audit-2024-001 is now stale and can confuse implementers. Prefer audit-YYYY-001 (or similarly year-agnostic format).

Suggested doc patch
-- `audit_id` — generate a short unique ID, e.g. `audit-2024-001`
+- `audit_id` — generate a short unique ID, e.g. `audit-YYYY-001`
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.claude/skills/doc-audit/SKILL.md at line 297, Update the example value for
`audit_id` (the inline text "`audit-2024-001`") to be year-agnostic—replace it
with a placeholder like `audit-YYYY-001` (or equivalent `audit-{YEAR}-001`) so
implementers don’t rely on a hard-coded year; ensure the surrounding text still
refers to `audit_id` as a short unique ID.
🧹 Nitpick comments (2)
.claude/skills/doc-audit-update/SKILL.md (2)

125-137: Add explicit envelope/schema validation before grouping findings.

At Line 125, the flow jumps from “Load findings.yaml” directly to grouping by doc_file. Add a required pre-step to validate the expected envelope (audit_run, scope, summary, findings) and required per-finding keys (doc_file, doc_locator, edit_type, edit_instructions) before processing; otherwise malformed input can produce unsafe or partial updates.

Suggested doc patch
 1. Load findings.yaml
+2. Validate schema envelope and finding requirements:
+   - top-level keys: audit_run, scope, summary, findings
+   - per-finding required keys before acting: doc_file, doc_locator, edit_type, edit_instructions
+   - if validation fails, skip invalid findings and record reason in report
-2. Group findings by doc_file
+3. Group findings by doc_file
-3. For each file:
+4. For each file:
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.claude/skills/doc-audit-update/SKILL.md around lines 125 - 137, Before
grouping findings by doc_file, add an explicit schema/envelope validation step
that checks the top-level keys (audit_run, scope, summary, findings) and
validates each finding contains required keys (doc_file, doc_locator, edit_type,
edit_instructions); implement this validation immediately after the "Load
findings.yaml" step and before "Group findings by doc_file" so malformed input
is rejected or reported early, returning a clear error/exception if required
fields are missing or types are wrong to prevent unsafe/partial updates.

98-100: Clarify stable ordering to keep runs deterministic.

Line 130 says “by line proximity if possible,” which is ambiguous versus the deterministic rule at Line 98. Define a fixed tie-break order (e.g., file path + original finding order + locator position) so repeated runs produce identical edits.

Also applies to: 130-130

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.claude/skills/doc-audit-update/SKILL.md around lines 98 - 100, The guidance
in "Deterministic edits" is ambiguous about tie-breaking; update the text where
it says "by line proximity if possible" to specify a fixed stable ordering rule
such as: sort findings by file path (lexicographically), then by original
finding index/order, then by locator position within the file (e.g., start line,
start column), and use this ordering as the deterministic tie-breaker for
applying edits; apply this wording change both at the existing deterministic
rule and at the later line that currently reads "by line proximity if possible"
so repeated runs produce identical edits.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In @.claude/skills/doc-audit/SKILL.md:
- Line 297: Update the example value for `audit_id` (the inline text
"`audit-2024-001`") to be year-agnostic—replace it with a placeholder like
`audit-YYYY-001` (or equivalent `audit-{YEAR}-001`) so implementers don’t rely
on a hard-coded year; ensure the surrounding text still refers to `audit_id` as
a short unique ID.

---

Nitpick comments:
In @.claude/skills/doc-audit-update/SKILL.md:
- Around line 125-137: Before grouping findings by doc_file, add an explicit
schema/envelope validation step that checks the top-level keys (audit_run,
scope, summary, findings) and validates each finding contains required keys
(doc_file, doc_locator, edit_type, edit_instructions); implement this validation
immediately after the "Load findings.yaml" step and before "Group findings by
doc_file" so malformed input is rejected or reported early, returning a clear
error/exception if required fields are missing or types are wrong to prevent
unsafe/partial updates.
- Around line 98-100: The guidance in "Deterministic edits" is ambiguous about
tie-breaking; update the text where it says "by line proximity if possible" to
specify a fixed stable ordering rule such as: sort findings by file path
(lexicographically), then by original finding index/order, then by locator
position within the file (e.g., start line, start column), and use this ordering
as the deterministic tie-breaker for applying edits; apply this wording change
both at the existing deterministic rule and at the later line that currently
reads "by line proximity if possible" so repeated runs produce identical edits.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: ce462d26-ff2a-4007-bea2-6ecd6d59880e

📥 Commits

Reviewing files that changed from the base of the PR and between 09d4cb1 and 06b809c.

📒 Files selected for processing (4)
  • .claude/skills/doc-audit-update/SKILL.md
  • .claude/skills/doc-audit/SKILL.md
  • CLAUDE.md
  • README.md
✅ Files skipped from review due to trivial changes (2)
  • CLAUDE.md
  • README.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant