Rework Sentinel into a change-risk and review-assurance engine

.github/workflows/ci.yml

@@ -0,0 +1,39 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: pnpm/action-setup@v4
+        with:
+          version: 10
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+          cache: pnpm
+
+      - name: Install dependencies
+        run: pnpm install --frozen-lockfile
+
+      - name: Lint
+        run: pnpm lint
+
+      - name: Typecheck
+        run: pnpm exec tsc --noEmit
+
+      - name: Test
+        run: pnpm test
+
+      - name: Build
+        run: pnpm build
+        env:
+          DATABASE_URL: postgresql://placeholder:placeholder@localhost:5432/placeholder
+          REDIS_URL: rediss://placeholder@localhost:6379

BUILD_SPEC.md

@@ -0,0 +1,149 @@
+# Sentinel rebuild: Change-Risk and Review-Assurance
+
+## Thesis
+
+Stop guessing who or what wrote a change. Measure two things that are real and
+verifiable, then combine them into one explainable score:
+
+- **Blast radius** of a change: how much damage it could do, from the files it touches.
+- **Review assurance** of a change: how well it was actually scrutinized before it merged.
+
+**Change Risk Score = Blast Radius x (1 - Review Assurance).**
+
+Risk is blast radius that went under-reviewed. A critical change that was reviewed well
+scores low. A critical change merged with no review scores high. A trivial test file merged
+with no review scores low, because its blast radius is low. Every score cites the concrete
+facts that produced it. Nothing is a vibe, nothing is provider-specific, nothing guesses
+authorship.
+
+This replaces the old premise entirely (detecting AI-authored code, "AI Code Percentage",
+"Verification Tax"). None of those concepts, and no tool names, survive.
+
+## Unit of analysis: a Change
+
+A `Change` is a merged or open pull request with its commits, files, reviews, and outcomes.
+A push straight to the default branch with no pull request is a synthetic direct-push change
+and is itself a review-assurance red flag (assurance = 0).
+
+## Signals (every signal produces a human evidence string)
+
+Each `RiskSignal` has: `name`, `axis` (`blast_radius` | `review_assurance` | `outcome`),
+`value` (number), `weight`, `matched` (bool), `evidence` (string citing the concrete fact).
+
+### Axis A: Blast Radius (BR), range 0..1
+
+Classify each touched file into the highest matching sensitivity class, take the max class
+weight, then add capped modifiers.
+
+- `critical` (1.0): auth, session, token, password, secret/credential/env handling, crypto,
+  payment, billing, database migrations (`*.sql`, `migrations/`, `drizzle/`), infrastructure
+  (`Dockerfile`, `docker-compose`, `k8s`, `terraform`, `.github/workflows/`), deserialization.
+- `sensitive` (0.7): api routes (`app/api/`, `route.ts`, `/api/`), middleware,
+  permission/role/access checks, data-access and raw db queries, outbound network clients.
+- `standard` (0.4): general application and business logic.
+- `low` (0.15): tests, specs, config files, type-only files, `.d.ts`, stories, mocks,
+  fixtures, docs, generated lockfiles.
+
+Modifiers (added to the max class weight, total capped at 1.0):
+- coupling: `+0.05` per distinct top-level module/dir touched beyond the first, capped `+0.15`.
+- churn: `+0.05` if additions+deletions > 400, `+0.10` if > 1000.
+- hot file: `+0.15` if any touched file has a prior incident link or high change frequency.
+
+### Axis B: Review Assurance (RA), range 0..1 (1 = well reviewed, 0 = unreviewed)
+
+Start at a baseline and apply grounded penalties and credits, clamp to 0..1.
+
+Hard floors:
+- `direct_push` (no pull request): RA = 0.
+- `unreviewed` (merged with 0 approving reviews): RA = 0.05.
+
+Penalties:
+- `self_merged` (author is the merger): -0.25.
+- `single_reviewer` on a sensitive or critical change: -0.20.
+- `rubber_stamp`: approval latency below `churn / 500` minutes (faster than plausibly
+  readable for the diff size): -0.30.
+- `stale_approval`: new commits pushed after the last approval, so the approval does not
+  cover the merged diff: -0.20.
+
+Credits:
+- `multiple_reviewers` (>= 2 distinct approving reviewers): +0.20.
+- `engaged_review` (review comments present, not just an approve click): +0.15.
+- `ci_passed_before_merge`: +0.10.
+
+Baseline for a reviewed change is 0.6; a critical change wants RA >= 0.7 (two independent
+reviewers) to count as assured.
+
+### Outcome linkage (O) - realized risk, does NOT feed the pre-merge score
+
+- `reverted`: a later commit or PR reverts this change (`Revert "<title>"`, or a revert of
+  its merge sha).
+- `hotfixed`: a fix/hotfix change to the same critical files within 48h of merge.
+- `incident_linked`: an incident references this change (sha or PR number) or touches the
+  same files during its window.
+
+Outcome is used for (1) a realized-risk view, (2) hot-file learning (a file that was
+reverted or incident-linked becomes hot, feeding future blast radius), and (3) calibration:
+do high Change-Risk-Score changes actually revert more often? That calibration number is the
+product's honesty metric and must be shown.
+
+## Change Risk Score (CRS)
+
+`CRS = BR * (1 - RA)`, range 0..1, deterministic.
+
+Tiers (keep the existing `riskTierEnum` values `T1..T4`, redefine their meaning; update all
+human-facing labels):
+- T1 `contained`: CRS < 0.15
+- T2 `moderate`: 0.15 <= CRS < 0.40
+- T3 `elevated`: 0.40 <= CRS < 0.70
+- T4 `critical`: CRS >= 0.70 (high blast radius that went under-reviewed)
+
+## Org-level metrics (replace AI% and Verification Tax)
+
+- **Change Risk Index (CRI)** 0..100: blast-radius-weighted mean CRS over merged changes in
+  the window. The headline number.
+- **Review Assurance %**: share of merged changes, weighted by blast radius, that cleared the
+  review bar (>= 1 independent approval with plausible latency; critical changes need >= 2).
+- **Unreviewed Critical Changes**: count of high-blast-radius changes merged with RA below the
+  assured threshold in the window. Grounded, not invented. This is the number for a review.
+- **Realized Risk / Revert-Linked Rate**: share of merged changes later reverted, hotfixed, or
+  incident-linked, plus model calibration (mean realized-failure rate by predicted tier).
+
+## Alerts (retarget `src/alerts/rules.ts`, delete cost logic)
+
+- `unreviewed_critical` (critical): a critical-blast-radius change merged with RA below floor;
+  message cites files, merger, review latency.
+- `cri_spike` (warning/critical): Change Risk Index rises materially over its baseline.
+- `review_assurance_drop` (warning): Review Assurance % falls below threshold.
+- `revert_cluster` (warning): N reverts/hotfixes within a window (realized risk surging).
+
+Delete `COST_PER_HOUR`, `formatCost`, `verificationTaxSpike`, `aiCodeHigh`, `aiCodeCritical`,
+and every dollar or AI-percentage concept.
+
+## Naming and hygiene (hard requirements)
+
+- Remove every occurrence of: Copilot, Cursor, Claude, ChatGPT, GPT-4, Gemini, CodeWhisperer,
+  Tabnine, "AI code", "AI-generated", "AI assist", "generated by", "auto-generated" as a
+  detection target, "verification tax", `COST_PER_HOUR`. The token "AI" must not appear in
+  source, docs, UI copy, or comments.
+- Rename: `detectAI` -> `scoreChange`; `DetectionResult` -> `ChangeScore`;
+  `DetectionSignal` -> `RiskSignal`; `detectionMethodEnum` -> `scoreMethodEnum`
+  (values `deterministic`, `manual_override`).
+- No em-dashes or en-dashes anywhere. Author every commit as SAY-5 / say.apm35@gmail.com,
+  single-line conventional messages, no Co-Authored-By lines.
+
+## Files to replace vs keep
+
+Keep: organizations, installations, repositories, events, incidents, alerts tables; tRPC;
+queue/workers; signed webhook ingestion of push / pull_request / pull_request_review /
+deployment_status; the dashboard shell and components.
+
+Replace: `src/analysis/detector.ts`, `src/analysis/risk.ts`, `src/analysis/types.ts`,
+everything under `src/analysis/signals/`, `src/alerts/rules.ts`, the metric queries and
+dashboard copy, README and all docs.
+
+Add: a Drizzle migration for the new change/review columns
+(reviewerLogins jsonb, approvingReviewCount, reviewLatencySeconds, selfMerged, directPush,
+staleApproval, ciPassedBeforeMerge, modulesTouched, churn, blastRadius, reviewAssurance,
+changeRiskScore, revertedByEventId, hotfixLinked, incidentLinked), plus unit tests for the
+blast-radius classifier, the review-assurance calculator, the CRS formula, and revert
+detection, and a GitHub Actions CI workflow (install, lint, typecheck/build, test).