Skip to content

feat: specialist historical baselining — 7-day baseline + classification#34

Merged
tallpsmith merged 14 commits into
mainfrom
011-specialist-baselining
Mar 10, 2026
Merged

feat: specialist historical baselining — 7-day baseline + classification#34
tallpsmith merged 14 commits into
mainfrom
011-specialist-baselining

Conversation

@tallpsmith

Copy link
Copy Markdown
Collaborator

Summary

  • Domain specialist prompts (CPU, Memory, Disk, Network, Process) now include a Baseline step that instructs the agent to fetch 7-day historical data and run pcp_detect_anomalies before analysis
  • Each finding is classified as ANOMALY, RECURRING, or BASELINE with baseline_context and severity_despite_baseline fields — captures "your normal is sick" scenarios
  • All 5 domain specialists gain baseline-aware heuristics in their domain knowledge (time-of-day patterns, scheduled jobs, working-set growth)
  • Cross-cutting specialist prioritises ANOMALY findings and flags correlated anomalies across subsystems (no Baseline step — it consumes classifications)
  • Coordinator synthesis ranks by classification tier (ANOMALY > RECURRING > BASELINE), then severity — output structure updated to "Findings by Classification & Severity"
  • Graceful degradation: falls back to threshold-only analysis when baseline data is insufficient
  • Documentation updated: investigation-flow.md diagrams reflect 5-step workflow and classification ranking

Changes

File What
src/pmmcp/prompts/specialist.py Baseline step, classification fields, baseline-aware heuristics, degradation fallback, cross-cutting classification prioritisation
src/pmmcp/prompts/coordinator.py Classification-based ranking in synthesis, updated output structure
tests/unit/test_prompts_specialist.py 14 new tests (baseline step, classification, heuristics, cross-cutting, degradation)
tests/unit/test_prompts_coordinator.py 3 new tests (ranking, baseline callout, recurring patterns)
docs/investigation-flow.md 5-step workflow diagram, classification ranking in synthesis

Test Plan

  • All 475 tests pass (43 prompt-specific)
  • Coverage 89% (≥80% gate)
  • Pre-push sanity green (lint + format + tests)
  • No interface changes — function signatures unchanged
  • No new dependencies — pure prompt text additions
  • Existing contract tests pass without modification
  • Docs impact: investigation-flow.md updated; README unaffected (no 4-step references)
  • E2E skipped (PMPROXY_URL not set) — noted here per convention

…ic narrative

Red phase: 3 new tests assert domain specialists include Baseline step
(pcp_fetch_timeseries + pcp_detect_anomalies + 7-day window), classification
fields (ANOMALY/RECURRING/BASELINE), and chronic problem narrative guidance.
Cross-cutting correctly excluded.
…t prompts

Domain subsystems (cpu, memory, disk, network, process) now include a
Baseline step that instructs the agent to fetch 7-day history and run
pcp_detect_anomalies. Report guidance includes classification (ANOMALY/
RECURRING/BASELINE), baseline_context, severity_despite_baseline, and
chronic problem narrative. Cross-cutting workflow unchanged.
Red phase: 5 tests assert each domain specialist's domain_knowledge
contains at least one baseline-aware heuristic referencing the 7-day
baseline, time-of-day patterns, or scheduled job detection.
…ledge

Each domain specialist now has a 9th heuristic referencing the 7-day
baseline: CPU (time-of-day saturation), Memory (leak vs working-set),
Disk (scheduled job I/O), Network (drop rate variance), Process
(count/context-switch patterns).
…US1)

Degradation fallback was included in the Baseline step text from US1.
Tests validate the contract: threshold-only fallback instruction and
"insufficient baseline" limitation note are present for all domain
subsystems.
Red phase: 3 tests assert cross-cutting specialist prioritises ANOMALY
over RECURRING/BASELINE, flags correlated anomalies across subsystems,
and notes mixed classification scenarios.
Cross-cutting domain knowledge now instructs the agent to prioritise
ANOMALY over RECURRING/BASELINE, flag correlated anomalies across
subsystems, and note mixed classification scenarios.
Red phase: 3 tests assert coordinator synthesis ranks ANOMALY above
BASELINE/RECURRING, calls out normal behaviour, and highlights
recurring pattern matches.
Synthesis phase now ranks ANOMALY > RECURRING > BASELINE (severity as
secondary sort within each tier). Output structure updated from
"Findings by Severity" to "Findings by Classification & Severity"
with sections for New Anomalies, Recurring Patterns, Baseline
Behaviour, and Normal Operation.
…sification ranking

Specialist workflow diagram updated from 4-step to 5-step (Discover →
Baseline → Fetch → Analyse → Report) with pcp_detect_anomalies call.
Cross-cutting noted as excluded from Baseline step. Synthesis phase
updated for ANOMALY > RECURRING > BASELINE ranking. Output structure
now shows "Findings by Classification & Severity". Test formatting
applied.
Spec, plan, tasks (all complete), research, data model, contracts,
checklist, and quickstart for the specialist historical baselining
feature.
Principle I now requires documentation impact review before PR merge.
Plan template updated with docs gate. Aligns with Documentation
Discipline section added to CLAUDE.md.
@tallpsmith tallpsmith merged commit e6ab370 into main Mar 10, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant