Skip to content

Commit 9ec1293

Browse files
base setup;
1 parent f3d2fda commit 9ec1293

16 files changed

Lines changed: 2430 additions & 257 deletions

File tree

.agent/workflows/test-quality-review.md

Lines changed: 250 additions & 198 deletions
Large diffs are not rendered by default.

.agent/workflows/write-tests-from-specs.md

Lines changed: 410 additions & 0 deletions
Large diffs are not rendered by default.
Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
name: analyze-mutation-survivors
3+
description: Analyze surviving mutations from Stryker mutation testing and write targeted tests to kill them. Use when mutation score is below threshold or when Stryker reports survivors.
4+
allowed-tools: Bash Read Edit Write Grep
5+
---
6+
7+
# Analyze Mutation Survivors
8+
9+
Analyze surviving mutants and write targeted tests to kill them.
10+
11+
## Instructions
12+
13+
### 1. Run Mutation Testing
14+
15+
```bash
16+
bun run test:mutate
17+
```
18+
19+
### 2. Open the Report
20+
21+
The HTML report is at: `reports/mutation/index.html`
22+
23+
Or read the console output which shows each survivor.
24+
25+
### 3. For Each Survivor, Analyze
26+
27+
Each surviving mutant shows:
28+
- **File and line number** - where the mutation was made
29+
- **Original code** - what the code looked like
30+
- **Mutated code** - what Stryker changed it to
31+
32+
Ask: "Why didn't the existing tests catch this change?"
33+
34+
### 4. Common Mutations and Fixes
35+
36+
| Mutation | Why It Survived | Fix |
37+
|----------|-----------------|-----|
38+
| `>=``>` | No boundary test | Add test with exact boundary value |
39+
| `&&``\|\|` | Only tested when both true | Add test where only one is true |
40+
| `if (x)``if (true)` | No test where x is falsy | Add test with falsy value |
41+
| Block removed | Side effect not verified | Assert the block's effect |
42+
| Return changed | Only checked `toBeDefined()` | Assert exact return value |
43+
44+
### 5. Write Targeted Test
45+
46+
Create a test specifically designed to fail if the mutation is applied.
47+
48+
Example:
49+
```typescript
50+
// Mutation: timestamp >= startTime → timestamp > startTime
51+
// Fix: Test with exact boundary
52+
test("includes message at exact startTime", () => {
53+
const exactTime = new Date();
54+
store.store(createMessage({ timestamp: exactTime }));
55+
const results = store.query({ startTime: exactTime });
56+
expect(results.length).toBe(1); // Would be 0 with >
57+
});
58+
```
59+
60+
### 6. Re-run Until Threshold Met
61+
62+
```bash
63+
bun run test:mutate
64+
```
65+
66+
Target: ≥ 80% mutation score
67+
68+
## Tips
69+
70+
- Focus on high-impact survivors first (core logic)
71+
- Some survivors are acceptable (defensive code, logging)
72+
- Property-based tests can kill many mutations at once
Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
---
2+
name: detect-test-antipatterns
3+
description: Detect fake safety patterns in tests that provide false confidence. Use when reviewing test quality, after automated checks pass, or when tests seem suspicious.
4+
allowed-tools: Read Grep Glob
5+
---
6+
7+
# Detect Test Anti-Patterns
8+
9+
Find patterns that create false confidence - tests that appear to verify behavior but don't.
10+
11+
## Instructions
12+
13+
### 1. Hidden Assertions (CRITICAL)
14+
15+
Search for assertions inside callbacks:
16+
```bash
17+
grep -rn "expect(" packages/*/src/*.test.ts | grep -E "\([^)]+\) =>"
18+
```
19+
20+
**Anti-pattern:**
21+
```typescript
22+
// ❌ BAD - expect may never run if callback doesn't execute
23+
pipeline.use(async (ctx) => {
24+
expect(ctx.value).toBe("something");
25+
});
26+
await pipeline.run(ctx);
27+
```
28+
29+
**Fix:**
30+
```typescript
31+
// ✅ GOOD - capture and assert after
32+
let capturedValue: string;
33+
let executed = false;
34+
pipeline.use(async (ctx) => {
35+
executed = true;
36+
capturedValue = ctx.value;
37+
});
38+
await pipeline.run(ctx);
39+
expect(executed).toBe(true);
40+
expect(capturedValue).toBe("something");
41+
```
42+
43+
### 2. Weak toBeDefined() Assertions
44+
45+
Search for:
46+
```bash
47+
grep -rn "toBeDefined()" packages/*/src/*.test.ts
48+
```
49+
50+
**Anti-pattern:**
51+
```typescript
52+
// ❌ BAD - passes even if value is wrong
53+
expect(event.id).toBeDefined();
54+
```
55+
56+
**Fix:**
57+
```typescript
58+
// ✅ GOOD - verify actual value
59+
expect(event.id).toMatch(/^[0-9a-f-]{36}$/i);
60+
```
61+
62+
### 3. Tautological Comparisons
63+
64+
Search for:
65+
```bash
66+
grep -rn "toBeGreaterThanOrEqual" packages/*/src/*.test.ts
67+
```
68+
69+
**Anti-pattern:**
70+
```typescript
71+
// ❌ BAD - always passes if unchanged
72+
expect(updated.getTime()).toBeGreaterThanOrEqual(original.getTime());
73+
```
74+
75+
**Fix:**
76+
```typescript
77+
// ✅ GOOD - strict with delay
78+
await new Promise(r => setTimeout(r, 5));
79+
expect(updated.getTime()).toBeGreaterThan(original.getTime());
80+
```
81+
82+
### 4. Length-Only Checks
83+
84+
Search for:
85+
```bash
86+
grep -rn "\.length).toBe" packages/*/src/*.test.ts
87+
```
88+
89+
**Anti-pattern:**
90+
```typescript
91+
// ❌ BAD - could be wrong items
92+
expect(sessions.length).toBe(3);
93+
```
94+
95+
**Fix:**
96+
```typescript
97+
// ✅ GOOD - verify content too
98+
expect(sessions.length).toBe(3);
99+
expect(sessions.map(s => s.id)).toContain(expected1.id);
100+
```
101+
102+
### 5. Existence-Only Tests
103+
104+
**Anti-pattern:**
105+
```typescript
106+
// ❌ BAD - doesn't test behavior
107+
it("exports SessionManager", () => {
108+
expect(SessionManager).toBeDefined();
109+
});
110+
```
111+
112+
**Fix:**
113+
```typescript
114+
// ✅ GOOD - verify functionality
115+
it("exports working SessionManager", () => {
116+
const manager = new SessionManager();
117+
const session = manager.create({ name: "test", transport: "stdio" });
118+
expect(session.state).toBe("CREATED");
119+
});
120+
```
121+
122+
## Output
123+
124+
For each anti-pattern found, document:
125+
1. File and line number
126+
2. The problematic code
127+
3. Suggested fix
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
name: map-spec-to-tests
3+
description: Map specification scenarios to test cases and identify coverage gaps. Use when verifying test completeness against requirements or during test quality reviews.
4+
allowed-tools: Read Grep Glob
5+
---
6+
7+
# Map Spec to Tests
8+
9+
Create a traceability matrix linking spec scenarios to test implementations.
10+
11+
## Instructions
12+
13+
### 1. Locate the Spec Document
14+
15+
Find the relevant spec file:
16+
```
17+
say2/3-how/multi-protocols/say2/specs/v1/02-architecture/shared/03-phased-implementation/
18+
```
19+
20+
### 2. Extract Test Scenarios
21+
22+
Look for sections with scenario lists:
23+
- Checkboxes: `- [ ]` or `- [x]`
24+
- Numbered lists describing expected behavior
25+
- Tables with acceptance criteria
26+
27+
### 3. For Each Scenario
28+
29+
Search for corresponding tests:
30+
```bash
31+
grep -rn "scenario keyword" packages/*/src/*.test.ts
32+
```
33+
34+
### 4. Create Traceability Matrix
35+
36+
Document in this format:
37+
38+
```markdown
39+
| Spec Scenario | Test Location | Status | Notes |
40+
|---------------|---------------|--------|-------|
41+
| Sessions have unique IDs | manager.test.ts:L25 || |
42+
| Messages preserve order | message-store.test.ts:L74 || |
43+
| Middleware chain order | pipeline.test.ts:L30 || |
44+
| Error state transition | manager.test.ts:L89 | ⚠️ | Only happy path |
45+
| Query by time range | - || Not implemented |
46+
```
47+
48+
### Status Legend
49+
50+
-**Fully covered** - Test exists and verifies all aspects
51+
- ⚠️ **Partially covered** - Test exists but missing edge cases
52+
-**Not covered** - No test found
53+
54+
### 5. For Missing or Partial Coverage
55+
56+
Document:
57+
1. What specific behavior is not tested
58+
2. Which edge cases are missing
59+
3. Recommended test to add
60+
61+
### 6. Update Spec Document
62+
63+
After analysis, update the spec with verification status:
64+
65+
```markdown
66+
## Test Scenarios
67+
68+
- [x] Sessions are created with unique IDs *(manager.test.ts)*
69+
- [x] Messages can be stored and retrieved *(message-store.test.ts)*
70+
- [ ] Query filtering by time range *(NOT TESTED)*
71+
```
72+
73+
## Example Output
74+
75+
```markdown
76+
## Traceability Report for Phase 0
77+
78+
**Coverage Summary:** 12/14 scenarios (86%)
79+
80+
### Fully Covered (10)
81+
- Session lifecycle management
82+
- Message storage and retrieval
83+
- Middleware execution order
84+
85+
### Partially Covered (2)
86+
- Error handling: missing timeout scenarios
87+
88+
### Not Covered (2)
89+
- Concurrent session handling
90+
```
Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
---
2+
name: run-quality-checks
3+
description: Run automated test quality checks including tests, coverage, assertion density, and linting. Use when you need to verify test quality metrics or when tests might have issues.
4+
allowed-tools: Bash Read
5+
---
6+
7+
# Run Quality Checks
8+
9+
Run the automated quality checks to assess test health.
10+
11+
## Instructions
12+
13+
### 1. Quick Quality Check
14+
15+
```bash
16+
bun run quality
17+
```
18+
19+
This runs:
20+
- All tests
21+
- Assertion density check (≥ 1.0 per test)
22+
- Lint checks
23+
24+
### 2. Full Quality Check (with mutation testing)
25+
26+
```bash
27+
bun run quality:full
28+
```
29+
30+
This adds mutation testing which is slower but more thorough.
31+
32+
### 3. Individual Checks
33+
34+
| Check | Command |
35+
|-------|---------|
36+
| Tests only | `bun test` |
37+
| Coverage | `bun test --coverage` |
38+
| Assertion density | `bun run test:density` |
39+
| Lint | `bun run lint` |
40+
| Mutation testing | `bun run test:mutate` |
41+
| Property tests | `bun run test:property` |
42+
43+
## On Failure
44+
45+
1. Read the output to identify which check failed
46+
2. Fix the issue in the test or source files
47+
3. Re-run until all checks pass
48+
49+
## Expected Output
50+
51+
```
52+
✅ All quality checks passed
53+
```
54+
55+
With metrics:
56+
- Mutation score: ≥ 80%
57+
- Assertion density: ≥ 1.0 per test
58+
- Line coverage: ≥ 80%

0 commit comments

Comments
 (0)