Skip to content

feat(bmm): add story-pipeline workflow for 60-70% token savings #1194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
tjetzinger wants to merge 3 commits into
base: main
Choose a base branch
from
Open

feat(bmm): add story-pipeline workflow for 60-70% token savings #1194

tjetzinger wants to merge 3 commits into from

Conversation

@tjetzinger
Copy link

@tjetzinger tjetzinger commented Dec 26, 2025

Summary

Single-session step-file architecture replacing the legacy 6-call pipeline with massive token efficiency gains.

Problem

The legacy pipeline made 6 separate claude -p calls per story (~71K tokens), reloading agent personas and re-reading documents each time.

Solution

  • 8 step files (~200 lines each) loaded just-in-time
  • Single Claude session with role switching (SM → TEA → DEV)
  • State YAML for checkpoint/resume capability
  • Adversarial validation and code review (must find issues)

Token Efficiency

Mode Tokens Savings
Interactive ~25K 65%
Batch ~30K 58%

What's Included

story-pipeline/
├── workflow.yaml          # Config, agent mapping, quality gates
├── workflow.md            # Interactive orchestration
├── batch-runner.sh        # Unattended YOLO mode
├── README.md              # Comprehensive documentation
├── steps/                 # 8 step files
├── checklists/            # Validation criteria
└── templates/             # State file templates

Steps Automated

  1. Init - Load context, detect mode
  2. Create Story (SM) - Generate story from epic
  3. Validate Story (SM) - Adversarial validation
  4. ATDD (TEA) - Generate failing tests (RED)
  5. Implement (DEV) - Code to pass tests (GREEN)
  6. Code Review (DEV) - Adversarial review
  7. Complete (SM) - Git commit
  8. Summary - Audit trail

Tested

Real-world test with story 1-4 (User Invitation system):

  • 17 files, +2800 lines
  • Context exhausted mid-session, resumed cleanly via state file
  • 5 code review issues found and fixed

Test Plan

  • Interactive mode execution (story 1-4)
  • Resume from checkpoint after context exhaustion
  • Adversarial review catches real issues
  • Batch mode execution
  • Multiple story runs for token baseline

🤖 Generated with Claude Code

@tjetzinger
feat(bmm): add story-pipeline workflow for 60-70% token savings
d2d6328 
Single-session step-file architecture replacing 6 separate claude calls:

Structure:
- 8 step files (~200 lines each) with just-in-time loading
- Role switching (SM → TEA → DEV) within session
- State YAML for checkpoint/resume capability
- Checklists for validation, ATDD, implementation, code review

Steps:
1. Init - Load context, detect mode
2. Create Story - SM generates story from epic
3. Validate Story - Adversarial validation (find issues)
4. ATDD - TEA generates failing tests (RED phase)
5. Implement - DEV implements to pass tests (GREEN)
6. Code Review - Adversarial review (must find 3-10 issues)
7. Complete - Git commit, status update
8. Summary - Audit trail generation

Token efficiency:
- Legacy (6 calls): ~71K tokens/story
- Interactive mode: ~25K tokens (~65% savings)
- Batch mode: ~30K tokens (~58% savings)

Tested with real story (1-4 User Invitation): 17 files, +2800 lines,
context exhaustion mid-session recovered via state file.
@tjetzinger
docs(story-pipeline): add comprehensive documentation
1347daa 
Covers:
- Problem statement and token efficiency gains
- What each step automates (8-step workflow)
- Usage: interactive, batch, and resume modes
- Configuration options (workflow.yaml)
- State management and checkpointing
- Quality gates and adversarial mode
- Troubleshooting and best practices
- Comparison with legacy pipeline
@tjetzinger tjetzinger force-pushed the feature/pipeline-step-file-architecture branch from 50ba962 to 1347daa Compare December 26, 2025 12:12
jschulte added a commit to jschulte/BMAD-METHOD that referenced this pull request Dec 27, 2025
@jschulte
feat: integrate story-pipeline with autonomous-epic for 65% token sav…
c5afd89 
…ings

This commit integrates the story-pipeline workflow (PR bmad-code-org#1194) with autonomous-epic
and adds post-implementation validation to catch false positives.

MAJOR CHANGES:
1. Merged story-pipeline workflow from upstream PR bmad-code-org#1194 (20 files, 4,564 additions)
2. Added post-validation step (step-05b) between implementation and code review
3. Integrated story-pipeline as the default workflow for autonomous-epic
4. Replaced super-dev-story with story-pipeline in batch mode

NEW FEATURES:
- Post-implementation validation (step-05b-post-validation.md)
  * Verifies completed tasks against actual codebase
  * Catches false positives (tasks marked done but not implemented)
  * Re-runs implementation if gaps found
  * Uses Glob/Grep/Read to verify file existence and completeness

BENEFITS:
- Token efficiency: 25-30K per story (vs 100-150K with super-dev-story)
- 65% token savings per story, 75% savings per epic
- All super-dev-story quality gates PLUS post-validation
- Checkpoint/resume capability for long stories
- Batch mode for fully unattended execution

ARCHITECTURE:
- autonomous-epic orchestrates epic-level processing
- story-pipeline handles single-story lifecycle (9 steps including 5b)
- Role-switching in same session (vs separate workflow calls)
- Single session per story = massive token savings

TIME ESTIMATES (updated):
- Small epic (3-5 stories): 2-4 hours (was 3-6 hours)
- Medium epic (6-10 stories): 4-8 hours (was 6-12 hours)
- Large epic (11+ stories): 8-16 hours (was 12-24 hours)

FILES MODIFIED:
- autonomous-epic/instructions.xml (integrated story-pipeline)
- autonomous-epic/workflow.yaml (updated settings, removed super-dev choice)
- story-pipeline/* (20 new files from PR bmad-code-org#1194)
- story-pipeline/steps/step-05-implement.md (points to step-05b)
- story-pipeline/workflow.md (added step 5b to map and gates)
- story-pipeline/workflow.yaml (added step 5b definition)
- story-pipeline/templates/*.yaml → *.yaml.template (renamed to avoid linting)

FILES ADDED:
- story-pipeline/steps/step-05b-post-validation.md (NEW)
- INTEGRATION-NOTES.md (comprehensive documentation)

TESTING:
- PR bmad-code-org#1194 validated with real User Invitation system story
- 17 files, 2,800+ lines generated successfully
- Context exhaustion recovery tested

See INTEGRATION-NOTES.md for full details.

Co-authored-by: tjetzinger (story-pipeline PR bmad-code-org#1194)
@jschulte jschulte mentioned this pull request Dec 28, 2025
Merged
tjetzinger pushed a commit to tjetzinger/BMAD-METHOD that referenced this pull request Dec 28, 2025
@jschulte @tjetzinger
feat: integrate story-pipeline with autonomous-epic for 65% token sav…
0dfaa1b 
…ings

This commit integrates the story-pipeline workflow (PR bmad-code-org#1194) with autonomous-epic
and adds post-implementation validation to catch false positives.

MAJOR CHANGES:
1. Merged story-pipeline workflow from upstream PR bmad-code-org#1194 (20 files, 4,564 additions)
2. Added post-validation step (step-05b) between implementation and code review
3. Integrated story-pipeline as the default workflow for autonomous-epic
4. Replaced super-dev-story with story-pipeline in batch mode

NEW FEATURES:
- Post-implementation validation (step-05b-post-validation.md)
  * Verifies completed tasks against actual codebase
  * Catches false positives (tasks marked done but not implemented)
  * Re-runs implementation if gaps found
  * Uses Glob/Grep/Read to verify file existence and completeness

BENEFITS:
- Token efficiency: 25-30K per story (vs 100-150K with super-dev-story)
- 65% token savings per story, 75% savings per epic
- All super-dev-story quality gates PLUS post-validation
- Checkpoint/resume capability for long stories
- Batch mode for fully unattended execution

ARCHITECTURE:
- autonomous-epic orchestrates epic-level processing
- story-pipeline handles single-story lifecycle (9 steps including 5b)
- Role-switching in same session (vs separate workflow calls)
- Single session per story = massive token savings

TIME ESTIMATES (updated):
- Small epic (3-5 stories): 2-4 hours (was 3-6 hours)
- Medium epic (6-10 stories): 4-8 hours (was 6-12 hours)
- Large epic (11+ stories): 8-16 hours (was 12-24 hours)

FILES MODIFIED:
- autonomous-epic/instructions.xml (integrated story-pipeline)
- autonomous-epic/workflow.yaml (updated settings, removed super-dev choice)
- story-pipeline/* (20 new files from PR bmad-code-org#1194)
- story-pipeline/steps/step-05-implement.md (points to step-05b)
- story-pipeline/workflow.md (added step 5b to map and gates)
- story-pipeline/workflow.yaml (added step 5b definition)
- story-pipeline/templates/*.yaml → *.yaml.template (renamed to avoid linting)

FILES ADDED:
- story-pipeline/steps/step-05b-post-validation.md (NEW)
- INTEGRATION-NOTES.md (comprehensive documentation)

TESTING:
- PR bmad-code-org#1194 validated with real User Invitation system story
- 17 files, 2,800+ lines generated successfully
- Context exhaustion recovery tested

See INTEGRATION-NOTES.md for full details.

Co-authored-by: tjetzinger (story-pipeline PR bmad-code-org#1194)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@tjetzinger