Skip to content

[rush reporter][R8] Qualify AI reporter auto-selection #5981

Description

Parent: #5858

Blocked by: #5977, #5978

Goal

Prove that the AI reporter improves agent outcomes before enabling environment-based automatic selection.

Scope

  • Build a deterministic corpus of representative install, configuration, dependency, build, cache, network/auth, and plugin failures.
  • Compare legacy/plaintext/AI output for remediation success, context consumption, determinism, secret handling, and full-log discoverability.
  • Validate built-in COPILOT_CLI detection plus configured agentEnvironmentVariables.
  • Define pass/fail thresholds and store machine-readable gate results.
  • Keep auto-selection disabled until the corpus gates pass.

Acceptance criteria

  • The corpus is deterministic and runnable in CI.
  • AI output never includes classified secrets and does not regress producer privacy.
  • Thresholds for remediation quality, output size, determinism, and log-path correctness are documented and enforced.
  • Passing the gate enables agent auto-selection only on the opt-in reporter path before Rush 6.
  • RUSH_REPORTER=legacy remains authoritative.

DAG

Unblocks R9.

Part of #5858.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementThe issue is asking for a new feature or design change

    Type

    No type

    Projects

    • Status
      Needs triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions