feat: add generic explore agent

[ascerra]

Summary

• Adds a generic explore agent for gathering technical context from GitHub, Jira, the target codebase, and public web sources
• Platform-aware pre/post scripts support both GitHub and Jira (ISSUE_SOURCE)
• Pipeline labels are optional via EXPLORE_READY_LABEL / EXPLORE_NEEDS_INFO_LABEL env vars (no hardcoded refinement labels)
• Includes public-research and jira-read skills for downstream customization via harness base: composition

Files added

File	Purpose
agents/explore.md	Agent prompt — research phases, confidence scoring, JSON output
harness/explore.yaml	Sandbox config, host_files, skills, env
policies/explore.yaml	Network policy (Vertex, GitHub, Jira read-only, Tavily)
schemas/explore-result.schema.json	Output validation schema
scripts/pre-explore.sh	Fetch Jira/GitHub issue context, clone referenced repos
scripts/post-explore.sh	Attach context, sticky comment, optional labels
scripts/comment-helpers.sh	Shared sticky comment + secret scanning
scripts/adf-to-markdown.py, markdown-to-adf.py	Jira ADF conversion
skills/public-research, skills/jira-read	Generic research skills

Security review

• No secrets, tokens, or org-specific references committed
• Credentials stay in runner env only (not sandbox)
• Input validation on ISSUE_SOURCE, ISSUE_KEY, REPO_FULL_NAME, and pipeline label names
• fullsend scan runs before posting comments

Code review fixes applied

• Export REFERENCED_REPOS_DIR to sandbox via env/explore.env
• Validate pipeline label env vars before GitHub/Jira mutations
• Validate REPO_FULL_NAME format in pre-script
• Set GITHUB_ISSUE_NUMBER from pre-script for GitHub runs

Follow-up (separate PRs)

• konflux-ci/refinement: thin harness with base: URL + jira-routing skill + Konflux pipeline labels
• Add test scripts for pre/post explore (similar to triage)

Test plan

• Merge this PR
• Register in refinement via thin harness + fullsend agent add
• Run explore workflow in dry-run against a GitHub issue
• Run explore against a Jira issue with credentials configured

Made with Cursor

[qodo-code-review]

PR Summary by Qodo

Add generic explore agent for issue research (GitHub/Jira)

✨ Enhancement ⚙️ Configuration changes 🕐 40+ Minutes

AI Description

• Add an explore agent that gathers technical context for downstream workflow agents.
• Fetch issue context and referenced repos via pre-script; publish results via post-script.
• Add harness, network policy, and JSON schema validation for consistent agent output.

Diagram

graph TD
  PRE["scripts/pre-explore.sh"] --> CTX[("issue-context.json")] --> AG(["Sandbox: explore agent"]) --> OUT[("agent-result.json")]
  PRE --> TAR[("referenced-repos.tar.gz")]
  TAR --> AG
  AG --> POST["scripts/post-explore.sh"] --> GH{{"GitHub API"}}
  POST --> JIRA{{"Jira API"}}
  subgraph Legend
    direction LR
    _script["Script"] ~~~ _sb(["Sandbox"]) ~~~ _file[("Artifact")] ~~~ _ext{{"External"}}
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Use an existing ADF/Markdown library

• ➕ Less custom parsing/formatting code to maintain
• ➕ Potentially more complete ADF coverage (edge-case nodes)
• ➖ Adds a dependency footprint (packaging/installation in runner/sandbox)
• ➖ Harder to control safety constraints (URL schemes, input limits, expand-node filtering)
• ➖ May be overkill for the limited comment formatting needs

2. Avoid cloning referenced repos; rely on GitHub API content reads

• ➕ No git operations on the runner; simpler pre-script and less disk usage
• ➕ Avoids tarball mounting complexity
• ➖ API rate limits and auth constraints reduce reliability
• ➖ Harder to do deep grep/find-style exploration across repos
• ➖ Less performant for large repos and multi-file context gathering

Recommendation: Current approach is reasonable for a generic explore workflow: a host-side pre-script safely gathers privileged context (Jira/GitHub + public repo clones without credentials), while the sandbox stays constrained by policy and produces schema-validated JSON. Keep the custom ADF converters unless maintenance becomes burdensome; they provide explicit safety limits and predictable output for sticky comments.

Files changed (12) +2451 / -0

Enhancement (3) +913 / -0

explore.mdAdd explore agent prompt with phased research workflow and JSON output contract +382/-0

Add explore agent prompt with phased research workflow and JSON output contract

• Introduces the explore agent prompt defining research phases (issue parsing, codebase analysis, related work, web research) and explicit constraints. Specifies required env inputs from pre-script and produces a structured JSON result including confidence and data source accounting.

agents/explore.md

adf-to-markdown.pyAdd Jira ADF to Markdown converter for readable context ingestion +227/-0

Add Jira ADF to Markdown converter for readable context ingestion

• Implements a bounded-input converter from Atlassian Document Format to Markdown, supporting common nodes (headings, lists, tables, panels, code blocks) and filtering expand nodes used for agent markers/history.

scripts/adf-to-markdown.py

markdown-to-adf.pyAdd Markdown to Jira ADF converter for Jira comment posting +304/-0

Add Markdown to Jira ADF converter for Jira comment posting

• Converts markdown-style text into Jira ADF JSON with safety limits and URL scheme validation. Supports headings, lists, tables, rules, panels, and multiline handling via hardBreak nodes, plus optional wrapping of details in an expand section.

scripts/markdown-to-adf.py

Documentation (2) +203 / -0

SKILL.mdDocument Jira read-only data model available to agents +106/-0

Document Jira read-only data model available to agents

• Adds a skill doc explaining what Jira-derived data the agent can read via issue-context.json, how issue levels are interpreted, and what the agent cannot do due to sandbox constraints.

skills/jira-read/SKILL.md

SKILL.mdDocument public research techniques for GitHub and web sources +97/-0

Document public research techniques for GitHub and web sources

• Adds a skill doc with patterns for using GitHub APIs/CLI, local repo inspection, and web search for public-only research, including guidance on targeted searches and anti-patterns.

skills/public-research/SKILL.md

Other (7) +1335 / -0

explore.envExport explore runtime env vars into the sandbox +3/-0

Export explore runtime env vars into the sandbox

• Adds exported environment variables for ISSUE_CONTEXT, FULLSEND_OUTPUT_DIR, and REFERENCED_REPOS_DIR so the sandboxed agent can consistently locate inputs and outputs.

env/explore.env

explore.yamlAdd explore harness wiring (host files, env bridging, schema validation) +64/-0

Add explore harness wiring (host files, env bridging, schema validation)

• Defines the explore harness configuration, mounting required env/credentials and optional issue/repo artifacts. Wires pre/post scripts, includes public-research and jira-read skills, and enables a validation loop against the explore-result schema.

harness/explore.yaml

explore.yamlAdd sandbox policy for explore (Vertex, GitHub, Jira read-only, web search) +82/-0

Add sandbox policy for explore (Vertex, GitHub, Jira read-only, web search)

• Introduces a network/filesystem policy allowing necessary endpoints (Anthropic/Google APIs, GitHub, Jira read-only, Tavily) while keeping the sandbox locked down with a minimal read/write filesystem set.

policies/explore.yaml

explore-result.schema.jsonDefine JSON schema for explore agent output +113/-0

Define JSON schema for explore agent output

• Adds a Draft-07 JSON schema that validates the explore agent result structure, including input metadata, technical landscape, related work, confidence scoring, optional gaps/data_sources, and a bounded summary field.

schemas/explore-result.schema.json

comment-helpers.shAdd shared sticky comment helpers for GitHub and Jira +188/-0

Add shared sticky comment helpers for GitHub and Jira

• Provides reusable functions to post/update a single sticky comment per agent, delegating GitHub to fullsend post-comment and implementing Jira sticky history with expand nodes. Includes secret redaction via fullsend scan and label add/remove helpers.

scripts/comment-helpers.sh

post-explore.shPublish explore results (attachment, sticky summary, optional labels) +222/-0

Publish explore results (attachment, sticky summary, optional labels)

• Reads the latest iteration output, validates JSON, and copies it to exploration_context.json. Attaches results to Jira, posts a sticky summary comment to GitHub/Jira, and optionally applies configured pipeline labels based on a confidence threshold with label name validation.

scripts/post-explore.sh

pre-explore.shFetch issue context and pre-clone referenced repos for deep exploration +663/-0

Fetch issue context and pre-clone referenced repos for deep exploration

• Implements platform-aware context fetching for Jira (REST + ADF conversion + hierarchy/linked issues) and GitHub (gh issue view + sub-issues). Discovers referenced repos from issue text/ADF/previous explore results, validates via GitHub API, shallow-clones public repos with credential helpers disabled, generates REPO-INDEX.md navigation manifests, and packages repos into a tarball for sandbox mounting.

scripts/pre-explore.sh

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 8:29 PM UTC · Completed 8:39 PM UTC
Commit: c338d84 · View workflow run →

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Missing label auto-created ✓ Resolved 🐞 Bug ⚙ Maintainability

Description

post-explore.sh adds SIGNAL_LABEL via the GitHub labels API without checking that the label
exists first, so a misconfigured env var can create unwanted labels or silently drift repo labeling
conventions.

Code

scripts/post-explore.sh[R113-118]

+# --- Add label when configured ---
+if [[ -n "$SIGNAL_LABEL" ]]; then
+  if [[ -n "${GITHUB_ISSUE_NUMBER:-}" && "${GITHUB_ISSUE_NUMBER}" != "N/A" ]]; then
+    gh api "repos/${REPO_FULL_NAME}/issues/${GITHUB_ISSUE_NUMBER}/labels" \
+      -f "labels[]=${SIGNAL_LABEL}" --silent 2>/dev/null || true
+    echo "::notice::Added label '${SIGNAL_LABEL}' to GitHub issue #${GITHUB_ISSUE_NUMBER}"

Evidence

Explore applies a label directly with gh api .../labels -f labels[]=... and ignores errors,
whereas triage explicitly fetches existing labels and skips any that don’t exist to avoid
auto-creating removed/unknown labels.

scripts/post-explore.sh[113-119]
scripts/post-triage.sh[310-343]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`scripts/post-explore.sh` applies pipeline labels without verifying they exist. This can unintentionally create labels (repo metadata drift) or mask configuration mistakes.

### Issue Context
`scripts/post-triage.sh` already implements a safe pattern: fetch repo labels once and skip non-existent labels to avoid accidental creation.

### Fix Focus Areas
- scripts/post-explore.sh[113-119]
- scripts/post-triage.sh[310-343]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Valid labels rejected ✓ Resolved 🐞 Bug ≡ Correctness

Description

post-explore.sh hard-fails when EXPLORE_READY_LABEL / EXPLORE_NEEDS_INFO_LABEL contains
characters like / or : even though those are used elsewhere in this repo’s label conventions,
which can prevent posting the explore comment and Jira attachment for otherwise successful runs.

Code

scripts/post-explore.sh[R19-25]

+validate_label_name() {
+  local label="$1"
+  if [[ ! "$label" =~ ^[a-zA-Z0-9][a-zA-Z0-9._-]*$ ]]; then
+    echo "ERROR: invalid label name: ${label}"
+    exit 1
+  fi
+}

Evidence

post-explore.sh rejects anything outside [a-zA-Z0-9._-] and exits, while the existing triage
pipeline explicitly allows / and : in label names and tests adding a label with /
(area/api).

scripts/post-explore.sh[19-25]
scripts/post-triage.sh[326-330]
scripts/post-triage-test.sh[301-304]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`scripts/post-explore.sh` validates optional pipeline labels with a regex that rejects common label characters (e.g. `/`, `:`). When a rejected-but-legitimate label is configured, the script `exit 1`s before posting the sticky comment/attachment.

### Issue Context
Other pipeline code in this repo supports labels containing `/` (example: `area/api`) and uses a broader allowlist.

### Fix Focus Areas
- scripts/post-explore.sh[19-25]
- scripts/post-explore.sh[101-110]
- scripts/post-triage.sh[326-330]
- scripts/post-triage-test.sh[301-304]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

3. History cap off-by-one ✓ Resolved 🐞 Bug ☼ Reliability

Description

comment-helpers.sh intends to cap Jira sticky-comment history via _CH_MAX_HISTORY, but it keeps
up to _CH_MAX_HISTORY old “Previous” expands and then prepends a new one, resulting in
_CH_MAX_HISTORY + 1 history entries.

Code

scripts/comment-helpers.sh[R70-92]

+  old_current=$(echo "$old_body" | jq '[.content[]? | select(.type != "expand")]')
+  old_history=$(echo "$old_body" | jq --argjson max "$_CH_MAX_HISTORY" \
+    '[.content[]? | select(.type == "expand" and (.attrs.title // "" | startswith("Previous")))] | .[:$max]')
+
+  timestamp=$(date -u +"%b %d, %H:%M UTC")
+
+  history_entry=$(jq -n --argjson content "$old_current" --arg title "Previous · ${timestamp}" \
+    '{"type": "expand", "attrs": {"title": $title}, "content": $content}')
+
+  final_adf=$(jq -n \
+    --argjson new_content "$new_content" \
+    --argjson history_entry "$history_entry" \
+    --argjson old_history "$old_history" \
+    --arg marker "$_CH_MARKER" \
+    '{body: {type: "doc", version: 1, content:
+      ($new_content
+       + [{"type": "rule"}]
+       + [$history_entry]
+       + $old_history
+       + [{"type": "expand", "attrs": {"title": ""}, "content":
+           [{"type": "paragraph", "content": [{"type": "text", "text": $marker}]}]}]
+      )
+    }}')

Evidence

The script sets _CH_MAX_HISTORY=3, slices old_history to that max, then appends history_entry
plus old_history into the final ADF, yielding 4 history expands when max is 3.

scripts/comment-helpers.sh[28-29]
scripts/comment-helpers.sh[70-92]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Jira sticky-comment history logic is off by one: it retains up to `_CH_MAX_HISTORY` prior history entries and then adds a new history entry, so the total history entries exceed the intended cap.

### Issue Context
If the intended meaning of `_CH_MAX_HISTORY` is “total previous snapshots to keep”, slice old history to `max-1` before prepending the new snapshot (or rename the variable/comment to reflect the current behavior).

### Fix Focus Areas
- scripts/comment-helpers.sh[28-29]
- scripts/comment-helpers.sh[70-92]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[fullsend-ai-review]

Review

Verdict: approve

All 14 findings from the prior review have been addressed in commit db4947e. The fixes are clean and complete:

#	Prior Finding	Status
1	Gap extraction used wrong field names (.category // .type → .dimension)	✅ Fixed
2	Schema version inconsistency (draft-07 → 2020-12, added $id)	✅ Fixed
3	impact_radius absent from schema	✅ Fixed
4	Summary maxLength mismatch (2000 → 1000, matches prompt)	✅ Fixed
5	Schema enum values not reflected in prompt (text, web, outcome added)	✅ Fixed
6	Missing null guard on related_work (// [] added)	✅ Fixed
7	Python injection via JIRA_EMAIL (now uses os.environ.get())	✅ Fixed
8	JIRA_HOST not validated (added validate_jira_host() with regex)	✅ Fixed
9	Silent fallback to no-op secret scanning (added ::warning::)	✅ Fixed
10	Jira attachment URL with -L redirect (URL host validation + removed -L)	✅ Fixed
11	Skill documentation field mismatches (removed assignee, updated project)	✅ Fixed
12	Missing user documentation (added docs/explore.md)	✅ Fixed
13	PR title not Conventional Commits (now feat:)	✅ Fixed
14	*.google.com wildcard too broad (narrowed to www.google.com)	✅ Fixed

Re-review assessment

Six review dimensions were evaluated against the current HEAD:

• Correctness — No logic errors, edge cases, or contract violations found. The _redact_secrets function correctly uses fullsend scan output as a stdin-to-stdout filter (verified against CLI help). Schema, prompt, and script field names are now consistent.
• Security — Credential isolation is solid (Jira/GitHub tokens stay on the runner). JIRA_HOST is validated against *.atlassian.net pattern. Python string interpolation injection is resolved. Attachment URL host validation prevents credential leakage. Network policy is appropriately scoped.
• Intent & coherence — The PR is a well-structured feature addition that follows the existing triage agent's architecture. No linked issue exists, but the PR body clearly documents scope, intent, and test plan, and the enhancement label provides implicit authorization.
• Style & conventions — Follows established patterns from the triage agent (harness structure, script idioms, schema format).
• Documentation — docs/explore.md, skills/jira-read/SKILL.md, and skills/public-research/SKILL.md accurately reflect the implementation.
• Cross-repo contracts — Purely additive (all new files). No existing interfaces or schemas are modified.

The code is clean, well-organized, and ready to merge.

---

Protected paths detected — this PR modifies files under one or more
protected paths. The review agent cannot approve PRs that touch these paths.
A human reviewer must approve this PR.

Protected files in this PR:

• agents/explore.md
• harness/explore.yaml
• policies/explore.yaml
• scripts/adf-to-markdown.py
• scripts/comment-helpers.sh
• scripts/markdown-to-adf.py
• scripts/post-explore.sh
• scripts/pre-explore.sh
• skills/jira-read/SKILL.md
• skills/public-research/SKILL.md

Previous run

Review

Verdict: request-changes

This PR adds a well-structured exploration agent that follows the existing triage agent's architecture. The credential isolation (keeping Jira/GitHub tokens out of the sandbox), input validation, and repo cloning safeguards are well done. However, there are several issues that should be addressed before merging — one logic error that silently breaks gap reporting in posted comments, schema/prompt mismatches that will cause downstream confusion, and security defense-in-depth gaps.

Findings

High

1. Gap extraction uses wrong field names (scripts/post-explore.sh, ~line 157): The jq expression .category // .type // "definition gap" extracts the gap label for the posted comment, but the schema defines .dimension (required) — not .category or .type. Every gap will show "definition gap" regardless of the agent's actual dimension value, losing information in the posted summary.

Medium

2. Schema version inconsistency (schemas/explore-result.schema.json, line 2): Uses http://json-schema.org/draft-07/schema# while the existing triage schema uses https://json-schema.org/draft/2020-12/schema. Also uses http:// instead of https://. Missing $id field present in the triage schema.

3. impact_radius absent from schema (agents/explore.md lines 319-325 vs schemas/explore-result.schema.json): The agent prompt instructs the agent to produce an impact_radius object, but the schema has no definition for it. Since additionalProperties is not false, it passes validation but provides no contract for downstream consumers.

4. Summary maxLength mismatch (schemas/explore-result.schema.json line 112 vs agents/explore.md line 387): Schema allows 2000 characters; prompt says "under 1000 characters." Pick one authoritative value.

5. Schema enum values not reflected in prompt (schemas/explore-result.schema.json lines 12-14): input.source includes "text" and "web" not mentioned in the prompt example. input.level includes "outcome" not in the prompt example. Either the schema has dead enum values or the prompt is incomplete.

6. Missing null guard on related_work (scripts/post-explore.sh, ~line 54): RELATED_COUNT=$(jq '.related_work | length' ...) lacks the // [] null guard used on the adjacent GAP_COUNT line. If the agent produces a non-conforming output where related_work is null, jq outputs null, and the subsequent arithmetic comparison fails under set -euo pipefail.

7. Python injection via JIRA_EMAIL (scripts/pre-explore.sh, ~line 128): JIRA_EMAIL is interpolated directly into an inline Python string literal: AGENT_EMAIL = '${JIRA_EMAIL}'. A value containing a single quote enables arbitrary Python execution on the runner host. Pass it as an environment variable or command-line argument instead.

8. JIRA_HOST not validated (scripts/pre-explore.sh, post-explore.sh, comment-helpers.sh): JIRA_HOST is used to construct API URLs with an attached Authorization: Basic header but is never validated against a pattern. An attacker who controls this variable can redirect authenticated requests to an arbitrary host. Validate against ^[a-zA-Z0-9.-]+\.atlassian\.net$ or an allowlist.

9. Silent fallback to no-op secret scanning (scripts/comment-helpers.sh, lines 57-62): _redact_secrets() falls back to cat when fullsend is not on $PATH, meaning no secret scanning occurs silently. At minimum, log a ::warning:: annotation so operators notice.

10. Jira attachment URL used with auth header without validation (scripts/pre-explore.sh, ~line 411): EXPLORE_URL from a Jira attachment's content field is passed to curl -L with the Authorization: Basic header. The -L flag follows redirects, potentially leaking credentials. Validate that the URL matches https://${JIRA_HOST}/... before requesting.

11. Skill documentation mismatches (skills/jira-read/SKILL.md): The example shows assignee (line 37) and project.hierarchy (line 73) fields that don't exist in the actual issue-context.json output from pre-explore.sh. The actual output has project.available_issue_types and project.team_usage instead.

12. Missing user documentation (docs/explore.md): The existing triage agent has docs/triage.md explaining usage. No corresponding docs file is added for the explore agent.

13. PR title does not follow Conventional Commits (AGENTS.md §4): Title "Add generic explore agent" is missing the required type prefix. Should be e.g., feat: add generic explore agent.

14. *.google.com wildcard overly broad (policies/explore.yaml, line 49): Allows connections to any Google subdomain including storage.googleapis.com. Restrict to specific subdomains needed for search.

---

Labels: PR adds a new explore agent with harness, scripts, schema, and skills.

[ascerra]

Additional review items from the summary (no inline thread) addressed in db4947e:

• impact_radius absent from schema — added impact_radius object to schemas/explore-result.schema.json matching the agent prompt
• Missing user documentation — added docs/explore.md and doc: field in harness/explore.yaml
• PR title Conventional Commits — retitled to feat: add generic explore agent

All 14 inline review threads have individual replies explaining the fix.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 8:48 PM UTC · Completed 9:00 PM UTC
Commit: db4947e · View workflow run →

Superseded by updated review