Replace Integration Data Updater agentic workflow with deterministic scripts#1351
Replace Integration Data Updater agentic workflow with deterministic scripts#1351IEvangelist wants to merge 3 commits into
Conversation
…scripts The gh-aw agentic Integration Data Updater spent Copilot inference on work that is entirely deterministic, and its 20-minute agent-step cap truncated the API-reference regeneration on catch-up runs. Replace it with an OS/CI-aware pwsh orchestrator (src/frontend/scripts/update-integration-data.ps1) that runs pnpm update:all, detects package version changes, conditionally regenerates the C#/TS API reference JSON and twoslash bundle, verifies the diff is well-scoped, and emits a PR title/body. A thin standard workflow (.github/workflows/update-integration-data.yml) runs it and opens a PR via the Aspire bot GitHub App, mirroring update-release-branch.yml. Removes update-integration-data.md and its generated .lock.yml; updates the update-integrations skill automation note. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR replaces the previous gh aw agentic “Integration Data Updater” automation with a deterministic PowerShell orchestrator and a standard GitHub Actions workflow that runs it, aiming to remove inference cost/limits while preserving the same update + conditional regen behavior.
Changes:
- Add
src/frontend/scripts/update-integration-data.ps1to runpnpm update:all, detect integration version changes, conditionally regenerate C#/TS API data + twoslash bundle, and emit workflow outputs. - Add
.github/workflows/update-integration-data.ymlto run on schedule/dispatch, commit only allowed generated files, open a PR via the Aspire bot GitHub App, and supersede older automation PRs. - Remove the prior agentic workflow sources (
.md+ compiled.lock.yml) and update theupdate-integrationsskill note to reflect the new deterministic flow.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/frontend/scripts/update-integration-data.ps1 | New deterministic orchestrator for data refresh + conditional API/twoslash regeneration + scope enforcement + PR metadata outputs. |
| .github/workflows/update-integration-data.yml | New scheduled/dispatch workflow to run the orchestrator, commit allowed outputs, and open/supersede PRs via GitHub App token. |
| .github/workflows/update-integration-data.md | Removes the prior agentic workflow definition. |
| .github/workflows/update-integration-data.lock.yml | Removes the compiled gh-aw lock workflow. |
| .agents/skills/update-integrations/SKILL.md | Updates the skill’s automation note to reference the new deterministic workflow/script. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Invoke-Git: emit git output as flat strings to the pipeline and normalize stderr ErrorRecords to their message text, instead of eturn , @() which wrapped output in a confusing nested array. - Change gate: check the FULL working tree (git status --porcelain) so an out-of-scope-only change still reaches the Phase 4 scope check and fails loudly instead of silently exiting 0. - PR date: use UTC + InvariantCulture ([DateTime]::UtcNow) so the PR title matches the UTC branch name and UTC schedules regardless of runner locale. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Frontend HTML artifact readyThe latest frontend build uploaded the This comment updates automatically when a new frontend build artifact is uploaded. |
Instead of creating a new timestamped branch and a brand-new PR on every run (and closing the previous day's PR), reuse a single long-lived branch chore/update-integration-data: - Force-push one fresh commit with the latest data on top of the newest main each run, so an unreviewed PR is refreshed in place rather than piling up a new PR per day. The single rebased snapshot also keeps the PR conflict-free. - Reuse the existing open PR when present (gh pr edit) so its review threads and history carry over; only open a new PR when none exists. - Still close any stray legacy timestamped automation PRs as a safety net. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
radical
left a comment
radical
left a comment
There was a problem hiding this comment.
[automated] Automated review pass (findings validated with a second model). Three inline notes: one Medium (TS-API partial-failure reporting), one Medium (force-push clobber on the rolling branch), one non-blocking (title-keyed version map + rollout reminder).
| } | ||
| exit 1 | ||
| } | ||
| $tsApiSummary = 'succeeded' |
There was a problem hiding this comment.
[automated] On a partial TS-API regen failure this opens a PR that claims success. generate-ts-api-json.ps1 counts per-package aspire sdk dump/transform failures into $failed and prints Complete: N failed, but it exits 0 (no exit/throw at the end), so update-ts-api.ts only fails on a catastrophic throw — not on individual package failures. So $tsExit is 0 here and we hardcode $tsApiSummary = 'succeeded', then open a PR with stale/missing ts-modules/* and a twoslash bundle rebuilt without them (hover tooltips fall back to any), with nothing red.
Two problems with that:
- It contradicts the "Hard-fails (red check, no PR) on … TS API regen failure" note in the PR description — that's only true for total failure.
- It's a reporting regression vs the old body, which had a
TS API JSON: S succeeded, F failedline, so a reviewer could see the count.
Suggest parsing the Complete: X succeeded, Y failed line the same way the C# path parses Done! Success:, surfacing Y in the body, and deciding whether Y > 0 should fail the run or just be reported.
| git commit \ | ||
| -m "$PR_TITLE" \ | ||
| -m "Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>" | ||
| git push --force origin "HEAD:refs/heads/$branch" |
There was a problem hiding this comment.
[automated] Force-pushing the shared chore/update-integration-data branch every run will clobber any human commits on the open automation PR. The old model opened a fresh PR per run, so that couldn't happen.
The intended safety net looks like the :octocat: auto-merge label, but I don't see any workflow in .github/workflows/ that consumes it — so same-day auto-merge isn't established in-repo, and a maintainer fix pushed onto the PR could be silently overwritten by the next scheduled run.
Worth confirming auto-merge is actually wired up (branch ruleset / external bot), or guarding the force-push when the branch has non-bot commits on top.
| foreach ($entry in @($entries)) { | ||
| if ($entry.PSObject.Properties.Name -contains 'title' -and | ||
| $entry.PSObject.Properties.Name -contains 'version') { | ||
| $map[[string]$entry.title] = [string]$entry.version |
There was a problem hiding this comment.
[automated] Non-blocking: this keys the version map on title, so two entries sharing a title would collapse (last wins) and a version move could be missed. It's fine today (164/164 entries, no duplicate titles) — just a latent assumption worth a dedup key or an assert.
Unrelated rollout reminder (not about this line): the reuse/stray-close logic in the workflow only matches the chore/update-integration-data branch family, so if any pre-existing agentic automation PR is open at merge time, close it by hand. None is open right now, so no action needed today.
Summary
Replaces the
gh awIntegration Data Updater agentic workflow with a deterministic script + a thin, standard GitHub Actions workflow. Every step the agent performed was already deterministic, so this removes Copilot inference cost and the 20‑minute agent‑step cap that truncated API‑reference regeneration on catch‑up runs (see run 29047474199, which timed out at 20m after the auth fix in #1342 let it run).What changed
src/frontend/scripts/update-integration-data.ps1— an OS/CI‑aware PowerShell orchestrator that:pnpm update:all,aspire-integrations.jsonvs. the freshly generated one bytitle→version; metadata‑only changes do not trigger regen),generate-package-json.ps1), the TypeScript API JSON (pnpm update:ts-api), and the chained twoslashaspire.d.tsbundle,GITHUB_OUTPUT.Reproducible locally:
pwsh src/frontend/scripts/update-integration-data.ps1(add-SkipRegenfor a fast data‑only run)..github/workflows/update-integration-data.yml— schedule +workflow_dispatch,timeout-minutes: 45. Sets up .NET / Aspire CLI / pnpm / Node, runs the orchestrator, and — only when data changed — commits only the allowed generated files and opens a PR through the Aspire bot GitHub App, superseding prior open automation PRs. Mirrors the existingupdate-release-branch.ymlapp‑token pattern..github/workflows/update-integration-data.mdand its generated.lock.yml.update-integrationsskill automation note to describe the deterministic flow.Behavior notes
update:allfailure, TS API regen failure, or any out‑of‑scope diff — so problems surface as CI rather than being silently "handled."pkgs/generation failures (common for meta‑packages) are tolerated and reported in the PR body, matching prior behavior.aspire-integrations.json,github-stats.json,samples.json,assets/samples/**,data/pkgs/**,data/ts-modules/**,data/twoslash/aspire.d.ts. Never workflow files, manifests, source, or docs (integration-docs.jsonreconciliation stays with the manual skill).Validation
aspire-integrations.json(164/164 entries mapped); allowed‑path matcher unit‑checked (correctly excludesintegration-docs.json,package.json, docs).workflow_dispatchon this branch to smoke‑test in CI before merge.Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com