Problem
cmd/verifyexamples runs the Go examples and checks their output, but no CI workflow currently runs it against live model providers.
Unit tests use mocked providers, so they do not catch authentication failures, provider API changes, or other problems that only appear in a real model call.
PR #431 added verifyexamples and left scheduled CI as follow-up work once provider access was available.
Proposed work
Add a scheduled and manually dispatched GitHub Actions workflow that:
- Runs a small, stable set of examples through
cmd/verifyexamples.
- Writes the Markdown report to the GitHub job summary.
- Uploads the detailed report as an artifact.
- Uses short-lived authentication where possible.
- Keeps API keys and tokens out of the repository.
- Starts as an informational check rather than a required PR check.
Provider investigation
Start by checking whether the GitHub Copilot provider can use authentication already available to GitHub Actions. Copilot-only coverage would still test real agent calls without requiring accounts for every supported provider.
verifyexamples currently uses Foundry for semantic output verification. If Copilot is the first provider tested, decide whether to:
- Use deterministic output checks.
- Add Copilot-backed semantic verification.
- Keep Foundry as the verifier and configure Azure OIDC.
For broader provider coverage, ask the Agent Framework team how they provision and manage credentials for their .NET and Python integration workflows. Anthropic and Gemini may require separate paid credentials.
Done when
- A scheduled and manually dispatched workflow runs selected examples.
- At least one live provider path runs end to end.
- Missing credentials fail or skip with a clear reason.
- The workflow publishes a report even when an example fails.
- Authentication and ownership are documented.
- No reusable credential is committed to the repository.
References
Problem
cmd/verifyexamplesruns the Go examples and checks their output, but no CI workflow currently runs it against live model providers.Unit tests use mocked providers, so they do not catch authentication failures, provider API changes, or other problems that only appear in a real model call.
PR #431 added
verifyexamplesand left scheduled CI as follow-up work once provider access was available.Proposed work
Add a scheduled and manually dispatched GitHub Actions workflow that:
cmd/verifyexamples.Provider investigation
Start by checking whether the GitHub Copilot provider can use authentication already available to GitHub Actions. Copilot-only coverage would still test real agent calls without requiring accounts for every supported provider.
verifyexamplescurrently uses Foundry for semantic output verification. If Copilot is the first provider tested, decide whether to:For broader provider coverage, ask the Agent Framework team how they provision and manage credentials for their .NET and Python integration workflows. Anthropic and Gemini may require separate paid credentials.
Done when
References