Skip to content

feat(benchmark): add optional Atlas VLM provider - #205

Open
binyangzhu000-sudo wants to merge 1 commit into
SharpAI:masterfrom
binyangzhu000-sudo:codex/add-atlas-vlm-provider
Open

feat(benchmark): add optional Atlas VLM provider#205
binyangzhu000-sudo wants to merge 1 commit into
SharpAI:masterfrom
binyangzhu000-sudo:codex/add-atlas-vlm-provider

Conversation

@binyangzhu000-sudo

Copy link
Copy Markdown

Summary

  • add opt-in Atlas Cloud VLM execution for the home-security benchmark
  • require explicit paid confirmation and environment-only credentials
  • disable SDK retries for Atlas and allow VLM-only runs without a local LLM

Validation

  • npm ci --ignore-scripts (0 vulnerabilities)
  • npm test (32 assertions passed)
  • node --check scripts/run-benchmark.cjs
  • CLI help and fail-fast gate checks
  • exactly one live Atlas smoke returned HTTP 402 (insufficient balance); it was not retried

Safety

  • local Aegis remains the default path
  • Atlas is enabled only with --atlas-vlm --confirm-paid-atlas
  • credentials are read from environment variables and are not accepted as CLI arguments

Signed-off-by: binyangzhu000-sudo <224954946+binyangzhu000-sudo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant