Skip to content

Add Positron API integration tests and CI workflow - #115

Merged
wch merged 5 commits into
mainfrom
jonv/positron-shiny-api-tests-b68e7b
Jul 26, 2026
Merged

Add Positron API integration tests and CI workflow#115
wch merged 5 commits into
mainfrom
jonv/positron-shiny-api-tests-b68e7b

Conversation

@jonvanausdeln

@jonvanausdeln jonvanausdeln commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes posit-dev/positron#14964

Adds a Positron-only integration test suite that runs the Shiny extension inside a real Positron build via @posit-dev/positron-test-electron, plus a CI workflow that runs it on every PR and push to main.

Part of the rollout tracked in posit-dev/positron#14531 (pattern established in quarto-dev/quarto#1058 and posit-dev/publisher#4298).

What's covered

The starter tests (src/test/positron/) exercise the extension's real Positron API surface — code paths unreachable by the plain VS Code suite:

  • extension.test.ts — Positron injects the acquirePositronApi global, the extension's own feature detection (isPositron() / getIdeName() in src/extension-api-utils/extensionHost.ts) recognizes Positron, and posit.shiny activates in the Positron extension host.
  • run-app-api.test.tsgetPositronRunAppApi() resolves the bundled positron-run-app extension with runApplicationInConsole — the exact gate rRunApp() uses to decide between running R apps in Positron's console vs. a terminal.
  • viewer-preview.test.tsPreviewSourceType.Terminal is exposed with the wire value the extension's fallback hardcodes, and a live previewUrl() round-trip returns a Viewer panel, mirroring openBrowser().

Harness

  • scripts/run-positron-tests.mjs downloads/caches a Positron build (stable by default, POSITRON_CHANNEL=daily supported) and runs the Mocha entry point (src/test/positron/index.ts) inside it. Bundled extensions stay enabled because the tests exercise positron-run-app.
  • Run locally with npm run test-positron (macOS only for now — an upstream positron-test-electron limitation; Windows/Linux follow once extractable archives are published).
  • The plain suite runner now ignores out/test/positron/ so npm test doesn't pick up Positron-only tests in vanilla VS Code.
  • src/test/positron/README.md documents how to run and add tests.

CI

.github/workflows/positron-api-tests.yaml — macos-latest, launched through posit-dev/setup-positron, with a date-keyed cache of the Positron download. Runs on PRs and pushes to main.

Found by the new tests: PreviewSourceType drift

The first CI run caught real API drift: the vendored typings (src/types/positron.d.ts) declared PreviewSourceType as a numeric enum (Terminal = 2), but Positron has shipped it as a string enum (Terminal = 'terminal') since before this extension's minimum supported version (2025.12.0). Runtime behavior was accidentally correct (the live enum value is passed through), but getPreviewSourceTypeTerminal()'s declared return type was wrong and buildPreviewSource()'s hardcoded fallback (2) would have sent Positron an invalid value if it ever fired. Fixed by syncing the vendored enum with upstream, correcting the return type, and changing the fallback to 'terminal'; the test now pins the wire value so future drift is caught again. (failing rungreen run)

Drive-by fixes: the plain npm test suite was silently broken

Since the esbuild migration, npm test always exited 0 without running anything meaningful:

  • import * as Mocha + new Mocha() throws "Mocha is not a constructor" under esbuild's ESM interop → now import Mocha = require("mocha").
  • The runner fired mocha.run() without awaiting it, so the extension host exited before any results and failures could never fail the run → now wrapped in a promise.
  • With results actually reported, the StreamingTagParser test turned out to be stale (asserts on a boolean return from process(), which is now async/void with results via contentHandler) — skipped with a comment pending a rewrite against the current API.

Testing

  • All 7 Positron tests pass on CI inside a live Positron: run 29868140475 (7 passing, 5s)
  • npm test (vanilla VS Code suite) passes locally on Windows with the Positron tests excluded
  • tsc, ESLint, and Prettier clean

Two pre-existing issues left npm test silently green since the esbuild
migration:

- import * as Mocha + new Mocha() throws "Mocha is not a constructor"
  under esbuild's ESM interop; use import = require so esbuild emits a
  plain require of the CJS constructor.
- run() fired mocha.run() without awaiting it, so the extension host
  exited 0 before any results; failures could never fail the run.

With results reported, the StreamingTagParser test turned out to be
stale (it asserts on a boolean return from process(), which is now
async returning void, with results delivered via contentHandler), so
it is skipped pending a rewrite against the current API.

The runner also now ignores out/test/positron/, home of the
Positron-only suite added in the next commit.
Part of the rollout tracked in posit-dev/positron#14531 (pattern
established in quarto-dev/quarto#1058 and posit-dev/publisher#4298).

Adds a Positron-only Mocha suite (src/test/positron/) that runs the
extension inside a real Positron build via
@posit-dev/positron-test-electron, exercising the Positron API surface
the plain VS Code suite can't reach:

- extension.test.ts: the acquirePositronApi global is injected, the
  extension's own feature detection (isPositron/getIdeName) recognizes
  Positron, and posit.shiny activates.
- run-app-api.test.ts: getPositronRunAppApi() resolves the bundled
  positron-run-app extension with runApplicationInConsole - the gate
  rRunApp() uses to run R apps in the console.
- viewer-preview.test.ts: PreviewSourceType.Terminal is exposed and a
  live previewUrl() round-trip returns a Viewer panel, mirroring
  openBrowser().

Run locally with npm run test-positron (macOS only for now, an
upstream positron-test-electron limitation); CI runs on every PR and
push to main via posit-dev/setup-positron with a date-keyed cache of
the Positron download. See src/test/positron/README.md.

Also adds the one-line 'export const version' to the vendored Positron
typings, matching upstream positron.d.ts.
Remove this commit (or just the branch trigger) before opening the PR;
it exists only to exercise the workflow without a PR.
Caught by the new Positron API tests on their first CI run: the
vendored typings declared PreviewSourceType as a numeric enum
(Terminal = 2), but Positron has shipped it as a string enum
(Terminal = 'terminal') since before this extension's minimum
supported version (2025.12.0).

At runtime the extension passes the live enum value through, so
current behavior was correct - but getPreviewSourceTypeTerminal()'s
declared return type (number) was wrong, and buildPreviewSource()'s
hardcoded fallback (2) would have sent an invalid value to Positron
if it ever fired. Sync the vendored enum with upstream, fix the
return type, change the fallback to 'terminal', and pin the test to
the real wire value so future drift is caught again.
@jonvanausdeln
jonvanausdeln requested review from gadenbuie and wch July 21, 2026 22:02
@wch
wch merged commit 1cdc0f6 into main Jul 26, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

testing: add API tests to posit.shiny extension

2 participants