Skip to content

Add snowflake-next preview emulator type - #464

Open
hovaesco wants to merge 4 commits into
mainfrom
lav-595-integration-with-lstk
Open

Add snowflake-next preview emulator type#464
hovaesco wants to merge 4 commits into
mainfrom
lav-595-integration-with-lstk

Conversation

@hovaesco

@hovaesco hovaesco commented Aug 18, 2026

Copy link
Copy Markdown

Motivation

lstk only knows the Python Snowflake emulator: type = "snowflake" means localstack/snowflake. Running the rewritten emulator meant hand-overriding image/tag and remapping env — a recipe that lived in a Slack thread and that every tester had to rediscover.

Solution

Adds snowflake-next as a preview emulator type, so lstk -t snowflake-next just works and records the choice in config like any other type. It is deliberately absent from the interactive first-run picker — a preview is reachable only by asking for it explicitly, so a fresh install's first choice stays a GA product.

The type is now a plain registry entry with no per-emulator special-casing. Earlier revisions carried adaptation code — rewriting the listen address onto the gateway port, bind-mounting the image's own VOLUME, placing PGDATA — because the image diverged from every other emulator. localstack/snowflake-rs#2245 made it a container-level drop-in for localstack/snowflake: it binds from GATEWAY_LISTEN, declares /var/lib/localstack as its volume, and picks its data dir from LOCALSTACK_PERSISTENCE. lstk's generic start path already covers all three, so that code is gone — which is why this diff is ~250 lines smaller than it was.

Found and fixed along the way: the IaC proxies and the emulator list handed to extensions enumerated only the selectable types, so lstk terraform against a running preview reported the misleading LocalStack AWS Emulator is not running.

Verified against the published image (revision 994f10d): start answers health on the configured host port, --persist survives a container recreation, and lstk volume clear still resets state.

Docs

User-facing, worth documenting:

  • New emulator type snowflake-next, a preview of the next Snowflake emulator. Selected with lstk start --type snowflake-next / lstk -t snowflake-next, or type = "snowflake-next" in config.toml. Displayed as "LocalStack Snowflake Preview Emulator". Default image localstack/snowflake-next:latest.
  • Intentionally absent from the interactive first-run picker — reachable only by asking for it explicitly. Worth stating plainly, since it is otherwise undiscoverable.
  • Behaves like the other emulators: --persist keeps state across restarts, lstk volume clear resets it, --endpoint-url can target a remote instance. No type-specific caveats.
  • Migration note for early testers: the hand-rolled image/tag/SNOWFLAKE_LISTEN_ADDR/PGDATA config from the preview period is no longer needed — replace it with type = "snowflake-next". A custom image remains a hard error when switching types.
  • A remote snowflake-next reached over --endpoint-url is reported under the GA Snowflake emulator's name; the two are indistinguishable from the health payload.

Review

Human review advised — new user-facing emulator type, and it depends on the image contract from localstack/snowflake-rs#2245 holding.

Towards LAV-595

Co-Authored-By: Claude noreply@anthropic.com

@hovaesco
hovaesco requested review from a team and peter-smith-phd as code owners August 18, 2026 14:19
@hovaesco hovaesco added semver: minor docs: needed Pull request requires documentation updates labels Aug 18, 2026
@hovaesco
hovaesco force-pushed the lav-595-integration-with-lstk branch from 27d3311 to 796e607 Compare August 18, 2026 14:26

@gtsiolis gtsiolis left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wires the snowflake-next preview emulator type end to end — reachable only via --type/config (never the first-run picker), with its port remap and PostgreSQL persistence handled on the start path, plus integration tests and every non-compiler-checked map/slice touch-point updated.

  1. suggestion(non-blocking): on internal/emulator/snowflake/snowflake.go / internal/container/start.go — the --persist uid-1000 caveat on Linux native Docker is well documented in code, but has no user-facing signal: when it bites, the user sees only an opaque PostgreSQL "refuses to start" surfacing as a health-check timeout. A note/warning on --persist for snowflake-next on Linux pointing at host-directory ownership would save a confusing debugging session. Preview-acceptable to defer.
  2. question(non-blocking): on cmd/status.gosnowflake-next maps to snowflake.NewClient(), but per the comment you added in internal/endpoint/target.go the preview's health payload carries a version and no services map. Does lstk status render correctly against that payload, or could it under-report? I did not verify this against a running preview image (the licensed image wasn't available to me).
  3. praise: thorough TDD — the integration tests pin the port remap, both persistence paths, the first-run picker omission, and the IaC-rejection naming, and every entry the compiler can't catch (knownImages, emulatorHealthPaths, ContainerPort, SelfValidatesLicense, emulatorDisplayNames, the cmd/status.go client map, tipsForType) is covered.

Automated review on behalf of @gtsiolis.


Generated by Claude Code

@peter-smith-phd peter-smith-phd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand what the change is trying achieve, but could we please have some upfront discussion on how to solve it? (Please forgive me if I missed the Slack message). I want to make sure we have a clean/elegant solution to this, rather than just asking Claude to make a whole ton of changes.

Some important questions I have:

  1. Are customers actively using the new Snowflake emulator, or is this for internal use only? What's the timeline for making the new Snowflake emulator the default?
  2. This PR touches 21 files and changes 601 lines of code, which seems excessive given that we're essentially making 3 small configuration changes. When I look at the code changes, I see a lot of places where a list of 3 things becomes a list of 4 things. That implies we should be centralizing those values in a common place, not just repeating the list everywhere.

@hovaesco

Copy link
Copy Markdown
Author
  1. The Snowflake Rust (Next) emulator is not yet being officially released to the public, but several prospects and customers are waiting to test it. We currently only have internal users and beta testers, and the lstk integration is one of the remaining pieces needed for a public release. The sooner we can launch the preview, the better. We’re tentatively targeting GA for early next year, depending on feedback from the preview phase.

  2. To be precise, the 603 added lines include approximately 406 lines of tests and snapshots. The runtime changes also extend beyond three configuration values. The accepted type set is centralized in KnownEmulatorTypes(), while the other occurrences represent behavior owned by different subsystems rather than duplicated copies of the same list. I’m open to consolidating more of this into a descriptor registry, but that would be a broader architectural refactor. If that is required for this PR, could you please suggest the specific abstraction you have in mind?

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adds a snowflake-next preview emulator type, reachable only through --type/config and deliberately never offered by the first-run picker; the plumbing is careful and the genuinely tricky edge cases (uid-1000 state dir, anonymous-volume-per-start leak, remote-payload collapse) are each handled and pinned by a test. I verified CI is green on 5814071 and that discovery is centralized via KnownEmulatorTypes()/KnownImageRepos().

  1. suggestion(non-blocking): per-type behavior still fans out across many sites — emulatorDisplayNames, emulatorHealthPaths, ContainerPort, SelfValidatesLicense, the cmd/status.go client map, tipsForType, and knownImages. Discovery is now centralized, but adding a type remains a set of compiler-unchecked edits in ~7 files (echoing @peter-smith-phd); a follow-up folding per-type config into one table would make the next type a single entry.
  2. thought(non-blocking): prepareNextStateDir in internal/container/start.go chmods the state dir 0777 (world-writable) under the managed volume. The rationale is sound (the emulator runs as uid 1000 and lstk cannot chown to a uid it does not own), but on a shared host it exposes the PostgreSQL cluster dir to other local users — worth noting since it is a security-relevant widening, even if acceptable for a dev tool.
  3. thought(non-blocking): a remote preview instance collapses to EmulatorSnowflake in internal/endpoint/target.go, so lstk status --endpoint-url reports "Snowflake", not "Snowflake Preview", for a preview instance. It's documented and tested (TestStatusEndpointURLSnowflakePreviewPayload) and only the display name differs — just confirming that display divergence is intended.
  4. praise: the documentation and test coverage of the failure modes is exceptional — the anonymous-volume-per-start leak, the uid-1000 PGDATA initialization failure on native Linux Docker, and the widen-an-existing-dir upgrade path are each explained on the declaration and pinned by a dedicated test.

Automated review on behalf of @gtsiolis.


Generated by Claude Code

hovaesco and others added 4 commits August 26, 2026 10:07
Co-Authored-By: Claude <noreply@anthropic.com>
… user

Co-Authored-By: Claude <noreply@anthropic.com>
…leanup

Co-Authored-By: Claude <noreply@anthropic.com>
localstack/snowflake-rs#2245 makes localstack/snowflake-next a
container-level drop-in for localstack/snowflake: it binds from
GATEWAY_LISTEN, declares /var/lib/localstack as its volume, picks its
data dir from LOCALSTACK_PERSISTENCE, and chowns a bind-mounted state
dir before dropping privileges. lstk's generic start path already
covers all of that, so the per-emulator branches, the Next* constants,
and the tests pinning them are redundant. The type stays a plain
registry entry.

Verified against the published image (revision 994f10d): start answers
health on the configured port, --persist survives a container
recreation, and volume clear still works.

Co-Authored-By: Claude <noreply@anthropic.com>
@hovaesco
hovaesco force-pushed the lav-595-integration-with-lstk branch from 5814071 to f891599 Compare August 26, 2026 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs: needed Pull request requires documentation updates semver: minor

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants