[VPEX][1/8] Add local-env foundation: result types and env-key mapping by rugpanov · Pull Request #5823 · databricks/cli

rugpanov · 2026-07-03T12:02:29Z

Why

The local-env feature needs a shared vocabulary before any behavior can be built: the result shape, the error taxonomy, and how a compute target maps to an environment key.
Landing these contract types first lets every later layer (resolve / fetch / merge / pipeline / command) depend on stable, reviewed definitions.
Kept deliberately minimal and dependency-free so it reviews on its own and stays unused/deadcode-clean with no consumers yet.

What

result.go — the --json / E_* output contract: Result, PipelineError, ErrorCode, PhaseName, PhaseStatus, Mode, TargetInfo, ResolvedInfo, Plan, Warning; plus the command-path constants (local-env / python / sync) defined in one place.
envkey.go — EnvKeyForServerless / EnvKeyForSparkVersion / NormalizeServerless, and PythonMinorFromRequires (clause-aware: returns the effective highest lower bound of a requires-python).
No wiring into cmd/, so the CLI is unchanged. Filesystem/artifact constants and the phase-order slice deliberately live with their consumer (PR 5).

Testing strategy

Unit tests for the error/type contract (result_test.go) and env-key mapping incl. multi-clause / strict-> / no-floor requires-python cases (envkey_test.go).
Gates: go build, go test, golangci-lint, deadcode, gofmt — all green.
Reviewed with codex across several rounds to convergence (all findings fixed or explicitly rejected as speculative).

About this stack

This is one of a series of small, stacked PRs that together add the databricks local-env python sync command — it provisions a local Python environment (Python version, databricks-connect pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack.

Review bottom-up. Each PR targets the previous one as its base branch, so its diff shows only that layer.

#	PR	What
1	#5823 ← you are here	foundation: result types + env-key mapping
2	#5824	compute-target resolution
3	#5826	constraint fetch + offline cache
4	#5827	formatting-preserving pyproject.toml merge
5	#5828	six-phase pipeline + detection + package-manager interface
6	#5832	uv backend + CLI command (registered hidden)
7	#5833	acceptance tests
8	#5835	unveil (unhide + help + changelog)

This pull request and its description were written by Isaac.

eng-dev-ecosystem-bot · 2026-07-03T12:22:13Z

Integration test report

Commit: afca77c

Run: 28855149237

	Env	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
💚	aws linux	4	4	230	1063	4:42
💚	aws windows	4	4	232	1061	5:49
💚	aws-ucws linux	4	4	314	981	5:26
💚	aws-ucws windows	4	4	316	979	6:21
💚	azure linux	4	4	230	1062	4:56
💚	azure windows	4	4	232	1060	5:53
💚	azure-ucws linux	4	4	316	978	5:33
💚	azure-ucws windows	4	4	318	976	6:15
💚	gcp linux	4	4	229	1064	4:59
💚	gcp windows	4	4	231	1062	5:52

8 interesting tests: 4 RECOVERED, 4 SKIP

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
🙈	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
💚	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
💚	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R
💚	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R	💚R

Top 10 slowest tests (at least 2 minutes):

duration	env	testname
5:21	aws-ucws windows	TestAccept
5:13	azure-ucws windows	TestAccept
5:02	azure windows	TestAccept
4:58	gcp windows	TestAccept
4:58	aws windows	TestAccept
2:59	aws linux	TestAccept
2:55	azure linux	TestAccept
2:54	gcp linux	TestAccept
2:50	aws-ucws linux	TestAccept
2:49	azure-ucws linux	TestAccept

First of a stacked series adding `databricks local-env python sync`, which provisions a local Python environment matched to a Databricks compute target. The feature lands across small, single-concern PRs; each layer is independently reviewable and adds no user-facing surface until the final PR wires the command in. This PR is the foundation the rest of the stack builds on: - result.go: the result types and the --json / E_* error contract that every phase reports through (Result, PipelineError, ErrorCode, PhaseName, PhaseStatus, Mode, TargetInfo, ResolvedInfo, Plan, Warning), plus the command-path constants (local-env / python / sync) defined once. - envkey.go: mapping a compute target to an environment key and parsing the Python minor from a requires-python specifier. Nothing imports this package yet, so the CLI is unchanged. The unexported filesystem/artifact constants and the canonical phase-order slice live with the pipeline that consumes them (a later PR in the stack). Co-authored-by: Isaac

…pecifiers Review of the foundation layer flagged that PythonMinorFromRequires took the first MAJOR.MINOR in the string via first-match regex. For a multi-clause requires-python where the exclusive upper bound comes first — e.g. "<3.13,>=3.10" — it returned 3.13, the version the "<3.13" clause forbids, because PEP 440 clause order is arbitrary. The result feeds PM.EnsurePython, so the tool could target a Python the constraint excludes. Prefer a lower-bound / pinning clause (>=, >, ==, ~=, ===) and only fall back to the first version when none is present. Adds multi-clause test coverage; the prior tests exercised only single-bound specifiers. Co-authored-by: Isaac

…dden version Round-2 review of the foundation layer noted that when a requires-python has no lower-bound/pin clause at all (only upper-bound or exclusion, e.g. "<3.13,!=3.12"), PythonMinorFromRequires fell back to the first number and returned 3.13 — a version the specifier forbids. Such a spec has no floor to install from, so it now errors rather than guessing. A bare "3.12" (no operator) is still accepted as a valid floor. Co-authored-by: Isaac

…ower bound Round-3 review found two edge cases in the regex-based PythonMinorFromRequires: with multiple lower bounds (">=3.8,>=3.11") it returned the first (3.8) rather than the effective floor (3.11), and a bare floor alongside an exclusion ("!=3.11,3.12") was wrongly rejected as having no floor. Replaced the layered regexes with a small clause parser: split on commas, classify each clause by operator (>=,>,==,~=,=== and bare = floor; <,<=,!= never a floor), and return the highest floor. A spec with no floor clause ("<3.13", "!=3.12") still errors. Covers multi-lower-bound, bare-floor + exclusion, ordering, and whitespace. Co-authored-by: Isaac

Round-4 review noted PythonMinorFromRequires returned 3.10 for ">3.10", but PEP 440's ">" excludes the entire given release series (neither 3.10 nor any 3.10.x satisfies ">3.10"), so the lowest installable minor is 3.11. The clause parser now bumps the minor by one for a strict ">" bound. Co-authored-by: Isaac

anton-107

Thanks for the clean, dependency-free foundation — easy to review in isolation. Two items I'd like addressed before this lands: one real (if narrow) correctness bug in PythonMinorFromRequires, and one --json contract nit. Details inline.

anton-107 · 2026-07-06T10:47:20Z

+		if op == ">" {
+			minor++


clauseRe captures only MAJOR.MINOR, so the patch component is invisible here. For a bare-minor strict bound like >3.10 this minor++ is correct (PEP 440 excludes all of 3.10.x, as the comment above explains). But for >3.10.5 the effective floor should stay 3.10 — 3.10.6 satisfies the constraint — yet this bumps it to 3.11, selecting the wrong Python version.

It's uncommon in real requires-python, but it's a genuinely incorrect result rather than a rounding choice. Could you either capture the patch in clauseRe and only bump the minor when the patch is absent, or document the limitation explicitly? A >3.10.5 → 3.10 case in TestPythonMinorFromRequires would lock it in.

Good catch — fixed. clauseRe now captures the patch component and the minor is only bumped for a strict > when the patch is absent, so >3.10.5 correctly stays 3.10 (3.10.6 satisfies it) while a bare >3.10 still yields 3.11. Added >3.10.5 / >3.10.5,<3.13 / >=3.10.2 cases to TestPythonMinorFromRequires. (commit 900518b)

anton-107 · 2026-07-06T10:47:20Z

+	Phases        []PhaseStatus  `json:"phases"`
+	Warnings      []Warning      `json:"warnings"`


Phases and Warnings have no omitempty and are slices, so a zero-value Result marshals them as "phases": null, "warnings": null rather than []. For a versioned --json contract, null-vs-empty-array is a classic ambiguity that trips up consumers (and golden-file diffs).

If the pipeline (#5828) always initializes both to non-nil before marshalling, this is moot — could you confirm that here, or otherwise guarantee they always emit as []?

Confirmed and hardened. The pipeline (#5828) always seeds both to non-nil (initialPhases() and []Warning{}), so --json never emits null in practice. To guarantee it at the type level rather than by convention, I added a NewResult() constructor here that seeds both slices, a doc note on the invariant, and a test asserting the JSON emits [] (and that a bare Result{} is the null case to avoid). #5828 now constructs its Result via NewResult(). (commit 900518b)

…json arrays Two items from review of the foundation layer: - PythonMinorFromRequires bumped the minor for any strict ">" bound, but a patch-qualified bound like ">3.10.5" is still satisfied by 3.10.6, so the floor should stay 3.10 (only a bare ">3.10" excludes the whole 3.10.x series). clauseRe now captures the patch component and the minor is bumped only when it is absent. Adds >3.10.5 / >=3.10.2 test cases. - Result.Phases and Result.Warnings are non-omitempty slices, so a bare Result{} would marshal them as "null" rather than "[]", an ambiguity in the --json contract. Added NewResult() which seeds both to empty slices, a doc note on the invariant, and a test asserting the JSON emits [] not null. The pipeline (later in the stack) constructs its Result through this. Co-authored-by: Isaac

rugpanov · 2026-07-06T11:49:21Z

@anton-107 both items addressed in commit 900518bcf — the >3.10.5 patch-bound fix (with tests) and a NewResult() constructor guaranteeing phases/warnings marshal as [] (with a test). Ready for another look.

anton-107

Both review items are addressed cleanly:

Patch-qualified strict bound — clauseRe now captures the patch component and the minor is bumped only for a bare >3.10 (not >3.10.5), with >3.10.5 / >3.10.5,<3.13 / >=3.10.2 test cases pinning it. Correct per PEP 440.
--json arrays — NewResult() seeds phases/warnings to non-nil, a struct doc-note records the invariant, and a test asserts [] rather than null (with a bare-Result{} sanity check). Confirmed the pipeline constructs through it.

LGTM — thanks for the quick turnaround.

eng-dev-ecosystem-bot · 2026-07-07T12:15:54Z

Integration test report

Commit: 4c2d06e

Run: 28858481053

	Env	❌FAIL	🟨KNOWN	🔄flaky	💚RECOVERED	🙈SKIP	✅pass	🙈skip	Time
❌	aws linux	6	1		3	4	523	992	40:55
❌	aws windows	6		4	4	4	476	1005	51:11
🔄	aws-ucws linux			8	6	2	980	807	115:18
🔄	aws-ucws windows			4	6	2	926	825	113:46
💚	azure linux				4	4	523	993	41:53
🔄	azure windows			6		4	478	1006	50:48
❌	azure-ucws linux	19	3		3	2	875	838	80:52
❌	azure-ucws windows	34	6			2	802	856	101:25
💚	gcp linux				4	4	514	998	44:10
🔄	gcp windows			3	3	4	469	1011	51:21

58 interesting tests: 44 FAIL, 6 KNOWN, 6 flaky, 2 SKIP

	Test Name	aws linux	aws windows	aws-ucws linux	aws-ucws windows	azure linux	azure windows	azure-ucws linux	azure-ucws windows	gcp linux	gcp windows
🟨	TestAccept	🟨K	💚R	💚R	💚R	💚R	🔄f	🟨K	🟨K	💚R	🔄f
❌	TestAccept/bundle/apps/job_permissions	🙈s	🙈s	✅p	✅p	🙈s	🙈s	❌F	❌F	🙈s	🙈s
❌	TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=direct			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/apps/job_permissions/DATABRICKS_BUNDLE_ENGINE=terraform			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/deployment/bind/vector_search_endpoint	🙈s	🙈s	✅p	✅p	🙈s	🙈s	❌F	❌F	🙈s	🙈s
❌	TestAccept/bundle/deployment/bind/vector_search_endpoint/DATABRICKS_BUNDLE_ENGINE=direct			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/generate/pipeline_and_deploy	✅p	🔄f	🔄f	🔄f	✅p	✅p	❌F	✅p	✅p	🔄f
❌	TestAccept/bundle/generate/pipeline_and_deploy/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	🔄f	🔄f	✅p	✅p	❌F	✅p	✅p	🔄f
🔄	TestAccept/bundle/generate/pipeline_and_deploy/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestAccept/bundle/generate/python_job_and_deploy	✅p	🔄f	🔄f	🔄f	✅p	🔄f	❌F	✅p	✅p	✅p
❌	TestAccept/bundle/generate/python_job_and_deploy/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	🔄f	🔄f	✅p	✅p	❌F	✅p	✅p	✅p
🔄	TestAccept/bundle/generate/python_job_and_deploy/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	🔄f	✅p	✅p	✅p	🔄f	✅p	✅p	✅p	✅p
❌	TestAccept/bundle/invariant/continue_293	🙈s	🙈s	✅p	✅p	🙈s	🙈s	❌F	❌F	🙈s	🙈s
❌	TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/invariant/continue_293/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/invariant/migrate	🙈s	🙈s	✅p	✅p	🙈s	🙈s	❌F	❌F	🙈s	🙈s
❌	TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl			✅p	✅p			❌F	❌F
🟨	TestAccept/bundle/invariant/no_drift	🙈S	🙈S	💚R	💚R	🙈S	🙈S	🟨K	🟨K	🙈S	🙈S
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl/READPLAN=			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=app.yml.tmpl/READPLAN=1			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl/READPLAN=			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl/READPLAN=1			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl/READPLAN=			✅p	✅p			❌F	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl/READPLAN=1			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=vector_search_endpoint.yml.tmpl/READPLAN=			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=vector_search_endpoint.yml.tmpl/READPLAN=1			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/resources/apps/inline_config	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
❌	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
❌	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
❌	TestAccept/bundle/resources/apps/lifecycle-started	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
❌	TestAccept/bundle/resources/apps/lifecycle-started-omitted	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
❌	TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
❌	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	✅p	✅p	✅p	✅p	✅p	❌F	✅p	✅p
🔄	TestAccept/bundle/resources/clusters/deploy/simple	✅p	✅p	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🔄	TestAccept/bundle/resources/clusters/deploy/simple/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🔄	TestAccept/bundle/resources/clusters/lifecycle-started	✅p	✅p	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p
🔄	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct	✅p	✅p	🔄f	✅p	✅p	✅p	✅p	✅p	✅p	✅p
❌	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions	❌F	❌F	✅p	✅p	🙈s	🙈s	🙈s	🙈s	🙈s	🙈s
❌	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct	❌F	❌F	✅p	✅p
❌	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	❌F	❌F	✅p	✅p
❌	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions	❌F	❌F	✅p	✅p	🙈s	🙈s	🙈s	🙈s	🙈s	🙈s
❌	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct	❌F	❌F	✅p	✅p
❌	TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform	❌F	❌F	✅p	✅p
❌	TestAccept/bundle/resources/synced_database_tables/basic	🙈s	🙈s	✅p	✅p	🙈s	🙈s	✅p	❌F	🙈s	🙈s
❌	TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform			✅p	✅p			✅p	❌F
❌	TestAccept/bundle/resources/vector_search_endpoints/basic	🙈s	🙈s	✅p	✅p	🙈s	🙈s	✅p	❌F	🙈s	🙈s
❌	TestAccept/bundle/resources/vector_search_endpoints/basic/DATABRICKS_BUNDLE_ENGINE=direct			✅p	✅p			✅p	❌F
🟨	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name	🙈S	🙈S	💚R	💚R	🙈S	🙈S	🟨K	🟨K	🙈S	🙈S
❌	TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name/DATABRICKS_BUNDLE_ENGINE=direct			✅p	✅p			❌F	❌F
🙈	TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🙈	TestAccept/ssh/connection	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S	🙈S
🟨	TestFetchRepositoryInfoAPI_FromRepo	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	🟨K	💚R	💚R
🟨	TestFetchRepositoryInfoAPI_FromRepo/root	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	🟨K	💚R	💚R
🟨	TestFetchRepositoryInfoAPI_FromRepo/subdir	💚R	💚R	💚R	💚R	💚R	🔄f	💚R	🟨K	💚R	💚R

Top 50 slowest tests (at least 2 minutes):

duration	env	testname
18:16	aws-ucws windows	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
13:40	aws-ucws windows	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
12:01	gcp linux	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
11:56	gcp windows	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
10:41	aws-ucws linux	TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
8:58	gcp linux	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:57	gcp windows	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
8:48	gcp windows	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:15	aws windows	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:05	aws-ucws windows	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
8:03	azure linux	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
7:48	aws windows	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:47	aws linux	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:33	aws-ucws linux	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:21	azure windows	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:16	azure-ucws linux	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:11	aws-ucws linux	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=1
7:10	azure linux	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
7:00	aws-ucws windows	TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:59	gcp windows	TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:57	gcp linux	TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
6:57	aws-ucws linux	TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=cluster_apply_policy_default_values.yml.tmpl/READPLAN=
6:57	gcp linux	TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:56	gcp linux	TestAccept/bundle/resources/clusters/deploy/local_ssd_count/DATABRICKS_BUNDLE_ENGINE=direct
6:53	azure-ucws linux	TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
6:45	aws linux	TestAccept/bundle/resources/apps/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:40	aws-ucws linux	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:37	aws-ucws windows	TestAccept/bundle/resources/clusters/resize-terminated-fallback/DATABRICKS_BUNDLE_ENGINE=direct
6:34	azure-ucws windows	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:26	gcp linux	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
6:06	azure windows	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
6:06	azure linux	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:59	aws-ucws linux	TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=direct
5:56	aws-ucws linux	TestAccept/bundle/resources/clusters/lifecycle-started-toggle/DATABRICKS_BUNDLE_ENGINE=direct
5:50	aws-ucws linux	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
5:30	azure-ucws linux	TestAccept/bundle/resources/clusters/lifecycle-started/DATABRICKS_BUNDLE_ENGINE=direct
5:16	aws-ucws linux	TestAccept/bundle/resources/clusters/deploy/data_security_mode/DATABRICKS_BUNDLE_ENGINE=direct
5:15	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
5:10	aws-ucws linux	TestAccept/bundle/config-remote-sync/multiple_resources/DATABRICKS_BUNDLE_ENGINE=terraform
5:07	aws-ucws windows	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
5:04	aws-ucws windows	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:57	aws linux	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:56	gcp linux	TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:52	gcp windows	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:46	gcp linux	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:33	gcp windows	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:32	aws linux	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=direct
4:30	gcp linux	TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct
4:30	azure-ucws windows	TestAccept/bundle/deploy/spark-jar-task/DATABRICKS_BUNDLE_ENGINE=terraform
4:21	gcp windows	TestAccept/bundle/resources/apps/lifecycle-started-omitted/DATABRICKS_BUNDLE_ENGINE=direct

## Why - `local-env` must turn the user's compute selection into a single environment key before it can fetch anything, and the selection can come from several places with a defined precedence. - Isolating resolution behind a narrow seam keeps it testable without a live workspace and keeps SDK details out of the engine. ## What - **`target.go`** — `ResolveTarget` with ordered precedence `--cluster` → `--serverless` → `--job` → bundle target, producing a `TargetInfo` + env key. - Compute lookups go through the narrow `ComputeClient` interface (stubbable in tests). - `ValidateTargetFlags` rejects more than one target flag; `ResolveTarget` runs it up front so a non-Cobra caller can't silently resolve the wrong target. - Classic-compute jobs read the Spark version from the documented first return of `GetJobSparkVersion` (not the recorded-version third return). ## Testing strategy - Unit tests against a stub `ComputeClient` covering each precedence branch, the mutually-exclusive-flags error, and the job classic-compute contract (`target_test.go`). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex to a clean pass. --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | databricks#5823 | foundation: result types + env-key mapping | | 2 | **databricks#5824 ← you are here** | compute-target resolution | | 3 | databricks#5826 | constraint fetch + offline cache | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.

…icks#5826) ## Why - Once a target resolves to an env key, `local-env` needs the pinned Python version, `databricks-connect` version, and dependency constraints published for that key. - The fetch must degrade gracefully offline and distinguish “this environment isn't published” from “the network is down,” because those call for different user action. - The artifact host must be a Databricks-owned, access-controlled location and must never default to a personal repo — whoever controls the host controls what the CLI installs. ## What - **`constraints.go`** — fetches the per-environment `pyproject.toml`, parses `requires-python`, the `databricks-connect` pin, and `[tool.uv]` `constraint-dependencies`, and caches it on disk. - **Host parameterization** — no host is hardcoded: `RepoConstraintBaseURL` reads the repo (`owner/name`) from the temporary `DATABRICKS_LOCALENV_CONSTRAINT_REPO` env var and builds a `raw.githubusercontent.com/<repo>/main` URL; the built-in default is empty. When unset it returns `""` and `FetchConstraints` reports the missing source as a fetch-phase `E_FETCH` error (so there is no untrusted default, and the failure flows through the normal phase/JSON reporting). Once `databricks/environments` can publish, that becomes the hardcoded default and the env var is no longer required. - Failure classification: **404** → `E_ENV_UNSUPPORTED` (no cache fallback — a distinct non-transient condition); **transport / non-404** → `E_FETCH` with fallback to the last-good cached copy. - Robustness: validate the body (parse + require `requires-python`) **before** caching so a bad 2xx can't poison the cache; atomic cache write (mkdir + temp-file + rename); dedicated `http.Client` with a 30s timeout; body read bounded by `io.LimitReader` at 1 MiB; `databricks-connect` matched by leading package name under PEP 503 normalization (so `Databricks_Connect` matches, `databricks-connectors` does not); cache filename = readable slug + sha256 suffix to prevent collisions. ## Testing strategy - Unit tests with an `httptest` server: 200-parse, 404 → `E_ENV_UNSUPPORTED`, transport failure + cache fallback, missing-`requires-python` rejection, PEP 503 name matching, cache-dir creation, collision-free filenames, oversized-body rejection (`constraints_test.go`). - Host resolution: `TestRepoConstraintBaseURL` (env var → URL, unset → `""`, whitespace treated as unset) and `TestFetchConstraintsNoSourceConfigured` (empty host → `E_FETCH` naming the env var). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex to a clean pass (several fetch/cache edge-case fixes landed from review). --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | databricks#5823 | foundation: result types + env-key mapping | | 2 | databricks#5824 | compute-target resolution | | 3 | **databricks#5826 ← you are here** | constraint fetch + offline cache | | 4 | databricks#5827 | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.

…atabricks#5827) ## Why - `local-env` must apply the resolved Python version and constraints to the user's `pyproject.toml` without disturbing their own content — comments, ordering, formatting, and unrelated config must survive untouched. - Re-running must be safe and idempotent, and a greenfield project needs a sensible file created from scratch. - This is the most intricate logic in the feature, so it lands in its own PR for focused review. ## What - **`merge.go`** — a formatting-preserving merge that rewrites only the env-owned regions (`requires-python`, the `databricks-connect` entry in `[dependency-groups].dev`, and a marker-bracketed managed `[tool.uv]` block) and preserves every other byte incl. CRLF; idempotent. `RenderFreshPyproject` builds a complete managed file for a greenfield project. - Scoping/robustness: managed `constraint-dependencies` nests header-less inside an existing user `[tool.uv]` (never a duplicate header); single- vs multi-line array detection tracks real bracket depth outside strings/comments; the `databricks-connect` rewrite is confined to `dev` and leaves trailing comments alone; `requires-python`'s inline comment is preserved; table-header parsing tolerates inline comments and recognizes `[[array.of.tables]]`. ## Testing strategy - Unit tests that parse the merged output as TOML (not just substring checks), covering idempotency, CRLF preservation, user-key preservation, the duplicate-`[tool.uv]` case, bracket-in-element arrays, sibling-group/comment non-clobbering, and `[[tool.uv.index]]` children (`merge_test.go`). - Gates: `go build`, `go test`, `golangci-lint`, `deadcode`, `gofmt` — all green. - Reviewed with codex to a clean pass (multiple TOML-corruption edge cases were caught and fixed). --- ## About this stack This is one of a series of small, stacked PRs that together add the `databricks local-env python sync` command — it provisions a local Python environment (Python version, `databricks-connect` pin, and dependency constraints) matched to a selected Databricks compute target. The work was split from one large branch into single-concern layers so each is independently reviewable; the command is kept hidden until the final PR so nothing is user-visible mid-stack. **Review bottom-up.** Each PR targets the previous one as its base branch, so its diff shows only that layer. | # | PR | What | |---|----|------| | 1 | databricks#5823 | foundation: result types + env-key mapping | | 2 | databricks#5824 | compute-target resolution | | 3 | databricks#5826 | constraint fetch + offline cache | | 4 | **databricks#5827 ← you are here** | formatting-preserving pyproject.toml merge | | 5 | databricks#5828 | six-phase pipeline + detection + package-manager interface | | 6 | databricks#5832 | uv backend + CLI command (registered hidden) | | 7 | databricks#5833 | acceptance tests | | 8 | databricks#5835 | unveil (unhide + help + changelog) | This pull request and its description were written by Isaac.

rugpanov added the ai-assisted label Jul 3, 2026

rugpanov temporarily deployed to test-trigger-is July 3, 2026 12:02 — with GitHub Actions Inactive

rugpanov force-pushed the dbconnect/01-engine branch from 87c371d to 865a2cd Compare July 3, 2026 12:23

rugpanov changed the title ~~Add dbconnect engine library (types, resolve, fetch, merge, pipeline)~~ Add dbconnect foundation: result types and env-key mapping Jul 3, 2026

rugpanov temporarily deployed to test-trigger-is July 3, 2026 12:23 — with GitHub Actions Inactive

rugpanov force-pushed the dbconnect/01-engine branch from 865a2cd to 22f99d9 Compare July 3, 2026 13:20

rugpanov changed the title ~~Add dbconnect foundation: result types and env-key mapping~~ [VPEX][1/8] Add local-env foundation: result types and env-key mapping Jul 3, 2026

rugpanov temporarily deployed to test-trigger-is July 3, 2026 13:20 — with GitHub Actions Inactive

rugpanov temporarily deployed to test-trigger-is July 3, 2026 15:27 — with GitHub Actions Inactive

rugpanov temporarily deployed to test-trigger-is July 3, 2026 18:15 — with GitHub Actions Inactive

rugpanov temporarily deployed to test-trigger-is July 3, 2026 18:27 — with GitHub Actions Inactive

rugpanov temporarily deployed to test-trigger-is July 3, 2026 19:14 — with GitHub Actions Inactive

rugpanov mentioned this pull request Jul 3, 2026

[VPEX][6/8] Add local-env uv backend and hidden CLI command #5832

Open

This was referenced Jul 3, 2026

[VPEX][7/8] Add local-env acceptance tests #5833

Open

[VPEX][8/8] Unveil local-env: make the command visible, add help + changelog #5835

Open

anton-107 requested changes Jul 6, 2026

View reviewed changes

rugpanov temporarily deployed to test-trigger-is July 6, 2026 11:47 — with GitHub Actions Inactive

anton-107 approved these changes Jul 6, 2026

View reviewed changes

Merge branch 'main' into dbconnect/01-engine

afca77c

rugpanov enabled auto-merge July 7, 2026 09:14

rugpanov temporarily deployed to test-trigger-is July 7, 2026 09:14 — with GitHub Actions Inactive

rugpanov added this pull request to the merge queue Jul 7, 2026

Merged via the queue into main with commit 4c2d06e Jul 7, 2026
23 checks passed

rugpanov deleted the dbconnect/01-engine branch July 7, 2026 10:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[VPEX][1/8] Add local-env foundation: result types and env-key mapping#5823

[VPEX][1/8] Add local-env foundation: result types and env-key mapping#5823
rugpanov merged 7 commits into
mainfrom
dbconnect/01-engine

rugpanov commented Jul 3, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Jul 3, 2026 •

edited

Loading

Uh oh!

anton-107 left a comment

Uh oh!

anton-107 Jul 6, 2026

Uh oh!

rugpanov Jul 6, 2026

Uh oh!

anton-107 Jul 6, 2026

Uh oh!

rugpanov Jul 6, 2026

Uh oh!

rugpanov commented Jul 6, 2026

Uh oh!

anton-107 left a comment

Uh oh!

Uh oh!

eng-dev-ecosystem-bot commented Jul 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		Phases []PhaseStatus `json:"phases"`
		Warnings []Warning `json:"warnings"`

Uh oh!

Conversation

rugpanov commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why

What

Testing strategy

About this stack

Uh oh!

eng-dev-ecosystem-bot commented Jul 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Integration test report

Uh oh!

anton-107 left a comment

Choose a reason for hiding this comment

Uh oh!

anton-107 Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

rugpanov Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

anton-107 Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

rugpanov Jul 6, 2026

Choose a reason for hiding this comment

Uh oh!

rugpanov commented Jul 6, 2026

Uh oh!

anton-107 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

eng-dev-ecosystem-bot commented Jul 7, 2026

Integration test report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

rugpanov commented Jul 3, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Jul 3, 2026 •

edited

Loading