Skip to content

Add runner to run tests on DBR#4416

Open
shreyas-goenka wants to merge 21 commits intomainfrom
dbr-runner-(fuse)
Open

Add runner to run tests on DBR#4416
shreyas-goenka wants to merge 21 commits intomainfrom
dbr-runner-(fuse)

Conversation

@shreyas-goenka
Copy link
Contributor

@shreyas-goenka shreyas-goenka commented Feb 2, 2026

Changes

This:

  1. Adds a runner to run tests on DBR.
  2. Sets the RunOnDbr option for all currrent eligible tests. Note: This option cannot be used with RecordRequest = true because serverless does not allow localhost connections.

builds upon #4450

Logs from the test run:

li-dbr-runner-(fuse) git:(dbr-runner-(fuse)) make dbr-test               
deco env run -i -n aws-prod-ucws -- make dbr-integration
2026/02/05 23:50:38 [INFO] Listing secrets from https://deco-aws-prod-ucws-is.vault.azure.net/
DBR_ENABLED=true go test -v -timeout 4h -run TestDbrAcceptance$ ./acceptance
=== RUN   TestDbrAcceptance
    dbr_test.go:221: 
    dbr_test.go:222: === DBR Test Run ===
    dbr_test.go:226:   Cloud tests: (all)
    dbr_test.go:233:   Cluster: serverless
    dbr_test.go:274: 
    dbr_test.go:275: === Job Started ===
    dbr_test.go:276:   Run URL: https://dbc-1232e87d-9384.cloud.databricks.com/?o=470576644108500#job/804912706766727/run/211783203787729
    dbr_test.go:277:   Debug logs: https://dbc-1232e87d-9384.cloud.databricks.com/editor/files/3036332921412433
    dbr_test.go:278: 
    dbr_test.go:279: Waiting for completion (timeout: 3h0m0s)...
    dbr_test.go:304: Job completed. Status: SUCCESS
    dbr_test.go:323: All tests passed!
--- PASS: TestDbrAcceptance (1008.27s)
PASS
ok      github.com/databricks/cli/acceptance    (cached)

The CI job will be added once the runner is merged.

@shreyas-goenka shreyas-goenka changed the title fix and update test configuration Add runner to run tests on DBR Feb 2, 2026
@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Feb 4, 2026

Commit: f0ec698

Run: 21746956937

Env 🟨​KNOWN 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
🟨​ aws linux 7 1 7 451 732 29:52
🟨​ aws windows 7 1 7 417 742 31:24
🟨​ aws-ucws linux 3 9 5 704 585 66:02
🟨​ aws-ucws windows 3 9 5 667 596 62:24
💚​ azure linux 2 9 451 731 25:34
💚​ azure windows 2 9 417 741 25:06
💚​ azure-ucws linux 6 7 666 596 57:29
💚​ azure-ucws windows 6 7 629 607 51:58
💚​ gcp linux 2 9 440 737 27:32
💚​ gcp windows 2 9 406 747 25:39
17 interesting tests: 7 KNOWN, 5 SKIP, 5 RECOVERED
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
🟨​ TestAccept 🟨​K 🟨​K 🟨​K 🟨​K 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/deployment/bind/alert 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/generate/alert 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/alerts/basic 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/alerts/with_file 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/permissions 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions 🟨​K 🟨​K 🟨​K 🟨​K 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 🟨​K 🟨​K
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/with_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions 🟨​K 🟨​K 💚​R 💚​R 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=direct 🟨​K 🟨​K 💚​R 💚​R
🟨​ TestAccept/bundle/resources/permissions/jobs/destroy_without_mgmtperms/without_permissions/DATABRICKS_BUNDLE_ENGINE=terraform 🟨​K 🟨​K 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform 💚​R 💚​R 💚​R 💚​R
💚​ TestAccept/ssh/connection 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
Top 50 slowest tests (at least 2 minutes):
duration env testname
11:48 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
7:52 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
7:51 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:41 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
7:31 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
7:30 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
7:30 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:21 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
7:16 aws-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
7:12 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
7:07 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
6:56 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
6:47 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:45 aws-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
6:25 aws-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
6:19 azure windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
6:04 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:56 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
5:47 azure-ucws windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
5:43 aws-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
5:42 aws-ucws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:41 aws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:40 aws-ucws linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:38 aws-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
5:29 aws windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:26 aws-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
5:24 gcp windows TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
5:18 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
5:16 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
5:11 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=terraform
5:02 gcp linux TestAccept/bundle/resources/clusters/deploy/update-after-create/DATABRICKS_BUNDLE_ENGINE=direct
4:58 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
4:40 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
4:36 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
4:27 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
4:26 azure-ucws windows TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=direct
4:23 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
4:22 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
4:11 azure linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:07 gcp linux TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
4:01 azure-ucws linux TestAccept/bundle/resources/synced_database_tables/basic/DATABRICKS_BUNDLE_ENGINE=terraform
3:45 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:41 azure windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:40 azure-ucws linux TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_catalog.yml.tmpl
3:37 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=direct
3:36 azure-ucws windows TestAccept/bundle/invariant/migrate/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:30 gcp windows TestAccept/bundle/resources/apps/inline_config/DATABRICKS_BUNDLE_ENGINE=terraform
3:21 azure-ucws windows TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl
3:20 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=synced_database_table.yml.tmpl
3:11 azure-ucws linux TestAccept/bundle/invariant/no_drift/DATABRICKS_BUNDLE_ENGINE=direct/INPUT_CONFIG=database_instance.yml.tmpl

// OR
// make dbr-test
func TestDbrAcceptance(t *testing.T) {
if os.Getenv("DBR_ENABLED") != "true" {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DBR tests will run in a separate "env" and will not be a part of normal integration test run. This option enables that.

@shreyas-goenka shreyas-goenka changed the title Add runner to run tests on DBR Add runner to run tests on DBR; Use FUSE on DBR for bundel commands Feb 5, 2026
@shreyas-goenka shreyas-goenka changed the title Add runner to run tests on DBR; Use FUSE on DBR for bundel commands Add runner to run tests on DBR Feb 5, 2026
shreyas-goenka added a commit that referenced this pull request Feb 6, 2026
## Changes
This PR updates the CLI to use the FUSE to read and write notebooks
instead of the workspace APIs, when the DBR version is serverless 2.5+.
This has the following benefits:
1. Much faster bundle deployments / summaries etc for DABs in the
workspace.
2. Fixes commands like `bundle generate --bind` which were previously
failing on DABs in the workspace due to both the workspace APIs and FUSE
being used simultaneously.
3. Enables running tests on DBR. With the previous logic, running tests
on DBR would fail because of mixing workspace API access and FUSE
access.

## Tests
Manually verified that both bundle deploy / bundle generate continue to
work correctly on DBR. Also verified that `bundle generate --bind` works
on DBR.

Acceptance tests will be added once
#4416 lands.

----

bundle generate --bind works, i.e. all notebooks are written
successfully via FUSE:
<img width="821" height="293" alt="Screenshot 2026-02-05 at 23 30 08"
src="https://github.com/user-attachments/assets/55d61b57-39a7-4119-8275-f38df17c6aba"
/>

----
bundle deploy works, for all sorts of notebooks, i.e. they are read
correctly.

<img width="819" height="232" alt="Screenshot 2026-02-05 at 23 30 15"
src="https://github.com/user-attachments/assets/badc91ae-1fee-41e8-9429-4cc8b6d2be97"
/>

----

This was tested and run on serverless `client.2.5` (standard v2). The UI
does not allow me to select a minor version so I could not test other
client.2 versions like 2.4. That's why we only use FUSE from 2.5
onwards.

In practice this is fine because customers wil also likely be using
`2.5` or above it they are on serverless.
@shreyas-goenka shreyas-goenka marked this pull request as ready for review February 6, 2026 10:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants