Skip to content

Conversation

@andrewnester
Copy link
Contributor

Changes

Added support for --bind flag in bundle generate

Why

This allows us to generate and bind the resources using one command instead of 2 separate commands.

Tests

Added an acceptance test

@eng-dev-ecosystem-bot
Copy link
Collaborator

eng-dev-ecosystem-bot commented Oct 17, 2025

Run: 18873802576

Env ❌​FAIL 🟨​KNOWN 🔄​flaky 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip
💚​ aws linux 1 2 320 582
💚​ aws windows 1 2 321 581
💚​ aws-ucws linux 2 1 440 478
💚​ aws-ucws windows 2 1 441 477
💚​ azure linux 1 2 320 581
💚​ azure windows 1 2 321 580
🟨​ azure-ucws linux 2 1 438 477
🟨​ azure-ucws windows 2 2 1 437 476
❌​ gcp linux 58 1 2 265 571
❌​ gcp windows 22 1 2 302 578
73 failing tests:
Test Name aws linux aws windows aws-ucws linux aws-ucws windows azure linux azure windows azure-ucws linux azure-ucws windows gcp linux gcp windows
TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R 🟨​K 🟨​K 🟨​K 🟨​K
TestAccept/bundle/deploy/files/no-snapshot-sync ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/deploy/files/no-snapshot-sync/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/destroy/jobs-and-pipeline ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/destroy/jobs-and-pipeline/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/jobs/check-metadata ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/resources/jobs/check-metadata/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/resources/jobs/double-underscore-keys ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/jobs/double-underscore-keys/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/jobs/fail-on-active-runs ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/jobs/fail-on-active-runs/DATABRICKS_BUNDLE_ENGINE=direct-exp ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/models/basic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/models/basic/DATABRICKS_BUNDLE_ENGINE=direct-exp ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/resources/pipelines/update ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/resources/pipelines/update/DATABRICKS_BUNDLE_ENGINE=terraform ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/resources/synced_database_tables/basic 🙈​S 🙈​S 💚​R 💚​R 🙈​S 🙈​S 🟨​K 🟨​K 🙈​S 🙈​S
TestAccept/bundle/run/app-with-job 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
TestAccept/bundle/templates/default-python/combinations/classic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=no/PY=no ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=no/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=yes/PY=no ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=yes/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=no ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=yes/PY=no ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=yes/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=no ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/classic/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/combinations/serverless ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=no/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s ❌​F
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=no/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=yes/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=no/NBOOK=yes/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=no/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=yes/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=direct-exp/DLT=yes/NBOOK=yes/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s ❌​F
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=no/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=no/NBOOK=yes/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=no/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=no 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p ❌​F 🙈​s
TestAccept/bundle/templates/default-python/combinations/serverless/DATABRICKS_BUNDLE_ENGINE=terraform/DLT=yes/NBOOK=yes/PY=yes 🙈​s 🙈​s ✅​p ✅​p 🙈​s 🙈​s ✅​p ✅​p 🙈​s ❌​F
TestAccept/bundle/templates/default-python/integration_classic ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.12 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.13 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=direct-exp/UV_PYTHON=3.9 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.10 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.11 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.12 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.13 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestAccept/bundle/templates/default-python/integration_classic/DATABRICKS_BUNDLE_ENGINE=terraform/UV_PYTHON=3.9 ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F
TestExport ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestExportWithFileFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestImportDirDoesNotOverwrite ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestImportDirWithOverwriteFlag ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestImportFileFormatAuto ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestImportFileFormatSource ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestImportFileUsingContentFormatAuto ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestImportFileUsingContentFormatSource ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestFilerRecursiveDelete ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestFilerRecursiveDelete/workspace_files ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestFilerRecursiveDelete/workspace_files_extensions ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestFilerWorkspaceFilesExtensionsRead ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestFilerWorkspaceFilesExtensionsReadDir ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestFilerWorkspaceNotebook ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ❌​F ✅​p
TestFilerWorkspaceNotebook/scalaNb.scala ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p 🔄​f ✅​p
TestWorkspaceFilesExtensions_ExportFormatIsPreserved ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p
TestWorkspaceFilesExtensions_ExportFormatIsPreserved/source_python ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ✅​p ❌​F ✅​p

@@ -0,0 +1,26 @@
# This test is using a workspace import API to load a notebook file.
# This API has a logic on how to accept notebook files and distinguishes them from regular python files.
# To succeed locally we would need to replicate this logic in the fake_workspace
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth doing some time?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Certainly, there are a bunch of other tests that need this, worth doing this as a separate PR

@andrewnester andrewnester requested a review from pietern October 27, 2025 12:46
Copy link
Contributor

@shreyas-goenka shreyas-goenka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

forgot to publish my review last week.

Comment on lines +26 to +33
Two-step workflow (manual bind):
1. Generate: databricks bundle generate job --existing-job-id 123 --key my_job
2. Bind: databricks bundle deployment bind my_job 123
3. Deploy: databricks bundle deploy
One-step workflow (automatic bind):
1. Generate and bind: databricks bundle generate job --existing-job-id 123 --key my_job --bind
2. Deploy: databricks bundle deploy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
Two-step workflow (manual bind):
1. Generate: databricks bundle generate job --existing-job-id 123 --key my_job
2. Bind: databricks bundle deployment bind my_job 123
3. Deploy: databricks bundle deploy
One-step workflow (automatic bind):
1. Generate and bind: databricks bundle generate job --existing-job-id 123 --key my_job --bind
2. Deploy: databricks bundle deploy
Three-step workflow (manual bind):
1. Generate: databricks bundle generate job --existing-job-id 123 --key my_job
2. Bind: databricks bundle deployment bind my_job 123
3. Deploy: databricks bundle deploy
Two-step workflow (automatic bind):
1. Generate and bind: databricks bundle generate job --existing-job-id 123 --key my_job --bind
2. Deploy: databricks bundle deploy

// This function is shared between the bind command and generate commands with --bind flag.
func BindResource(cmd *cobra.Command, resourceKey, resourceId string, autoApprove, forceLock bool) error {
ctx := cmd.Context()
b := utils.ConfigureBundleWithVariables(cmd)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: This means functions like root.MustConfigureBundle end up being called twice on the command struct.

}

if !exists {
return fmt.Errorf("%s with an id '%s' is not found", resource.ResourceDescription().SingularName, resourceId)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grammer nit:

Suggested change
return fmt.Errorf("%s with an id '%s' is not found", resource.ResourceDescription().SingularName, resourceId)
return fmt.Errorf("%s with an id '%s' was not found", resource.ResourceDescription().SingularName, resourceId)

@andrewnester andrewnester merged commit bc10093 into main Oct 28, 2025
12 of 13 checks passed
@andrewnester andrewnester deleted the feat/auto-bind branch October 28, 2025 13:56
deco-sdk-tagging bot added a commit that referenced this pull request Oct 29, 2025
## Release v0.275.0

### Notable Changes
* Python support for Databricks Asset Bundles is now generally available.

### CLI
* Remove `inplace` mode for the `--progress-format` flag ([#3811](#3811))
* Remove `json` mode for the `--progress-format` flag ([#3812](#3812))
* Deprecate the `--progress-format` flag ([#3819](#3819))

### Bundles
* Add support for `--bind` flag in `bundle generate` ([#3782](#3782))
* Add `pydabs` template replacing `experimental-jobs-as-code` template ([#3806](#3806))
* You can now use the top-level `python` section instead of `experimental/python` ([#3540](#3540))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants