Added support for --bind flag in bundle generate (#3782)

andrewnester · web-flow · commit bc10093fbe45 · 2025-10-28T14:56:01.000+01:00
## Changes
Added support for --bind flag in `bundle generate`

## Why
This allows us to generate and bind the resources using one command
instead of 2 separate commands.

## Tests
Added an acceptance test

&lt;!-- If your PR needs to be included in the release notes for next
release,
add a separate entry in NEXT_CHANGELOG.md as part of your PR. --&gt;
diff --git a/NEXT_CHANGELOG.md b/NEXT_CHANGELOG.md
@@ -13,6 +13,7 @@
 ### Dependency updates
 
 ### Bundles
+* Added support for --bind flag in `bundle generate` ([#3782](https://github.com/databricks/cli/pull/3782))
 * Add `pydabs` template replacing `experimental-jobs-as-code` template ([#3806](https://github.com/databricks/cli/pull/3806))
 * You can now use `python` section instead of `experimental/python` ([#3540](https://github.com/databricks/cli/pull/3540))
 
diff --git a/acceptance/bundle/deployment/bind/alert/output.txt b/acceptance/bundle/deployment/bind/alert/output.txt
@@ -3,7 +3,8 @@
 
 >>> [CLI] bundle deployment bind my_alert [UUID] --auto-approve
 Updating deployment state...
-Successfully bound alert with an id '[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound alert with an id '[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle summary
 Name: test-bundle-$UNIQUE_NAME
diff --git a/acceptance/bundle/deployment/bind/cluster/output.txt b/acceptance/bundle/deployment/bind/cluster/output.txt
@@ -6,7 +6,8 @@
 
 >>> [CLI] bundle deployment bind cluster1 [CLUSTER-ID] --auto-approve
 Updating deployment state...
-Successfully bound cluster with an id '[CLUSTER-ID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound cluster with an id '[CLUSTER-ID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deployment unbind cluster1
 Updating deployment state...
diff --git a/acceptance/bundle/deployment/bind/dashboard/output.txt b/acceptance/bundle/deployment/bind/dashboard/output.txt
@@ -1,7 +1,8 @@
 
 >>> [CLI] bundle deployment bind dashboard1 [DASHBOARD_ID] --auto-approve
 Updating deployment state...
-Successfully bound dashboard with an id '[DASHBOARD_ID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound dashboard with an id '[DASHBOARD_ID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
diff --git a/acceptance/bundle/deployment/bind/dashboard/recreation/output.txt b/acceptance/bundle/deployment/bind/dashboard/recreation/output.txt
@@ -1,7 +1,8 @@
 
 >>> [CLI] bundle deployment bind dashboard1 [DASHBOARD_ID] --auto-approve
 Updating deployment state...
-Successfully bound dashboard with an id '[DASHBOARD_ID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound dashboard with an id '[DASHBOARD_ID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> errcode [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
diff --git a/acceptance/bundle/deployment/bind/database_instance/output.txt b/acceptance/bundle/deployment/bind/database_instance/output.txt
@@ -1,7 +1,8 @@
 
 >>> [CLI] bundle deployment bind database_instance1 [UUID] --auto-approve
 Updating deployment state...
-Successfully bound database_instance with an id '[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound database_instance with an id '[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle summary
 Name: test-bundle-$UNIQUE_NAME
diff --git a/acceptance/bundle/deployment/bind/experiment/output.txt b/acceptance/bundle/deployment/bind/experiment/output.txt
@@ -3,7 +3,8 @@
 === Substitute variables in the template
 === Create a pre-defined experiment
 === Bind experiment: Updating deployment state...
-Successfully bound experiment with an id '[NUMID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound experiment with an id '[NUMID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 === Deploy bundle: Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
 Deploying resources...
diff --git a/acceptance/bundle/deployment/bind/job/generate-and-bind/output.txt b/acceptance/bundle/deployment/bind/job/generate-and-bind/output.txt
@@ -18,7 +18,8 @@ test.py
 
 >>> [CLI] bundle deployment bind test_job_key [NUMID] --auto-approve
 Updating deployment state...
-Successfully bound job with an id '[NUMID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound job with an id '[NUMID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-generate-bind-[UNIQUE_NAME]/files...
diff --git a/acceptance/bundle/deployment/bind/job/job-spark-python-task/output.txt b/acceptance/bundle/deployment/bind/job/job-spark-python-task/output.txt
@@ -5,7 +5,8 @@ Created job with ID: [NUMID]
 === Bind job:
 >>> [CLI] bundle deployment bind foo [NUMID] --auto-approve
 Updating deployment state...
-Successfully bound job with an id '[NUMID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound job with an id '[NUMID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 === Remove .databricks directory to simulate fresh deployment:
 >>> rm -rf .databricks
diff --git a/acceptance/bundle/deployment/bind/job/noop-job/output.txt b/acceptance/bundle/deployment/bind/job/noop-job/output.txt
@@ -3,7 +3,8 @@
 
 >>> [CLI] bundle deployment bind job_1 [NUMID] --auto-approve
 Updating deployment state...
-Successfully bound job with an id '[NUMID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound job with an id '[NUMID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/my_project/default/files...
diff --git a/acceptance/bundle/deployment/bind/job/python-job/output.txt b/acceptance/bundle/deployment/bind/job/python-job/output.txt
@@ -3,7 +3,8 @@
 
 >>> uv run --with [DATABRICKS_BUNDLES_WHEEL] -q [CLI] bundle deployment bind job_1 [NUMID] --auto-approve
 Updating deployment state...
-Successfully bound job with an id '[NUMID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound job with an id '[NUMID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> uv run --with [DATABRICKS_BUNDLES_WHEEL] -q [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/my_project/default/files...
diff --git a/acceptance/bundle/deployment/bind/model-serving-endpoint/output.txt b/acceptance/bundle/deployment/bind/model-serving-endpoint/output.txt
@@ -29,7 +29,8 @@ resources:
 
 >>> [CLI] bundle deployment bind endpoint1 test-endpoint-[UUID]
 Updating deployment state...
-Successfully bound model_serving_endpoint with an id 'test-endpoint-[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound model_serving_endpoint with an id 'test-endpoint-[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
diff --git a/acceptance/bundle/deployment/bind/quality-monitor/output.txt b/acceptance/bundle/deployment/bind/quality-monitor/output.txt
@@ -13,7 +13,8 @@
 
 >>> [CLI] bundle deployment bind monitor1 catalog.schema.table
 Updating deployment state...
-Successfully bound quality_monitor with an id 'catalog.schema.table'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound quality_monitor with an id 'catalog.schema.table'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/bind-quality-monitor-test-localonly/default/files...
diff --git a/acceptance/bundle/deployment/bind/registered-model/output.txt b/acceptance/bundle/deployment/bind/registered-model/output.txt
@@ -15,7 +15,8 @@ resources:
 
 >>> [CLI] bundle deployment bind model1 main.test-schema-rmodel-[UUID].test-registered-model-[UUID]
 Updating deployment state...
-Successfully bound registered_model with an id 'main.test-schema-rmodel-[UUID].test-registered-model-[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound registered_model with an id 'main.test-schema-rmodel-[UUID].test-registered-model-[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
diff --git a/acceptance/bundle/deployment/bind/schema/output.txt b/acceptance/bundle/deployment/bind/schema/output.txt
@@ -7,7 +7,8 @@
 }
 
 === Bind schema: Updating deployment state...
-Successfully bound schema with an id 'main.test-schema-[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound schema with an id 'main.test-schema-[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 === Deploy bundle: Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
 Deploying resources...
diff --git a/acceptance/bundle/deployment/bind/secret-scope/output.txt b/acceptance/bundle/deployment/bind/secret-scope/output.txt
@@ -3,7 +3,8 @@
 
 >>> [CLI] bundle deployment bind secret_scope1 test-secret-scope-[UUID] --auto-approve
 Updating deployment state...
-Successfully bound secret_scope with an id 'test-secret-scope-[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound secret_scope with an id 'test-secret-scope-[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/bind-secret-scope-test-[UNIQUE_NAME]/default/files...
diff --git a/acceptance/bundle/deployment/bind/sql_warehouse/output.txt b/acceptance/bundle/deployment/bind/sql_warehouse/output.txt
@@ -1,7 +1,8 @@
 
 >>> [CLI] bundle deployment bind sql_warehouse1 [SQL-WAREHOUSE-ID] --auto-approve
 Updating deployment state...
-Successfully bound sql_warehouse with an id '[SQL-WAREHOUSE-ID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound sql_warehouse with an id '[SQL-WAREHOUSE-ID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle summary
 Name: test-bundle-$UNIQUE_NAME
diff --git a/acceptance/bundle/deployment/bind/volume/output.txt b/acceptance/bundle/deployment/bind/volume/output.txt
@@ -8,7 +8,8 @@
 === Create a pre-defined volume:
 >>> [CLI] bundle deployment bind volume1 main.test-schema-[UUID].volume-[UUID] --auto-approve
 Updating deployment state...
-Successfully bound volume with an id 'main.test-schema-[UUID].volume-[UUID]'. Run 'bundle deploy' to deploy changes to your workspace
+Successfully bound volume with an id 'main.test-schema-[UUID].volume-[UUID]'
+Run 'bundle deploy' to deploy changes to your workspace
 
 >>> [CLI] bundle deploy
 Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/test-bundle-[UNIQUE_NAME]/default/files...
diff --git a/acceptance/bundle/generate/auto-bind/databricks.yml.tmpl b/acceptance/bundle/generate/auto-bind/databricks.yml.tmpl
@@ -0,0 +1,8 @@
+bundle:
+  name: auto-bind-test
+
+workspace:
+  root_path: /tmp/${UNIQUE_NAME}
+
+include:
+  - resources/*.yml
diff --git a/acceptance/bundle/generate/auto-bind/out.test.toml b/acceptance/bundle/generate/auto-bind/out.test.toml
diff --git a/acceptance/bundle/generate/auto-bind/output.txt b/acceptance/bundle/generate/auto-bind/output.txt
@@ -0,0 +1,48 @@
+
+=== Create a pre-defined job:
+Created job with ID: [NUMID]
+
+>>> [CLI] workspace mkdirs /Workspace/Users/[USERNAME]/python-[UNIQUE_NAME]
+
+>>> [CLI] workspace import /Workspace/Users/[USERNAME]/python-[UNIQUE_NAME]/test --file test.py --language PYTHON
+
+=== Generate and bind in one step:
+>>> [CLI] bundle generate job --key test_job --existing-job-id [NUMID] --config-dir resources --source-dir src --bind
+File successfully saved to src/test.py
+Job configuration successfully saved to resources/test_job.job.yml
+Updating deployment state...
+Successfully bound job with an id '[NUMID]'
+
+>>> ls src/
+test.py
+
+>>> cat resources/test_job.job.yml
+      name: auto-bind-job-[UNIQUE_NAME]
+
+=== Deploy the bound job:
+>>> [CLI] bundle deploy
+Uploading bundle files to /Workspace/tmp/[UNIQUE_NAME]/files...
+Deploying resources...
+Updating deployment state...
+Deployment complete!
+
+=== Destroy the bundle:
+>>> [CLI] bundle destroy --auto-approve
+The following resources will be deleted:
+  delete job test_job
+
+All files and directories at the following location will be deleted: /Workspace/tmp/[UNIQUE_NAME]
+
+Deleting files...
+Destroy complete!
+
+=== Check that job is bound and does not exist after bundle is destroyed:
+>>> errcode [CLI] jobs get [NUMID] --output json
+Error: Job [NUMID] does not exist.
+
+Exit code: 1
+
+=== Delete the tmp folder:
+>>> [CLI] workspace delete /Workspace/Users/[USERNAME]/python-[UNIQUE_NAME]/test
+
+>>> [CLI] workspace delete /Workspace/Users/[USERNAME]/python-[UNIQUE_NAME]
diff --git a/acceptance/bundle/generate/auto-bind/script b/acceptance/bundle/generate/auto-bind/script
@@ -0,0 +1,51 @@
+title "Create a pre-defined job:\n"
+
+PYTHON_NOTEBOOK_DIR="/Workspace/Users/${CURRENT_USER_NAME}/python-${UNIQUE_NAME}"
+PYTHON_NOTEBOOK="${PYTHON_NOTEBOOK_DIR}/test"
+
+JOB_ID=$($CLI jobs create --json '
+{
+  "name": "auto-bind-job-'${UNIQUE_NAME}'",
+  "tasks": [
+    {
+      "task_key": "test",
+      "new_cluster": {
+        "spark_version": "'${DEFAULT_SPARK_VERSION}'",
+        "node_type_id": "'${NODE_TYPE_ID}'",
+        "num_workers": 1
+      },
+      "notebook_task": {
+        "notebook_path": "'${PYTHON_NOTEBOOK}'"
+      }
+    }
+  ]
+}' | jq -r '.job_id')
+
+echo "Created job with ID: $JOB_ID"
+
+envsubst < databricks.yml.tmpl > databricks.yml
+
+cleanup() {
+    title "Delete the tmp folder:"
+    trace $CLI workspace delete ${PYTHON_NOTEBOOK}
+    trace $CLI workspace delete ${PYTHON_NOTEBOOK_DIR}
+}
+trap cleanup EXIT
+
+trace $CLI workspace mkdirs "${PYTHON_NOTEBOOK_DIR}"
+trace $CLI workspace import "${PYTHON_NOTEBOOK}" --file test.py --language PYTHON
+
+title "Generate and bind in one step:"
+trace $CLI bundle generate job --key test_job --existing-job-id $JOB_ID --config-dir resources --source-dir src --bind
+trace ls src/
+# The output of the job is difference per cloud so we only check the name.
+trace cat resources/test_job.job.yml | grep "name: auto-bind-job-${UNIQUE_NAME}"
+
+title "Deploy the bound job:"
+trace $CLI bundle deploy
+
+title "Destroy the bundle:"
+trace $CLI bundle destroy --auto-approve
+
+title "Check that job is bound and does not exist after bundle is destroyed:"
+trace errcode $CLI jobs get "${JOB_ID}" --output json
diff --git a/acceptance/bundle/generate/auto-bind/test.py b/acceptance/bundle/generate/auto-bind/test.py
@@ -0,0 +1,2 @@
+# Databricks notebook source
+print("Test notebook")
diff --git a/acceptance/bundle/generate/auto-bind/test.toml b/acceptance/bundle/generate/auto-bind/test.toml
@@ -0,0 +1,26 @@
+# This test is using a workspace import API to load a notebook file.
+# This API has a logic on how to accept notebook files and distinguishes them from regular python files.
+# To succeed locally we would need to replicate this logic in the fake_workspace
+Local = false
+Cloud = true
+
+Ignore = [
+    "databricks.yml",
+    "resources/*",
+    "src/*",
+    ".databricks",
+]
+
+[EnvMatrix]
+  DATABRICKS_BUNDLE_ENGINE = ["terraform"]
+
+
+[Env]
+# MSYS2 automatically converts absolute paths like /Users/$username/$UNIQUE_NAME to
+# C:/Program Files/Git/Users/$username/UNIQUE_NAME before passing it to the CLI
+# Setting this environment variable prevents that conversion on windows.
+MSYS_NO_PATHCONV = "1"
+
+[[Repls]]
+Old = '\\'
+New = '/'
diff --git a/acceptance/bundle/help/bundle-generate-dashboard/output.txt b/acceptance/bundle/help/bundle-generate-dashboard/output.txt
@@ -36,6 +36,7 @@ Usage:
   databricks bundle generate dashboard [flags]
 
 Flags:
+  -b, --bind                   automatically bind the generated dashboard config to the existing dashboard
   -s, --dashboard-dir string   directory to write the dashboard representation to (default "src")
       --existing-id string     ID of the dashboard to generate configuration for
       --existing-path string   workspace path of the dashboard to generate configuration for
diff --git a/acceptance/bundle/help/bundle-generate-job/output.txt b/acceptance/bundle/help/bundle-generate-job/output.txt
@@ -13,6 +13,9 @@ Examples:
   databricks bundle generate job --existing-job-id 67890 \
     --key data_pipeline --config-dir resources --source-dir src
 
+  # Generate and automatically bind to the existing job
+  databricks bundle generate job --existing-job-id 12345 --key my_etl_job --bind
+
 What gets generated:
 - Job configuration YAML file in the resources directory
 - Any associated notebook or Python files in the source directory
@@ -25,6 +28,7 @@ Usage:
   databricks bundle generate job [flags]
 
 Flags:
+  -b, --bind                  automatically bind the generated resource to the existing resource
   -d, --config-dir string     Dir path where the output config will be stored (default "resources")
       --existing-job-id int   Job ID of the job to generate config for
   -f, --force                 Force overwrite existing files in the output directory
diff --git a/acceptance/bundle/help/bundle-generate-pipeline/output.txt b/acceptance/bundle/help/bundle-generate-pipeline/output.txt
@@ -14,6 +14,9 @@ Examples:
   databricks bundle generate pipeline --existing-pipeline-id def456 \
     --key data_transformation --config-dir resources --source-dir src
 
+  # Generate and automatically bind to the existing pipeline
+  databricks bundle generate pipeline --existing-pipeline-id abc123 --key etl_pipeline --bind
+
 What gets generated:
 - Pipeline configuration YAML file with settings and libraries
 - Pipeline notebooks downloaded to the source directory
@@ -25,6 +28,7 @@ Usage:
   databricks bundle generate pipeline [flags]
 
 Flags:
+  -b, --bind                          automatically bind the generated resource to the existing resource
   -d, --config-dir string             Dir path where the output config will be stored (default "resources")
       --existing-pipeline-id string   ID of the pipeline to generate config for
   -f, --force                         Force overwrite existing files in the output directory
diff --git a/acceptance/bundle/help/bundle-generate/output.txt b/acceptance/bundle/help/bundle-generate/output.txt
@@ -4,16 +4,24 @@ Generate bundle configuration from existing Databricks resources.
 
 Common patterns:
   databricks bundle generate job --existing-job-id 123 --key my_job
+  databricks bundle generate pipeline --existing-pipeline-id abc123 --key etl_pipeline
   databricks bundle generate dashboard --existing-path /my-dashboard --key sales_dash
   databricks bundle generate dashboard --resource my_dashboard --watch --force  # Keep local copy in sync. Useful for development.
   databricks bundle generate dashboard --resource my_dashboard --force # Do a one-time sync.
 
-Complete migration workflow:
-  1. Generate: databricks bundle generate job --existing-job-id 123 --key my_job
-  2. Bind: databricks bundle deployment bind my_job 123
-  3. Deploy: databricks bundle deploy
+Migration workflows:
+
+  Two-step workflow (manual bind):
+    1. Generate: databricks bundle generate job --existing-job-id 123 --key my_job
+    2. Bind: databricks bundle deployment bind my_job 123
+    3. Deploy: databricks bundle deploy
+
+  One-step workflow (automatic bind):
+    1. Generate and bind: databricks bundle generate job --existing-job-id 123 --key my_job --bind
+    2. Deploy: databricks bundle deploy
 
 Use --key to specify the resource name in your bundle configuration.
+Use --bind to automatically bind the generated resource to the existing workspace resource.
 
 Usage:
   databricks bundle generate [command]
diff --git a/cmd/bundle/deployment/bind.go b/cmd/bundle/deployment/bind.go
diff --git a/cmd/bundle/deployment/bind_resource.go b/cmd/bundle/deployment/bind_resource.go
diff --git a/cmd/bundle/generate.go b/cmd/bundle/generate.go
diff --git a/cmd/bundle/generate/app.go b/cmd/bundle/generate/app.go
diff --git a/cmd/bundle/generate/dashboard.go b/cmd/bundle/generate/dashboard.go
diff --git a/cmd/bundle/generate/job.go b/cmd/bundle/generate/job.go
diff --git a/cmd/bundle/generate/pipeline.go b/cmd/bundle/generate/pipeline.go
diff --git a/cmd/bundle/utils/utils.go b/cmd/bundle/utils/utils.go
