Use direct map lookup in FindResourceByConfigKey#4445
Open
Use direct map lookup in FindResourceByConfigKey#4445
FindResourceByConfigKey#4445Conversation
Implements postgres_project as a new bundle resource type with full support for direct deployment and Terraform conversion. Includes config resource, direct deployment handler, and Terraform converter. Key features: - Uses display_name field for bundle summary output - Supports project_id as immutable identifier (triggers recreation on change) - Handles long-running operations with waiter pattern - Custom state wrapper for fields not in SDK types Also adds postgres_branch config resource (implementation in progress). Includes three acceptance tests: - single-project: basic deployment happy path - without-project-id: validates required field error handling - recreate-on-id-change: verifies immutable field recreation behavior Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements postgres_branch as a bundle resource with full support for direct deployment and Terraform conversion. Branches represent isolated database environments with copy-on-write storage within a postgres project. Key features: - Hierarchical relationship: branches belong to projects - Immutable identifiers: branch_id and parent trigger recreation on change - Long-running operations with waiter pattern - Custom state wrapper for creation-only fields (branch_id) Includes single-branch acceptance test: - Deploys project and branch together - Verifies branch creation and READY state - Normalizes non-deterministic branch UIDs and LSN values Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Adds support for Lakebase v2 (Postgres) endpoints in Databricks Asset Bundles:
- Config resource: bundle/config/resources/postgres_endpoint.go
- Implements ConfigResource interface with hierarchical URL parsing
- Supports endpoint_id field for user-specified IDs
- Direct deployment: bundle/direct/dresources/postgres_endpoint.go
- State wrapper for creation-time fields
- Handles long-running operations with proper waiting
- Creates endpoints with parent branch reference
- Terraform converter: bundle/deploy/terraform/tfdyn/convert_postgres_endpoint.go
- Normalizes endpoint spec for Terraform provider
- Registration:
- bundle/config/resources.go: PostgresEndpoints map
- bundle/direct/dresources/all.go: Added to SupportedResources
- bundle/direct/dresources/resources.yml: Immutable fields configuration
- Acceptance test: acceptance/bundle/resources/postgres_endpoints/single-endpoint/
- Tests full stack: project + branch + endpoint deployment
- Verifies endpoint creation with ENDPOINT_TYPE_READ_WRITE
- Includes cleanup delay for backend reconciliation
Key implementation details:
- Endpoint types use full enum names: ENDPOINT_TYPE_READ_WRITE, ENDPOINT_TYPE_READ_ONLY
- Immutable fields: endpoint_id, parent, spec.endpoint_type
- Hierarchical naming: projects/{project_id}/branches/{branch_id}/endpoints/{endpoint_id}
- Endpoint UIDs follow pattern: ep-{adjective}-{noun}-{alphanum}
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Adds comprehensive test coverage for required fields and immutable field behavior: **Branches (2 new tests):** - without-branch-id: Validates that branch_id is required - recreate-on-id-change: Verifies branch_id immutability triggers recreation **Endpoints (2 new tests):** - without-endpoint-id: Validates that endpoint_id is required - recreate-on-endpoint-type-change: Verifies endpoint_type immutability **Immutable Fields Documentation:** Updated bundle/direct/dresources/resources.yml to document additional immutable fields: - Branch source fields (source_branch, source_branch_lsn, source_branch_time) These define the point-in-time data source and cannot be changed after creation - Endpoint type (endpoint_type) already documented **Test Coverage:** All 9 postgres tests now passing: - 3 project tests (5.66s, 19.92s, 27.42s) - 3 branch tests (16.95s, 19.91s, 33.74s) - 3 endpoint tests (12.85s, 29.36s, 68.89s) Total test time: ~72s (parallel execution) **Key Findings:** 1. Changing endpoint_type triggers recreation as expected 2. Test verifies recreation via DELETE/POST in request logs 3. Cannot verify "old endpoint gone" when reusing same endpoint_id (both old and new have same hierarchical name) 4. Branch source fields are immutable by nature (define data lineage) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Refactor postgres resources to embed *Spec types directly instead of
embedding the full resource types. This eliminates the unnecessary "spec:"
nesting level in bundle YAML configuration, making postgres resources
consistent with other DAB resources (jobs, clusters, pipelines, etc.).
Changes:
- PostgresProject embeds postgres.ProjectSpec (not postgres.Project)
- PostgresBranch embeds postgres.BranchSpec (not postgres.Branch)
- PostgresEndpoint embeds postgres.EndpointSpec (not postgres.Endpoint)
- Updated PrepareState() to reconstruct API types from flattened config
- Updated all 9 acceptance tests to remove spec: nesting
- Regenerated bundle schema
User-facing config changes from:
postgres_projects:
my_project:
spec:
display_name: "Test"
To:
postgres_projects:
my_project:
display_name: "Test"
All acceptance tests passing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The Name field is output-only but was included in StateType structs via
embedding, causing variable references like ${...name} to resolve to
empty strings during planning instead of delaying until after API response.
Restructured StateType to only include user-configurable fields.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add postgres resource types to GroupToTerraformName mapping to enable
reference resolution (e.g., ${resources.postgres_projects.x.name}).
Update converters to restructure flattened bundle config fields into
nested spec blocks expected by the Terraform provider schema.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The SDK's duration.Duration type uses custom JSON marshaling to represent
durations as strings (e.g., "300s", "604800s"). Previously, the dyn/convert
package treated it as a struct, causing duration fields to be silently
dropped with "expected map, found string" warnings.
This change adds special handling for duration.Duration in:
- normalize.go: Treat as string during normalization
- to_typed.go: Convert string to duration using JSON unmarshaling
- from_typed.go: Convert duration to string using JSON marshaling
- from_type.go: Represent as string type in JSON schema
Also updates postgres acceptance tests to use protobuf duration format
("604800s" instead of "7d") which is what the SDK expects.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Track filtered postgres API requests in out.requests.json files for test verification while keeping full request logs ignored. This enables validation of postgres resource CRUD operations in acceptance tests. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implement full CRUD operations for postgres projects, branches, and endpoints in the testserver. This enables local acceptance testing without requiring cloud connectivity. Key changes: - Add postgres.go with handlers for projects, branches, and endpoints - Add postgres_test.go with unit tests for the handlers - Register postgres routes in handlers.go - Fix WorkspaceDelete to also remove directories (not just files) - Update test.toml files with correct regex patterns for UIDs and LSNs - Add conditional sleep in endpoint recreate test for cloud mode only Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
After endpoint create/update operations complete, poll the endpoint status until PendingState is empty. Also retry DELETE operations that fail with 409 ABORTED due to reconciliation still in progress. This ensures subsequent operations don't fail because the endpoint is still reconciling internally. - Add waitForReconciliation helper to poll until PendingState is empty - Add retry loop in DoDelete for 409 ABORTED errors - Remove sleep commands from acceptance tests (no longer needed) - Deduplicate requests in test output to handle retry variability Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add postgres resource field definitions to refschema output - Update sql_warehouses destroy output to reflect no active deployment Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements test cases for postgres_project, postgres_branch, and postgres_endpoint resources in the no_drift test suite. The key fix is in RemapState functions: extract resource IDs from hierarchical names and populate spec fields from status (effective values) instead of the sparse spec returned by the API. This ensures state comparison accurately reflects deployed resources and prevents false drift detection. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Implements conversion support for SDK native types across the dyn package conversion layer (FromTyped, ToTyped, Normalize). Supports three SDK types that use custom JSON marshaling: - duration.Duration: Protobuf duration format (e.g., "300s") - time.Time: RFC3339 timestamp format (e.g., "2023-12-25T10:30:00Z") - fieldmask.FieldMask: Comma-separated paths (e.g., "name,age,email") The implementation uses a generic approach with all SDK-specific code in sdk_native_types.go and sdk_native_types_test.go. All SDK type imports are prefixed with "sdk" for clarity. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Adds comprehensive round-trip tests using actual SDK types from the postgres service (BranchSpec and UpdateBranchRequest) to verify that serialization and deserialization work correctly in real-world usage. Tests cover: - BranchSpec with time.Time and duration.Duration fields - UpdateRequest with fieldmask.FieldMask field - Full round-trip FromTyped -> dyn.Value -> ToTyped - Normalization with SDK types Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Update test code to follow linter recommendations including struct field alignment and using assert.True for boolean assertions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
1. Clarified that all SDK native types marshal to JSON strings with
improved comments explaining the conversion process.
2. Fixed JSON encoding in toTypedSDKNative to use json.Marshal instead
of fmt.Sprintf("%q") for proper JSON string literal creation.
3. Fixed TestFromTypedDurationPointer to actually test with a pointer
instead of a value.
4. Added comprehensive table-driven roundtrip tests with reusable
equality check functions for all three SDK types (Duration, Time,
FieldMask) testing both value and pointer variants.
5. Removed redundant basic tests that are now covered by the roundtrip
tests, while keeping edge case tests for nil, zero, error cases,
normalization, and in-struct scenarios.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Using .id ensures that parent references resolve to stable identifiers rather than computed name values, which prevents false drift detection and unnecessary resource recreation in dependent resources. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Removed 18 tests that are now covered by the comprehensive roundtrip tests. The remaining tests focus on unique edge cases: - Nil value handling (one per type) - Zero values - Error cases (invalid format, wrong type) - Empty values (FieldMask) - Normalize with nil (important edge case) - End-to-end tests with real postgres types This keeps test coverage while improving maintainability. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Tests that display_name can be updated without recreating the project. Follows the pattern: create → verify idempotency → update → verify → restore → verify → destroy. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Converted 13 individual edge case tests into 3 table-driven tests: - TestSDKNativeTypesNilValues: Tests nil handling for FromTyped and Normalize across all 3 SDK types (6 subtests) - TestSDKNativeTypesErrors: Tests error handling for ToTyped with invalid inputs across all 3 types (4 subtests) - TestSDKNativeTypesSpecialCases: Tests zero values and empty FieldMask (3 subtests) Final test suite for SDK native types: - 1 comprehensive roundtrip test (18 subtests) - 3 edge case table tests (13 subtests) - 3 end-to-end tests with real postgres types This improves maintainability while preserving full test coverage. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This test verifies that the is_protected field on a postgres branch can be updated in place without recreating the branch. The test covers four stages: - create: Initial deployment with is_protected: false - no_change: Verify idempotency (no unnecessary updates) - update: Change is_protected to true (PATCH request) - restore: Change is_protected back to false (PATCH request) Each stage captures both the plan output (focused on the branch resource) and the HTTP requests made during deployment, allowing verification that updates use PATCH requests rather than recreate (DELETE + POST). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This test verifies that the autoscaling_limit_max_cu field on a postgres endpoint can be updated in place without recreating the endpoint. The test covers four stages: - create: Initial deployment with autoscaling_limit_max_cu: 8 - no_change: Verify idempotency (no unnecessary updates) - update: Change autoscaling_limit_max_cu to 4 (PATCH request) - restore: Change autoscaling_limit_max_cu back to 8 (PATCH request) Each stage captures both the plan output (focused on the endpoint resource) and the HTTP requests made during deployment, allowing verification that updates use PATCH requests rather than recreate (DELETE + POST). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…utput Apply the same structural improvements as the other update tests: - Add focused plan output using jq for the project resource - Rename "idempotency" to "no_change" for clarity - Update file naming to out.requests.<stage>.json and out.plan.<stage>.json - Capture plan output at all four stages (create, no_change, update, restore) This allows verification of the exact changes being made at each stage and confirms that updates use PATCH requests rather than recreate. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Use `sync.paths: []` instead of `sync.exclude: ["*"]` to prevent the "no files to sync" warning. An empty paths array is treated as explicitly set, so no default is applied and no warning is shown. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When updating postgres resources (projects, branches, endpoints), the code was using update_mask="*" which requires ALL spec fields to be provided in the request. The real cloud API is stricter than the test server and rejects updates with missing fields. Changed to derive the precise update_mask based on which fields actually changed, using the Changes map that already tracks field-level changes. Code changes: - Added collectUpdatePaths() in dresources/util.go to extract paths with action=Update from the Changes map - Modified DoUpdate() in postgres_project.go to use precise field paths - Modified DoUpdate() in postgres_branch.go to use precise field paths - Modified DoUpdate() in postgres_endpoint.go to use precise field paths Test changes: - Fixed postgres_branches/update-protected test by adding no_expiry field and updating branch UID replacement pattern to be more flexible - Fixed postgres_endpoints/update-autoscaling test by updating endpoint UID and branch UID replacement patterns - Updated expected test outputs to reflect new precise update_mask behavior All postgres acceptance tests now pass against real cloud instance: - postgres_projects: 4/4 passing - postgres_branches: 4/4 passing - postgres_endpoints: 4/4 passing Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Moved common Repl patterns from individual test.toml files to parent test.toml files in each postgres category (branches, endpoints, projects). This eliminates duplication of patterns like URL cleanup, UID normalization, and LSN values across multiple tests. Individual tests now only define test-specific patterns (e.g., TraceId in recreate-on-id-change tests). Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Changes the postgres resource ID fields (project_id, branch_id, endpoint_id) from optional to required, matching the API's behavior. Previously these fields were marked as omitempty but the API requires them for resource creation. Changes: - Mark project_id, branch_id, endpoint_id, and parent fields as required in bundle config - Remove manual validation from direct deployment code (now handled by config validation) - Update generated required_fields.go and JSON schema - Add validation in test server to return 400 errors matching real API behavior - Update acceptance test outputs to reflect new warning messages and error responses The test server now properly validates required ID fields and returns the same error message as the real API: "Field 'X' is required, expected non-default value (not \"\")!" All acceptance tests pass both locally and against real AWS workspace. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
The logical_size_bytes field can be 0 or a larger value depending on timing, causing flaky test failures. Filter it out with jq to make tests deterministic. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The API requires an expiration setting when creating branches. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update the TestAll CRUD test to observe ignore_remote_changes from both resources.yml and resources.generated.yml when comparing input state against remapped remote state. This replaces hardcoded field skipping with configuration-driven behavior. Also update postgres resource struct literals to use the new PostgresProjectConfig, PostgresBranchConfig, and PostgresEndpointConfig wrapper types. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduce testIgnoreFilter helper to encapsulate field filtering logic based on ignore_remote_changes configuration from resources.yml and resources.generated.yml. Replace hardcoded field skipping and direct equality checks with configuration-driven filtering throughout the CRUD test. Key changes: - Add testIgnoreFilter type with shouldIgnore, filterChanges, and requireEqual methods to centralize filtering logic - Update all state comparisons in testCRUD to use ignoreFilter.requireEqual instead of direct require.Equal calls - Apply filtering to field-by-field walk comparison - Remove hardcoded "updated_at" special case from test code - Clear output-only timestamp fields (CreatedAt, UpdatedAt, CreatedBy, UpdatedBy) in RegisteredModel.RemapState instead of filtering in tests - Add dashboard test configuration with SerializedDashboard as map This makes the test behavior fully configuration-driven and eliminates technical debt from hardcoded field names in test assertions. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Restructure postgres resource types to embed SDK spec types directly instead of having a nested Spec field. This simplifies the YAML config by allowing users to write fields like display_name directly instead of nesting them under spec. Changes: - PostgresProjectConfig now embeds postgres.ProjectSpec directly - PostgresBranchConfig now embeds postgres.BranchSpec directly - PostgresEndpointConfig now embeds postgres.EndpointSpec directly - State types are now aliases to their respective Config types - Add ignore_remote_changes for all spec fields since the read API only returns status, not spec values - Prefix update mask paths with "spec." since the API expects paths relative to the Project/Branch/Endpoint object - Update apitypes.yml to use Spec types for field behavior extraction - Update all test files to use new struct layout Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The merged testcrud-ignore-remote branch handles this differently. Remove the redundant field-checking logic. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The logical_size_bytes field in remote_state.status can vary based on timing, causing flaky test failures. Filter it out with jq to make tests deterministic. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change from iterating over all map entries to direct O(1) lookup using the comma-ok idiom. This improves performance from O(n) to O(1) per resource type when searching for a resource by key. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
pietern
requested review from
andrewnester,
anton-107,
denik and
shreyas-goenka
as code owners
pietern
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
|
Commit: fcdcb3a
28 interesting tests: 11 flaky, 7 KNOWN, 5 SKIP, 5 RECOVERED
Top 50 slowest tests (at least 2 minutes):
|
andrewnester
approved these changes
Feb 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
This came up in both of the PRs where new resources were added.