Skip to content

[datadog_integration_gcp_sts] Add granular namespace filters to GCP V2 Terraform Provider #3299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 11, 2025
Merged

[datadog_integration_gcp_sts] Add granular namespace filters to GCP V2 Terraform Provider #3299

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4eeea78
Add metric namespace config filters to GCP V2 terraform resource
ash-ddog Oct 23, 2025
1f3a048
Update cassettes
ash-ddog Oct 30, 2025
94ca105
Run make docs
ash-ddog Oct 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
117 changes: 63 additions & 54 deletions datadog/fwprovider/resource_datadog_integration_gcp_sts.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,9 @@ var (
AttrTypes: map[string]attr.Type{
"id": types.StringType,
"disabled": types.BoolType,
"filters": types.SetType{
ElemType: types.StringType,
},
Comment on lines +29 to +31
Copy link
Contributor Author

@ash-ddog ash-ddog Oct 30, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just want to highlight that while this field is actually optional in terms of our API, it's required for terraform and I don't think we have a choice here 😞 .

From my understanding/research, this has to do with terraform provider framework v5 vs v6. In v5, you cannot have a definition for nested types, but you can in v6. Unfortunately, Datadog is stuck on v5 until every team updates to v6 and we stop using the terraform "mux" server.

So, ideally, our schema definition would have:

			"metric_namespace_configs": schema.SetNestedAttribute{
				Optional:    true,
				Computed:    true,
				Description: "Configurations for GCP metric namespaces.",
				NestedObject: schema.NestedAttributeObject{
					Attributes: map[string]schema.Attribute{
						"id": schema.StringAttribute{
							Required:    true,
							Description: "The namespace ID.",
						},
						"disabled": schema.BoolAttribute{
							Optional:    true,
							Description: "Whether the namespace is disabled.",
						},
						"filters": schema.SetAttribute{
							Optional:    true,
							Description: "Filters for the namespace.",
							ElementType: types.StringType,
						},
					},
				},
			},

But this fails our tests with a message like "attributes are not allowed in v5" or something of that sort. Note that our code is actually using v6, but the terraform "mux" server verifies our v6 code is backwards-compatible with v5.

Functionally, what this means is that for customers that currently have metric_namespace_configs defined, once they update their terraform version, they'll have to manually add filters = [] to all of their objects.

Copy link
Contributor

@katherinekim-51 katherinekim-51 Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 aligns with https://developer.hashicorp.com/terraform/plugin/terraform-plugin-protocol#protocol-version-6 and this seems like the best case thinking of other options (ex: deprecate this field and make a new block-based one, make filters a top-level attribute, etc)

Unfortunately, Datadog is stuck on v5 until every team updates to v6 and we stop using the terraform "mux" server

what's the/is there a timeline for this OOC?

},
}

Expand All @@ -47,7 +50,9 @@ type integrationGcpStsResource struct {
type MetricNamespaceConfigModel struct {
ID types.String `tfsdk:"id"`
Disabled types.Bool `tfsdk:"disabled"`
Filters types.Set `tfsdk:"filters"`
}

type MonitoredResourceConfigModel struct {
Type types.String `tfsdk:"type"`
Filters types.Set `tfsdk:"filters"`
Expand Down Expand Up @@ -183,6 +188,7 @@ func (r *integrationGcpStsResource) Read(ctx context.Context, request resource.R
if response.Diagnostics.HasError() {
return
}

resp, httpResp, err := r.Api.ListGCPSTSAccounts(r.Auth)
if err != nil {
if httpResp != nil && httpResp.StatusCode == 404 {
Expand All @@ -201,7 +207,7 @@ func (r *integrationGcpStsResource) Read(ctx context.Context, request resource.R
for _, account := range resp.GetData() {
if account.GetId() == state.ID.ValueString() {
found = true
r.updateState(ctx, &state, &account)
r.parseGcpStsResponseBody(ctx, &state, &account)
break
}
}
Expand All @@ -216,12 +222,11 @@ func (r *integrationGcpStsResource) Read(ctx context.Context, request resource.R
}

func (r *integrationGcpStsResource) Create(ctx context.Context, request resource.CreateRequest, response *resource.CreateResponse) {
var state integrationGcpStsModel
response.Diagnostics.Append(request.Plan.Get(ctx, &state)...)
var plan integrationGcpStsModel
Copy link
Contributor

@katherinekim-51 katherinekim-51 Oct 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

response.Diagnostics.Append(request.Plan.Get(ctx, &plan)...)
if response.Diagnostics.HasError() {
return
}

integrationGcpStsMutex.Lock()
defer integrationGcpStsMutex.Unlock()

Expand All @@ -236,11 +241,11 @@ func (r *integrationGcpStsResource) Create(ctx context.Context, request resource
return
}
delegateEmail := delegateResponse.Data.Attributes.GetDelegateAccountEmail()
state.DelegateAccountEmail = types.StringValue(delegateEmail)
plan.DelegateAccountEmail = types.StringValue(delegateEmail)

attributes, diags := r.buildIntegrationGcpStsRequestBody(ctx, &state)
if !state.ClientEmail.IsNull() {
attributes.SetClientEmail(state.ClientEmail.ValueString())
attributes, diags := r.buildGcpStsRequestBody(ctx, &plan)
if !plan.ClientEmail.IsNull() {
attributes.SetClientEmail(plan.ClientEmail.ValueString())
}

body := datadogV2.NewGCPSTSServiceAccountCreateRequestWithDefaults()
Expand All @@ -260,26 +265,26 @@ func (r *integrationGcpStsResource) Create(ctx context.Context, request resource
response.Diagnostics.AddError("response contains unparsedObject", err.Error())
return
}
r.updateState(ctx, &state, resp.Data)

r.parseGcpStsResponseBody(ctx, &plan, resp.Data)

// Save data into Terraform state
response.Diagnostics.Append(response.State.Set(ctx, &state)...)
response.Diagnostics.Append(response.State.Set(ctx, &plan)...)
}

func (r *integrationGcpStsResource) Update(ctx context.Context, request resource.UpdateRequest, response *resource.UpdateResponse) {
var state integrationGcpStsModel

response.Diagnostics.Append(request.Plan.Get(ctx, &state)...)
var plan integrationGcpStsModel
response.Diagnostics.Append(request.Plan.Get(ctx, &plan)...)
if response.Diagnostics.HasError() {
return
}

integrationGcpStsMutex.Lock()
defer integrationGcpStsMutex.Unlock()

id := state.ID.ValueString()
id := plan.ID.ValueString()

attributes, diags := r.buildIntegrationGcpStsRequestBody(ctx, &state)
attributes, diags := r.buildGcpStsRequestBody(ctx, &plan)
body := datadogV2.NewGCPSTSServiceAccountUpdateRequestWithDefaults()
body.Data = datadogV2.NewGCPSTSServiceAccountUpdateRequestDataWithDefaults()
body.Data.SetAttributes(attributes)
Expand All @@ -294,14 +299,16 @@ func (r *integrationGcpStsResource) Update(ctx context.Context, request resource
response.Diagnostics.Append(utils.FrameworkErrorDiag(err, "error retrieving Integration Gcp Sts"))
return
}

if err := utils.CheckForUnparsed(resp); err != nil {
response.Diagnostics.AddError("response contains unparsedObject", err.Error())
return
}
r.updateState(ctx, &state, resp.Data)

r.parseGcpStsResponseBody(ctx, &plan, resp.Data)

// Save data into Terraform state
response.Diagnostics.Append(response.State.Set(ctx, &state)...)
response.Diagnostics.Append(response.State.Set(ctx, &plan)...)
}

func (r *integrationGcpStsResource) Delete(ctx context.Context, request resource.DeleteRequest, response *resource.DeleteResponse) {
Expand All @@ -326,99 +333,101 @@ func (r *integrationGcpStsResource) Delete(ctx context.Context, request resource
}
}

func (r *integrationGcpStsResource) updateState(ctx context.Context, state *integrationGcpStsModel, resp *datadogV2.GCPSTSServiceAccount) {
state.ID = types.StringValue(resp.GetId())
func (r *integrationGcpStsResource) parseGcpStsResponseBody(ctx context.Context, model *integrationGcpStsModel, resp *datadogV2.GCPSTSServiceAccount) {
model.ID = types.StringValue(resp.GetId())

attributes := resp.GetAttributes()
if automute, ok := attributes.GetAutomuteOk(); ok {
state.Automute = types.BoolValue(*automute)
model.Automute = types.BoolValue(*automute)
}
if clientEmail, ok := attributes.GetClientEmailOk(); ok {
state.ClientEmail = types.StringValue(*clientEmail)
model.ClientEmail = types.StringValue(*clientEmail)
}
if isCspmEnabled, ok := attributes.GetIsCspmEnabledOk(); ok {
state.IsCspmEnabled = types.BoolValue(*isCspmEnabled)
model.IsCspmEnabled = types.BoolValue(*isCspmEnabled)
}
if isSecurityCommandCenterEnabled, ok := attributes.GetIsSecurityCommandCenterEnabledOk(); ok {
state.IsSecurityCommandCenterEnabled = types.BoolValue(*isSecurityCommandCenterEnabled)
model.IsSecurityCommandCenterEnabled = types.BoolValue(*isSecurityCommandCenterEnabled)
}
if isResourceChangeCollectionEnabled, ok := attributes.GetIsResourceChangeCollectionEnabledOk(); ok {
state.IsResourceChangeCollectionEnabled = types.BoolValue(*isResourceChangeCollectionEnabled)
model.IsResourceChangeCollectionEnabled = types.BoolValue(*isResourceChangeCollectionEnabled)
}
if isPerProjectQuotaEnabled, ok := attributes.GetIsPerProjectQuotaEnabledOk(); ok {
state.IsPerProjectQuotaEnabled = types.BoolValue(*isPerProjectQuotaEnabled)
model.IsPerProjectQuotaEnabled = types.BoolValue(*isPerProjectQuotaEnabled)
}
if resourceCollectionEnabled, ok := attributes.GetResourceCollectionEnabledOk(); ok {
state.ResourceCollectionEnabled = types.BoolValue(*resourceCollectionEnabled)
model.ResourceCollectionEnabled = types.BoolValue(*resourceCollectionEnabled)
}

if accountTags := attributes.GetAccountTags(); len(accountTags) > 0 {
state.AccountTags, _ = types.SetValueFrom(ctx, types.StringType, accountTags)
model.AccountTags, _ = types.SetValueFrom(ctx, types.StringType, accountTags)
}

mncs := make([]*MetricNamespaceConfigModel, 0)
for _, mnc := range attributes.GetMetricNamespaceConfigs() {
mncs = append(mncs, &MetricNamespaceConfigModel{
ID: types.StringValue(mnc.GetId()),
Disabled: types.BoolValue(mnc.GetDisabled()),
})
for _, cfg := range attributes.GetMetricNamespaceConfigs() {
var mdl MetricNamespaceConfigModel
mdl.ID = types.StringValue(cfg.GetId())
mdl.Disabled = types.BoolValue(cfg.GetDisabled())
mdl.Filters, _ = types.SetValueFrom(ctx, types.StringType, cfg.GetFilters())
mncs = append(mncs, &mdl)
}
state.MetricNamespaceConfigs, _ = types.SetValueFrom(ctx, MetricNamespaceConfigSpec, mncs)
model.MetricNamespaceConfigs, _ = types.SetValueFrom(ctx, MetricNamespaceConfigSpec, mncs)

state.HostFilters, _ = types.SetValueFrom(ctx, types.StringType, attributes.GetHostFilters())
state.CloudRunRevisionFilters, _ = types.SetValueFrom(ctx, types.StringType, attributes.GetCloudRunRevisionFilters())
model.HostFilters, _ = types.SetValueFrom(ctx, types.StringType, attributes.GetHostFilters())
model.CloudRunRevisionFilters, _ = types.SetValueFrom(ctx, types.StringType, attributes.GetCloudRunRevisionFilters())
mrcs := make([]*MonitoredResourceConfigModel, 0)
for _, mrc := range attributes.GetMonitoredResourceConfigs() {
var mdl MonitoredResourceConfigModel
mdl.Type = types.StringValue(string(mrc.GetType()))
mdl.Filters, _ = types.SetValueFrom(ctx, types.StringType, mrc.GetFilters())
mrcs = append(mrcs, &mdl)
}
state.MonitoredResourceConfigs, _ = types.SetValueFrom(ctx, MonitoredResourceConfigSpec, mrcs)
model.MonitoredResourceConfigs, _ = types.SetValueFrom(ctx, MonitoredResourceConfigSpec, mrcs)
}

func (r *integrationGcpStsResource) buildIntegrationGcpStsRequestBody(ctx context.Context, state *integrationGcpStsModel) (datadogV2.GCPSTSServiceAccountAttributes, diag.Diagnostics) {
func (r *integrationGcpStsResource) buildGcpStsRequestBody(ctx context.Context, model *integrationGcpStsModel) (datadogV2.GCPSTSServiceAccountAttributes, diag.Diagnostics) {
diags := diag.Diagnostics{}
attributes := datadogV2.GCPSTSServiceAccountAttributes{}

if !state.Automute.IsNull() {
attributes.SetAutomute(state.Automute.ValueBool())
if !model.Automute.IsNull() {
attributes.SetAutomute(model.Automute.ValueBool())
}
if !state.IsCspmEnabled.IsNull() {
attributes.SetIsCspmEnabled(state.IsCspmEnabled.ValueBool())
if !model.IsCspmEnabled.IsNull() {
attributes.SetIsCspmEnabled(model.IsCspmEnabled.ValueBool())
}
if !state.IsSecurityCommandCenterEnabled.IsUnknown() {
attributes.SetIsSecurityCommandCenterEnabled(state.IsSecurityCommandCenterEnabled.ValueBool())
if !model.IsSecurityCommandCenterEnabled.IsUnknown() {
attributes.SetIsSecurityCommandCenterEnabled(model.IsSecurityCommandCenterEnabled.ValueBool())
}
if !state.IsResourceChangeCollectionEnabled.IsUnknown() {
attributes.SetIsResourceChangeCollectionEnabled(state.IsResourceChangeCollectionEnabled.ValueBool())
if !model.IsResourceChangeCollectionEnabled.IsUnknown() {
attributes.SetIsResourceChangeCollectionEnabled(model.IsResourceChangeCollectionEnabled.ValueBool())
}
if !state.ResourceCollectionEnabled.IsUnknown() {
attributes.SetResourceCollectionEnabled(state.ResourceCollectionEnabled.ValueBool())
if !model.ResourceCollectionEnabled.IsUnknown() {
attributes.SetResourceCollectionEnabled(model.ResourceCollectionEnabled.ValueBool())
}
if !state.IsPerProjectQuotaEnabled.IsUnknown() {
attributes.SetIsPerProjectQuotaEnabled(state.IsPerProjectQuotaEnabled.ValueBool())
if !model.IsPerProjectQuotaEnabled.IsUnknown() {
attributes.SetIsPerProjectQuotaEnabled(model.IsPerProjectQuotaEnabled.ValueBool())
}

attributes.SetAccountTags(tfCollectionToSlice[string](ctx, diags, state.AccountTags))
attributes.SetAccountTags(tfCollectionToSlice[string](ctx, diags, model.AccountTags))

// only set this field if the user explicitly sets the field
// otherwise we want to omit it so that the API server can populate defaults when applicable
if cfgs := state.MetricNamespaceConfigs; !cfgs.IsUnknown() {
if cfgs := model.MetricNamespaceConfigs; !cfgs.IsUnknown() {
mncs := make([]datadogV2.GCPMetricNamespaceConfig, 0)
for _, mnc := range tfCollectionToSlice[*MetricNamespaceConfigModel](ctx, diags, cfgs) {
mncs = append(mncs, datadogV2.GCPMetricNamespaceConfig{
Id: mnc.ID.ValueStringPointer(),
Disabled: mnc.Disabled.ValueBoolPointer(),
Filters: tfCollectionToSlice[string](ctx, diags, mnc.Filters),
})
}
attributes.SetMetricNamespaceConfigs(mncs)
}

attributes.SetHostFilters(tfCollectionToSlice[string](ctx, diags, state.HostFilters))
attributes.SetCloudRunRevisionFilters(tfCollectionToSlice[string](ctx, diags, state.CloudRunRevisionFilters))
attributes.SetHostFilters(tfCollectionToSlice[string](ctx, diags, model.HostFilters))
attributes.SetCloudRunRevisionFilters(tfCollectionToSlice[string](ctx, diags, model.CloudRunRevisionFilters))
mrcs := make([]datadogV2.GCPMonitoredResourceConfig, 0)
for _, mrc := range tfCollectionToSlice[*MonitoredResourceConfigModel](ctx, diags, state.MonitoredResourceConfigs) {
for _, mrc := range tfCollectionToSlice[*MonitoredResourceConfigModel](ctx, diags, model.MonitoredResourceConfigs) {
mrcs = append(mrcs, datadogV2.GCPMonitoredResourceConfig{
Type: ptrTo(datadogV2.GCPMonitoredResourceConfigType(mrc.Type.ValueString())),
Filters: tfCollectionToSlice[string](ctx, diags, mrc.Filters),
Expand Down
2 changes: 1 addition & 1 deletion datadog/tests/cassettes/TestAccIntGcpStsMetricNamespaceConfigs.freeze
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2025-10-28T13:40:01.25536-04:00
2025-10-30T11:38:29.608552-04:00
Loading
Loading