Test assertion errors suppress values even with nonsensitive()

[xyl0o]

Terraform Version

Terraform v1.9.8
on linux_amd64

Terraform Configuration Files

# main.tf
variable "some_var" {
  type = string
}

variable "another_var" {
  type = string
}

variable "secret_var" {
  type      = string
  sensitive = true
}

# tests/sensitive_values.tftest.hcl
variables {
  some_var         = "some_value"
  another_var      = "another_value"
  secret_var       = "secret_value"
}

run "bare" {
  command = plan

  assert {
    condition     = var.some_var == "doesn't match"
    error_message = "some_var doesn't match static"
  }

  assert {
    condition     = var.some_var == var.another_var
    error_message = "some_var doesn't match another_var"
  }

  assert {
    condition     = var.secret_var == "doesn't match"
    error_message = "secret_var doesn't match static"
  }

  assert {
    condition     = var.some_var == var.secret_var
    error_message = "some_var doesn't match secret_var"
  }
}


run "with_non_sensitive" {
  command = plan

  assert {
    condition     = var.some_var == "doesn't match"
    error_message = "some_var doesn't match static"
  }

  assert {
    condition     = var.some_var == var.another_var
    error_message = "some_var doesn't match another_var"
  }

  assert {
    condition     = nonsensitive(var.secret_var) == "doesn't match"
    error_message = "secret_var doesn't match static"
  }

  assert {
    condition     = var.some_var == nonsensitive(var.secret_var)
    error_message = "some_var doesn't match secret_var"
  }
}

Debug Output

https://gist.github.com/xyl0o/8302b1e48630543b4b21fcf4b50a13db

Expected Behavior

The failed assertion doesn't print values that are sensitive:

╷
│ Error: Test assertion failed
│
│   on tests/sensitive_values.tftest.hcl line 26, in run "bare":
│   26:     condition     = var.some_var == var.secret_var
│     ├────────────────
│     │ var.some_var is "some_value"
│
│ some_var doesn't match secret_var
╵

But I'd expect to be able to see the value of both operands when using nonsensitive() - so something like this:

╷
│ Error: Test assertion failed
│
│   on tests/sensitive_values.tftest.hcl line 50, in run "with_non_sensitive":
│   50:     condition     = var.some_var == nonsensitive(var.secret_var)
│     ├────────────────
│     │ var.some_var is "some_value"
│     │ var.secret_var is "secret_value"
│
│ some_var doesn't match secret_var
╵

Actual Behavior

Using nonsensitive() has no effect:

╷
│ Error: Test assertion failed
│
│   on tests/sensitive_values.tftest.hcl line 50, in run "with_non_sensitive":
│   50:     condition     = var.some_var == nonsensitive(var.secret_var)
│     ├────────────────
│     │ var.some_var is "some_value"
│
│ some_var doesn't match secret_var
╵

Steps to Reproduce

terraform test --filter tests/sensitive_values.tftest.hcl

Additional Context

No response

References

No response

[liamcervante]

Hi @xyl0o, thanks for filing this!

I'll just note this behaviour isn't unique to the testing framework. All custom condition checks (including test assertions, pre and post conditions, variable validations, and check blocks) show the same behaviour. The nonsensitive function isn't actually modifying the underlying value, it creates a new value that isn't sensitive and returns that. This means the variable references that the diagnostic looks at when deducing values is still sensitive and hence it isn't shown.

With that in mind, I don't believe this is a bug as the everything is working as intended. However, I do think it could be valuable for the testing framework to be able expose sensitive variables when requested by the user. The testing framework is likely to be using dummy values anyway, and allowing the user easier access to debug values is useful. I'm going to recategorise this as an enhancement request rather than a bug. We'll have to think about the best way to allow the exposure of sensitive values in test files, as I wouldn't want to modify the behaviour of the other custom conditions as an accidental side effect of this.

[xyl0o]

The nonsensitive function isn't actually modifying the underlying value, it creates a new value that isn't sensitive and returns that

Why isn't the error message showing that new value?

I wouldn't necessary scope this just to terraform test. Just used it as an example. I'd like to be able to see these values generally when marking them as non-sensitive in the expression. Isn't that the purpose of nonsensitive()?

Nonetheless if this is improved just for terraform test I'm not complaining 😅

[liamcervante]

The output of functions called inside an expression are not available externally. Terraform simply doesn't have access to the nonsensitive value returned by the nonsensitive call inside the expression, and so it can't show the value for it. Supporting this would be a significant change to the HCL library, and we'd be unlikely to make such a change to support a single function that is unique to Terraform and not useful to other consumers of HCL (sensitive values are a Terraform concept, rather than something built into HCL).

In regular Terraform, you can work around this by aliasing the nonsensitive value outside of the expression and then using that.

locals {
  nonsensitive = nonsensitive(var.sensitive)
}

resource "type" "name" {
  lifecycle {
    postcondition {
      condition = var.nonsensitive == "something"
      error_message = "wrong value"
    }
  }
}

This currently isn't possible within the testing framework, as there is no way to perform intermediary processing of values from within the module. We do have an existing feature request for local blocks within test files: #34629. It is possible that implementing that would help with this somewhat, but leaving this ticket open would allow us to see if there is more interest in a more targeted solution.