Expose provider config #164

Zaid-Ajaj · 2025-02-28T22:04:46Z

Initial idea to expose provider config is first by exposing which providers can be configured: when inferring the schema, we can access ProviderRequirements.RequiredProviders and can see which provider is used in the module (e.g. aws).

Using this information, we extend the schema of the provider resource to accept a free-form object for every required provider e.g. aws : mapType(anyType). In TypeScript it looks like this:

const provider = new bucket.Provider("test-provider", {
    aws: {
        "region": "us-west-2"
    }
})

We keep track of provider configuration in memory via CheckConfig and Configure because CheckConfig is not always called during Delete. Then we match the used explicit provider during Construct(...)

During construct, we figure out which provider config to use, if any and pass it down to the TF sandbox in the form of map[string]resource.PropertyMap where each entry is a provider config block that has inputs and the module inputs gets a new providers field to reference these config blocks i.e. providers = { "aws" = aws }

See generated TF files per operation on the test TestS3BucketWithExplicitProvider

EDIT: problem of the program hanging no longer happens thanks to #171 ❤️ @t0yv0

t0yv0 · 2025-02-28T22:28:22Z

Hah. Deadlock may be of my doing, I'll dig with you on Monday. I can at least add timeouts to all the places.

t0yv0 · 2025-03-03T18:15:06Z

pkg/tfsandbox/tf_file.go

@@ -152,6 +155,10 @@ func CreateTFFile(
 	}
 	inputsMap := inputs.MapRepl(nil, locals.decode)

+	for providerName, config := range providerConfig {
+		providers[providerName] = config.MapRepl(nil, locals.decode)


locals.decode here is interesting. It makes one think what happens when first-class secrets are passed in here. I wonder if we already handle all there is to handle or there's some residual issues.

Same for unknowns. Wonder what happens when provider configs are not known.

When getting the values via CheckConfig, I actually made it such that we don't keep secrets nor unknowns:

config, err := plugin.UnmarshalProperties(req.GetNews(), plugin.MarshalOptions{ KeepUnknowns: false, KeepSecrets: false, KeepResources: false, KeepOutputValues: false, })

Actually changed this to:

config, err := plugin.UnmarshalProperties(req.GetNews(), plugin.MarshalOptions{ KeepUnknowns: true, RejectAssets: true, KeepSecrets: true, KeepOutputValues: false, })

since tf seems to be able to handle secrets and unknowns but that needs more testing

Ok so sounds like I need to write test tickets for that.

#187 not hugely important but listing for completeness.

pkg/tfsandbox/tf_file.go

t0yv0 · 2025-03-03T18:18:30Z

pkg/modprovider/tfmodules.go

+	}
+
+	if module.ProviderRequirements != nil {
+		for providerName := range module.ProviderRequirements.RequiredProviders {


This looks reasonable!

pkg/modprovider/state.go

pkg/modprovider/server.go

t0yv0 · 2025-03-03T18:40:16Z

Do you know why we have RegisterComponentResource call in the code under NewModuleComponentResource?

https://github.com/pulumi/pulumi-terraform-module-provider/blob/48adf03578ac0dcc3cc61779ce2028ec6f04974c/pkg/modprovider/module_component.go#L71

I thought we should not have it as the SDK is going to be calling that.

t0yv0 · 2025-03-03T18:49:10Z

Ah that's a red herring. Looks like the RegisterComponentResource call gets back a URN for us.

t0yv0 · 2025-03-03T19:05:27Z

Still it looks like deadlock is due to breaking assumptions about the process model. We have:

35353 s003  S+     0:00.29 pulumi preview
35356 s003  S      0:00.10 /run/current-system/sw/bin/pulumi-language-nodejs -packagemanager=npm -root=/Users/anton/code/pulumi-terraform-module/ex1 127.0.0.1:59321
35359 s003  S      0:00.05 /Users/anton/code/pulumi-terraform-module/bin/pulumi-resource-terraform-module 127.0.0.1:59321
35362 s003  S      0:00.05 /Users/anton/code/pulumi-terraform-module/bin/pulumi-resource-terraform-module 127.0.0.1:59321

There are two instances of the provider. We expect there to be one. One sec.

t0yv0 · 2025-03-03T20:42:42Z

#171 perhaps? Gets me past the hang on preview.

pkg/modprovider/server.go

t0yv0 · 2025-03-03T20:48:10Z

pkg/modprovider/server.go

+//
+//		{
+//	       propertyKey("version"): stringProperty("0.1.0"),
+//		   propertyKey("aws"): stringProperty("{\"region\": \"us-west-2\"}")


Can you explain step by step, what's going on here? Are we affected by the JSON encoding again :/

Are we affected by the JSON encoding again :/

Yes. Config values arrive to the provider JSON-stringified so here we change them back to an object format, if necessary. When they are already an object, we use that shape.

If the value is a string, then it must be a stringified JSON object since the schema for each field specifies a free-form object so it is same to assume.

…tion Amendments to exposing provider configurations PR

t0yv0 · 2025-03-06T14:39:34Z

pkg/modprovider/module_component.go

@@ -84,15 +86,28 @@ func NewModuleComponentResource(
 		planStore.Forget(urn)
 	}()

+	var providerSelfRef pulumi.ProviderResource
+	if providerSelfURN != "" {


Remind me why it's ever empty. Is it only empty in tests? If so can we add a comment to that purpose?

It is empty if only CheckConfig is called until we can access the URN from Configure. Right now, URN from Configure is not strictly needed and the check here is a workaround for TestSavingModuleState

OK yeah TestSavingModuleState we might want to remove that, together with an incomplete engine, in favor of just trusting full blown integration tests.

t0yv0 · 2025-03-06T14:47:53Z

pkg/modprovider/module_component.go

@@ -242,3 +278,16 @@ func NewModuleComponentResource(

 	return &urn, marshalledOutputs, nil
 }
+
+func newProviderSelfReference(ctx *pulumi.Context, urn1 pulumi.URN) pulumi.ProviderResource {


@Frassle if I could have your 5 min to eyeball this.

We feed the result of this function to pulumi.Provider(newProviderSelfReference(..)). This is a workaround. I tried and tired to find a way to pass URN directly to pulumi.Provider() but could not :/

We could drop to gRPC level of calling RegisterResource to workaround this but that negates benefits of using the SDK.

While this workaround seems to work, you were mentioning that self-identifying a provider instance requires both a self-URN and self-ID, and self-URN by itself is not specific enough.

Ideally something like this maybe would be public?
https://github.com/pulumi/pulumi/blob/68295c45f3f3c8f6aadbd76d141a4dcf4f0a55d2/sdk/go/pulumi/resource.go#L181

I found no way to call it though due to internal constraints.

It tried extending the struct and overriding ID() and URN() outputs. That worked EXCEPT I cannot seem to ever be able to set this highly private pkg field, not through reflection, not through subtyping, nor can I override getPackage so I am stuck not being able to make it work with the Provider option.

https://github.com/pulumi/pulumi/blob/68295c45f3f3c8f6aadbd76d141a4dcf4f0a55d2/sdk/go/pulumi/resource.go#L193

I guess we could use what we use here, the code as is, and then wrap{prov} with

type wrap struct { pulumi.ProviderResourceState id pulumi.ID } func (*wrap) ID() pulumi.IDOutput{ return toOutput(pulumi.ID) }

t0yv0 · 2025-03-06T20:44:51Z

pkg/modprovider/schema.go

@@ -41,6 +41,9 @@ func pulumiSchemaForModule(pargs *ParameterizeArgs, inferredModule *InferredModu
 		Name:    string(packageName),
 		Version: string(pkgVer),
 		Types:   inferredModule.SupportingTypes,
+		Provider: schema.ResourceSpec{
+			InputProperties: inferredModule.ProvidersConfig.Variables,


Good call, we forgot this before.

t0yv0 · 2025-03-06T20:46:50Z

pkg/modprovider/server.go

+		KeepUnknowns:     true,
+		RejectAssets:     true,
+		KeepSecrets:      true,
+		KeepOutputValues: false,


What this does is rewires first-class outputs to Secret or Computed cases, but it also drops dependencies inherent in first-class outputs. Whether that messes things up for Pulumi or not, I am not sure. We could test this out later on. I'll add a ticket.

t0yv0 · 2025-03-06T20:48:36Z

pkg/modprovider/state.go

-		[]tfsandbox.TFOutputSpec{}, /*outputs*/
+		inputs,                            /*inputs*/
+		[]tfsandbox.TFOutputSpec{},        /*outputs*/
+		map[string]resource.PropertyMap{}, /*providersConfig*/


This should be a TODO here right? When read executes tofu refresh, it does need to have the provider config?

t0yv0 · 2025-03-06T20:51:12Z

pkg/tfsandbox/tf_file.go

+	// An environment variable that can be set to a path of a directory
+	// to which we write the generated Terraform JSON file.
+	// mainly used for debugging purposes and being able to see the generated code.
+	writeDir := os.Getenv("PULUMI_TERRAFORM_MODULE_WRITE_TF_FILE")


This is cool, we can do better though exposing the entire workdir with downloaded tofu modules and all, for debugging purposes I think.

Ah I see it is handy for testing to just write the TF file nothing else.

Yeah just for tests 😄

t0yv0 · 2025-03-06T20:53:07Z

tests/acc_test.go

@@ -338,7 +338,7 @@ func TestTerraformAwsModulesVpcIntoTypeScript(t *testing.T) {

 func TestS3BucketModSecret(t *testing.T) {
 	localProviderBinPath := ensureCompiledProvider(t)
-	skipLocalRunsWithoutCreds(t)
+	//skipLocalRunsWithoutCreds(t)


Restore this I think? Did you have trouble with it?

No trouble, just wanted to run it locally so I removed the line 😄 lemme bring it back

It would run locally if you have AWS_PROFILE env var set or some such. AWS_REGION.

t0yv0 · 2025-03-06T20:55:34Z

tests/acc_test.go

+		})
+	})
+
+	t.Run("pulumi up", func(t *testing.T) {


Refresh missing here yes?

#186 can backfill in separate change.

t0yv0 · 2025-03-06T20:58:04Z

tests/testdata/programs/ts/s3bucket-explicit-provider/tf_files_preview/test-bucket.tf.json

+      "version": "4.5.0"
+    }
+  },
+  "provider": {


Does this support detecting and emitting alias configuration?

https://developer.hashicorp.com/terraform/language/providers/configuration#alias-multiple-provider-configurations

Or do we need a ticket to take care of it later?

I think we don't need aliases because in my understanding, they are used to define different configurations for the same provider. In Pulumi, you would define another explicit provider with different config if you wanted the same provider with different config

I don't think that quite squares with what I understand they accomplish. A single module instance m may request several copies of the aws provider, like aws-east and aws-west, etc. If we defined several Pulumi explicit providers we'd have a tough time sending 2+ of them to the same module instance.

t0yv0 · 2025-03-06T21:05:25Z

pkg/modprovider/server.go

+
+			if len(deserialized) > 0 {
+				providersConfig[string(propertyKey)] = resource.NewPropertyMapFromMap(deserialized)
+			}


Missing continue here? I don't think it should fall through to the next case.

Added the continue so that the final fall through is a hard failure in case we get something we don't understand ✅

t0yv0 · 2025-03-06T21:06:12Z

pkg/modprovider/server.go

+			// we might later get a new behaviour where programs no longer send serialized JSON
+			// but send the actual object instead
+			providersConfig[string(propertyKey)] = serializedConfig.ObjectValue()
+		}


Let's assert here if we fall through and we failed to parse it, we fail loud that we don't understand the encoding. This would be helpful weeding out some edge cases. I think secrets are encoded funny in JSON config encoding and we will need to tweak this a bit to fix that.

t0yv0 · 2025-03-06T21:07:42Z

pkg/modprovider/server.go

+		}
+
+		if serializedConfig.IsObject() {
+			// we might later get a new behaviour where programs no longer send serialized JSON


We already have this behavior in YAML and Go. This entire section should reference https://github.com/pulumi/home/issues/3705 or some such.

Added the issue for reference in the code ✅

t0yv0 · 2025-03-06T21:08:11Z

pkg/modprovider/server.go

+			value := serializedConfig.StringValue()
+			deserialized := map[string]interface{}{}
+			if err := json.Unmarshal([]byte(value), &deserialized); err != nil {
+				contract.Failf("failed to deserialize provider config into a map: %v", err)


So this thinks if it's a string it's always JSON-encoded string. That's probably OK since the type of these values is map<any>.

t0yv0 · 2025-03-06T21:09:06Z

pkg/modprovider/server.go

+func cleanProvidersConfig(config resource.PropertyMap) map[string]resource.PropertyMap {
+	providersConfig := make(map[string]resource.PropertyMap)
+	for propertyKey, serializedConfig := range config {
+		if string(propertyKey) == "version" {


I think another one is "pluginDownloadURL" https://github.com/pulumi/pulumi-terraform-bridge/blob/4ee0d00c1d1a3cd3dd1b50b0a5227c816cbac4dd/pkg/tfbridge/provider.go#L712

t0yv0 · 2025-03-06T22:00:52Z

Let's address the small comments like missing continue etc and

I've stood up 5 follow up small tickets on various aspects discussed here.

#186 is most important of these.

t0yv0 · 2025-03-07T14:57:48Z

pulumi-bot · 2025-03-07T17:17:03Z

This PR has been shipped in release v0.0.5.

Zaid-Ajaj added 5 commits February 28, 2025 18:28

init

d32b3be

Merge branch 'main' into zaid/exposing-provider-configuration

b2a9cc3

initial implementation writing out config to TF files

e6eaeda

Merge branch 'main' into zaid/exposing-provider-configuration

17ed559

remove in-place test artifacts

c8f8b19

Zaid-Ajaj marked this pull request as draft February 28, 2025 22:05

lint

4d7cac8

t0yv0 self-requested a review March 3, 2025 15:31

add unit test for cleanProvidersConfig

d68e485

t0yv0 requested a review from a team March 3, 2025 18:09