Skip to content

Commit afa7fe0

Browse files
vijtrip2vijtrip2
authored
Initial implementation of AWSResourceManager.LateInitialize() method. (#138)
Issue #, if available: aws-controllers-k8s/community#812 Description of changes: * Initial implementation of `AWSResourceManager.LateInitialize()` method code generator * Adds two new hooks `late_initialize_pre_read_one` & `late_initialize_post_read_one` * Introduces new LateInitializeConfig with DelaySeconds as single parameter * Corresponding runtime PR : aws-controllers-k8s/runtime#37 ------------------------------------ Testing * Unit Tests Successful ``` ➜ code-generator git:(li-mlp) ✗ make test go test -tags codegen ./... ? github.com/aws-controllers-k8s/code-generator/cmd/ack-generate [no test files] ? github.com/aws-controllers-k8s/code-generator/cmd/ack-generate/command [no test files] ok github.com/aws-controllers-k8s/code-generator/pkg/generate/ack 2.346s ok github.com/aws-controllers-k8s/code-generator/pkg/generate/code 17.798s ? github.com/aws-controllers-k8s/code-generator/pkg/generate/config [no test files] ? github.com/aws-controllers-k8s/code-generator/pkg/generate/crossplane [no test files] ? github.com/aws-controllers-k8s/code-generator/pkg/generate/olm [no test files] ? github.com/aws-controllers-k8s/code-generator/pkg/generate/templateset [no test files] ? github.com/aws-controllers-k8s/code-generator/pkg/metadata [no test files] ok github.com/aws-controllers-k8s/code-generator/pkg/model 14.671s ok github.com/aws-controllers-k8s/code-generator/pkg/model/multiversion 5.152s ok github.com/aws-controllers-k8s/code-generator/pkg/names (cached) ? github.com/aws-controllers-k8s/code-generator/pkg/testutil [no test files] ? github.com/aws-controllers-k8s/code-generator/pkg/util [no test files] ? github.com/aws-controllers-k8s/code-generator/pkg/version [no test files] ``` * Controller generation successful ``` ➜ code-generator git:(li-mlp) ✗ make build-controller building ack-generate ... ok. installing controller-gen v0.6.1 ... ok. Copying common custom resource definitions into ecr Building Kubernetes API objects for ecr Generating deepcopy code for ecr Generating custom resource definitions for ecr Building service controller for ecr Generating RBAC manifests for ecr Running gofmt against generated code for ecr ``` * Sample generated code for ecr repository ```go // LateInitialize returns an acktypes.AWSResource after setting the late initialized // fields from the readOne call. This method will initialize the optional fields // which were not provided by the k8s user but were defaulted by the AWS service. // If there are no such fields to be initialized, the returned object is similar to // object passed in the parameter. func (rm *resourceManager) LateInitialize( ctx context.Context, res acktypes.AWSResource, ) (acktypes.AWSResource, error) { rlog := ackrtlog.FromContext(ctx) // If there are no fields to late initialize, do nothing if len(lateInitializeFieldNames) == 0 { rlog.Debug("no late initialization required.") return res, nil } r := rm.concreteResource(res) if r.ko == nil { // Should never happen... if it does, it's buggy code. panic("resource manager's LateInitialize() method received resource with nil CR object") } koWithDefaults := r.ko.DeepCopy() latestWithDefaults := &resource{koWithDefaults} lateInitConditionReason := "" lateInitConditionMessage := "" observed, err := rm.sdkFind(ctx, latestWithDefaults) if err != nil { lateInitConditionMessage = "Unable to complete Read operation required for late initialization" lateInitConditionReason = "Late Initialization Failure" ackcondition.SetLateInitialized(latestWithDefaults, corev1.ConditionFalse, &lateInitConditionMessage, &lateInitConditionReason) return latestWithDefaults, ackrequeue.NeededAfter(err, time.Duration(0)*time.Second) } observedKo := observed.ko if observedKo.Spec.Name != nil && koWithDefaults.Spec.Name == nil { koWithDefaults.Spec.Name = observedKo.Spec.Name } incompleteInitialization := rm.incompleteLateInitialization(latestWithDefaults) if incompleteInitialization { // Add the condition with LateInitialized=False lateInitConditionMessage = "Late initialization did not complete, requeuing with delay of 5 seconds" lateInitConditionReason = "Delayed Late Initialization" ackcondition.SetLateInitialized(latestWithDefaults, corev1.ConditionFalse, &lateInitConditionMessage, &lateInitConditionReason) return latestWithDefaults, ackrequeue.NeededAfter(nil, time.Duration(5)*time.Second) } // Set LateIntialized condition to True lateInitConditionMessage = "Late initialization successful" lateInitConditionReason = "Late initialization successful" ackcondition.SetLateInitialized(latestWithDefaults, corev1.ConditionTrue, &lateInitConditionMessage, &lateInitConditionReason) return latestWithDefaults, nil } // incompleteLateInitialization return true if there are fields which were supposed to be // late initialized but are not. If all the fields are late initialized, false is returned func (rm *resourceManager) incompleteLateInitialization( latestWithDefaults *resource, ) bool { ko := latestWithDefaults.ko if ko.Spec.Name == nil { return true } return false } ``` * local-kind-e2e-test logs ``` │ │ 2021-07-26T19:38:08.701Z DEBUG ackrt > r.Sync {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "kind": "Repository" │ │ 2021-07-26T19:38:08.701Z DEBUG ackrt >> rm.ReadOne {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": f │ │ 2021-07-26T19:38:08.701Z DEBUG ackrt >>> rm.sdkFind {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": │ │ 2021-07-26T19:38:09.641Z DEBUG ackrt <<< rm.sdkFind {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": │ │ 2021-07-26T19:38:09.641Z DEBUG ackrt << rm.ReadOne {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": f │ │ 2021-07-26T19:38:09.641Z DEBUG ackrt >> r.setResourceManaged {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_a │ │ 2021-07-26T19:38:09.641Z DEBUG ackrt >>> kc.Patch (metadata + spec) {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2" │ │ 2021-07-26T19:38:09.694Z DEBUG ackrt <<< kc.Patch (metadata + spec) {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2" │ │ 2021-07-26T19:38:09.694Z DEBUG ackrt marked resource as managed {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "i │ │ 2021-07-26T19:38:09.694Z DEBUG ackrt << r.setResourceManaged {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_a │ │ 2021-07-26T19:38:09.694Z DEBUG ackrt >> rm.Create {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": fa │ │ 2021-07-26T19:38:09.694Z DEBUG ackrt >>> rm.sdkCreate {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted" │ │ 2021-07-26T19:38:09.725Z DEBUG ackrt <<< rm.sdkCreate {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted" │ │ 2021-07-26T19:38:09.725Z DEBUG ackrt << rm.Create {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": fa │ │ 2021-07-26T19:38:09.725Z INFO ackrt created new resource {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopt │ │ 2021-07-26T19:38:09.725Z DEBUG ackrt >> rm.LateInitialize {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adop │ │ 2021-07-26T19:38:09.725Z INFO ackrt calculated late initialization delay is 0 seconds {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "re │ │ 2021-07-26T19:38:09.725Z DEBUG ackrt >>> rm.sdkFind {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": │ │ 2021-07-26T19:38:09.748Z DEBUG ackrt <<< rm.sdkFind {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": │ │ 2021-07-26T19:38:09.748Z DEBUG ackrt << rm.LateInitialize {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adop │ │ 2021-07-26T19:38:09.748Z DEBUG ackrt >> r.patchResourceMetadataAndSpec {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west │ │ 2021-07-26T19:38:09.748Z DEBUG ackrt >>> kc.Patch (metadata + spec) {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2" │ │ 2021-07-26T19:38:09.758Z DEBUG ackrt <<< kc.Patch (metadata + spec) {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2" │ │ 2021-07-26T19:38:09.759Z DEBUG ackrt patched resource metadata and spec {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-wes │ │ 2021-07-26T19:38:09.759Z DEBUG ackrt << r.patchResourceMetadataAndSpec {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west │ │ 2021-07-26T19:38:09.759Z DEBUG ackrt >> r.patchResourceStatus {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_ │ │ 2021-07-26T19:38:09.759Z DEBUG ackrt >>> kc.Patch (status) {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_ado │ │ 2021-07-26T19:38:09.794Z DEBUG ackrt <<< kc.Patch (status) {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_ado │ │ 2021-07-26T19:38:09.795Z DEBUG ackrt patched resource status {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_a │ │ 2021-07-26T19:38:09.795Z DEBUG ackrt << r.patchResourceStatus {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_ │ │ 2021-07-26T19:38:09.795Z DEBUG ackrt < r.Sync {"kind": "Repository", "namespace": "default", "name": "ecr-repository-wy6n6wjm8", "generation": 1, "account": "309117047740", "role": "", "region": "us-west-2", "is_adopted": false, │ │ 2021-07-26T19:38:09.795Z DEBUG controller-runtime.controller Successfully Reconciled {"controller": "repository", "request": "default/ecr-repository-wy6n6wjm8"} ``` By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
1 parent 309d704 commit afa7fe0

13 files changed

+746
-5
lines changed

‎Makefile

+1-1
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ GO111MODULE=on
66
AWS_SERVICE=$(shell echo $(SERVICE) | tr '[:upper:]' '[:lower:]')
77

88
# Build ldflags
9-
VERSION ?= "v0.9.2"
9+
VERSION ?= "v0.10.0"
1010
GITCOMMIT=$(shell git rev-parse HEAD)
1111
BUILDDATE=$(shell date -u +'%Y-%m-%dT%H:%M:%SZ')
1212
IMPORT_PATH=github.com/aws-controllers-k8s/code-generator

‎go.mod

+1-1
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ module github.com/aws-controllers-k8s/code-generator
33
go 1.14
44

55
require (
6-
github.com/aws-controllers-k8s/runtime v0.9.2
6+
github.com/aws-controllers-k8s/runtime v0.10.0
77
github.com/aws/aws-sdk-go v1.37.10
88
github.com/dlclark/regexp2 v1.4.0
99
// pin to v0.1.1 due to release problem with v0.1.2

‎go.sum

+2-2
Original file line numberDiff line numberDiff line change
@@ -67,8 +67,8 @@ github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5 h1:0CwZNZbxp69SHPd
6767
github.com/armon/go-socks5 v0.0.0-20160902184237-e75332964ef5/go.mod h1:wHh0iHkYZB8zMSxRWpUBQtwG5a7fFgvEO+odwuTv2gs=
6868
github.com/asaskevich/govalidator v0.0.0-20180720115003-f9ffefc3facf/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
6969
github.com/asaskevich/govalidator v0.0.0-20190424111038-f61b66f89f4a/go.mod h1:lB+ZfQJz7igIIfQNfa7Ml4HSf2uFQQRzpGGRXenZAgY=
70-
github.com/aws-controllers-k8s/runtime v0.9.2 h1:53ahm38Cn6DTfQdHNrTgbXmUbCjuKntvhkuWGHkAQ18=
71-
github.com/aws-controllers-k8s/runtime v0.9.2/go.mod h1:kG2WM4JAmLgf67cgZV9IZUkY2DsrUzsaNbmhFMfb05c=
70+
github.com/aws-controllers-k8s/runtime v0.10.0 h1:MPZ4mPeap2mP/EKU6Pk7a7phiBSYaeZ9QJX38OPecXo=
71+
github.com/aws-controllers-k8s/runtime v0.10.0/go.mod h1:kG2WM4JAmLgf67cgZV9IZUkY2DsrUzsaNbmhFMfb05c=
7272
github.com/aws/aws-sdk-go v1.37.10 h1:LRwl+97B4D69Z7tz+eRUxJ1C7baBaIYhgrn5eLtua+Q=
7373
github.com/aws/aws-sdk-go v1.37.10/go.mod h1:hcU610XS61/+aQV88ixoOzUoG7v3b31pl2zKMmprdro=
7474
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=

‎pkg/generate/ack/controller.go

+9
Original file line numberDiff line numberDiff line change
@@ -118,6 +118,15 @@ var (
118118
"GoCodeSetResourceIdentifiers": func(r *ackmodel.CRD, sourceVarName string, targetVarName string, indentLevel int) string {
119119
return code.SetResourceIdentifiers(r.Config(), r, sourceVarName, targetVarName, indentLevel)
120120
},
121+
"GoCodeFindLateInitializedFieldNames": func(r *ackmodel.CRD, resVarName string, indentLevel int) string {
122+
return code.FindLateInitializedFieldNames(r.Config(), r, resVarName, indentLevel)
123+
},
124+
"GoCodeLateInitializeFromReadOne": func(r *ackmodel.CRD, sourceResVarName string, targetResVarName string, indentLevel int) string {
125+
return code.LateInitializeFromReadOne(r.Config(), r, sourceResVarName, targetResVarName, indentLevel)
126+
},
127+
"GoCodeIncompleteLateInitialization": func(r *ackmodel.CRD, resVarName string, indentLevel int) string {
128+
return code.IncompleteLateInitialization(r.Config(), r, resVarName, indentLevel)
129+
},
121130
}
122131
)
123132

‎pkg/generate/ack/hook.go

+8
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,8 @@ code paths:
5959
* sdk_file_end
6060
* delta_pre_compare
6161
* delta_post_compare
62+
* late_initialize_pre_read_one
63+
* late_initialize_post_read_one
6264
6365
The "pre_build_request" hooks are called BEFORE the call to construct
6466
the Input shape that is used in the API operation and therefore BEFORE
@@ -102,6 +104,12 @@ compares two resources.
102104
The "delta_post_compare" hooks are called AFTER the generated code that
103105
compares two resources.
104106
107+
The "late_initialize_pre_read_one" hooks are called BEFORE making the
108+
readOne call inside AWSResourceManager.LateInitialize() method
109+
110+
The "late_initialize_post_read_one" hooks are called AFTER making the
111+
readOne call inside AWSResourceManager.LateInitialize() method
112+
105113
*/
106114

107115
// ResourceHookCode returns a string with custom callback code for a resource

‎pkg/generate/ack/runtime_test.go

+4
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,10 @@ func (frm *fakeRM) Delete(context.Context, acktypes.AWSResource) (acktypes.AWSRe
117117

118118
func (frm *fakeRM) ARNFromName(string) string { return "" }
119119

120+
func (frm *fakeRM) LateInitialize(context.Context, acktypes.AWSResource) (acktypes.AWSResource, error) {
121+
return nil, nil
122+
}
123+
120124
// This test is mostly just a hack to introduce a Go module dependency between
121125
// the ACK runtime library and the code generator. The code generator doesn't
122126
// actually depend on Go code in the ACK runtime, but it *produces* templated

0 commit comments

Comments
 (0)