Skip to content

Fix ExtractItems to preserve empty maps and lists #293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mrIncompetent
Copy link

What

ExtractItems now preserves empty collections ({} and []) instead of converting them to null.

Why

In our workflow, we extract apply configurations to compare with desired state before making API calls. When empty collections are converted to null, this comparison incorrectly shows
differences, causing unnecessary API requests.

This affects resources like emptyDir volumes and container resources specifications:

// Resource has: {"emptyDir": {}} or {"resources": {}}
extracted := resource.ExtractItems(fieldSet.Leaves())
// Before: {"emptyDir": null}
// After:  {"emptyDir": {}}

Changes

  • Modified typed/remove.go to preserve empty maps and lists during extraction
  • Added test cases for empty struct, list, and nested collection preservation
  • Maintains distinction between null and empty values

Fixes issue reported in #259 regarding consistent empty collection handling.

Additional

V4(that's what my codebase imports) vs PR Compare example
package internal

import (
	"encoding/json"
	"testing"

	corev1apply "k8s.io/client-go/applyconfigurations/core/v1"
	typedv4 "sigs.k8s.io/structured-merge-diff/v4/typed"
	typedv6 "sigs.k8s.io/structured-merge-diff/v6/typed"
)

var (
	podSpecApplyConfig = corev1apply.PodSpec().
		WithContainers(
			corev1apply.Container().
				WithName("main").
				WithImage("nginx:latest").
				WithVolumeMounts(
					corev1apply.VolumeMount().
						WithName("data").
						WithMountPath("/data"),
				),
		).
		WithVolumes(
			corev1apply.Volume().
				WithName("data").
				WithEmptyDir(corev1apply.EmptyDirVolumeSource()),
		)
)

func TestStructuredMergeDiffExtraction_Pod_v6_PATCHED(t *testing.T) {
	parser, _ := typedv6.NewParser(schemaYAML)

	specType := parser.Type("io.k8s.api.core.v1.PodSpec")
	specTypeValue, _ := specType.FromStructured(podSpecApplyConfig)

	originalJSON, _ := json.Marshal(podSpecApplyConfig)
	t.Logf("Original JSON: \n%s", string(originalJSON))

	specTypeValueFields, _ := specTypeValue.ToFieldSet()
	t.Logf("FieldSet Leaves: \n%v", specTypeValueFields.Leaves())

	extractedSpecTypeValue := specTypeValue.ExtractItems(specTypeValueFields.Leaves())
	extractedJSON, _ := json.Marshal(extractedSpecTypeValue.AsValue().Unstructured())
	t.Logf("Extracted JSON: \n%s", string(extractedJSON))

	diff, _ := specTypeValue.Compare(extractedSpecTypeValue)

	if !diff.IsSame() {
		t.Logf("Diff: \n%s", diff.String())
		t.Errorf("The extracted TypedValue differs from the original.")
	}
}

func TestStructuredMergeDiffExtraction_Pod_v4(t *testing.T) {
	schemaYAMLv4 := typedv4.YAMLObject(schemaYAML)
	parser, _ := typedv4.NewParser(schemaYAMLv4)

	specType := parser.Type("io.k8s.api.core.v1.PodSpec")
	specTypeValue, _ := specType.FromStructured(podSpecApplyConfig)

	originalJSON, _ := json.Marshal(podSpecApplyConfig)
	t.Logf("Original JSON: \n%s", string(originalJSON))

	specTypeValueFields, _ := specTypeValue.ToFieldSet()
	t.Logf("FieldSet Leaves: \n%v", specTypeValueFields.Leaves())

	extractedSpecTypeValue := specTypeValue.ExtractItems(specTypeValueFields.Leaves())
	extractedJSON, _ := json.Marshal(extractedSpecTypeValue.AsValue().Unstructured())
	t.Logf("Extracted JSON: \n%s", string(extractedJSON))

	diff, _ := specTypeValue.Compare(extractedSpecTypeValue)

	if !diff.IsSame() {
		t.Logf("Diff: \n%s", diff.String())
		t.Errorf("The extracted TypedValue differs from the original.")
	}
}

The test output:

=== RUN   TestStructuredMergeDiffExtraction_Pod_v6_PATCHED
    ssa_test.go:38: Original JSON: 
        {"volumes":[{"name":"data","emptyDir":{}}],"containers":[{"name":"main","image":"nginx:latest","volumeMounts":[{"name":"data","mountPath":"/data"}]}]}
    ssa_test.go:41: FieldSet Leaves: 
        .containers[name="main"].image
        .containers[name="main"].name
        .containers[name="main"].volumeMounts[mountPath="/data"].mountPath
        .containers[name="main"].volumeMounts[mountPath="/data"].name
        .volumes[name="data"].emptyDir
        .volumes[name="data"].name
    ssa_test.go:45: Extracted JSON: 
        {"containers":[{"image":"nginx:latest","name":"main","volumeMounts":[{"mountPath":"/data","name":"data"}]}],"volumes":[{"emptyDir":{},"name":"data"}]}
--- PASS: TestStructuredMergeDiffExtraction_Pod_v6_PATCHED (0.01s)
=== RUN   TestStructuredMergeDiffExtraction_Pod_v4
    ssa_test.go:63: Original JSON: 
        {"volumes":[{"name":"data","emptyDir":{}}],"containers":[{"name":"main","image":"nginx:latest","volumeMounts":[{"name":"data","mountPath":"/data"}]}]}
    ssa_test.go:66: FieldSet Leaves: 
        .containers[name="main"].image
        .containers[name="main"].name
        .containers[name="main"].volumeMounts[mountPath="/data"].mountPath
        .containers[name="main"].volumeMounts[mountPath="/data"].name
        .volumes[name="data"].emptyDir
        .volumes[name="data"].name
    ssa_test.go:70: Extracted JSON: 
        {"containers":[{"image":"nginx:latest","name":"main","volumeMounts":[{"mountPath":"/data","name":"data"}]}],"volumes":[{"emptyDir":null,"name":"data"}]}
    ssa_test.go:75: Diff: 
        - Modified Fields:
        .volumes[name="data"].emptyDir
    ssa_test.go:76: The extracted TypedValue differs from the original.
--- FAIL: TestStructuredMergeDiffExtraction_Pod_v4 (0.01s)
FAIL
FAIL    github.com/niledatabase/operator/pkg/client/applyconfigurations/internal        0.024s
FAIL

Previously, ExtractItems converted empty collections ({} and []) to null.
This change preserves the distinction between null and empty values.
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 17, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: mrIncompetent
Once this PR has been reviewed and has the lgtm label, please assign jpbetz for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot requested review from apelisse and yliaog June 17, 2025 18:04
@k8s-ci-robot
Copy link
Contributor

Welcome @mrIncompetent!

It looks like this is your first PR to kubernetes-sigs/structured-merge-diff 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/structured-merge-diff has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot
Copy link
Contributor

Hi @mrIncompetent. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 17, 2025
@mrIncompetent
Copy link
Author

cc @jpbetz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants