Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for more K8s resources #456

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 95 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Reflector
Reflector is a Kubernetes addon designed to monitor changes to resources (secrets and configmaps) and reflect changes to mirror resources in the same or other namespaces.
Reflector is a Kubernetes addon designed to monitor changes to resources (secrets, configmaps, service accounts, roles and rolebindings) and reflect changes to mirror resources in the same or other namespaces.

[![Pipeline](https://github.com/emberstack/kubernetes-reflector/actions/workflows/pipeline.yaml/badge.svg)](https://github.com/emberstack/kubernetes-reflector/actions/workflows/pipeline.yaml)
[![Release](https://img.shields.io/github/release/emberstack/kubernetes-reflector.svg?style=flat-square)](https://github.com/emberstack/kubernetes-reflector/releases/latest)
Expand Down Expand Up @@ -70,7 +70,7 @@ $ kubectl -n kube-system apply -f https://github.com/emberstack/kubernetes-refle

## Usage

### 1. Annotate the source `secret` or `configmap`
### 1. Annotate the source resource

- Add `reflector.v1.k8s.emberstack.com/reflection-allowed: "true"` to the resource annotations to permit reflection to mirrors.
- Add `reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "<list>"` to the resource annotations to permit reflection from only the list of comma separated namespaces or regular expressions. Note: If this annotation is omitted or is empty, all namespaces are allowed.
Expand Down Expand Up @@ -110,7 +110,60 @@ $ kubectl -n kube-system apply -f https://github.com/emberstack/kubernetes-refle
...
```

### 2. Annotate the mirror secret or configmap
Example source service account:
```yaml
piVersion: v1
kind: ServiceAccount
metadata:
name: source-service-account
namespace: default
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: 'true'
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "namespace-1,namespace-2,namespace-[0-9]*"
...
```
Example source role:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: source-role
namespace: default
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "namespace-1,namespace-2,namespace-[0-9]*"
rules:
- verbs:
- '*'
apiGroups:
- '*'
resources:
- '*'
...
```
Example source rolebindings:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: source-role-binding
namespace: default
annotations:
reflector.v1.k8s.emberstack.com/reflection-allowed: "true"
reflector.v1.k8s.emberstack.com/reflection-allowed-namespaces: "namespace-1,namespace-2,namespace-[0-9]*"
...
subjects:
- kind: ServiceAccount
name: source-service-account
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: source-role
```


### 2. Annotate the mirror resource

- Add `reflector.v1.k8s.emberstack.com/reflects: "<source namespace>/<source name>"` to the mirror object. The value of the annotation is the full name of the source object in `namespace/name` format.

Expand Down Expand Up @@ -140,16 +193,54 @@ $ kubectl -n kube-system apply -f https://github.com/emberstack/kubernetes-refle
...
```

Example mirror service account:
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: mirror-service-account
annotations:
reflector.v1.k8s.emberstack.com/reflects: "default/source-service-account"
data:
...
```

Example mirror role:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: mirror-role
annotations:
reflector.v1.k8s.emberstack.com/reflects: "default/source-role"
data:
...
```

Example mirror rolebinding:
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: mirror-role-binding
annotations:
reflector.v1.k8s.emberstack.com/reflects: "default/source-role-binding"
data:
...
```

### 3. Done!
Reflector will monitor any changes done to the source objects and copy the following fields:
- `data` for secrets
- `data` and `binaryData` for configmaps
- nothing for service accounts
- `rules` for roles
- `subjects` only for rolebindings, as `roleRef` is immutable
Reflector keeps track of what was copied by annotating mirrors with the source object version.

- - - -



## `cert-manager` support

> Since version 1.5 of cert-manager you can annotate secrets created from certificates for mirroring using `secretTemplate` (see https://cert-manager.io/docs/usage/certificate/).
Expand Down
5 changes: 3 additions & 2 deletions src/ES.Kubernetes.Reflector/Core/Mirroring/ResourceMirror.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using System.Collections.Concurrent;
using System.Collections.Concurrent;
using System.Net;
using ES.Kubernetes.Reflector.Core.Extensions;
using ES.Kubernetes.Reflector.Core.Json;
Expand All @@ -13,6 +13,7 @@
using Microsoft.AspNetCore.JsonPatch;
using Microsoft.AspNetCore.JsonPatch.Operations;
using Newtonsoft.Json;
using CloudNimble.EasyAF.NewtonsoftJson.Compatibility;

namespace ES.Kubernetes.Reflector.Core.Mirroring;

Expand Down Expand Up @@ -478,7 +479,7 @@ private async Task ResourceReflect(KubeRef sourceId, KubeRef targetId, TResource

await OnResourceConfigurePatch(source, patchDoc);

var patch = JsonConvert.SerializeObject(patchDoc, Formatting.Indented);
var patch = JsonConvert.SerializeObject(patchDoc, Formatting.Indented, new JsonSerializerSettings { ContractResolver = new SystemTextJsonContractResolver() });
await OnResourceApplyPatch(new V1Patch(patch, V1Patch.PatchType.JsonPatch), targetId);
Logger.LogInformation("Patched {id} as a reflection of {sourceId}", targetId, sourceId);
}
Expand Down
61 changes: 61 additions & 0 deletions src/ES.Kubernetes.Reflector/Core/RoleBindingMirror.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
using ES.Kubernetes.Reflector.Core.Mirroring;
using ES.Kubernetes.Reflector.Core.Resources;
using k8s;
using k8s.Models;
using Microsoft.AspNetCore.JsonPatch;
using System.Text.Json;
using System.Diagnostics;

namespace ES.Kubernetes.Reflector.Core;

public class RoleBindingMirror : ResourceMirror<V1RoleBinding>
{
public RoleBindingMirror(ILogger<RoleBindingMirror> logger, IKubernetes client) : base(logger, client)
{
}

protected override async Task<V1RoleBinding[]> OnResourceWithNameList(string itemRefName)
{
return (await Client.RbacAuthorizationV1.ListRoleBindingForAllNamespacesAsync(fieldSelector: $"metadata.name={itemRefName}")).Items
.ToArray();
}

protected override Task OnResourceApplyPatch(V1Patch patch, KubeRef refId)
{
return Client.RbacAuthorizationV1.PatchNamespacedRoleBindingWithHttpMessagesAsync(patch, refId.Name, refId.Namespace);
}

protected override Task OnResourceConfigurePatch(V1RoleBinding source, JsonPatchDocument<V1RoleBinding> patchDoc)
{
// Roleref is immutable by design, so we only patch the Subjects list
patchDoc.Replace(e => e.Subjects, source.Subjects);
return Task.CompletedTask;
}

protected override Task OnResourceCreate(V1RoleBinding item, string ns)
{
item.Metadata.ResourceVersion = null;
return Client.RbacAuthorizationV1.CreateNamespacedRoleBindingAsync(item, ns);
}

protected override Task<V1RoleBinding> OnResourceClone(V1RoleBinding sourceResource)
{
return Task.FromResult(new V1RoleBinding
{
ApiVersion = sourceResource.ApiVersion,
Kind = sourceResource.Kind,
Subjects = sourceResource.Subjects,
RoleRef = sourceResource.RoleRef
});
}

protected override Task OnResourceDelete(KubeRef resourceId)
{
return Client.RbacAuthorizationV1.DeleteNamespacedRoleBindingAsync(resourceId.Name, resourceId.Namespace);
}

protected override Task<V1RoleBinding> OnResourceGet(KubeRef refId)
{
return Client.RbacAuthorizationV1.ReadNamespacedRoleBindingAsync(refId.Name, refId.Namespace);
}
}
24 changes: 24 additions & 0 deletions src/ES.Kubernetes.Reflector/Core/RoleBindingWatcher.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
using ES.Kubernetes.Reflector.Core.Configuration;
using ES.Kubernetes.Reflector.Core.Watchers;
using k8s;
using k8s.Autorest;
using k8s.Models;
using MediatR;
using Microsoft.Extensions.Options;

namespace ES.Kubernetes.Reflector.Core;

public class RoleBindingWatcher : WatcherBackgroundService<V1RoleBinding, V1RoleBindingList>
{
public RoleBindingWatcher(ILogger<RoleBindingWatcher> logger, IMediator mediator, IKubernetes client,
IOptionsMonitor<ReflectorOptions> options) :
base(logger, mediator, client, options)
{
}

protected override Task<HttpOperationResponse<V1RoleBindingList>> OnGetWatcher(CancellationToken cancellationToken)
{
return Client.RbacAuthorizationV1.ListRoleBindingForAllNamespacesWithHttpMessagesAsync(watch: true, timeoutSeconds: WatcherTimeout,
cancellationToken: cancellationToken);
}
}
60 changes: 60 additions & 0 deletions src/ES.Kubernetes.Reflector/Core/RoleMirror.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
using ES.Kubernetes.Reflector.Core.Mirroring;
using ES.Kubernetes.Reflector.Core.Resources;
using k8s;
using k8s.Models;
using Microsoft.AspNetCore.JsonPatch;
using System.Text.Json;

namespace ES.Kubernetes.Reflector.Core;

public class RoleMirror : ResourceMirror<V1Role>
{
public RoleMirror(ILogger<RoleMirror> logger, IKubernetes client) : base(logger, client)
{
}

protected override async Task<V1Role[]> OnResourceWithNameList(string itemRefName)
{
return (await Client.RbacAuthorizationV1.ListRoleForAllNamespacesAsync(fieldSelector: $"metadata.name={itemRefName}")).Items
.ToArray();
}

protected override Task OnResourceApplyPatch(V1Patch patch, KubeRef refId)
{
return Client.RbacAuthorizationV1.PatchNamespacedRoleWithHttpMessagesAsync(patch, refId.Name, refId.Namespace);
}

protected override Task OnResourceConfigurePatch(V1Role source, JsonPatchDocument<V1Role> patchDoc)
{
// Replace with new List of Rules
patchDoc.Replace(e => e.Rules, source.Rules);
return Task.CompletedTask;
}

protected override Task OnResourceCreate(V1Role item, string ns)
{
item.Metadata.ResourceVersion = null;
return Client.RbacAuthorizationV1.CreateNamespacedRoleAsync(item, ns);
}

protected override Task<V1Role> OnResourceClone(V1Role sourceResource)
{
return Task.FromResult(new V1Role
{
ApiVersion = sourceResource.ApiVersion,
Kind = sourceResource.Kind,
Rules = sourceResource.Rules
});
}

protected override Task OnResourceDelete(KubeRef resourceId)
{
return Client.RbacAuthorizationV1.DeleteNamespacedRoleAsync(resourceId.Name, resourceId.Namespace);
}

protected override Task<V1Role> OnResourceGet(KubeRef refId)
{
return Client.RbacAuthorizationV1.ReadNamespacedRoleAsync(refId.Name, refId.Namespace);
}

}
24 changes: 24 additions & 0 deletions src/ES.Kubernetes.Reflector/Core/RoleWatcher.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
using ES.Kubernetes.Reflector.Core.Configuration;
using ES.Kubernetes.Reflector.Core.Watchers;
using k8s;
using k8s.Autorest;
using k8s.Models;
using MediatR;
using Microsoft.Extensions.Options;

namespace ES.Kubernetes.Reflector.Core;

public class RoleWatcher : WatcherBackgroundService<V1Role, V1RoleList>
{
public RoleWatcher(ILogger<RoleWatcher> logger, IMediator mediator, IKubernetes client,
IOptionsMonitor<ReflectorOptions> options) :
base(logger, mediator, client, options)
{
}

protected override Task<HttpOperationResponse<V1RoleList>> OnGetWatcher(CancellationToken cancellationToken)
{
return Client.RbacAuthorizationV1.ListRoleForAllNamespacesWithHttpMessagesAsync(watch: true, timeoutSeconds: WatcherTimeout,
cancellationToken: cancellationToken);
}
}
56 changes: 56 additions & 0 deletions src/ES.Kubernetes.Reflector/Core/ServiceAccountMirror.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
using ES.Kubernetes.Reflector.Core.Mirroring;
using ES.Kubernetes.Reflector.Core.Resources;
using k8s;
using k8s.Models;
using Microsoft.AspNetCore.JsonPatch;

namespace ES.Kubernetes.Reflector.Core;

public class ServiceAccountMirror : ResourceMirror<V1ServiceAccount>
{
public ServiceAccountMirror(ILogger<ServiceAccountMirror> logger, IKubernetes client) : base(logger, client)
{
}

protected override async Task<V1ServiceAccount[]> OnResourceWithNameList(string itemRefName)
{
return (await Client.CoreV1.ListServiceAccountForAllNamespacesAsync(fieldSelector: $"metadata.name={itemRefName}")).Items
.ToArray();
}

protected override Task OnResourceApplyPatch(V1Patch patch, KubeRef refId)
{
return Client.CoreV1.PatchNamespacedServiceAccountWithHttpMessagesAsync(patch, refId.Name, refId.Namespace);
}

protected override Task OnResourceConfigurePatch(V1ServiceAccount source, JsonPatchDocument<V1ServiceAccount> patchDoc)
{
// Just update annotations.
return Task.CompletedTask;
}

protected override Task OnResourceCreate(V1ServiceAccount item, string ns)
{
item.Metadata.ResourceVersion = null;
return Client.CoreV1.CreateNamespacedServiceAccountAsync(item, ns);
}

protected override Task<V1ServiceAccount> OnResourceClone(V1ServiceAccount sourceResource)
{
return Task.FromResult(new V1ServiceAccount
{
ApiVersion = sourceResource.ApiVersion,
Kind = sourceResource.Kind
});
}

protected override Task OnResourceDelete(KubeRef resourceId)
{
return Client.CoreV1.DeleteNamespacedServiceAccountAsync(resourceId.Name, resourceId.Namespace);
}

protected override Task<V1ServiceAccount> OnResourceGet(KubeRef refId)
{
return Client.CoreV1.ReadNamespacedServiceAccountAsync(refId.Name, refId.Namespace);
}
}
Loading