Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove runc drivers #103

Merged
merged 7 commits into from
Feb 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion gremlin/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: gremlin
version: 0.19.1
version: 0.20.0
description: The Gremlin Inc client application
apiVersion: v1
home: https://www.gremlin.com
Expand Down
120 changes: 60 additions & 60 deletions gremlin/README.md

Large diffs are not rendered by default.

3 changes: 0 additions & 3 deletions gremlin/agent_apparmor.profile
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,9 @@ profile gremlin-agent flags=(attach_disconnected,mediate_deleted) {
/dev/null rwlix,

# Container runtime
/run/docker/runtime-runc/moby rwix,
/var/run/docker.sock rwix,
/run/crio/crio.sock rwix,
/run/runc rwix,
/run/containers/containers.sock rwix,
/run/containerd/runc/k8s.io rwix,

# We need access to Pid 1's real pid to resolve the container driver
# we're just taking a read perm here to accomlish this
Expand Down
12 changes: 4 additions & 8 deletions gremlin/examples/drivers/README.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,14 @@
# Gremlin's recommended container drivers

Gremlin has 3 different recommended container drivers: `docker-runc`, `crio-runc`, and `containerd-runc`, each of which
Gremlin has 3 different recommended container drivers: `docker-linux`, `crio-linux`, and `containerd-linux`, each of which
is used to integrate with Docker, Cri-O, and Containerd respectively.

In order to preserve the behaviors of previous versions of this chart, none of the above drivers are enabled by
default. Instead, Gremlin will try to run under the legacy `docker` driver, which has [some limitations][cgroup-driver].

You also have the option of specifying `any` which will mount the locations for all supported drivers and then gremlin
will attempt to determine the correct one. This can be a good option if you don't know which driver you have
will attempt to determine the correct one. This can be a good option if you don't know which driver you have.

## Requirements

In order to use one of the recommended container drivers, you must run the Gremlin Daemonset in the host's PID namespace,
so `gremlin.hostPID=true`.
In order to use one of the recommended container drivers, you must run the Gremlin Daemonset in the host's PID and network namespaces: `gremlin.hostPID=true`, and `gremlin.hostNetwork=true`. These are both `true` by default.

## Usage

Expand All @@ -22,7 +18,7 @@ To use one of the recommended container drivers, set the name in `gremlin.contai
helm install gremlin gremlin/gremlin \
--namespace gremlin \
--set gremlin.hostPID=true \
--set gremlin.container.driver=crio-runc \
--set gremlin.container.driver=crio-linux \
--set gremlin.secret.managed=true \
--set gremlin.secret.teamID=$GREMLIN_TEAM_ID \
--set gremlin.secret.clusterID=$GREMLIN_CLUSTER_ID \
Expand Down
32 changes: 7 additions & 25 deletions gremlin/templates/_helpers.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -64,26 +64,24 @@ Create a computed value for the intended Gremlin secret type which can either be

{{- define "containerDriverWithDefaultOrError" -}}
{{- if .Values.gremlin.container.driver -}}
{{- $valid := list "docker" "docker-runc" "crio-runc" "containerd-runc" "any" "docker-linux" "containerd-linux" "crio-linux" "linux" -}}
{{- $valid := list "any" "docker-linux" "containerd-linux" "crio-linux" "linux" -}}
{{- if has .Values.gremlin.container.driver $valid -}}
{{- .Values.gremlin.container.driver -}}
{{- else -}}
{{- fail (printf "unknown container driver: %s (must be one of %s)" .Values.gremlin.container.driver (join ", " $valid)) -}}
{{- end -}}
{{- else -}}
{{- "docker" -}}
{{- "docker-linux" -}}
{{- end -}}
{{- end -}}

{{- define "containerMounts" -}}
{{- $selectedDriver := (include "containerDriverWithDefaultOrError" .) -}}
{{- $mountPaths := (dict "docker-runc" (dict "name" "docker" "socket" "/var/run/docker.sock" "runc" "/run/docker/runtime-runc/moby") "docker" (dict "name" "docker" "socket" "/var/run/docker.sock") "crio-runc" (dict "name" "crio" "socket" "/run/crio/crio.sock" "runc" "/run/runc") "containerd-runc" (dict "name" "containerd" "socket" "/run/containerd/containerd.sock" "runc" "/run/containerd/runc/k8s.io") "docker-linux" (dict "name" "docker" "socket" "/var/run/docker.sock") "containerd-linux" (dict "name" "containerd" "socket" "/run/containerd/containerd.sock") "crio-linux" (dict "name" "crio" "socket" "/run/crio/crio.sock")) -}}
{{- $mountPaths := (dict "docker-linux" (dict "name" "docker" "socket" "/var/run/docker.sock") "containerd-linux" (dict "name" "containerd" "socket" "/run/containerd/containerd.sock") "crio-linux" (dict "name" "crio" "socket" "/run/crio/crio.sock")) -}}
{{- range $key, $val := .Values.containerDrivers -}}
{{- /* create a list of values to match against customer selection */ -}}
{{- /* this is the current driver or all drivers in the case of "any" */ -}}
{{- /* to prevent docker from apearing twice tho, we remove any from the valid */ -}}
{{- /* list just for the key "docker" */ -}}
{{- $validDrivers := (ternary (list $key) (list $key "any") (or (eq $key "docker") (hasSuffix "linux" $key))) }}
{{- $validDrivers := (ternary (list $key) (list $key "any") (or (hasSuffix "linux" $key))) }}
{{- if hasSuffix "-linux" $key -}}
{{- $validDrivers = append $validDrivers "linux" -}}
{{- end -}}
Expand All @@ -93,24 +91,17 @@ Create a computed value for the intended Gremlin secret type which can either be
mountPath: {{ (get $mountPaths $key).socket }}
readOnly: true
{{- end -}}
{{- if $val.runtimeRunc }}
- name: {{ $val.name }}-runc
mountPath: {{ (get $mountPaths $key).runc }}
readOnly: false
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{- define "containerMountsPSP" -}}
{{- $selectedDriver := (include "containerDriverWithDefaultOrError" .) -}}
{{- $mountPaths := (dict "docker-runc" (dict "name" "docker" "socket" "/var/run/docker.sock" "runc" "/run/docker/runtime-runc/moby") "docker" (dict "name" "docker" "socket" "/var/run/docker.sock") "crio-runc" (dict "name" "crio" "socket" "/run/crio/crio.sock" "runc" "/run/runc") "containerd-runc" (dict "name" "containerd" "socket" "/run/containerd/containerd.sock" "runc" "/run/containerd/runc/k8s.io") "docker-linux" (dict "name" "docker" "socket" "/var/run/docker.sock") "containerd-linux" (dict "name" "containerd" "socket" "/run/containerd/containerd.sock") "crio-linux" (dict "name" "crio" "socket" "/run/crio/crio.sock")) -}}
{{- $mountPaths := (dict "docker-linux" (dict "name" "docker" "socket" "/var/run/docker.sock") "containerd-linux" (dict "name" "containerd" "socket" "/run/containerd/containerd.sock") "crio-linux" (dict "name" "crio" "socket" "/run/crio/crio.sock")) -}}
{{- range $key, $val := .Values.containerDrivers -}}
{{- /* create a list of values to match against customer selection */ -}}
{{- /* this is the current driver or all drivers in the case of "any" */ -}}
{{- /* to prevent docker from apearing twice tho, we remove any from the valid */ -}}
{{- /* list just for the key "docker" */ -}}
{{- $validDrivers := (ternary (list $key) (list $key "any") (or (eq $key "docker") (hasSuffix "linux" $key))) }}
{{- $validDrivers := (ternary (list $key) (list $key "any") (or (hasSuffix "linux" $key))) }}
{{- if hasSuffix "-linux" $key -}}
{{- $validDrivers = append $validDrivers "linux" -}}
{{- end -}}
Expand All @@ -119,18 +110,14 @@ Create a computed value for the intended Gremlin secret type which can either be
- pathPrefix: {{ (get $mountPaths $key).socket }}
readOnly: true
{{- end -}}
{{- if $val.runtimeRunc }}
- pathPrefix: {{ (get $mountPaths $key).runc }}
readOnly: false
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}

{{- define "containerVolumes" -}}
{{- $selectedDriver := (include "containerDriverWithDefaultOrError" .) -}}
{{- range $key, $val := .Values.containerDrivers -}}
{{- $validDrivers := (ternary (list $key) (list $key "any") (or (eq $key "docker") (hasSuffix "linux" $key))) }}
{{- $validDrivers := (ternary (list $key) (list $key "any") (or (hasSuffix "linux" $key))) }}
{{- if hasSuffix "-linux" $key -}}
{{- $validDrivers = append $validDrivers "linux" -}}
{{- end -}}
Expand All @@ -140,11 +127,6 @@ Create a computed value for the intended Gremlin secret type which can either be
hostPath:
path: {{ $val.runtimeSocket }}
{{- end -}}
{{- if $val.runtimeRunc }}
- name: {{ $val.name }}-runc
hostPath:
path: {{ $val.runtimeRunc }}
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}
Expand Down
2 changes: 0 additions & 2 deletions gremlin/templates/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,6 @@ spec:
fieldPath: spec.nodeName
- name: GREMLIN_CLIENT_TAGS
value: {{ .Values.gremlin.client.tags }}
- name: GREMLIN_DOCKER_IMAGE
value: {{ .Values.image.repository }}:{{ .Values.image.tag }}
- name: GREMLIN_COLLECT_DNS
value: {{ .Values.gremlin.collect.dns | quote }}
- name: GREMLIN_SERVICE_URL
Expand Down
75 changes: 12 additions & 63 deletions gremlin/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,8 @@ gremlin:
dns: true

# gremlin.hostPID -
# This must be true for all Gremlin container drivers exept for the legacy `docker` driver. It is also required
# for any Gremlin installation that wishes to carry out Process Killer or attacks at the host-level
# This must be true for all Gremlin container drivers. It is also required for any Gremlin
# installation that wishes to carry out Process Killer or attacks at the host-level
#
hostPID: true

Expand All @@ -78,38 +78,21 @@ gremlin:
container:
# gremlin.container.driver -
# Specifies which container driver with which to run Gremlin. Possible values and their meanings:
# docker = Provides legacy integration with Docker containers
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker is still in the dictionary in the helpers template, and container drivers list

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch

# Implies host mounts [/var/run/docker.sock]
# Does not support the systemd cgroup driver
#
# docker-runc = Provides full integration with Docker containers
# Implies host mounts [/var/run/docker.sock, /run/docker/runtime-runc/moby]
#
# crio-runc = Provides full integration with Cri-O containers
# Implies host mounts [/run/crio/crio.sock, /run/runc]
#
# containerd-runc = Provides full integration with Containerd containers
# Implies host mounts [/run/containers/containers.sock, /run/containerd/runc/k8s.io]
#
# any = Useful when the Cluster's container runtime is not known.
# Helm will provide file access to all drivers so that Gremlin can choose.
# Implies host mounts [/var/run/docker.sock, /run/docker/runtime-runc/moby,
# /run/crio/crio.sock, /run/runc,
# /run/containers/containers.sock, /run/containerd/runc/k8s.io]
# Implies host mounts [/var/run/docker.sock, /run/crio/crio.sock, /run/containers/containers.sock]
#
# docker-linux = Provides full integration with Docker containers without runc
# docker-linux = Provides full integration with Docker containers
# Implies host mounts [/var/run/docker.sock]
#
# containerd-linux = Provides full integration with Containerd containers without runc
# containerd-linux = Provides full integration with Containerd containers
# Implies host mounts [/run/containers/containers.sock]
#
# crio-linux = Provides full integration with Cri-O containers without runc
# crio-linux = Provides full integration with Cri-O containers
# Implies host mounts [/run/crio/crio.sock]
#
# linux = Useful when the Cluster's container runtime is not known.
# Helm will provide file access to all linux drivers so that Gremlin can choose.
# Implies host mounts [/var/run/docker.sock, /run/crio/crio.sock,
# /run/containers/containers.sock]
# linux = Alias for `any`
# Implies host mounts [/var/run/docker.sock, /run/crio/crio.sock, /run/containers/containers.sock]
driver: linux

cgroup:
Expand Down Expand Up @@ -142,30 +125,11 @@ gremlin:
- NET_ADMIN # Required to run network attacks
- SYS_BOOT # Required to run Shutdown attacks
- SYS_TIME # Required to run Time Travel attacks
- DAC_READ_SEARCH # Required to run Certificate Expiry attacks, and dependency discovery features
- DAC_READ_SEARCH # Required to run Certificate Expiry attacks with CIDR address arguments, and to discover dependencies
- SYS_RESOURCE # Required to run Process Exhaustion attacks against containers

- SYS_ADMIN # Required by container drivers: docker-runc, crio-runc, containerd-runc
# to run attacks against running containers

- SYS_PTRACE # Required by container drivers: docker-runc, crio-runc, containerd-runc
# to determine if Gremlin is in the host's pid namespace

- SETFCAP # Required by container drivers: docker-runc, crio-runc, containerd-runc
# to set capabilities on Gremlin attack sidecars

- AUDIT_WRITE # Required by container drivers: docker-runc, crio-runc, containerd-runc
# to write to the Kernel's audit log

- MKNOD # Required by container drivers: docker-runc, crio-runc, containerd-runc
# to create new devices for Gremlin attack sidecars

- SYS_CHROOT # Required by container drivers: docker-runc, crio-runc, containerd-runc
# to create and enter new namespaces for Gremlin attack sidecars

- NET_RAW # Required by container drivers: docker-runc, crio-runc, containerd-runc
# Not actively used by Gremlin but requested by sidecars
# This capability will be removed in a later release
- SYS_ADMIN # Required to apply impact within container namespaces
- SYS_PTRACE # Required to to identify charactistics of container processes (such as associated network)
- NET_RAW # Required to discover dependencies

# gremlin.podSecurity.seLinuxOptions -
# Specifies SELinux options to apply to the Gremlin Daemonset container securityContext.
Expand Down Expand Up @@ -349,21 +313,6 @@ ssl:
certDir:

containerDrivers:
docker-runc:
runtimeSocket: "/var/run/docker.sock"
runtimeRunc: "/run/docker/runtime-runc/moby"
name: "docker"
containerd-runc:
runtimeSocket: "/run/containerd/containerd.sock"
runtimeRunc: "/run/containerd/runc/k8s.io"
name: "containerd"
crio-runc:
runtimeSocket: "/run/crio/crio.sock"
runtimeRunc: "/run/runc"
name: "crio"
docker:
runtimeSocket: "/var/run/docker.sock"
name: "docker"
docker-linux:
runtimeSocket: "/var/run/docker.sock"
name: "docker"
Expand Down