This repository was archived by the owner on Mar 14, 2024. It is now read-only.
This repository was archived by the owner on Mar 14, 2024. It is now read-only.
grpc traffic allowed when accessControlMode=deny #76
Open
Description
I deployed NSM with accessControlMode=deny, and gRPC still works perfectly.
ACTUAL RESULTS
Transactions work with gRPC, when they should fail.
EXPECTED
I expected that traffic would be forbidden. This is the case with HTTP, but not with gRPC:
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.21.6</center>
</body>
</html>STEPS (General)
- Deploy NSM
- Deploy server application
- Deploy client application
- Redeploy NSM with
accessControlMode=denyand verifykubectl delete --namespace "nginx-mesh" \ $(kubectl get pods \ --namespace "nginx-mesh" \ --selector "app.kubernetes.io/name=nginx-mesh-api" \ --output name) # verify deny nginx-meshctl config | jq -r .accessControlMode
- Attempt gRPC traffic member within service mesh
STEPS (Specific)
These are steps from within my project, but really anything similar should work. I used grpcurl to build this:
URLS=(https://docs.nginx.com/nginx-service-mesh/examples/{prometheus,grafana,otel-collector,jaeger}.yaml)
for URL in ${URLS[*]}; do curl -sOL $URL; done
for FILE in {prometheus,grafana,otel-collector,jaeger}.yaml; do kubectl apply -f $FILE; done
cat << EOF > nsm.yaml
# nsm.yaml
prometheusAddress: prometheus.nsm-monitoring.svc:9090
telemetry:
exporters:
otlp:
host: otel-collector.nsm-monitoring.svc
port: 4317
samplerRatio: 1
tracing: null
mtls:
mode: permissive
autoInjection:
disable: false
disabledNamespaces:
- nsm-monitoring
EOF
cat << EOF > dgraph.yaml
# dgraph.yaml
image:
tag: v21.03.2
alpha:
configFile:
config.yaml: |
security:
whitelist: "0.0.0.0/0"
EOF
cat << EOF > pydgraph.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: pydgraph-client
spec:
replicas: 1
selector:
matchLabels:
app: pydgraph-client
template:
metadata:
labels:
app: pydgraph-client
spec:
containers:
- name: pydgraph-client
image: darknerd/pydgraph-client:latest
env:
- name: DGRAPH_ALPHA_SERVER
value: dgraph-dgraph-alpha.dgraph.svc.cluster.local
resources:
requests:
memory: "64Mi"
cpu: "80m"
limits:
memory: "128Mi"
cpu: "250m"
EOF
###################
# deploy mesj
########################
helm repo add nginx https://helm.nginx.com/stable
helm install nginx-mesh nginx-stable/nginx-service-mesh --values nsm.yaml
###################
# deploy server to mesh
########################
kubectl get namespace "dgraph" > /dev/null 2> /dev/null \
|| kubectl create namespace "dgraph" \
&& kubectl label namespaces "dgraph" name="dgraph"
helm repo add dgraph https://charts.dgraph.io
helm template dgraph dgraph/dgraph --values dgraph.yaml \
| nginx-meshctl inject \
--ignore-incoming-ports 5080,7080 \
--ignore-outgoing-ports 5080,7080 \
| kubectl apply --namespace "dgraph" --filename
###################
# deploy client to mesh
########################
kubectl get namespace "pydgraph-client" > /dev/null 2> /dev/null \
|| kubectl create namespace "pydgraph-client" \
&& kubectl label namespaces "pydgraph-client" name="pydgraph-client"
cat pydgraph-client \
| nginx-meshctl inject \
| kubectl apply --namespace "pydgraph-client" --filename -
###################
# set accessControlMode=deny
########################
cat << EOF > nsm_deny .yaml
# nsm.yaml
prometheusAddress: prometheus.nsm-monitoring.svc:9090
telemetry:
exporters:
otlp:
host: otel-collector.nsm-monitoring.svc
port: 4317
samplerRatio: 1
tracing: null
mtls:
mode: permissive
autoInjection:
disable: false
disabledNamespaces:
- nsm-monitoring
accessControlMode: deny
EOF
# patch configuration
helm install nginx-mesh nginx-stable/nginx-service-mesh --values nsm.yaml
# delete pod using old configuration
kubectl delete --namespace "nginx-mesh" \
$(kubectl get pods --namespace "nginx-mesh" --selector "app.kubernetes.io/name=nginx-mesh-api" --output name)
# few seconds later, verify configuration is now deny
nginx-meshctl config | jq -r .accessControlMode
###################
# exec into client
########################
export CLIENT_NAMESPACE="pydgraph-client"
PYDGRAPH_POD=$(kubectl get pods --namespace $CLIENT_NAMESPACE --output name)
kubectl exec -ti --container "pydgraph-client" --namespace $CLIENT_NAMESPACE ${PYDGRAPH_POD} -- bash
###################
# NOTE: this is run in the pydgraph-client container
########################
# verify HTTP doesn't work (these fail as expected)
curl --silent ${DGRAPH_ALPHA_SERVER}:8080/health
curl --silent ${DGRAPH_ALPHA_SERVER}:8080/state
# verify GRPC doesn't work (these will succeed, not expected)
grpcurl -plaintext -proto api.proto ${DGRAPH_ALPHA_SERVER}:9080 api.Dgraph/CheckVersion
# GRPC using python works as well, should not work:
python3 load_data.py --plaintext --alpha ${DGRAPH_ALPHA_SERVER}:9080 \
--files ./sw.nquads.rdf --schema ./sw.schema