Skip to content

Remove note in TAR docs#2097

Open
GTRekter wants to merge 1 commit intolinkerd:mainfrom
GTRekter:main
Open

Remove note in TAR docs#2097
GTRekter wants to merge 1 commit intolinkerd:mainfrom
GTRekter:main

Conversation

@GTRekter
Copy link
Contributor

@GTRekter GTRekter commented Feb 27, 2026

The documentation mention that it does not support trafficDistribution, but we do support it.

Summary

If we enable spec.trafficDistribution for a Service, Kubernetes will populate hints.forZones on the corresponding EndpointSlice endpoints. The Destination controller watches these Services resources and processes keep track of the hints. (Source code).
At that point, endpointTranslator kicks in, specifically filterAddresses, and processes endpoints as follows:

  • If this is endpoint discovery for a remote cluster, it returns all endpoints.
  • If this is endpoint discovery for the local cluster, it applies these filters:
  • If the Service has .spec.internalTrafficPolicy: Local, it returns only endpoints on the same node. (Source code)
  • If any endpoint address does not have a hint, then all hints are ignored and all available addresses are returned. (Source code)
  • If the addresses have hints (i.e., TAR is enabled), it returns only addresses whose hint matches the node’s zone (topology.kubernetes.io/zone). (Source code)
  • As a fallback, if the result is empty after filtering, it returns all endpoints. (Source code)

Reproduction

Consider the following setup:
A Fortio client generating traffic (10 concurrent connections, 100 QPS) deployed on a node in zone apne2-az1.

apiVersion: v1
kind: Pod
metadata:
  name: fortio
  namespace: simple-app
  annotations:
    linkerd.io/inject: enabled
spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
          - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                  - apne2-az1
  containers:
  - name: fortio
    image: fortio/fortio:latest
    imagePullPolicy: IfNotPresent
    args: [
        "load", 
        "-t", "0", 
        "-qps", "100", 
        "-c", "10",
        "-payload-size", "10000",
        "http://simple-app-v1.simple-app.svc.cluster.local/"
      ]  

A simple HTTP echo server with 5 replicas, spread across all (zones) nodes via podAntiAffinity. The Service uses trafficDistribution: PreferSameZone.

apiVersion: apps/v1 
kind: Deployment 
metadata: 
  name: simple-app-v1 
  namespace: simple-app 
spec: 
  replicas: 5 
  selector: 
    matchLabels: 
      app: simple-app-v1 
      version: v1 
  template: 
    metadata: 
      labels: 
        app: simple-app-v1 
        version: v1 
    spec: 
      affinity: 
        podAntiAffinity: 
          requiredDuringSchedulingIgnoredDuringExecution: 
            - labelSelector: 
                matchLabels: 
                  app: simple-app-v1 
                  version: v1 
              topologyKey: kubernetes.io/hostname 
      containers: 
        - name: http-app 
          image: hashicorp/http-echo:latest 
          args: 
            - "-text=Simple App v1 - CLUSTER_NAME" 
          ports: 
            - containerPort: 5678 
---
apiVersion: v1 
kind: Service 
metadata: 
  name: simple-app-v1 
  namespace: simple-app 
spec: 
  trafficDistribution: PreferSameZone 
  selector: 
    app: simple-app-v1 
    version: v1 
  ports: 
    - port: 80 
      targetPort: 5678 

The resulting pods look like:

kubectl get pods -A -o wide
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE     IP          NODE              NOMINATED NODE   READINESS GATES
(linkerd components)
...
simple-app    fortio                                    2/2     Running   0          56s     10.23.4.9   k3d-01-agent-0    <none>           <none>
simple-app    simple-app-v1-658f5cffc4-7v4l8            2/2     Running   0          3m47s   10.23.6.2   k3d-01-agent-3    <none>           <none>
simple-app    simple-app-v1-658f5cffc4-9xk75            2/2     Running   0          3m47s   10.23.2.2   k3d-01-agent-4    <none>           <none>
simple-app    simple-app-v1-658f5cffc4-blnc4            2/2     Running   0          3m47s   10.23.1.2   k3d-01-agent-2    <none>           <none>
simple-app    simple-app-v1-658f5cffc4-fhsnf            2/2     Running   0          2m55s   10.23.4.7   k3d-01-agent-0    <none>           <none>
simple-app    simple-app-v1-658f5cffc4-tbtfd            2/2     Running   0          2m56s   10.23.5.7   k3d-01-agent-1    <none>           <none>

You can also see that the EndpointSlice is populated with hints.forZones by kubernetes, which Linkerd destination will later consume:

kubectl -n simple-app get endpointslice -n simple-app simple-app-v1-6rbnc  -o yaml
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
  name: simple-app-v1-6rbnc
  namespace: simple-app
addressType: IPv4
ports:
- name: ""
  port: 5678
  protocol: TCP
endpoints:
- addresses:
  - 10.23.1.2
  conditions:
    ready: true
    serving: true
    terminating: false
  hints:
    forZones:
    - name: apne2-az3
  nodeName: k3d-01-agent-2
  targetRef:
    kind: Pod
    name: simple-app-v1-658f5cffc4-blnc4
    namespace: simple-app
    uid: 1f147c2d-3386-4d51-a7da-cc534b9124c8
  zone: apne2-az3
- addresses:
  - 10.23.2.2
  conditions:
    ready: true
    serving: true
    terminating: false
  hints:
    forZones:
    - name: apne2-az5
  nodeName: k3d-01-agent-4
  targetRef:
    kind: Pod
    name: simple-app-v1-658f5cffc4-9xk75
    namespace: simple-app
    uid: b0d192b3-e0a5-46d2-acb5-f5909144cba4
  zone: apne2-az5
- addresses:
  - 10.23.6.2
  conditions:
    ready: true
    serving: true
    terminating: false
  hints:
    forZones:
    - name: apne2-az4
  nodeName: k3d-01-agent-3
  targetRef:
    kind: Pod
    name: simple-app-v1-658f5cffc4-7v4l8
    namespace: simple-app
    uid: bc44030c-1e08-4d8a-a527-c2ef3bfdc1a6
  zone: apne2-az4
- addresses:
  - 10.23.4.7
  conditions:
    ready: true
    serving: true
    terminating: false
  hints:
    forZones:
    - name: apne2-az1
  nodeName: k3d-01-agent-0
  targetRef:
    kind: Pod
    name: simple-app-v1-658f5cffc4-fhsnf
    namespace: simple-app
    uid: 265c18b1-1110-4278-b34c-4f340d9da1fc
  zone: apne2-az1
- addresses:
  - 10.23.5.7
  conditions:
    ready: true
    serving: true
    terminating: false
  hints:
    forZones:
    - name: apne2-az2
  nodeName: k3d-01-agent-1
  targetRef:
    kind: Pod
    name: simple-app-v1-658f5cffc4-tbtfd
    namespace: simple-app
    uid: 2e64f1b6-6878-42ca-9b81-742d4677b9f2
  zone: apne2-az2

If we check the Destination logs, we can see we are hitting the “hints” path in endpointTranslator:

kubectl logs -n linkerd linkerd-destination-b9c8c6669-4jzxv -c destination | grep "Filtering through addresses"
...
time="2026-02-26T11:47:53Z" level=debug msg="Filtering through addresses that should be consumed by zone apne2-az1" addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=curl-test remote="10.23.0.8:34878" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:49:09Z" level=debug msg="Filtering through addresses that should be consumed by zone apne2-az1" addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=curl-test remote="10.23.0.8:34878" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:49:55Z" level=debug msg="Filtering through addresses that should be consumed by zone apne2-az1" addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=curl-test remote="10.23.0.8:34878" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:53:50Z" level=debug msg="Filtering through addresses that should be consumed by zone apne2-az1" addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=fortio remote="10.23.0.8:34878" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:58:50Z" level=debug msg="Filtering through addresses that should be consumed by zone apne2-az1" addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=fortio remote="10.23.0.8:34878" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T15:45:09Z" level=debug msg="Filtering through addresses that should be consumed by zone apne2-az1" addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=fortio remote="10.23.0.8:34878" service="simple-app-v1.simple-app.svc.cluster.local:80"

For convenience, I used the Linkerd Dashboard to visualize the traffic:

helm upgrade -i -n monitoring --create-namespace linkerd-dashboard oci://ghcr.io/buoyantio/charts/linkerd-dashboard

Even with 10 parallel connections at 100 QPS, all traffic flows to simple-app-v1-658f5cffc4-fhsnf, which is deployed in the same zone as the client (apne2-az1).

If we remove trafficDistribution: PreferSameZone from the Service, the zone becomes empty in Destination logs and endpointTranslator falls back to returning all endpoints. In this case (because I enabled and then disabled trafficDistribution), the logs show:

kubectl logs -n linkerd linkerd-destination-b9c8c6669-4jzxv -c destination | grep "Filtering through addresses"
time="2026-02-26T11:28:42Z" level=debug msg="Filtering through addresses that should be consumed by zone " addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=traffic-ffb7b6c8c-bgpm6 remote="10.23.0.8:40752" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:28:45Z" level=debug msg="Filtering through addresses that should be consumed by zone " addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=traffic-ffb7b6c8c-bgpm6 remote="10.23.0.8:40752" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:28:48Z" level=debug msg="Filtering through addresses that should be consumed by zone " addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=traffic-ffb7b6c8c-bgpm6 remote="10.23.0.8:40752" service="simple-app-v1.simple-app.svc.cluster.local:80"
time="2026-02-26T11:28:51Z" level=debug msg="Filtering through addresses that should be consumed by zone " addr=":8086" component=endpoint-translator context-ns=simple-app context-pod=traffic-ffb7b6c8c-bgpm6 remote="10.23.0.8:40752" service="simple-app-v1.simple-app.svc.cluster.local:80 (http://simple-app-v1.simple-app.svc.cluster.local:80)"

As result the traffic is spread across all available endpoints.

Reference:

Signed-off-by: Ivan Porta <porta.ivan@outlook.com>
@travisbeckham
Copy link
Collaborator

The 2.18 and edge docs were updated with this PR, but I assume the 2.19 docs need to be updated too?

This note in the docs goes back to 2.12. At what point was trafficDistribution supported?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants