Merge pull request #45322 from gyliu513/podantiaffinity

Automatic merge from submit-queue (batch tested with PRs 45322, 44770, 45411)

Put podAntiAffinity in PodSpec.

**What this PR does / why we need it**:

**Which issue this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close that issue when PR gets merged)*: fixes #

**Special notes for your reviewer**:

In 1.6, we moved the `affinity` from `annotations` to `PodSpec`, but the `annotations` still works with `AffinityInAnnotations=true`. Please check https://docs.google.com/document/d/1YP3OJTKMpXkWAhipPjx9-bDMxJ87JZ6AcVak7c3SGos/edit for detail.

**Release note**:

```release-note
```

Fixed #45318

Author: Kubernetes Submit Queue

cockroachdb/cockroachdb-statefulset.yaml

@@ -78,22 +78,6 @@ spec:
       labels:
         app: cockroachdb
       annotations:
-        scheduler.alpha.kubernetes.io/affinity: >
-            {
-              "podAntiAffinity": {
-                "preferredDuringSchedulingIgnoredDuringExecution": [{
-                  "weight": 100,
-                  "labelSelector": {
-                    "matchExpressions": [{
-                      "key": "app",
-                      "operator": "In",
-                      "values": ["cockroachdb"]
-                    }]
-                  },
-                  "topologyKey": "kubernetes.io/hostname"
-                }]
-              }
-            }
         # Init containers are run only once in the lifetime of a pod, before
         # it's started up for the first time. It has to exit successfully
         # before the pod's main containers are allowed to start.
@@ -133,6 +117,18 @@ spec:
             }
         ]'
     spec:
+      affinity:
+        podAntiAffinity:
+          preferredDuringSchedulingIgnoredDuringExecution:
+          - weight: 100
+            podAffinityTerm:
+              labelSelector:
+                matchExpress:
+                - key: app
+                  operator: In
+                  values:
+                  - cockroachdb
+              topologyKey: kubernetes.io/hostname
       containers:
       - name: cockroachdb
         # Runs the master branch. Not recommended for production, but since