-
Notifications
You must be signed in to change notification settings - Fork 93
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add simple tests for openvswitch bridge (#220)
* Install nm and ovs on all the nodes. Adding ovs installation and extending nm installation to all nodes. Signed-off-by: Federico Paolinelli <[email protected]> * Add ovs bridge state functions and rename linux bridge ones. Now linux bridge ones have the linuxBr prefix. Signed-off-by: Federico Paolinelli <[email protected]> * Add simple ovs bridge tests. Signed-off-by: Federico Paolinelli <[email protected]> * Add tutorial on how to configure ovs bridge. Signed-off-by: Federico Paolinelli <[email protected]> * Skip ovs tests for non k8s providers. Signed-off-by: Federico Paolinelli <[email protected]> * Remove unused var() Rename OVS tests as "Simple OVS bridge" Signed-off-by: Federico Paolinelli <[email protected]> * Remove unused var() Rename OVS tests as "Simple OVS bridge" Remove test with ovs bridge and no ports Signed-off-by: Federico Paolinelli <[email protected]> * Add test for ovs bridge with internal interface. Signed-off-by: Federico Paolinelli <[email protected]> * Refer to ovs as Open vSwitch Signed-off-by: Federico Paolinelli <[email protected]> * Add new line to install-ovs.sh Signed-off-by: Federico Paolinelli <[email protected]> * Mention the ovs tutorial in the readme and in the user guide Signed-off-by: Federico Paolinelli <[email protected]> * Use SSH and KUBECTL exported by the Makefile Signed-off-by: Federico Paolinelli <[email protected]> * Change ovs skipping comment, fix a typo in the readme Signed-off-by: Federico Paolinelli <[email protected]> * Remove unnecessary context Signed-off-by: Federico Paolinelli <[email protected]> * Extend ovs tutorial with internal interface Signed-off-by: Federico Paolinelli <[email protected]> * Remove unnecessary empty lines Signed-off-by: Federico Paolinelli <[email protected]> * Reword the internal interface part without mentioning the loose of connectivity Signed-off-by: Federico Paolinelli <[email protected]> * Install network manager only on k8s providers Signed-off-by: Federico Paolinelli <[email protected]> * Refactor skipping test behaviour. Have a variable holding all the skips, append to it any local change and extras coming from outside, and then add the skip to the args. Signed-off-by: Federico Paolinelli <[email protected]>
- Loading branch information
Showing
13 changed files
with
290 additions
and
35 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,121 @@ | ||
| # Tutorial: Create a Open vSwitch Bridge and Connect it to a Node Interface | ||
|
|
||
| Use Node Network Configuration Policy to configure a new ovs bridge `br1` connected | ||
| to node interface `eth1`. | ||
|
|
||
| ## Requirements | ||
|
|
||
| Before we start, please make sure that you have your Kubernetes/OpenShift | ||
| cluster ready. In order to do that, you can follow the guides of deployment on | ||
| [local cluster](deployment-local-cluster.md) or your | ||
| [arbitrary cluster](deployment-arbitrary-cluster.md). | ||
|
|
||
| You must also be sure that [OpenVSwitch](https://www.openvswitch.org/) and [NetworkManager OpenVSwitch plugin](https://developer.gnome.org/NetworkManager/stable/nm-openvswitch.html) are installed in the nodes. | ||
|
|
||
| Please also note that on OpenShift `openvswitch` is already installed as a daemon, so only `nm-openvswitch` is required. | ||
|
|
||
| ## Configure bridge | ||
|
|
||
| All you have to do in order to create the bridge on all nodes across cluster is | ||
| to apply the following policy: | ||
|
|
||
| ```yaml | ||
| cat <<EOF | kubectl create -f - | ||
| apiVersion: nmstate.io/v1alpha1 | ||
| kind: NodeNetworkConfigurationPolicy | ||
| metadata: | ||
| name: br1-eth1-policy | ||
| spec: | ||
| desiredState: | ||
| interfaces: | ||
| - name: br1 | ||
| description: ovs bridge with eth1 as a port | ||
| type: ovs-bridge | ||
| state: up | ||
| bridge: | ||
| options: | ||
| stp: false | ||
| port: | ||
| - name: eth1 | ||
| EOF | ||
| ``` | ||
|
|
||
| By doing this though, we will be able to create an Open vSwitch bridge but the bridge won't have an IP endpoint. | ||
| In order to provide an ip address an Open vSwitch internal interface must be used. | ||
|
|
||
| ```yaml | ||
| cat <<EOF | kubectl create -f - | ||
| apiVersion: nmstate.io/v1alpha1 | ||
| kind: NodeNetworkConfigurationPolicy | ||
| metadata: | ||
| name: br1-eth1-policy | ||
| spec: | ||
| desiredState: | ||
| interfaces: | ||
| - name: ovs0 | ||
| type: ovs-interface | ||
| state: up | ||
| ipv4: | ||
| enabled: true | ||
| address: | ||
| - ip: 192.0.2.1 | ||
| prefix-length: 24 | ||
| - name: br1 | ||
| description: ovs bridge with eth1 as a port and ovs0 as an internal interface | ||
| type: ovs-bridge | ||
| state: up | ||
| bridge: | ||
| options: | ||
| stp: true | ||
| port: | ||
| - name: eth1 | ||
| type: system | ||
| - name: ovs0 | ||
| type: internal | ||
| EOF | ||
| ``` | ||
|
|
||
| You can also remove the bridge with the following command: | ||
|
|
||
| ```yaml | ||
| cat <<EOF | kubectl create -f - | ||
| apiVersion: nmstate.io/v1alpha1 | ||
| kind: NodeNetworkConfigurationPolicy | ||
| metadata: | ||
| name: br1-eth1-policy | ||
| spec: | ||
| desiredState: | ||
| interfaces: | ||
| - name: br1 | ||
| type: ovs-bridge | ||
| state: absent | ||
| EOF | ||
| ``` | ||
|
|
||
| ## Selecting nodes | ||
|
|
||
| `NodeNetworkConfigurationPolicy` supports node selectors. Thanks to them you can | ||
| select a subset of nodes or a specific node by its name: | ||
|
|
||
| ```yaml | ||
| cat <<EOF | kubectl create -f - | ||
| apiVersion: nmstate.io/v1alpha1 | ||
| kind: NodeNetworkConfigurationPolicy | ||
| metadata: | ||
| name: br1-eth1-policy | ||
| spec: | ||
| nodeSelector: | ||
| kubernetes.io/hostname: node01 | ||
| desiredState: | ||
| interfaces: | ||
| - name: ovs-br0 | ||
| description: Ovs bridge with eth1 as a port | ||
| type: ovs-bridge | ||
| state: up | ||
| bridge: | ||
| options: | ||
| stp: false | ||
| port: | ||
| - name: eth1 | ||
| EOF | ||
| ``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,17 @@ | ||
| #!/bin/bash -e | ||
|
|
||
| # TODO: Iterate all the nodes | ||
| script_dir=$(dirname "$(readlink -f "$0")") | ||
| ssh=$script_dir/../kubevirtci/cluster-up/ssh.sh | ||
| function install_nm_on_node() { | ||
| node=$1 | ||
| $SSH $node sudo -- yum install -y NetworkManager NetworkManager-ovs | ||
| $SSH $node sudo -- systemctl daemon-reload | ||
| $SSH $node sudo -- systemctl restart NetworkManager | ||
| echo "Check NetworkManager is working fine on node $node" | ||
| $SSH $node -- nmcli device show > /dev/null | ||
| } | ||
|
|
||
| if [[ "$KUBEVIRT_PROVIDER" =~ k8s ]]; then | ||
| echo 'Install NetworkManager on the node' | ||
| $ssh node01 -- sudo yum install -y NetworkManager | ||
| $ssh node01 -- sudo systemctl daemon-reload | ||
| $ssh node01 -- sudo systemctl restart NetworkManager | ||
|
|
||
| echo 'Check NetworkManager is working fine' | ||
| $ssh node01 -- nmcli device show > /dev/null | ||
| fi | ||
| if [[ "$KUBEVIRT_PROVIDER" =~ k8s- ]]; then | ||
| echo 'Install NetworkManager on nodes' | ||
| for node in $($KUBECTL get nodes --no-headers | awk '{print $1}'); do | ||
| install_nm_on_node "$node" | ||
| done | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| #!/bin/bash -e | ||
|
|
||
|
|
||
| function install_ovs_on_node() { | ||
| node=$1 | ||
| $SSH $node -- sudo yum install -y http://cbs.centos.org/kojifiles/packages/openvswitch/2.9.2/1.el7/x86_64/openvswitch-2.9.2-1.el7.x86_64.rpm http://cbs.centos.org/kojifiles/packages/openvswitch/2.9.2/1.el7/x86_64/openvswitch-devel-2.9.2-1.el7.x86_64.rpm http://cbs.centos.org/kojifiles/packages/dpdk/17.11/3.el7/x86_64/dpdk-17.11-3.el7.x86_64.rpm | ||
| $SSH $node -- sudo systemctl daemon-reload | ||
| $SSH $node -- sudo systemctl restart openvswitch | ||
| } | ||
|
|
||
| # we currently skip ovs for non k8s providers. | ||
| if [[ "$KUBEVIRT_PROVIDER" =~ k8s- ]]; then | ||
| echo 'Installing Open vSwitch on nodes' | ||
|
|
||
| for node in $($KUBECTL get nodes --no-headers | awk '{print $1}'); do | ||
| install_ovs_on_node "$node" | ||
| done | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| package e2e | ||
|
|
||
| import ( | ||
| . "github.com/onsi/ginkgo" | ||
| . "github.com/onsi/gomega" | ||
| ) | ||
|
|
||
| var _ = Describe("Simple OVS bridge", func() { | ||
| Context("when desiredState is configured with an ovs bridge up", func() { | ||
| BeforeEach(func() { | ||
| updateDesiredState(ovsBrUp(bridge1)) | ||
| }) | ||
| AfterEach(func() { | ||
| updateDesiredState(ovsBrAbsent(bridge1)) | ||
| for _, node := range nodes { | ||
| interfacesNameForNodeEventually(node).ShouldNot(ContainElement(bridge1)) | ||
| } | ||
| resetDesiredStateForNodes() | ||
| }) | ||
| It("should have the ovs bridge at currentState", func() { | ||
| for _, node := range nodes { | ||
| interfacesForNode(node).Should(ContainElement(SatisfyAll( | ||
| HaveKeyWithValue("name", bridge1), | ||
| HaveKeyWithValue("type", "ovs-bridge"), | ||
| HaveKeyWithValue("state", "up"), | ||
| ))) | ||
| } | ||
| }) | ||
| }) | ||
| Context("when desiredState is configured with an ovs bridge with internal port up", func() { | ||
| BeforeEach(func() { | ||
| updateDesiredState(ovsbBrWithInternalInterface(bridge1)) | ||
| }) | ||
| AfterEach(func() { | ||
| updateDesiredState(ovsBrAbsent(bridge1)) | ||
| for _, node := range nodes { | ||
| interfacesNameForNodeEventually(node).ShouldNot(ContainElement(bridge1)) | ||
| } | ||
| resetDesiredStateForNodes() | ||
| }) | ||
| It("should have the ovs bridge at currentState", func() { | ||
| for _, node := range nodes { | ||
| interfacesForNode(node).Should(SatisfyAll( | ||
| ContainElement(SatisfyAll( | ||
| HaveKeyWithValue("name", bridge1), | ||
| HaveKeyWithValue("type", "ovs-bridge"), | ||
| HaveKeyWithValue("state", "up"), | ||
| )), | ||
| ContainElement(SatisfyAll( | ||
| HaveKeyWithValue("name", "ovs0"), | ||
| HaveKeyWithValue("type", "ovs-interface"), | ||
| HaveKeyWithValue("state", "up"), | ||
| )), | ||
| )) | ||
| } | ||
| }) | ||
| }) | ||
| }) |
Oops, something went wrong.