[Task] Explore around "cluster-api-provider-libvirt" and get more info about the cluster ecosystem

[praveenkumar]

Right now https://github.com/openshift/cluster-api-provider-libvirt actually implement the cluster-api ( https://sigs.k8s.io/cluster-api ) provider for libvirt and good to explore this since we might need same kind of provider the hypervisor we want to support (hyperkit/Hyper-V).

This issue going to track our finding around 'libvirt' provider and generic around cluster-api side.

[praveenkumar]

How to test libvirt api provider using minikube.

$ ./minikube start --vm-driver kvm2 --kubernetes-version="v1.11.3" -v=5
$ git clone https://github.com/openshift/cluster-api-provider-libvirt.git && cd cluster-api-provider-libvirt
$ cd config
$ kubectl create -f crds/libvirtproviderconfig_v1alpha1_libvirtmachineproviderconfig.yaml 
customresourcedefinition.apiextensions.k8s.io/libvirtmachineproviderconfigs.libvirtproviderconfig.k8s.io created
$ kubectl create -f rbac/rbac_role.yaml 
clusterrole.rbac.authorization.k8s.io/manager-role created
$ kubectl create -f manager/manager.yaml 
namespace/system created
service/controller-manager-service created
statefulset.apps/controller-manager created

$ cat cluster.yaml 
apiVersion: cluster.k8s.io/v1alpha1
kind: Cluster
metadata:
  creationTimestamp: null
  name: test1
  namespace: openshift-cluster-api
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
      - 10.128.0.0/14
    serviceDomain: ""
    services:
      cidrBlocks:
      - 172.30.0.0/16
  providerSpec: {}
status: {}

$ cat machine.yaml 
apiVersion: v1
apiVersion: cluster.k8s.io/v1alpha1
kind: Machine
metadata:
  creationTimestamp: null
  labels:
    sigs.k8s.io/cluster-api-cluster: test1
    sigs.k8s.io/cluster-api-machine-role: worker
    sigs.k8s.io/cluster-api-machine-type: worker
    sigs.k8s.io/cluster-api-machineset: test1-worker-0
  name: test1-worker-0
  namespace: openshift-cluster-api
spec:
  metadata:
    creationTimestamp: null
  providerSpec:
    value:
      apiVersion: libvirtproviderconfig.k8s.io/v1alpha1
      autostart: false
      cloudInit: null
      domainMemory: 2048
      domainVcpu: 2
      ignKey: ""
      ignition:
        userDataSecret: worker-user-data
      kind: LibvirtMachineProviderConfig
      networkInterfaceAddress: 192.168.126.0/24
      networkInterfaceHostname: ""
      networkInterfaceName: test1
      networkUUID: ""
      uri: qemu+tcp://libvirt.default/system
      volume:
        baseVolumeID: /var/lib/libvirt/images/test1-base
        poolName: default
        volumeName: ""
  versions:
    kubelet: ""

$ cat userdata.yaml 
apiVersion: v1
kind: Secret
metadata:
  name: worker-user-data
  namespace: openshift-cluster-api
type: Opaque
data:
  userData: eyJpZ25pdGlvbiI6eyJjb25maWciOnsiYXBwZW5kIjpbeyJzb3VyY2UiOiJodHRwczovL3Rlc3QxLWFwaS50dC50ZXN0aW5nOjQ5NTAwL2NvbmZpZy93b3JrZXIiLCJ2ZXJpZmljYXRpb24iOnt9fV19LCJzZWN1cml0eSI6eyJ0bHMiOnsiY2VydGlmaWNhdGVBdXRob3JpdGllcyI6W3sic291cmNlIjoiZGF0YTp0ZXh0L3BsYWluO2NoYXJzZXQ9dXRmLTg7YmFzZTY0LExTMHRMUzFDUlVkSlRpQkRSVkpVU1VaSlEwRlVSUzB0TFMwdENrMUpTVVJEVkVORFFXWkhaMEYzU1VKQlowbENRVVJCVGtKbmEzRm9hMmxIT1hjd1FrRlJjMFpCUkVGdFRWSkpkMFZCV1VSV1VWRk1SWGRzZG1OSFZuVUtZekpvY0ZwdVVYaEZSRUZQUW1kT1ZrSkJUVlJDTTBwMllqTlJkRmt5UlhkSWFHTk9UVlJyZDAxVVNYcE5SR3Q1VFZSRmVGZG9ZMDVOYW10M1RWUkpkd3BOUkd0NVRWUkZlRmRxUVcxTlVrbDNSVUZaUkZaUlVVeEZkMngyWTBkV2RXTXlhSEJhYmxGNFJVUkJUMEpuVGxaQ1FVMVVRak5LZG1JelVYUlpNa1YzQ21kblJXbE5RVEJIUTFOeFIxTkpZak5FVVVWQ1FWRlZRVUUwU1VKRWQwRjNaMmRGUzBGdlNVSkJVVU0xU1djNU9VZFpjRkZRVGtNM2F5OTVXVEJ0Y0ZNS1puUnJja3BaYldwWFJIRmlWMU5RWkRWbE5VZEJhelpJWlVrd2NsbEhMM053ZW1reWFHRmlhemRDTTNKTVN6UjZVRFE1ZWxGeFRHdHNiQzlGZVdjMmVBcDVaREpuV0VkSmFtUklPUzlJTWtaVFNuaHRjbWhCZDIxRmQxUjBUelJ4YkZZMWFqUnBZMDk2VGpaVWFYSlRja05vT0M5MU1tRkVRV0pRTlcxdlpFbERDbTVtVFRCV1kybENla0ZsSzNReFRHRXhNbXQwUkhCWGFrRXdhV2R5YjBsRFYxcFhMMVZKVjBsQ2VFazNjelp1UTNnclIxSkZPRTl3T0RNcmRqWTRiblFLV0RreWNUUXhUMk5MZERkR1ZtTnlWRFIxYURWNVNUSlNSalJaYWpKQlZqQXpRbUoxUW1OaE4wOU9SR1J6YUU1dE9FbHVkRTlWY1VsSlZGZEdkRXRvZFFwM2VrWldOVW80V0Zad1VVazVWSEZMZWxJeGFXWnNibmhXU1ZSeGFHczJaR2hHU0VoS2J5dHhXbEpFYnpKakt5dGpORkZFVlZOVE1qTlZhMUpOVVZKWUNrRm5UVUpCUVVkcVVXcENRVTFCTkVkQk1WVmtSSGRGUWk5M1VVVkJkMGxEY0VSQlVFSm5UbFpJVWsxQ1FXWTRSVUpVUVVSQlVVZ3ZUVUl3UjBFeFZXUUtSR2RSVjBKQ1V6SnRkMjl2TDI5R1pHc3ZXbXBsV214S2NGWlVXRXB0ZFcwcmVrRk9RbWRyY1docmFVYzVkekJDUVZGelJrRkJUME5CVVVWQmNYbGpaZ3BWVEd4Q1owOTFTblpzYlRkcWNHcFJNV0pRUWxjclUwZGpaM1ZIZUdFM1pIZEJLMjU0VGxOQkx6RlJiRU5YZEhKc09WWlJVRGxPYm1Kd1ZEQndTSFpQQ25WeVowVmpMMk5yVEVWS1NtUldVbEJYYkhONkwxbGtjR3hyV2pGWU1EQlFTR1pSZEZGaFNIZGxRMnN3ZUd0WlpVUmhObGd2ZGtSb2VqQnFka2R5VjNNS1pqbHBVazlzTUVkRFpYWmlibXhSV2xGNGJERTNhbFIwVTJsSVlUUnJRemc1ZW5ZMmJXazJUbkJUUzFONFFrMVJkVEZDVG1salpGTnpibEZrY1Vwb1p3cEtlRkJTZHk5MWJYb3JkMVZqVkVvM1VscDFZbVZUYTBaVWJVRkZkaTl5ZDJkdVMwcFRNa2wzZEV4c00zaENlRVF3VHpOMGVHSTNjRlI2ZDFaVlRYZERDbUk1WjJzMk1WazVUVlJhVEdvMFNWaDZUbU5uYjBka1NGSnlWbWx1Y0VSVU1qa3hVV2xSYjFKd2VVWmhjbUYzSzJsb1VXaGxUalJSUVN0bWJXOUhabFlLVDFaWU5XSjNlRVpwWmxsQ1RHVlRRMlozUFQwS0xTMHRMUzFGVGtRZ1EwVlNWRWxHU1VOQlZFVXRMUzB0TFFvPSIsInZlcmlmaWNhdGlvbiI6e319XX19LCJ0aW1lb3V0cyI6e30sInZlcnNpb24iOiIyLjIuMCJ9LCJuZXR3b3JrZCI6e30sInBhc3N3ZCI6e30sInN0b3JhZ2UiOnt9LCJzeXN0ZW1kIjp7fX0=

$ ./libvirt-actuator --logtostderr create --kubeconfig ~/.kube/config -m machine.yaml -c cluster.yaml -u userdata.yaml --v 10
[...]
I0128 12:12:28.732590   30116 client.go:242] Domain ID: f64a761c-0f65-4fa7-b259-5de556e892ce
I0128 12:12:28.732636   30116 client.go:249] Lookup domain by name: "test1-worker-0"
I0128 12:12:28.733093   30116 actuator.go:327] Updating status for test1-worker-0
I0128 12:12:28.733728   30116 actuator.go:378] Machine test1-worker-0 status has changed: 
object.ProviderStatus:
  a: <nil>
  b: &runtime.RawExtension{Raw:[]uint8{0x7b, 0x22, 0x6b, 0x69, 0x6e, 0x64, 0x22, 0x3a...
object.Addresses:
  a: []v1.NodeAddress(nil)
  b: []v1.NodeAddress{}
I0128 12:12:28.733776   30116 client.go:140] Closing libvirt connection: 0xc42000e940
Machine creation was successful!

[praveenkumar]

Put initial bits to https://github.com/praveenkumar/cluster-api-provider-hyperkit but this is not working condition because hyperkit doesn't have any socket expose uri like libvirt or any other cloud providers have.