No ping to SSVM public ip

[dominar250]

ISSUE TYPE

• Other

COMPONENT NAME

SSVM

CLOUDSTACK VERSION

CloudStack 4.19.0.1

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

I'm using nested environment over vmware. promiscuous mode and MAC address change ,Forged Transmit are enabled in port group level. SSVM and console proxy agents are up.I'm having issue with packet loss from kvm machine to SSVM /Console proxy vm both private and public ip address. Unable to ping SSVM public ips .Ping from ssvm to management /kvm is good

root@cloudstack-kvm:~# brctl show
bridge name     bridge id               STP enabled     interfaces
cloud0          8000.fe00a9fe08c6       no              vnet24
                                                        vnet27
cloudbr0                8000.005056b4a6ca       yes             ens192
                                                        vnet25
                                                        vnet28
cloudbr1                8000.005056b48ea4       yes             ens224
                                                        vnet26
                                                        vnet29
root@cloudstack-kvm:~# ip addr show cloudbr0
5: cloudbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:50:56:b4:a6:ca brd ff:ff:ff:ff:ff:ff
    inet 10.158.65.11/25 brd 10.158.65.127 scope global cloudbr0
       valid_lft forever preferred_lft forever
    inet6 fe80::250:56ff:feb4:a6ca/64 scope link
       valid_lft forever preferred_lft forever
root@cloudstack-kvm:~#
root@cloudstack-kvm:~# ip addr show ens192
2: ens192: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master cloudbr0 state UP group default qlen 1000
    link/ether 00:50:56:b4:a6:ca brd ff:ff:ff:ff:ff:ff
    altname enp11s0
root@cloudstack-kvm:~#
root@cloudstack-kvm:~# ip addr show cloudbr1
6: cloudbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:50:56:b4:8e:a4 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::250:56ff:feb4:8ea4/64 scope link
       valid_lft forever preferred_lft forever
root@cloudstack-kvm:~# ip addr show ens224
3: ens224: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master cloudbr1 state UP group default qlen 1000
    link/ether 00:50:56:b4:8e:a4 brd ff:ff:ff:ff:ff:ff
    altname enp19s0

EXPECTED RESULTS

Able to ping ssvm public/private range from kvm/management server

ACTUAL RESULTS

root@cloudstack-kvm:~# ping -O 10.158.209.11
PING 10.158.209.11 (10.158.209.11) 56(84) bytes of data.
no answer yet for icmp_seq=1
no answer yet for icmp_seq=2
no answer yet for icmp_seq=3
no answer yet for icmp_seq=4

--- 10.158.209.11 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4081ms

[dominar250]

cloud.log-consoleproxy.txt
cloud.log-secondary storage.txt

[weizhouapache]

@dominar250
have you configured any routes to 10.158.209.x on your switch ?

[dominar250]

@weizhouapache No sir . I have assigned this network to a vm for testing , I can able to ssh and ping that ip from 10.158.65.0 network

[weizhouapache]

10.158.209.

@dominar250
does ping to 10.158.209.X gateway work inside the ssvm ?

[dominar250]

@weizhouapache no ping from ssvm to gateway
ping 10.158.209.1
PING 10.158.209.1 (10.158.209.1): 56 data bytes
92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable
92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable
92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable
92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable
--- 10.158.209.1 ping statistics ---

[weizhouapache]

@weizhouapache no ping from ssvm to gateway ping 10.158.209.1 PING 10.158.209.1 (10.158.209.1): 56 data bytes 92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable 92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable 92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable 92 bytes from s-10-VM (10.158.209.11): Destination Host Unreachable --- 10.158.209.1 ping statistics ---

I guess it is because the virtual nic (vnet25 ?) for public interface of s-10-VM is on cloudbr0
when you create a vm, the virtual nic is on cloudbr1

you can go to the Zone-> Physical network, and update the traffic label of Public traffic to "cloudbr1" and retry

[dominar250]

@weizhouapache Its already cloudbr1. guest is also in cloudbr1 and managment interface is on cloudbr0

[weizhouapache]

@weizhouapache Its already cloudbr1. guest is also in cloudbr1 and managment interface is on cloudbr0

destroy the system vm and re-check the new system vm ?

[dominar250]

@weizhouapache still having the same issue. no ping to gateways

[DaanHoogland]

@dominar250 can you run /usr/local/cloud/systemvm/ssvm-check.sh in the ssvm and show the output? It may give som clue.

[dominar250]

ssvm-script.txt
@DaanHoogland

[DaanHoogland]

There is obviously a network problem:

root@s-15-VM:~# /usr/local/cloud/systemvm/ssvm-check.sh
================================================
First DNS server is  66.173.144.10
PING 66.173.144.10 (66.173.144.10): 56 data bytes
92 bytes from 10.158.65.14: Destination Host Unreachable
92 bytes from 10.158.65.14: Destination Host Unreachable
--- 66.173.144.10 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
WARNING: cannot ping DNS server
route follows
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.158.209.1    0.0.0.0         UG    0      0        0 eth2
10.158.65.0     0.0.0.0         255.255.255.128 U     0      0        0 eth1
10.158.209.0    0.0.0.0         255.255.255.128 U     0      0        0 eth2
66.173.144.10   10.158.65.1     255.255.255.255 UGH   0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth0
================================================
ERROR: DNS not resolving cloudstack.apache.org
resolv.conf follows
nameserver 66.173.144.10
nameserver 139.55.95.240

What are your physical network traffic configurations?

[dominar250]

@DaanHoogland - This is my netplan configuration

network:
version: 2
ethernets:
ens192: {}
ens224: {}
bridges:
cloudbr0:
addresses:
- 10.158.65.11/25
dhcp4: false
routes:
- to: default
via: 10.158.65.1
nameservers:
addresses:
- 139.55.95.240
- 66.173.144.10
search: [winster.com]
interfaces:
- ens192
parameters:
stp: false
cloudbr1:
dhcp4: false
interfaces:
- ens224
parameters:
stp: false

[weizhouapache]

@DaanHoogland - This is my netplan configuration

network:
version: 2
ethernets:
ens192: {}
ens224: {}
bridges:
cloudbr0:
addresses:
- 10.158.65.11/25
dhcp4: false
routes:
- to: default
via: 10.158.65.1
nameservers:
addresses:
- 139.55.95.240
- 66.173.144.10
search: [winster.com]
interfaces:
- ens192
parameters:
stp: false
cloudbr1:
dhcp4: false
interfaces:
- ens224
parameters:
stp: false

@dominar250
Can you share agent.properties on the host?