Bridge interface not recognized for newly added host if they don't have a physical interface attached

[deajan]

problem

So I added a KVM hypervisor running AlmaLinux 9.5 to a Cloudstack Management Server via the UI, which failed with error Unable to add the host: Cannot find the server resources at <host>

While digging in the cloudstack management-server.log file, I noticed that my bridge br_npf0 is not found according to the management server:

2025-04-15 14:47:43,865 INFO  [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-7:[]) (logid:) PingMap for agent: 1 will not be updated because agent is no longer in the PingMap
2025-04-15 14:47:43,866 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentManager-Handler-7:[]) (logid:) Not processing PingRoutingCommand for agent id=0; can't find the host in the DB
2025-04-15 14:47:43,867 DEBUG [c.c.a.t.Request] (AgentManager-Handler-8:[]) (logid:) Seq 1-9217179587367141377: Processing:  { Ans: , MgmtId: 90520739542428, via: 1, Ver: v1, Flags: 110, [{"com.cloud.agent.api.CheckNetworkAnswer":{"_reconnect":"false","result":"false","details":"Can not find network: br_npf0","wait":"0","bypassHostMaintenance":"false"}}] }
2025-04-15 14:47:43,867 DEBUG [c.c.a.m.ClusteredAgentAttache] (AgentManager-Handler-8:[]) (logid:) Seq 1-9217179587367141377: No more commands found
2025-04-15 14:47:43,867 DEBUG [c.c.a.t.Request] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) Seq 1-9217179587367141377: Received:  { Ans: , MgmtId: 90520739542428, via: 1(reacted_host_name.local), Ver: v1, Flags: 110, { CheckNetworkAnswer } }
2025-04-15 14:47:43,867 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) Details from executing class com.cloud.agent.api.CheckNetworkCommand: Can not find network: br_npf0
2025-04-15 14:47:43,867 WARN  [o.a.c.e.o.NetworkOrchestrator] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) Unable to setup agent 1 due to Can not find network: br_npf0
2025-04-15 14:47:43,870 WARN  [c.c.a.AlertManagerImpl] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) alertType=[7] dataCenterId=[1] podId=[1] clusterId=[null] message=[Incorrect Network setup on agent, Reinitialize agent after network names are setup, details : Can not find network: br_npf0].
2025-04-15 14:47:43,878 WARN  [c.c.a.AlertManagerImpl] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) No recipients set in global setting 'alert.email.addresses', skipping sending alert with subject [Incorrect Network setup on agent, Reinitialize agent after network names are setup, details : Can not find network: br_npf0] and content [Incorrect Network setup on agent, Reinitialize agent after network names are setup, details : Can not find network: br_npf0].
2025-04-15 14:47:43,878 INFO  [c.c.u.e.CSExceptionErrorCode] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) Could not find exception: com.cloud.exception.ConnectionException in error code list for exceptions
[...]
2025-04-15 14:47:43,890 DEBUG [c.c.a.m.ClusteredAgentManagerImpl] (AgentConnectTaskPool-596:[ctx-58c65884]) (logid:c8f5bf3e) Failed to handle host connection: com.cloud.exception.ConnectionException: Incorrect Network setup on agent, Reinitialize agent after network names are setup, details : Can not find network: br_npf0
        at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.processConnect(NetworkOrchestrator.java:4321)
        at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:553)
        at com.cloud.agent.manager.AgentManagerImpl.sendReadyAndGetAttache(AgentManagerImpl.java:1116)
        at com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1135)
        at com.cloud.agent.manager.AgentManagerImpl$HandleAgentConnectTask.runInContext(AgentManagerImpl.java:1227)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

Looking at my configuration on the KVM host, the bridge br_npf0 exists, is up, has an IP, and can ping the management server.

ip a | grep br_npf0
10: br_npf0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.13.37.2/24 brd 10.13.37.255 scope global noprefixroute br_npf0
28: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br_npf0 state UP group default qlen 1000

# pinging the cloudstack management server
ping cloudstack01i.npf.local -c 1
PING cloudstack01i.npf.local (10.13.37.250) 56(84) bytes of data.
64 bytes from cloudstack01i.npf.local (10.13.37.250): icmp_seq=1 ttl=64 time=0.687 ms

--- cloudstack01i.npf.local ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.687/0.687/0.687/0.000 ms

The management server zone is setup with that exact same bridge name:

Is there any direction to point me to ?

versions

Cloudstack 4.20 running on AlmaLinux 9.5
KVM host AlmaLinux 9.5 with bridge setup via NetworkManager

The steps to reproduce the bug

What to do about it?

No response

[weizhouapache]

@deajan
can you run some commands on the kvm host ?

bridge=brXXX
ls -l /sys/devices/virtual/net/$bridge/brif
ls -ld /sys/class/net/$bridge/

[deajan]

Sure

export bridge=br_npf0
ls -l /sys/devices/virtual/net/${bridge}/brif
total 0
lrwxrwxrwx. 1 root root 0 Apr 15 15:32 eno2 -> ../../../../pci0000:00/0000:00:1c.0/0000:02:00.1/net/eno2/brport
ls -ld /sys/class/net/${bridge}/
drwxr-xr-x. 7 root root 0 Apr 15 15:28 /sys/class/net/br_npf0/

Dates are recent because I disconnected / reconnected eno2 just to make sure (and yes, the management server still complains)

[weizhouapache]

@deajan
looks good
can you share the agent.log ?

[deajan]

Sure:
It looks like it complains about lscpu (and iscsi I haven't setup, going to use local storage for tests), but nothing about the bridges.

2025-04-15 15:49:59,027 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from "host capabilities"
2025-04-15 15:49:59,031 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Retrieved value [2095080000] from "host capabilities". This corresponds to a CPU speed of [2095] MHz.
2025-04-15 15:49:59,031 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching host CPU arch
2025-04-15 15:49:59,063 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Host uses control group [cgroup2fs].
2025-04-15 15:49:59,063 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Calculating the max shares of the host.
2025-04-15 15:49:59,063 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) The max shares of the host is [167600].
2025-04-15 15:49:59,256 WARN  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Execution of process [28600] for command [sudo grep InitiatorName= /etc/iscsi/initiatorname.iscsi ] failed.
2025-04-15 15:49:59,256 WARN  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Process [28600] for command [sudo grep InitiatorName= /etc/iscsi/initiatorname.iscsi ] encountered the error: [grep: /etc/iscsi/initiatorname.iscsi: Aucun fichier ou dossier de ce type].
2025-04-15 15:49:59,429 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Attempting to create storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d (Filesystem) in libvirt
2025-04-15 15:49:59,431 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Found existing defined storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d, using it.
2025-04-15 15:49:59,431 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Trying to fetch storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d from libvirt
2025-04-15 15:49:59,549 INFO  [cloud.agent.Agent] (Agent-Handler-2:[]) (logid:) Process agent startup answer, agent id = 0
2025-04-15 15:49:59,549 INFO  [cloud.agent.Agent] (Agent-Handler-2:[]) (logid:) Startup Response Received: agent id = 0
2025-04-15 15:49:59,703 WARN  [cloud.agent.Agent] (UgentTask-5:[]) (logid:) Unable to send request: null
2025-04-15 15:50:03,833 INFO  [cloud.agent.Agent] (Agent-Handler-4:[]) (logid:) Connected to the host: 10.131.37.250
2025-04-15 15:50:04,663 INFO  [cloud.agent.Agent] (Agent-Handler-4:[]) (logid:) Lost connection to host: 10.131.37.250. Attempting reconnection while we still have 0 commands in progress.
2025-04-15 15:50:04,663 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) NioClient connection closed
2025-04-15 15:50:04,663 INFO  [cloud.agent.Agent] (Agent-Handler-4:[]) (logid:) Reconnecting to host:10.131.37.250
2025-04-15 15:50:04,664 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) Connecting to 10.131.37.250:8250
2025-04-15 15:50:04,664 INFO  [utils.nio.Link] (Agent-Handler-4:[]) (logid:) Conf file found: /etc/cloudstack/agent/agent.properties
2025-04-15 15:50:04,786 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) SSL: Handshake done
2025-04-15 15:50:04,786 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) Connected to 10.131.37.250:8250
2025-04-15 15:50:04,878 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from command "lscpu".
2025-04-15 15:50:05,000 ERROR [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Unable to retrieve the CPU speed from lscpu. java.lang.NullPointerException

2025-04-15 15:50:05,000 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from file [/sys/devices/system/cpu/cpu0/cpufreq/base_frequency].
2025-04-15 15:50:05,000 ERROR [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Unable to retrieve the CPU speed from file [/sys/devices/system/cpu/cpu0/cpufreq/base_frequency] java.io.FileNotFoundException: /sys/devices/system/cpu/cpu0/cpufreq/base_frequency (Aucun fichier ou dossier de ce type)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:216)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:111)
        at java.base/java.io.FileReader.<init>(FileReader.java:60)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.getCpuSpeedFromFile(KVMHostInfo.java:152)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.getCpuSpeed(KVMHostInfo.java:121)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.getHostInfoFromLibvirt(KVMHostInfo.java:196)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.<init>(KVMHostInfo.java:69)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.initialize(LibvirtComputingResource.java:3672)
        at com.cloud.agent.Agent.sendStartup(Agent.java:458)
        at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1128)
        at com.cloud.utils.nio.Task.call(Task.java:83)
        at com.cloud.utils.nio.Task.call(Task.java:29)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

2025-04-15 15:50:05,001 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from "host capabilities"
2025-04-15 15:50:05,004 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Retrieved value [2095080000] from "host capabilities". This corresponds to a CPU speed of [2095] MHz.
2025-04-15 15:50:05,004 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching host CPU arch
2025-04-15 15:50:05,038 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Host uses control group [cgroup2fs].
2025-04-15 15:50:05,038 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Calculating the max shares of the host.
2025-04-15 15:50:05,038 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) The max shares of the host is [167600].
2025-04-15 15:50:05,243 WARN  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Execution of process [32149] for command [sudo grep InitiatorName= /etc/iscsi/initiatorname.iscsi ] failed.
2025-04-15 15:50:05,244 WARN  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Process [32149] for command [sudo grep InitiatorName= /etc/iscsi/initiatorname.iscsi ] encountered the error: [grep: /etc/iscsi/initiatorname.iscsi: Aucun fichier ou dossier de ce type].
2025-04-15 15:50:05,436 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Attempting to create storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d (Filesystem) in libvirt
2025-04-15 15:50:05,438 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Found existing defined storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d, using it.
2025-04-15 15:50:05,439 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Trying to fetch storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d from libvirt
2025-04-15 15:50:05,558 INFO  [cloud.agent.Agent] (Agent-Handler-2:[]) (logid:) Process agent startup answer, agent id = 0
2025-04-15 15:50:05,558 INFO  [cloud.agent.Agent] (Agent-Handler-2:[]) (logid:) Startup Response Received: agent id = 0
2025-04-15 15:50:05,720 WARN  [cloud.agent.Agent] (UgentTask-5:[]) (logid:) Unable to send request: null
2025-04-15 15:50:09,786 INFO  [cloud.agent.Agent] (Agent-Handler-4:[]) (logid:) Connected to the host: 10.131.37.250
2025-04-15 15:50:10,681 INFO  [cloud.agent.Agent] (Agent-Handler-4:[]) (logid:) Lost connection to host: 10.131.37.250. Attempting reconnection while we still have 0 commands in progress.
2025-04-15 15:50:10,682 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) NioClient connection closed
2025-04-15 15:50:10,682 INFO  [cloud.agent.Agent] (Agent-Handler-4:[]) (logid:) Reconnecting to host:10.131.37.250
2025-04-15 15:50:10,682 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) Connecting to 10.131.37.250:8250
2025-04-15 15:50:10,683 INFO  [utils.nio.Link] (Agent-Handler-4:[]) (logid:) Conf file found: /etc/cloudstack/agent/agent.properties
2025-04-15 15:50:10,815 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) SSL: Handshake done
2025-04-15 15:50:10,815 INFO  [utils.nio.NioClient] (Agent-Handler-4:[]) (logid:) Connected to 10.131.37.250:8250
2025-04-15 15:50:10,889 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from command "lscpu".
2025-04-15 15:50:11,005 ERROR [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Unable to retrieve the CPU speed from lscpu. java.lang.NullPointerException

2025-04-15 15:50:11,005 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from file [/sys/devices/system/cpu/cpu0/cpufreq/base_frequency].
2025-04-15 15:50:11,006 ERROR [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Unable to retrieve the CPU speed from file [/sys/devices/system/cpu/cpu0/cpufreq/base_frequency] java.io.FileNotFoundException: /sys/devices/system/cpu/cpu0/cpufreq/base_frequency (Aucun fichier ou dossier de ce type)
        at java.base/java.io.FileInputStream.open0(Native Method)
        at java.base/java.io.FileInputStream.open(FileInputStream.java:216)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
        at java.base/java.io.FileInputStream.<init>(FileInputStream.java:111)
        at java.base/java.io.FileReader.<init>(FileReader.java:60)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.getCpuSpeedFromFile(KVMHostInfo.java:152)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.getCpuSpeed(KVMHostInfo.java:121)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.getHostInfoFromLibvirt(KVMHostInfo.java:196)
        at org.apache.cloudstack.utils.linux.KVMHostInfo.<init>(KVMHostInfo.java:69)
        at com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.initialize(LibvirtComputingResource.java:3672)
        at com.cloud.agent.Agent.sendStartup(Agent.java:458)
        at com.cloud.agent.Agent$ServerHandler.doTask(Agent.java:1128)
        at com.cloud.utils.nio.Task.call(Task.java:83)
        at com.cloud.utils.nio.Task.call(Task.java:29)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)

2025-04-15 15:50:11,006 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching CPU speed from "host capabilities"
2025-04-15 15:50:11,009 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Retrieved value [2095080000] from "host capabilities". This corresponds to a CPU speed of [2095] MHz.
2025-04-15 15:50:11,009 INFO  [utils.linux.KVMHostInfo] (Agent-Handler-1:[]) (logid:) Fetching host CPU arch
2025-04-15 15:50:11,036 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Host uses control group [cgroup2fs].
2025-04-15 15:50:11,036 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Calculating the max shares of the host.
2025-04-15 15:50:11,036 INFO  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) The max shares of the host is [167600].
2025-04-15 15:50:11,237 WARN  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Execution of process [35023] for command [sudo grep InitiatorName= /etc/iscsi/initiatorname.iscsi ] failed.
2025-04-15 15:50:11,237 WARN  [kvm.resource.LibvirtComputingResource] (Agent-Handler-1:[]) (logid:) Process [35023] for command [sudo grep InitiatorName= /etc/iscsi/initiatorname.iscsi ] encountered the error: [grep: /etc/iscsi/initiatorname.iscsi: Aucun fichier ou dossier de ce type].
2025-04-15 15:50:11,398 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Attempting to create storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d (Filesystem) in libvirt
2025-04-15 15:50:11,400 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Found existing defined storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d, using it.
2025-04-15 15:50:11,400 INFO  [kvm.storage.LibvirtStorageAdaptor] (Agent-Handler-1:[]) (logid:) Trying to fetch storage pool 3fd9d9aa-40fc-42d5-b99e-12fe590fd42d from libvirt
2025-04-15 15:50:11,514 INFO  [cloud.agent.Agent] (Agent-Handler-2:[]) (logid:) Process agent startup answer, agent id = 0
2025-04-15 15:50:11,515 INFO  [cloud.agent.Agent] (Agent-Handler-2:[]) (logid:) Startup Response Received: agent id = 0
2025-04-15 15:50:11,666 WARN  [cloud.agent.Agent] (UgentTask-5:[]) (logid:) Unable to send request: null

[deajan]

@weizhouapache While digging into my problem, I am thinking of a potential issue

Do the bridges need to be physically connected ?

I have one bridge (br_bgp0) that is "fed" by a vxlan interface (not connected to a physical interface but to a wireguard instance in order to transport some public IPs into my lab) on this KVM host.
This allows me to have my public IPs directly bridged from my bgp routers into my lab host.

Is this "allowed" ?

If not, that could perhaps explain why cloudstack complains about my bridges, even though they work.

[weizhouapache]

change the log level and restart cloudstack-agent ?

sed -i "s/INFO/DEBUG/g" /etc/cloudstack/agent/log4j-cloud.xml 

Is this "allowed" ?

yes, that's ok

[deajan]

Hmmm... interesting.

2025-04-15 15:59:16,309 DEBUG [kvm.resource.LibvirtComputingResource] (agentRequest-Handler-5:[]) (logid:04967b5d) failing to get physical interface from bridge br_bgp0, did not find an eth*, bond*, team*, vlan*, em*, p*p*, ens*, eno*, enp*, or enx* in /sys/devices/virtual/net/br_bgp0/brif

Looks like indeed using a bridge in cloudstack is limited to physical interfaces.
I've done a (stupid) check:

nmcli c add type dummy ifname ethdummy0 con-name ethdummy0
nmcli c mod ethdummy0 master br_bgp0
nmcli c up ethdummy0

Now the agent doesn't complain anymore, since there's now a eth* in /sys/devices/virtual/net/br_bgp0/brif

I guess that indeed the bridge tests are a bit too restrictive.
I wonder if the test should be less restrictive, like checking if there is at least one non vtnet* interface in the bridge, so people would be allowed to use whatever they want behind the bridge (like vxlans, gretap or whatever).
This would also allow make cloudstack future proof for ethernet driver names. Perhaps one day we'll have igb* or whatever driver names.

I can do python and bash PRs, but I really am not java fluent.
Is there anyway this could be discussed for perhaps some next release ?

@weizhouapache Big thanks for the hints.

[weizhouapache]

Now the agent doesn't complain anymore, since there's now a eth* in /sys/devices/virtual/net/br_bgp0/brif

that's strange.
in your previous reply, it showed that the eno2 is present

ls -l /sys/devices/virtual/net/${bridge}/brif
total 0
lrwxrwxrwx. 1 root root 0 Apr 15 15:32 eno2 -> ../../../../pci0000:00/0000:00:1c.0/0000:02:00.1/net/eno2/brport

[deajan]

in your previous reply, it showed that the eno2 is present

I think I mixed up br_npf0 and br_bgp0 in my tests (deleted & recreated bridges multiple times in order to diagnose my issue).

Point is, my bgp interface, the bridge br_bgp0 with a vxlan interface, wasn't accepted by Cloudstack agent since the bridge test complained that there wasn't a physical interface. There's still no physical interface yet, but I fooled the agent test by creating a dummy interface slave in that bridge and naming it ethdummy0.

ls /sys/devices/virtual/net/br_bgp0/brif
ethdummy0  vx_wg0

I think those tests should be relaxed, as I think they only should test that there are other interfaces than vnet* connected to the bridge.

[weizhouapache]

in your previous reply, it showed that the eno2 is present

I think I mixed up br_npf0 and br_bgp0 in my tests (deleted & recreated bridges multiple times in order to diagnose my issue).

Point is, my bgp interface, the bridge br_bgp0 with a vxlan interface, wasn't accepted by Cloudstack agent since the bridge test complained that there wasn't a physical interface. There's still no physical interface yet, but I fooled the agent test by creating a dummy interface slave in that bridge and naming it ethdummy0.

ls /sys/devices/virtual/net/br_bgp0/brif
ethdummy0  vx_wg0

I think those tests should be relaxed, as I think they only should test that there are other interfaces than vnet* connected to the bridge.

hmm, has the kvm host been added to cloudstack ?

[deajan]

Yes, host is now visible and manageable in cloudstack.

Anything I need to check ?

[weizhouapache]

Anything I need to check ?

Are the system vms Running and agent states are Up ?

[weizhouapache]

@deajan
just to confirm, is br_bgp0 used as the kvm network label of a physical network ?

[deajan]

@weizhouapache You mean like this ?

I don't have system VMs yet (it's a fresh cloudstack lab setup), but host is up and running according to management server.
I've redone a specific check:

• Disconnect eno2 physical interface and remove it from bridge br_npf0
• Host status goes to disabled
• Add ethdummy to br_npf0
• Host status goes to enabled

Of course, this will prevent me from using cloudstack properly until I reconnect eno2, but at it validates the assumption.