Troubleshooting the Virtual Network

OVN has one failure mode that catches everybody, so it is worth knowing before anything else.

---

The rule that exists and does nothing

The OVN northbound database validates very little. It will accept a rule whose expression the data plane cannot compile. When that happens:

• the object is created,
• the API reports success,
• the page lists it as healthy,
• and it has no effect whatsoever.

Nothing in Vapor can tell you this, because as far as the database is concerned the object is fine. The only evidence is on the host, in ovn-controller's log:

grep -iE "error parsing|Syntax error" /var/log/ovn/ovn-controller.log

Make this your first check whenever an object exists but does not do what it says. Two examples produce it:

• a - in a port group or address set name, which OVN reads as subtraction
• a malformed option value, such as a list of DNS servers in the wrong form

A second signal for the same class of problem: the rule appears in ovn-sbctl lflow-list but has no corresponding entry in ovs-ofctl dump-flows br-int. The logical rule was generated and the switch refused it.

---

Symptoms and where to look

A virtual machine gets no DHCP address

• Does the MAC OVN recorded for the port match the MAC the guest sends from? They must agree; OVN's DHCP responder matches on the source MAC.
• Is DHCP enabled on that subnet, and does the port have the subnet's DHCP options attached?
• Check ovn-controller.log as above — a malformed DHCP option stops DHCP for every port on the switch, not just one.
• Has the guest actually requested an address? A hot-plugged interface is not configured automatically.

A machine reaches its gateway and same-subnet hosts, but nothing routed

Almost always guest routing rather than OVN. A guest with a second interface usually has a default route with a lower metric there, so routed traffic leaves the wrong way. Check the guest's routing table before suspecting the network.

Hosts on the same logical switch cannot reach each other

• Are both ports bound? A port shows as bound on the Logical Switch Ports page once ovn-controller has claimed it.
• Are the two chassis' tunnel addresses on a subnet they share? A chassis with an unreachable tunnel address registers fine and never forms a tunnel.
• Zero tunnel ports on a host with no logical ports is normal. OVN builds them on demand.

Attaching a machine to a logical switch fails when it starts

The host needs ovs-vsctl, which Vapor uses to put the interface on the integration bridge. Without it the machine fails to start with a message about adding a port to br-int. The OVN status card reports whether the client is present.

A load balancer returns nothing

• Attach it to the logical switch, not to a distributed router without a gateway port. A router-attached balancer translates inbound traffic and not the replies.
• Give the VIP an address outside the client's subnet. An in-subnet VIP has no ARP responder, so the connection hangs before a packet is sent.

An ACL does not block what it should

• Check the naming rule for port groups and address sets, then ovn-controller.log.
• Confirm the remote was actually recorded. An ACL with no source restriction matches every source, which is broader than intended and looks like the rule "not working" when it is in fact working on everything.
• allow is stateless. For anything connection-oriented use allow-related, or replies are dropped.

Objects disappear after a while

On a shared kube-ovn deployment, this is kube-ovn's garbage collection. Check:

kubectl -n kube-system logs deploy/kube-ovn-controller | grep gc.go

and whether the controller restarted. See the mode 3 section of Activating the Virtual Network for the three conditions that prevent it.

A router vanished on a kube-ovn deployment

If --enable-external-vpc=true is set, each of Vapor's routers also exists as a Kubernetes Vpc resource, and deleting that resource deletes the router. Check whether anything pruned it.

---

Cluster-specific checks

A member refuses connections on its client port

Expected on a follower. OVSDB clients talk only to the leader, so a query aimed at one follower is refused even though the member is healthy. Vapor addresses a cluster by all of its members for this reason. To query a follower directly for diagnosis, use --no-leader-only.

Leadership keeps moving

Normal after a restart or a brief loss of contact. The deployment stays usable. Leadership settling on a different member than the one you bootstrapped is not a fault.

A central's databases will not start

Check the ordering: ovn-northd requires the two database services to be up, and starting them together races. Start ovn-ovsdb-server-nb and ovn-ovsdb-server-sb first, confirm they are active, then ovn-northd.

After a reboot a central has no databases

The cluster parameters live in the OVN service units' configuration and those units must be enabled. Vapor enables them during activation and logs a warning if it could not.

---

Useful commands

Run these on the host. On a kube-ovn node, ovn-nbctl and ovs-vsctl may live inside a pod rather than on the host.

# What this host thinks its role is
ovn-nbctl --db=<nb-address> show

# Is the port bound, and to which chassis
ovn-sbctl --db=<sb-address> find Port_Binding logical_port=<port>

# Cluster membership for one database
ovs-appctl -t /var/run/ovn/ovnnb_db.ctl cluster/status OVN_Northbound

# The generated logical rules for a switch
ovn-sbctl lflow-list <switch>

# What the switch actually installed
ovs-ofctl dump-flows br-int

# The log that explains silent failures
grep -iE "error parsing|Syntax error" /var/log/ovn/ovn-controller.log

---

When reporting a problem

Include the mode this host was activated in, which source supplied the database addresses (the status card shows it), whether the object exists in the northbound database, whether a corresponding flow exists on the switch, and any error parsing lines from ovn-controller.log. Those five answers separate a configuration mistake from a genuine fault faster than anything else.