Re: How to receive broadcast messages in VPP?

Neale Ranns

Hi Elias,

´╗┐On 14/02/2020 13:35, "Elias Rudberg" <Elias.Rudberg@...> wrote:

Hi Neale and Dave,

Thanks for your answers!
I was able to make it work using multicast as Neale suggested.

Here is roughly what I did to make it work using multicast instead of

On the sending side, to make it send multicast packets:

adj_index_t adj_index_for_multicast = adj_mcast_add_or_lock
(FIB_PROTOCOL_IP4, VNET_LINK_IP4, sw_if_index);

Note the 'and_lock' part here. You need to call adj_unlock too. But not before your packet has been sent...
In other words, it's best to do this once in the control plane and use it in the DP.

and then when a message is to be sent, use the above created adj_index
before invoking ip4_rewrite_node (instead of ip4_lookup_node):

vnet_buffer (b)->ip.adj_index[VLIB_TX] = adj_index_for_multicast;
vlib_put_frame_to_node (vm, ip4_rewrite_node.index, f);

you need the mcast version of the rewrite so the correct mcast mac is added, which is: ip4_rewrite_mcast_node. Or better still use: adj->ia_node_index.


On the receiving side the following config was needed:

ip mroute add via MyInterface Accept
ip mroute add via local Forward

After that it works using multicast. Thanks for your help!
(Please let me know if the above is not the right way to do it)

Best regards,

On Thu, 2020-02-06 at 13:45 +0000, Neale Ranns via Lists.Fd.Io wrote:
> Hi Elias,
> Please see inline.
> On 06/02/2020 12:41, "vpp-dev@... on behalf of Elias
> Rudberg" <vpp-dev@... on behalf of elias.rudberg@...>
> wrote:
> Hello everyone,
> I am trying to figure out how to receive broadcast messages in
> VPP (vpp
> version 19.08 in case that matters).
> This is in the context of some changes we are considering in the
> NAT HA functionality. That code in e.g. plugins/nat/nat_ha.c uses
> messages to communicate information about NAT sessions between
> different VPP servers. It is currently using unicast messages,
> but we
> are considering the possibility of using broadcast messages
> instead,
> hoping that could be more efficient in case there are more than
> two
> servers involved. For example, when a new NAT session has been
> created,
> we could send a broadcast message about the new session, that
> would
> reach several other VPP servers, without need to send a separate
> unicast message to each server.
> The code in plugins/nat/nat_ha.c calls udp_register_dst_port() to
> register that it wants to receive UDP traffic, like this:
> udp_register_dst_port (ha->vlib_main, port,
> nat_ha_handoff_node.index, 1);
> This works fine for unicast messages; when such packets arrive at
> the
> given port, they get handled by the nat_ha_handoff_node as
> desired.
> However, if broadcast packets arrive, those packets are dropped
> instead, they do not arrive at the nat_ha_handoff_node.
> For example, if the IP address of the relevant interface on the
> receiving side is then unicast UDP messages with
> destination are handled fine. However, if the
> destination is
> (the broadcast address for that /24 subnet) then the
> packets are dropped. Here is an example of a packet trace when
> such a
> packet is received from
> 02:41:19:250212: rdma-input
> rdma: Interface101 (3) next-node bond-input
> 02:41:19:250214: bond-input
> src 02:fe:ff:76:e4:5d, dst ff:ff:ff:ff:ff:ff, Interface101 ->
> BondEthernet0
> 02:41:19:250214: ethernet-input
> IP4: 02:fe:ff:76:e4:5d -> ff:ff:ff:ff:ff:ff 802.1q vlan 1015
> 02:41:19:250215: ip4-input
> UDP: ->
> tos 0x80, ttl 254, length 92, checksum 0x02fa
> fragment id 0x0002, flags DONT_FRAGMENT
> UDP: 1234 -> 2345
> length 72, checksum 0x0000
> 02:41:19:250216: ip4-lookup
> fib 0 dpo-idx 0 flow hash: 0x00000000
> UDP: ->
> tos 0x80, ttl 254, length 92, checksum 0x02fa
> fragment id 0x0002, flags DONT_FRAGMENT
> UDP: 1234 -> 2345
> length 72, checksum 0x0000
> 02:41:19:250217: ip4-drop
> UDP: ->
> tos 0x80, ttl 254, length 92, checksum 0x02fa
> fragment id 0x0002, flags DONT_FRAGMENT
> UDP: 1234 -> 2345
> length 72, checksum 0x0000
> if you check:
> sh ip fib
> you'll see an explicit entry to drop. You can't override this.
> 02:41:19:250217: error-drop
> rx:BondEthernet0.1015
> 02:41:19:250217: drop
> ethernet-input: no error
> So the packet ends up at ip4-drop when I would have liked it to
> come to
> nat_ha_handoff_node.
> Does anyone have a suggestion about how to make this work?
> Is some special configuration of the receiving interface needed
> to tell
> VPP that we want it to receive broadcast packets?
> I'd suggest you multicast it instead. Pick an address and have the
> other servers listen.
> See:
> for programming multicast.
> Multicast will also work with IPv6 interface addresses.
> /neale

Join to automatically receive all group messages.