Topics

VPP DPDK build failure with Mellanox interface(aarch64)

Sirshak Das
 

Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined. */

/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined. */

/* HAVE_IBV_MLX5_MOD_MPW is not defined. */

/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_100G is not defined. */

/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-h.sh
$Q $(RM) -f -- '$@'
$Q sh -- '$<' '$@' \
HAVE_IBV_DEVICE_VXLAN_SUPPORT \
infiniband/verbs.h \
enum IBV_DEVICE_VXLAN_SUPPORT \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_IBV_WQ_FLAG_RX_END_PADDING \
infiniband/verbs.h \
enum IBV_WQ_FLAG_RX_END_PADDING \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_IBV_MLX5_MOD_MPW \
infiniband/mlx5dv.h \
enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_IBV_MLX5_MOD_CQE_128B_COMP \
infiniband/mlx5dv.h \
enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_ETHTOOL_LINK_MODE_25G \
/usr/include/linux/ethtool.h \
enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_ETHTOOL_LINK_MODE_50G \
/usr/include/linux/ethtool.h \
enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_ETHTOOL_LINK_MODE_100G \
/usr/include/linux/ethtool.h \
enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT \
$(AUTOCONF_OUTPUT)
$Q sh -- '$<' '$@' \
HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT \
infiniband/verbs.h \
enum IBV_FLOW_SPEC_ACTION_COUNT \
$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das

Marco Varlese
 

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

$Q $(RM) -f --
'$@'


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


infiniband/verbs.h
\


enum IBV_DEVICE_VXLAN_SUPPORT
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


infiniband/verbs.h
\


enum IBV_WQ_FLAG_RX_END_PADDING
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_MPW
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_25G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_50G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_100G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

infiniband/verbs.h
\


enum IBV_FLOW_SPEC_ACTION_COUNT
\


$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das


--
Marco V

SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg

Sirshak Das
 

Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a
I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
JVPP GEN jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
JVPP GEN jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
JVPP GEN jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
JVPP GEN jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
JVPP GEN jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
JVPP GEN jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
JVPP GEN jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
JVPP GEN jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
JVPP GEN jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp4NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp6NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

$Q $(RM) -f --
'$@'


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


infiniband/verbs.h
\


enum IBV_DEVICE_VXLAN_SUPPORT
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


infiniband/verbs.h
\


enum IBV_WQ_FLAG_RX_END_PADDING
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_MPW
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_25G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_50G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_100G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

infiniband/verbs.h
\


enum IBV_FLOW_SPEC_ACTION_COUNT
\


$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das


Dave Wallace
 

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:
Hi Marco,

Thanks for replying. 
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a

I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/                                                                     
                                                                              
    
                                                                              
                                                                              

/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              

/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/                                                                     
                                                                              
            
                                                                              
                                                                              

/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              

/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/                                                                     
                                                                              

                                                                              
                                                                              

/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/                                                                     
                                                                              
      
                                                                              
                                                                              

/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */ 

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh                                                                   
                                                                       
        $Q $(RM) -f --
'$@'                                                                   
                                                                              

        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                infiniband/verbs.h
\                                                                             
                                                                              

                enum IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                infiniband/verbs.h
\                                                                             
                                                                              

                enum IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_IBV_MLX5_MOD_MPW
\                                                                             
                                                                              
              
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\                                                                             
                                                                            
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\                                                                             
                                                                              
    
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\                                                                             
                                                                          
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_ETHTOOL_LINK_MODE_25G
\                                                                             
                                                                              

                /usr/include/linux/ethtool.h
\                                                                             
                                                                              

                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\                                                                             
                                                                      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_ETHTOOL_LINK_MODE_50G
\                                                                             
                                                                              

                /usr/include/linux/ethtool.h
\                                                                             
                                                                              

                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\                                                                             

                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_ETHTOOL_LINK_MODE_100G
\                                                                             
                                                                              
        
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              

                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\                                                                             
                                                                    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              

                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\                                                                             
                                                                       
                infiniband/verbs.h
\                                                                             
                                                                              

                enum IBV_FLOW_SPEC_ACTION_COUNT
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h 
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das






Marco Varlese
 

On Mon, 2018-03-05 at 16:44 -0600, Sirshak Das wrote:
Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.Indeed I missed your last part of email... :(

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a
I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
I reported to you what's stated by the DPDK folks on their guide
(http://dpdk.org/doc/guides/nics/mlx5.html):

<<
22.6.1.1. RMDA Core with Linux Kernel
Minimal kernel version : v4.14 or the most recent 4.14-rc (see Linux
installation documentation)
Minimal rdma-core version: v15+ commit 0c5f5765213a (“Merge pull request #227
from yishaih/tm”) (see RDMA Core installation documentation)
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-
native/vpp/vpp-api/java'
JVPP GEN jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
JVPP GEN jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
JVPP GEN jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
JVPP GEN jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
JVPP GEN jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
JVPP GEN jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
JVPP GEN jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
JVPP GEN jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
JVPP GEN jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-
core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177:
error: cannot find symbol
public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats
request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws
io.fd.vpp.jvpp.VppInvocationException {

^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-
core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183:
error: cannot find symbol
public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats
request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws
io.fd.vpp.jvpp.VppInvocationException {

^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368:
error: cannot find symbol
void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request,
io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws
io.fd.vpp.jvpp.VppInvocationException;

^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369:
error: cannot find symbol
void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request,
io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws
io.fd.vpp.jvpp.VppInvocationException;

^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815:
error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp4NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820:
error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp6NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.
I see Dave W already replied to you about this and can confirm that the revert
of that patch fixes the above issue.


Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/









/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/









/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/









/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/









/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/









/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/









/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh



$Q $(RM) -f --
'$@'




$Q sh -- '$<' '$@'
\




HAVE_IBV_DEVICE_VXLAN_SUPPORT
\




infiniband/verbs.h
\




enum IBV_DEVICE_VXLAN_SUPPORT
\




$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_IBV_WQ_FLAG_RX_END_PADDING
\




infiniband/verbs.h
\




enum IBV_WQ_FLAG_RX_END_PADDING
\




$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_IBV_MLX5_MOD_MPW
\




infiniband/mlx5dv.h
\




enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\



$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\




infiniband/mlx5dv.h
\




enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\


$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_ETHTOOL_LINK_MODE_25G
\




/usr/include/linux/ethtool.h
\




enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\


$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_ETHTOOL_LINK_MODE_50G
\




/usr/include/linux/ethtool.h
\




enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\


$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_ETHTOOL_LINK_MODE_100G
\




/usr/include/linux/ethtool.h
\




enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\


$(AUTOCONF_OUTPUT)




$Q sh -- '$<' '$@'
\




HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\



infiniband/verbs.h
\




enum IBV_FLOW_SPEC_ACTION_COUNT
\




$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das


--
Marco V

SUSE LINUX GmbH | GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg) Maxfeldstr. 5, D-90409, Nürnberg

Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) <mgradzki@...>
 

Hi,

 

verify of 10920 would fail if it was rebased before submission.

 

JVPP generation was updated recently to use service definitions

Instead of inferring message type based on its name.

 

10920 was verified against previous version.

 

Here is updated version of 10920 with updated service definitions:

https://gerrit.fd.io/r/#/c/10973/

 

Regards,

Marek

 

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

 

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:

Hi Marco,
 
Thanks for replying. 
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
 
I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:
 
# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes
 
I was able to go past the issue but currently I am facing issue with
jvpp building:
 
$ javac -version
javac 1.8.0_161
 
$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
 
 
Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
 
 
If you or anybody else could help me with the issue it will be great.
 
Thank you
Sirshak Das
 
 
Marco Varlese writes:
 
Hi Sirshak,
 
Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219
 
Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...
 
Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x
 
Hope this helps.
 
 
Cheers,
Marco
 
On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,
 
I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.
 
The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:
 
Kernel Version:
$ uname -r
4.10.0-28-generic
 
Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)
 
gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)
 
Target Machine:
Qualcomm Centriq
 
lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]
 
Steps I followed:
 
$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y
 
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {
 
 
By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h
 
Which looks like this:
 
/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/                                                                     
                                                                              
    
                                                                              
                                                                              
 
/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/                                                                     
                                                                              
            
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */
 
/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/                                                                     
                                                                              
 
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/                                                                     
                                                                              
      
                                                                              
                                                                              
 
/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */ 
 
So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh                                                                   
                                                                       
        $Q $(RM) -f --
'$@'                                                                   
                                                                              
 
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_MPW
\                                                                             
                                                                              
              
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\                                                                             
                                                                            
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\                                                                             
                                                                              
    
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\                                                                             
                                                                          
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_25G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\                                                                             
                                                                      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_50G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\                                                                             
 
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_100G
\                                                                             
                                                                              
        
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\                                                                             
                                                                    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\                                                                             
                                                                       
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_FLOW_SPEC_ACTION_COUNT
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)
 
Which ideally shouldnt because all these files exist:
 
$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h
 
$ ls -l /usr/include/linux/ethtool.h 
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h
 
 
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.
 
Thank you
Sirshak Das
 
 
 
 
 
 
 

 

Dave Wallace
 

Hi Marek,

Thanks for the analysis.  VPP's gerrit "Submit Type" is "Rebase if Necessary".  This means that occasionally we may encounter build breakage due to dependent patches.  Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:

Hi,

 

verify of 10920 would fail if it was rebased before submission.

 

JVPP generation was updated recently to use service definitions

Instead of inferring message type based on its name.

 

10920 was verified against previous version.

 

Here is updated version of 10920 with updated service definitions:

https://gerrit.fd.io/r/#/c/10973/

 

Regards,

Marek

 

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

 

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:

Hi Marco,
 
Thanks for replying. 
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
 
I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:
 
# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes
 
I was able to go past the issue but currently I am facing issue with
jvpp building:
 
$ javac -version
javac 1.8.0_161
 
$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
 
 
Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
 
 
If you or anybody else could help me with the issue it will be great.
 
Thank you
Sirshak Das
 
 
Marco Varlese writes:
 
Hi Sirshak,
 
Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219
 
Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...
 
Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x
 
Hope this helps.
 
 
Cheers,
Marco
 
On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,
 
I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.
 
The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:
 
Kernel Version:
$ uname -r
4.10.0-28-generic
 
Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)
 
gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)
 
Target Machine:
Qualcomm Centriq
 
lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]
 
Steps I followed:
 
$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y
 
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {
 
 
By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h
 
Which looks like this:
 
/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/                                                                     
                                                                              
    
                                                                              
                                                                              
 
/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/                                                                     
                                                                              
            
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */
 
/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/                                                                     
                                                                              
 
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/                                                                     
                                                                              
      
                                                                              
                                                                              
 
/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */ 
 
So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh                                                                   
                                                                       
        $Q $(RM) -f --
'$@'                                                                   
                                                                              
 
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_MPW
\                                                                             
                                                                              
              
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\                                                                             
                                                                            
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\                                                                             
                                                                              
    
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\                                                                             
                                                                          
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_25G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\                                                                             
                                                                      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_50G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\                                                                             
 
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_100G
\                                                                             
                                                                              
        
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\                                                                             
                                                                    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\                                                                             
                                                                       
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_FLOW_SPEC_ACTION_COUNT
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)
 
Which ideally shouldnt because all these files exist:
 
$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h
 
$ ls -l /usr/include/linux/ethtool.h 
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h
 
 
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.
 
Thank you
Sirshak Das
 
 
 
 
 
 
 

 

Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) <mgradzki@...>
 

Dave,

 

these patches were not dependent, but rather in conflict.

 

And this is the responsibility of merge job to detect them.

If such situation happens patch should be reverted.

So I think everything was done right.

 

I am totally fine with “Rebase if necessary”.

But personally I often rebase patches before submitting.

Especially these that might affect several components.

 

Regards,

Marek

 

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

 

Hi Marek,

Thanks for the analysis.  VPP's gerrit "Submit Type" is "Rebase if Necessary".  This means that occasionally we may encounter build breakage due to dependent patches.  Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:

Hi,

 

verify of 10920 would fail if it was rebased before submission.

 

JVPP generation was updated recently to use service definitions

Instead of inferring message type based on its name.

 

10920 was verified against previous version.

 

Here is updated version of 10920 with updated service definitions:

https://gerrit.fd.io/r/#/c/10973/

 

Regards,

Marek

 

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

 

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:

Hi Marco,
 
Thanks for replying. 
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
 
I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:
 
# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes
 
I was able to go past the issue but currently I am facing issue with
jvpp building:
 
$ javac -version
javac 1.8.0_161
 
$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
 
 
Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
 
 
If you or anybody else could help me with the issue it will be great.
 
Thank you
Sirshak Das
 
 
Marco Varlese writes:
 
Hi Sirshak,
 
Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219
 
Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...
 
Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x
 
Hope this helps.
 
 
Cheers,
Marco
 
On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,
 
I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.
 
The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:
 
Kernel Version:
$ uname -r
4.10.0-28-generic
 
Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)
 
gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)
 
Target Machine:
Qualcomm Centriq
 
lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]
 
Steps I followed:
 
$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y
 
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {
 
 
By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h
 
Which looks like this:
 
/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/                                                                     
                                                                              
    
                                                                              
                                                                              
 
/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/                                                                     
                                                                              
            
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */
 
/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/                                                                     
                                                                              
 
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/                                                                     
                                                                              
      
                                                                              
                                                                              
 
/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */ 
 
So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh                                                                   
                                                                       
        $Q $(RM) -f --
'$@'                                                                   
                                                                              
 
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_MPW
\                                                                             
                                                                              
              
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\                                                                             
                                                                            
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\                                                                             
                                                                              
    
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\                                                                             
                                                                          
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_25G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\                                                                             
                                                                      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_50G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\                                                                             
 
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_100G
\                                                                             
                                                                              
        
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\                                                                             
                                                                    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\                                                                             
                                                                       
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_FLOW_SPEC_ACTION_COUNT
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)
 
Which ideally shouldnt because all these files exist:
 
$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h
 
$ ls -l /usr/include/linux/ethtool.h 
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h
 
 
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.
 
Thank you
Sirshak Das
 
 
 
 
 
 
 

 

 

Dave Wallace
 

Marek,

Thanks for the clarification. 
I'm trying to understand if there is a way to avoid the branch build breakage.

Cheers,
-daw-

On 3/6/2018 11:12 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) wrote:

Dave,

 

these patches were not dependent, but rather in conflict.

 

And this is the responsibility of merge job to detect them.

If such situation happens patch should be reverted.

So I think everything was done right.

 

I am totally fine with “Rebase if necessary”.

But personally I often rebase patches before submitting.

Especially these that might affect several components.

 

Regards,

Marek

 

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

 

Hi Marek,

Thanks for the analysis.  VPP's gerrit "Submit Type" is "Rebase if Necessary".  This means that occasionally we may encounter build breakage due to dependent patches.  Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:

Hi,

 

verify of 10920 would fail if it was rebased before submission.

 

JVPP generation was updated recently to use service definitions

Instead of inferring message type based on its name.

 

10920 was verified against previous version.

 

Here is updated version of 10920 with updated service definitions:

https://gerrit.fd.io/r/#/c/10973/

 

Regards,

Marek

 

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

 

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:

Hi Marco,
 
Thanks for replying. 
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
 
I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:
 
# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes
 
I was able to go past the issue but currently I am facing issue with
jvpp building:
 
$ javac -version
javac 1.8.0_161
 
$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
 
 
Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
 
 
If you or anybody else could help me with the issue it will be great.
 
Thank you
Sirshak Das
 
 
Marco Varlese writes:
 
Hi Sirshak,
 
Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219
 
Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...
 
Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x
 
Hope this helps.
 
 
Cheers,
Marco
 
On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,
 
I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.
 
The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:
 
Kernel Version:
$ uname -r
4.10.0-28-generic
 
Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)
 
gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)
 
Target Machine:
Qualcomm Centriq
 
lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]
 
Steps I followed:
 
$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y
 
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {
 
 
By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h
 
Which looks like this:
 
/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/                                                                     
                                                                              
    
                                                                              
                                                                              
 
/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/                                                                     
                                                                              
            
                                                                              
                                                                              
 
/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/                                                                     
                                                                              
  
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */
 
/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/                                                                     
                                                                              
 
                                                                              
                                                                              
 
/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/                                                                     
                                                                              
      
                                                                              
                                                                              
 
/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */ 
 
So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh                                                                   
                                                                       
        $Q $(RM) -f --
'$@'                                                                   
                                                                              
 
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_DEVICE_VXLAN_SUPPORT
\                                                                             
                                                                              
      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_WQ_FLAG_RX_END_PADDING
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_MPW
\                                                                             
                                                                              
              
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\                                                                             
                                                                            
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\                                                                             
                                                                              
    
                infiniband/mlx5dv.h
\                                                                             
                                                                              
                
                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\                                                                             
                                                                          
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_25G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\                                                                             
                                                                      
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_50G
\                                                                             
                                                                              
 
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\                                                                             
 
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_ETHTOOL_LINK_MODE_100G
\                                                                             
                                                                              
        
                /usr/include/linux/ethtool.h
\                                                                             
                                                                              
 
                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\                                                                             
                                                                    
                $(AUTOCONF_OUTPUT)                                     
                                                                              
                                                    
        $Q sh -- '$<' '$@'
\                                                                             
                                                                              
 
                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\                                                                             
                                                                       
                infiniband/verbs.h
\                                                                             
                                                                              
 
                enum IBV_FLOW_SPEC_ACTION_COUNT
\                                                                             
                                                                              
    
                $(AUTOCONF_OUTPUT)
 
Which ideally shouldnt because all these files exist:
 
$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h
 
$ ls -l /usr/include/linux/ethtool.h 
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h
 
 
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.
 
Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a
 
I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.
 
Thank you
Sirshak Das
 
 
 
 
 
 
 

 

 

Ole Troan
 

Thanks for the clarification.
I'm trying to understand if there is a way to avoid the branch build breakage.
You could imagine that you could be more clever estimating the chance of a conflict.
In this particular case though, there was no overlap in changeset, so I'd imagine that automatic detection would have been hard.
Let's assume it's a rare event.

Cheers,
Ole

On 3/6/2018 11:12 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) wrote:
Dave,

these patches were not dependent, but rather in conflict.

And this is the responsibility of merge job to detect them.
If such situation happens patch should be reverted.
So I think everything was done right.

I am totally fine with “Rebase if necessary”.
But personally I often rebase patches before submitting.
Especially these that might affect several components.

Regards,
Marek

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Hi Marek,

Thanks for the analysis. VPP's gerrit "Submit Type" is "Rebase if Necessary". This means that occasionally we may encounter build breakage due to dependent patches. Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:
Hi,

verify of 10920 would fail if it was rebased before submission.

JVPP generation was updated recently to use service definitions
Instead of inferring message type based on its name.

10920 was verified against previous version.

Here is updated version of 10920 with updated service definitions:
https://gerrit.fd.io/r/#/c/10973/

Regards,
Marek

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job. I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:
Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a


I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
JVPP GEN jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
JVPP GEN jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
JVPP GEN jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
JVPP GEN jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
JVPP GEN jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
JVPP GEN jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
JVPP GEN jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
JVPP GEN jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
JVPP GEN jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp4NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp6NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

$Q $(RM) -f --
'$@'


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


infiniband/verbs.h
\


enum IBV_DEVICE_VXLAN_SUPPORT
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


infiniband/verbs.h
\


enum IBV_WQ_FLAG_RX_END_PADDING
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_MPW
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_25G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_50G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_100G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

infiniband/verbs.h
\


enum IBV_FLOW_SPEC_ACTION_COUNT
\


$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das








Dave Wallace
 

Ole,

That is what I thought, but I wanted to see if there's any way to close the gap. In this case it is clear that the cost of automatic detection or avoidance is way to high.  I'm fine with leaving things as they are.

Thanks,
-daw-

On 03/06/2018 02:21 PM, Ole Troan wrote:
Thanks for the clarification.
I'm trying to understand if there is a way to avoid the branch build breakage.
You could imagine that you could be more clever estimating the chance of a conflict.
In this particular case though, there was no overlap in changeset, so I'd imagine that automatic detection would have been hard.
Let's assume it's a rare event.

Cheers,
Ole
On 3/6/2018 11:12 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) wrote:
Dave,

these patches were not dependent, but rather in conflict.

And this is the responsibility of merge job to detect them.
If such situation happens patch should be reverted.
So I think everything was done right.

I am totally fine with “Rebase if necessary”.
But personally I often rebase patches before submitting.
Especially these that might affect several components.

Regards,
Marek

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Hi Marek,

Thanks for the analysis.  VPP's gerrit "Submit Type" is "Rebase if Necessary".  This means that occasionally we may encounter build breakage due to dependent patches.  Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:
Hi,

verify of 10920 would fail if it was rebased before submission.

JVPP generation was updated recently to use service definitions
Instead of inferring message type based on its name.

10920 was verified against previous version.

Here is updated version of 10920 with updated service definitions:
https://gerrit.fd.io/r/#/c/10973/

Regards,
Marek

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:
Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a


I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

        $Q $(RM) -f --
'$@'


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


                infiniband/verbs.h
\


                enum IBV_DEVICE_VXLAN_SUPPORT
\


                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


                infiniband/verbs.h
\


                enum IBV_WQ_FLAG_RX_END_PADDING
\


                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_MLX5_MOD_MPW
\


                infiniband/mlx5dv.h
\


                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


                infiniband/mlx5dv.h
\


                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_ETHTOOL_LINK_MODE_25G
\


                /usr/include/linux/ethtool.h
\


                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_ETHTOOL_LINK_MODE_50G
\


                /usr/include/linux/ethtool.h
\


                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_ETHTOOL_LINK_MODE_100G
\


                /usr/include/linux/ethtool.h
\


                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

                infiniband/verbs.h
\


                enum IBV_FLOW_SPEC_ACTION_COUNT
\


                $(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das













Ole Troan
 

Dave,

That is what I thought, but I wanted to see if there's any way to close the gap. In this case it is clear that the cost of automatic detection or avoidance is way to high. I'm fine with leaving things as they are.
One tooling improvement we could make, would be to send out a big fat alert when the merge build fails.

Cheers,
Ole

On 03/06/2018 02:21 PM, Ole Troan wrote:
Thanks for the clarification.
I'm trying to understand if there is a way to avoid the branch build breakage.
You could imagine that you could be more clever estimating the chance of a conflict.
In this particular case though, there was no overlap in changeset, so I'd imagine that automatic detection would have been hard.
Let's assume it's a rare event.

Cheers,
Ole

On 3/6/2018 11:12 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) wrote:

Dave,

these patches were not dependent, but rather in conflict.

And this is the responsibility of merge job to detect them.
If such situation happens patch should be reverted.
So I think everything was done right.

I am totally fine with “Rebase if necessary”.
But personally I often rebase patches before submitting.
Especially these that might affect several components.

Regards,
Marek

From:
vpp-dev@... [mailto:vpp-dev@...
] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To:
vpp-dev@...

Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Hi Marek,

Thanks for the analysis. VPP's gerrit "Submit Type" is "Rebase if Necessary". This means that occasionally we may encounter build breakage due to dependent patches. Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:
Hi,

verify of 10920 would fail if it was rebased before submission.

JVPP generation was updated recently to use service definitions
Instead of inferring message type based on its name.

10920 was verified against previous version.

Here is updated version of 10920 with updated service definitions:

https://gerrit.fd.io/r/#/c/10973/


Regards,
Marek

From:
vpp-dev@... [mailto:vpp-dev@...
] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To:
vpp-dev@...

Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job. I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:
Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a


I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
JVPP GEN jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
JVPP GEN jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
JVPP GEN jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
JVPP GEN jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
JVPP GEN jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
JVPP GEN jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
JVPP GEN jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
JVPP GEN jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
JVPP GEN jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp4NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp6NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:

https://lists.fd.io/g/vpp-dev/message/8219


Also, take a look at
http://dpdk.org/doc/guides/nics/mlx5.html
for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

$Q $(RM) -f --
'$@'


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


infiniband/verbs.h
\


enum IBV_DEVICE_VXLAN_SUPPORT
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


infiniband/verbs.h
\


enum IBV_WQ_FLAG_RX_END_PADDING
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_MPW
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_25G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_50G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_100G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

infiniband/verbs.h
\


enum IBV_FLOW_SPEC_ACTION_COUNT
\


$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das











Dave Wallace
 

Yep.  Ed Kern and I are investigating how to make that happen.

Thanks,
-daw-

On 03/06/2018 05:09 PM, Ole Troan wrote:
Dave,

That is what I thought, but I wanted to see if there's any way to close the gap. In this case it is clear that the cost of automatic detection or avoidance is way to high.  I'm fine with leaving things as they are.
One tooling improvement we could make, would be to send out a big fat alert when the merge build fails.

Cheers,
Ole

On 03/06/2018 02:21 PM, Ole Troan wrote:
Thanks for the clarification.
I'm trying to understand if there is a way to avoid the branch build breakage.

You could imagine that you could be more clever estimating the chance of a conflict.
In this particular case though, there was no overlap in changeset, so I'd imagine that automatic detection would have been hard.
Let's assume it's a rare event.

Cheers,
Ole

On 3/6/2018 11:12 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) wrote:

Dave,

these patches were not dependent, but rather in conflict.

And this is the responsibility of merge job to detect them.
If such situation happens patch should be reverted.
So I think everything was done right.

I am totally fine with “Rebase if necessary”.
But personally I often rebase patches before submitting.
Especially these that might affect several components.

Regards,
Marek

From:
vpp-dev@... [mailto:vpp-dev@...
] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To:
vpp-dev@...

Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Hi Marek,

Thanks for the analysis.  VPP's gerrit "Submit Type" is "Rebase if Necessary".  This means that occasionally we may encounter build breakage due to dependent patches.  Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:
Hi,

verify of 10920 would fail if it was rebased before submission.

JVPP generation was updated recently to use service definitions
Instead of inferring message type based on its name.

10920 was verified against previous version.

Here is updated version of 10920 with updated service definitions:

https://gerrit.fd.io/r/#/c/10973/


Regards,
Marek

From:
vpp-dev@... [mailto:vpp-dev@...
] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To:
vpp-dev@...

Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job.  I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:
Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a


I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
  JVPP GEN  jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
  JVPP GEN  jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
  JVPP GEN  jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
  JVPP GEN  jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
  JVPP GEN  jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
  JVPP GEN  jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
  JVPP GEN  jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
  JVPP GEN  jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
  JVPP GEN  jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
    public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
    public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
                                                                                                                   ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
    void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp4NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
    void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
                                                                                                      ^
  symbol:   class WantIp6NbrStatsCallback
  location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp4NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
        return send(request);
                   ^
    equality constraints: WantIp6NbrStats
    upper bounds: JVppReply<REQ>
  where REPLY,REQ are type-variables:
    REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
    REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:

https://lists.fd.io/g/vpp-dev/message/8219


Also, take a look at
http://dpdk.org/doc/guides/nics/mlx5.html
 for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
 struct ibv_flow_spec_counter_action {
        ^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
 struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

        $Q $(RM) -f --
'$@'


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


                infiniband/verbs.h
\


                enum IBV_DEVICE_VXLAN_SUPPORT
\


                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


                infiniband/verbs.h
\


                enum IBV_WQ_FLAG_RX_END_PADDING
\


                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_MLX5_MOD_MPW
\


                infiniband/mlx5dv.h
\


                enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


                infiniband/mlx5dv.h
\


                enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_ETHTOOL_LINK_MODE_25G
\


                /usr/include/linux/ethtool.h
\


                enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_ETHTOOL_LINK_MODE_50G
\


                /usr/include/linux/ethtool.h
\


                enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_ETHTOOL_LINK_MODE_100G
\


                /usr/include/linux/ethtool.h
\


                enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

                $(AUTOCONF_OUTPUT)


        $Q sh -- '$<' '$@'
\


                HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

                infiniband/verbs.h
\


                enum IBV_FLOW_SPEC_ACTION_COUNT
\


                $(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root  1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root  2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root  5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root  4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root  1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
  CC mlx5.o
  PMDINFO mlx5.o.pmd.c
  CC mlx5.o.pmd.o
  LD mlx5.o
  CC mlx5_glue.o
  CC mlx5_rxq.o
  CC mlx5_txq.o
  CC mlx5_rxtx.o
  CC mlx5_rxtx_vec.o
  CC mlx5_trigger.o
  CC mlx5_ethdev.o
  CC mlx5_mac.o
  CC mlx5_rxmode.o
  CC mlx5_vlan.o
  CC mlx5_stats.o
  CC mlx5_rss.o
  CC mlx5_mr.o
  CC mlx5_flow.o
  CC mlx5_socket.o
  AR librte_pmd_mlx5.a
  INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das











          




      

    

Sirshak Das
 

Going back to the original discussion:
I have VPP working now on aarch64 with Mellanox Card.

Disclaimer:
$ uname -r
4.10.0-28-generic
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)
I am aware that the published supported version for mellanox with dpdk
is 4.14+

I am listing down my steps along with errors I faced and the work
arounds I used.

They may not be committable workarounds and I am not familiar with code
base of either vpp or dpdk to guarentee that they are the best workarounds
either.

Prereq for anything:
MOFED Installation(Very Important):

Download from:

http://www.mellanox.com/page/products_dyn?product_family=26

First VPP downloading DPDK does not work for me:
Error:
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {
Workaround:

I use the same DPDK downloaded by VPP master
and make VPP use an external DPDK.

http://fast.dpdk.org/rel/dpdk-18.02.tar.xz

$ make config T=arm64-armv8a-linuxapp-gcc

Edit .config file at build/.config and mark this as yes:
CONFIG_RTE_LIBRTE_MLX5_PMD=y

$ make

VPP Compile:

Errors:
$ sudo ./bin/vpp -c ./etc/startup.conf

./bin/vpp[5054]: vnet_feature_arc_init:205: feature node 'acl-plugin-out-ip6-fa' not found
./bin/vpp[5054]: vnet_feature_arc_init:205: feature node 'acl-plugin-out-ip4-fa' not found

./bin/vpp[5054]: dpdk_config:1260: EAL init args: -c 1 -n 4 --huge-dir /run/vpp/hugepages --file-prefix vpp -w 0004:01:00.1 -w 0004:01:00.2 --master-lcore 0 --socket-mem 64
EAL: DPAA2: DPRC not available
EAL: 2048 hugepages of size 2097152 reserved, but no mounted hugetlbfs found for that size
EAL: VFIO support initialized
./bin/vpp: symbol lookup error: /home/sirdas/code/commita/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins/dpdk_plugin.so: undefined symbol: mlx5dv_query_device

Note: Before this another error also comes up due to which you will have
compile dpdk_plugin.so with -libverbs its again some symbol
resolution issue. The above error was fixed adding -lmlx5.


Workaround:

I apply the following patch(Dont Copy-Paste, Edit <Install Dir> in your patch):

$ cat mlx5_vpp.diff
diff --git a/build-data/platforms/vpp.mk b/build-data/platforms/vpp.mk
index 320609d..4587e19 100644
--- a/build-data/platforms/vpp.mk
+++ b/build-data/platforms/vpp.mk
@@ -29,13 +29,13 @@ vpp_uses_dpdk = yes
vpp_root_packages = vpp

# DPDK configuration parameters
-# vpp_uses_dpdk_mlx5_pmd = yes
-# vpp_uses_external_dpdk = yes
-# vpp_dpdk_inc_dir = /usr/include/dpdk
-# vpp_dpdk_lib_dir = /usr/lib
+vpp_uses_dpdk_mlx5_pmd = yes
+vpp_uses_external_dpdk = yes
+vpp_dpdk_inc_dir = <Install DIR>/dpdk-18.02/build/include
+vpp_dpdk_lib_dir = <Install Dir>/dpdk-18.02/build/lib
# vpp_dpdk_shared_lib = yes

-vpp_configure_args_vpp =
+vpp_configure_args_vpp += --disable-japi

# load balancer plugin is not portable on 32 bit platform
ifeq ($(MACHINE),i686)
diff --git a/src/plugins/dpdk.am b/src/plugins/dpdk.am
index 10f2fe4..7a1f2be 100644
--- a/src/plugins/dpdk.am
+++ b/src/plugins/dpdk.am
@@ -26,7 +26,7 @@ if WITH_ISA_L_CRYPTO_LIB
dpdk_plugin_la_LDFLAGS += -Wl,--exclude-libs,libisal_crypto.a,-l:libisal_crypto.a
endif
dpdk_plugin_la_CFLAGS = $(AM_CFLAGS)
-dpdk_plugin_la_LDFLAGS += -Wl,-lnuma
+dpdk_plugin_la_LDFLAGS += -Wl,-lnuma,-libverbs,-lmlx5

dpdk_plugin_la_LDFLAGS += -Wl,-lm,-ldl
dpdk_plugin_la_LIBADD =


+vpp_configure_args_vpp += --disable-japi
- I choose to disable this because at the time of writing, japi build
was broken.


$ make build-release

These steps work for me. If somebody has faced similar issues and has a
better workaround then let me know.

P.S. I tried an alternate solution of building dpdk libraries as shared
it didnt work out for me. This is the error I got:

ERROR: failed to parse device "0004:01:00.1"
EAL: Unable to parse device '0004:01:00.1'

But this seems like more of DPDK issue as running DPDK examples also
gives similar errors:

sudo LD_LIBRARY_PATH=<DPDK Install Path>/dpdk-18.02/build/lib:$LD_LIBRARY_PATH ./l2fwd -l 4-7 -n 4 -w 0004:01:00.1 -w 0004:01:00.2 -- -p 0x3
EAL: Detected 46 lcore(s)
ERROR: failed to parse device "0004:01:00.1"
EAL: Unable to parse device '0004:01:00.1'
EAL: Error - exiting with code: 1 Cause: Invalid EAL arguments

$ lspci
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4]
0004:01:00.1 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function]
0004:01:00.2 Ethernet controller: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function]

Thank you
Sirshak Das


Dave Wallace writes:

Ole,

That is what I thought, but I wanted to see if there's any way to close
the gap. In this case it is clear that the cost of automatic detection
or avoidance is way to high. I'm fine with leaving things as they are.

Thanks,
-daw-

On 03/06/2018 02:21 PM, Ole Troan wrote:
Thanks for the clarification.
I'm trying to understand if there is a way to avoid the branch build breakage.
You could imagine that you could be more clever estimating the chance of a conflict.
In this particular case though, there was no overlap in changeset, so I'd imagine that automatic detection would have been hard.
Let's assume it's a rare event.

Cheers,
Ole
On 3/6/2018 11:12 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) wrote:
Dave,

these patches were not dependent, but rather in conflict.

And this is the responsibility of merge job to detect them.
If such situation happens patch should be reverted.
So I think everything was done right.

I am totally fine with “Rebase if necessary”.
But personally I often rebase patches before submitting.
Especially these that might affect several components.

Regards,
Marek

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 16:59
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Hi Marek,

Thanks for the analysis. VPP's gerrit "Submit Type" is "Rebase if Necessary". This means that occasionally we may encounter build breakage due to dependent patches. Changing the type to "Fast forward only" is NOT worth the extra time/effort to avoid this.

In general, dependent patches are not recommended, but sometimes it can be better to separate the change sets into smaller patches. It would also be helpful if the merge jobs would send an email to the reviewers list on the patch.

Thanks,
-daw-

On 3/6/18 4:20 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) wrote:
Hi,

verify of 10920 would fail if it was rebased before submission.

JVPP generation was updated recently to use service definitions
Instead of inferring message type based on its name.

10920 was verified against previous version.

Here is updated version of 10920 with updated service definitions:
https://gerrit.fd.io/r/#/c/10973/

Regards,
Marek

From: vpp-dev@... [mailto:vpp-dev@...] On Behalf Of Dave Wallace
Sent: 6 marca 2018 00:19
To: vpp-dev@...
Subject: Re: [vpp-dev] VPP DPDK build failure with Mellanox interface(aarch64)

Sirshak,

The JVPP error was caused by a patch (10920) that somehow passed the Verify job, but failed during the merge job. I have just merged the patch (10965) which reverts the patch which introduced the JVPP build errors.

Thanks,
-daw-

On 3/5/2018 5:44 PM, Sirshak Das wrote:
Hi Marco,

Thanks for replying.
I am not sure if you read the last part of my email:
Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a


I dont know if Kernel Version is still an issue as dpdk(mlx5 pmd) works
for me on its own.
So in order to go past the issue I tried this change to vpp.mk:

# DPDK configuration parameters
vpp_uses_dpdk_mlx5_pmd = yes
vpp_uses_external_dpdk = yes
vpp_dpdk_inc_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/include
vpp_dpdk_lib_dir = <DPDK INSTALL DIR>/dpdk-18.02/build/lib
vpp_dpdk_shared_lib = yes

I was able to go past the issue but currently I am facing issue with
jvpp building:

$ javac -version
javac 1.8.0_161

$ java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)


Entering directory '/home/sirdas/code/common/vpp/build-root/build-vpp-native/vpp/vpp-api/java'
JVPP GEN jvpp-registry/io_fd_vpp_jvpp_VppJNIConnection.h
JVPP GEN jvpp-gtpu/io_fd_vpp_jvpp_gtpu_JVppGtpuImpl.h
JVPP GEN jvpp-pppoe/io_fd_vpp_jvpp_pppoe_JVppPppoeImpl.h
JVPP GEN jvpp-core/io_fd_vpp_jvpp_core_JVppCoreImpl.h
JVPP GEN jvpp-ioampot/io_fd_vpp_jvpp_ioampot_JVppIoampotImpl.h
JVPP GEN jvpp-acl/io_fd_vpp_jvpp_acl_JVppAclImpl.h
JVPP GEN jvpp-ioamexport/io_fd_vpp_jvpp_ioamexport_JVppIoamexportImpl.h
JVPP GEN jvpp-ioamtrace/io_fd_vpp_jvpp_ioamtrace_JVppIoamtraceImpl.h
JVPP GEN jvpp-nat/io_fd_vpp_jvpp_nat_JVppNatImpl.h
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2177: error: cannot find symbol
public final void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCoreFacade.java:2183: error: cannot find symbol
public final void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException {
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:368: error: cannot find symbol
void wantIp4NbrStats(io.fd.vpp.jvpp.core.dto.WantIp4NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp4NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp4NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/callfacade/CallbackJVppCore.java:369: error: cannot find symbol
void wantIp6NbrStats(io.fd.vpp.jvpp.core.dto.WantIp6NbrStats request, io.fd.vpp.jvpp.core.callback.WantIp6NbrStatsCallback callback) throws io.fd.vpp.jvpp.VppInvocationException;
^
symbol: class WantIp6NbrStatsCallback
location: package io.fd.vpp.jvpp.core.callback
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1815: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp4NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)
jvpp-core/target/io/fd/vpp/jvpp/core/future/FutureJVppCoreFacade.java:1820: error: incompatible types: inference variable REPLY has incompatible bounds
return send(request);
^
equality constraints: WantIp6NbrStats
upper bounds: JVppReply<REQ>
where REPLY,REQ are type-variables:
REPLY extends JVppReply<REQ> declared in method <REQ,REPLY>send(REQ)
REQ extends JVppRequest declared in method <REQ,REPLY>send(REQ)


If you or anybody else could help me with the issue it will be great.

Thank you
Sirshak Das


Marco Varlese writes:

Hi Sirshak,

Can you please take a look at this thread:
https://lists.fd.io/g/vpp-dev/message/8219

Also, take a look at http://dpdk.org/doc/guides/nics/mlx5.html for detailed
explanations and steps for DPDK to work with Mellanox NICs...

Likely, it's a kernel issue: for Mellanox latest PMD drivers (DPDK 17.11 and
above) you're required to have 4.15.x whilst I see you're running 4.10.x

Hope this helps.


Cheers,
Marco

On Thu, 2018-03-01 at 18:02 -0600, Sirshak Das wrote:
Hi all,

I am facing an issue with building VPP with DPDK (Mellanox Drivers
enabled) on aarch64.

The OFED version installed:
$ ofed_info -s
MLNX_OFED_LINUX-4.2-1.2.0.0:

Kernel Version:
$ uname -r
4.10.0-28-generic

Operating System Version:
Ubuntu 16.04.4 LTS (GNU/Linux 4.10.0-28-generic aarch64)

gcc version:
gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)

Target Machine:
Qualcomm Centriq

lspci:
0004:01:00.0 Ethernet controller: Mellanox Technologies MT27700 Family
[ConnectX-4]

Steps I followed:

$ git clone ssh://<my-user-id>@gerrit.fd.io:29418/vpp.git
$ cd vpp/
$ sed -i '/vpp_uses_dpdk_mlx5_pmd/s/^# //g' build-data/platforms/vpp.mk
$ make dpdk-install-dev DPDK_MLX5_PMD=y

<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:38:8: error: redefinition of ‘struct
ibv_flow_spec_counter_action’
struct ibv_flow_spec_counter_action {
^
In file included from <INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-
18.02/drivers/net/mlx5/mlx5_flow.c:14:0:
/usr/include/infiniband/verbs.h:1360:8: note: originally defined here
struct ibv_flow_spec_counter_action {


By looking at this file:
<INSTALL
DIR>/vpp/dpdk/deb/_build/dpdk-18.02/arm64-armv8a-linuxapp-
gcc/build/drivers/net/mlx5/mlx5_autoconf.h

Which looks like this:

/* HAVE_IBV_DEVICE_VXLAN_SUPPORT is not defined.
*/





/* HAVE_IBV_WQ_FLAG_RX_END_PADDING is not defined.
*/





/* HAVE_IBV_MLX5_MOD_MPW is not defined.
*/





/* HAVE_IBV_MLX5_MOD_CQE_128B_COMP is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_25G is not defined. */

/* HAVE_ETHTOOL_LINK_MODE_50G is not defined.
*/





/* HAVE_ETHTOOL_LINK_MODE_100G is not defined.
*/





/* HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT is not defined. */

So I moved my attentions to this file:
<INSTALL DIR>/vpp/dpdk/deb/_build/dpdk-18.02/drivers/net/mlx5/Makefile
This whole section fails:
mlx5_autoconf.h.new: $(RTE_SDK)/buildtools/auto-config-
h.sh

$Q $(RM) -f --
'$@'


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_VXLAN_SUPPORT
\


infiniband/verbs.h
\


enum IBV_DEVICE_VXLAN_SUPPORT
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_WQ_FLAG_RX_END_PADDING
\


infiniband/verbs.h
\


enum IBV_WQ_FLAG_RX_END_PADDING
\


$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_MPW
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_MLX5_MOD_CQE_128B_COMP
\


infiniband/mlx5dv.h
\


enum MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_25G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_25000baseCR_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_50G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_50000baseCR2_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_ETHTOOL_LINK_MODE_100G
\


/usr/include/linux/ethtool.h
\


enum ETHTOOL_LINK_MODE_100000baseKR4_Full_BIT
\

$(AUTOCONF_OUTPUT)


$Q sh -- '$<' '$@'
\


HAVE_IBV_DEVICE_COUNTERS_SET_SUPPORT
\

infiniband/verbs.h
\


enum IBV_FLOW_SPEC_ACTION_COUNT
\


$(AUTOCONF_OUTPUT)

Which ideally shouldnt because all these files exist:

$ ls -l /usr/include/infiniband/
total 156
-rw-r--r-- 1 root root 1939 Nov 25 12:17 arch.h
-rw-r--r-- 1 root root 2897 Nov 25 12:17 ib.h
-rw-r--r-- 1 root root 27353 Nov 25 12:17 kern-abi.h
-rw-r--r-- 1 root root 12895 Nov 25 12:17 mlx4dv.h
-rw-r--r-- 1 root root 17121 Nov 25 12:17 mlx5dv.h
-rw-r--r-- 1 root root 5694 Nov 25 12:17 opcode.h
-rw-r--r-- 1 root root 4074 Nov 25 12:17 sa.h
-rw-r--r-- 1 root root 1622 Nov 25 12:17 sa-kern-abi.h
-rw-r--r-- 1 root root 67777 Nov 25 12:17 verbs.h

$ ls -l /usr/include/linux/ethtool.h
-rw-r--r-- 1 root root 65636 Feb 12 17:44 /usr/include/linux/ethtool.h


Finally what is strange is if I copy dpdk-18.02.tar.gz to seperate
location and turn MLx5 config and build this builds fine.

Build drivers/net/mlx5
CC mlx5.o
PMDINFO mlx5.o.pmd.c
CC mlx5.o.pmd.o
LD mlx5.o
CC mlx5_glue.o
CC mlx5_rxq.o
CC mlx5_txq.o
CC mlx5_rxtx.o
CC mlx5_rxtx_vec.o
CC mlx5_trigger.o
CC mlx5_ethdev.o
CC mlx5_mac.o
CC mlx5_rxmode.o
CC mlx5_vlan.o
CC mlx5_stats.o
CC mlx5_rss.o
CC mlx5_mr.o
CC mlx5_flow.o
CC mlx5_socket.o
AR librte_pmd_mlx5.a
INSTALL-LIB librte_pmd_mlx5.a

I feel I am missing some trivial step in vpp build process please help
me out I am new to vpp. Let me know if you need any more infor to debug
this issue.

Thank you
Sirshak Das