[dpdk-dev,v1] i40: fix the VXLAN TSO issue

Message ID 1467752375-25984-1-git-send-email-zhe.tao@intel.com (mailing list archive)
State Changes Requested, archived
Delegated to: Bruce Richardson
Headers

Commit Message

Zhe Tao July 5, 2016, 8:59 p.m. UTC
  Problem:
when using the TSO + VXLAM feature in i40e, the outer UDP len will
sometimes be a invalid value for the multiple UDP segements which
are TSOed by the i40e for the inner TCP.

Fix this problem by add the tunnel type field in the i40e descriptor
which is missed before.

Fixes: 77b8301733c3 ("i40e: VXLAN Tx checksum offload")

Signed-off-by: Zhe Tao <zhe.tao@intel.com>
---
 app/test-pmd/csumonly.c      | 26 +++++++++++++++++++-------
 drivers/net/i40e/i40e_rxtx.c | 10 ++++++++--
 lib/librte_mbuf/rte_mbuf.h   | 11 +++++++++++
 3 files changed, 38 insertions(+), 9 deletions(-)
  

Comments

Jingjing Wu July 6, 2016, 5:38 a.m. UTC | #1
> --- a/drivers/net/i40e/i40e_rxtx.c
> +++ b/drivers/net/i40e/i40e_rxtx.c
> @@ -801,6 +801,12 @@ i40e_txd_enable_checksum(uint64_t ol_flags,
>  			union i40e_tx_offload tx_offload,
>  			uint32_t *cd_tunneling)
>  {
> +	/* Tx pkts tunnel type*/
> +	if ((ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_VXLAN)
> +		*cd_tunneling |= I40E_TXD_CTX_UDP_TUNNELING;
> +	else if ((ol_flags & PKT_TX_TUNNEL_MASK) ==
> PKT_TX_TUNNEL_GRE)
> +		*cd_tunneling |= I40E_TXD_CTX_GRE_TUNNELING;
> +
>  	/* UDP tunneling packet TX checksum offload */
>  	if (ol_flags & PKT_TX_OUTER_IP_CKSUM) {
> 
Besides the tunnel type programming, please also check the L4TUNLEN in tunnel parameters.

> @@ -1510,7 +1516,8 @@ i40e_calc_context_desc(uint64_t flags)
> 
>  /* set i40e TSO context descriptor */
>  static inline uint64_t
> -i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_offload)
> +i40e_set_tso_ctx(struct rte_mbuf *mbuf,
> +		 union i40e_tx_offload tx_offload)
>  {
>  	uint64_t ctx_desc = 0;
>  	uint32_t cd_cmd, hdr_len, cd_tso_len;
> @@ -1537,7 +1544,6 @@ i40e_set_tso_ctx(struct rte_mbuf *mbuf, union
> i40e_tx_offload tx_offload)

Please have a look at the calculation of tso length here. Only tunnel type added may not be enogh.
 
>  		 I40E_TXD_CTX_QW1_TSO_LEN_SHIFT) |
>  		((uint64_t)mbuf->tso_segsz <<
>  		 I40E_TXD_CTX_QW1_MSS_SHIFT);
> -
>  	return ctx_desc;
>  }
> 

This patch contains changes on testpmd, driver and mbuf definition. It's better to split them and sent to the maintainers.
  
Jianfeng Tan Aug. 1, 2016, 3:56 a.m. UTC | #2
Patch 1: mbuf: add Tx side tunneling type
Patch 2: net/i40e: add TSO support on tunneling packet
Patch 3: app/testpmd: fix Tx offload on tunneling packet

v4:
  - According to tunnel type flag to parse tunneling parameters.
  - Add new capabilities to indicate support of TSO on tunneling packets.
  - Add check to see if TSO on tunneling packets are supported for the
    specified NIC.
  - Add support for geneve (as i40e does not differentiate UDP tunneling.
  - Split into three patches.

v3:
  - added external IP offload flag when TSO is enabled for tunnelling packets
v2:
  - edited the comments

Signed-off-by: Zhe Tao <zhe.tao@intel.com>
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>

Jianfeng Tan (3):
  mbuf: add Tx side tunneling type
  net/i40e: add TSO support on tunneling packet
  app/testpmd: fix Tx offload on tunneling packet

 app/test-pmd/cmdline.c         | 42 +++++++++++++++++---
 app/test-pmd/csumonly.c        | 37 +++++++++++++----
 drivers/net/i40e/i40e_ethdev.c |  6 ++-
 drivers/net/i40e/i40e_rxtx.c   | 90 +++++++++++++++++++++++++++++-------------
 lib/librte_ether/rte_ethdev.h  |  4 ++
 lib/librte_mbuf/rte_mbuf.c     |  4 ++
 lib/librte_mbuf/rte_mbuf.h     | 17 +++++++-
 7 files changed, 157 insertions(+), 43 deletions(-)
  
Ananyev, Konstantin Sept. 27, 2016, 5:29 p.m. UTC | #3
> > > -----Original Message-----
> > > From: Tan, Jianfeng
> > > Sent: Monday, August 1, 2016 11:57 AM
> > > To: dev@dpdk.org
> > > Cc: thomas.monjalon@6wind.com; De Lara Guarch, Pablo; Ananyev,
> > > Konstantin; Wu, Jingjing; Zhang, Helin; Tan, Jianfeng; Tao, Zhe
> > > Subject: [PATCH v4 0/3] Add TSO on tunneling packet
> > >
> > > Patch 1: mbuf: add Tx side tunneling type Patch 2: net/i40e: add TSO
> > > support on tunneling packet Patch 3: app/testpmd: fix Tx offload on
> > > tunneling packet
> > >
> > > v4:
> > >   - According to tunnel type flag to parse tunneling parameters.
> > >   - Add new capabilities to indicate support of TSO on tunneling packets.
> > >   - Add check to see if TSO on tunneling packets are supported for the
> > >     specified NIC.
> > >   - Add support for geneve (as i40e does not differentiate UDP tunneling.
> > >   - Split into three patches.
> > >
> > > v3:
> > >   - added external IP offload flag when TSO is enabled for
> > > tunnelling packets
> > > v2:
> > >   - edited the comments
> > >
> > > Signed-off-by: Zhe Tao <zhe.tao@intel.com>
> > > Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> > >
> > > Jianfeng Tan (3):
> > >   mbuf: add Tx side tunneling type
> > >   net/i40e: add TSO support on tunneling packet
> > >   app/testpmd: fix Tx offload on tunneling packet
> > >
> > >  app/test-pmd/cmdline.c         | 42 +++++++++++++++++---
> > >  app/test-pmd/csumonly.c        | 37 +++++++++++++----
> > >  drivers/net/i40e/i40e_ethdev.c |  6 ++-
> > >  drivers/net/i40e/i40e_rxtx.c   | 90 +++++++++++++++++++++++++++++-----
> > > --------
> > >  lib/librte_ether/rte_ethdev.h  |  4 ++
> > >  lib/librte_mbuf/rte_mbuf.c     |  4 ++
> > >  lib/librte_mbuf/rte_mbuf.h     | 17 +++++++-
> > >  7 files changed, 157 insertions(+), 43 deletions(-)
> > >
> > > --

Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
I think you need to rebase your first one: mbuf: add Tx side tunneling type
against the mainline.
Also 3-rd one is v5 actually.

> > > 2.7.4
  
Jianfeng Tan Sept. 27, 2016, 5:52 p.m. UTC | #4
Hi Konstantin,


On 9/28/2016 1:29 AM, Ananyev, Konstantin wrote:
>
[...]
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> I think you need to rebase your first one: mbuf: add Tx side tunneling type
> against the mainline.

The first one can be applied by 3-way merge, git am -k -3, so I did not 
rebase the first two.

Thomas, shall I do that?

Thanks,
Jianfeng

> Also 3-rd one is v5 actually.
>
>>>> 2.7.4
  
Thomas Monjalon Sept. 27, 2016, 7:47 p.m. UTC | #5
2016-09-28 01:52, Tan, Jianfeng:
> Hi Konstantin,
> 
> 
> On 9/28/2016 1:29 AM, Ananyev, Konstantin wrote:
> >
> [...]
> > Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > I think you need to rebase your first one: mbuf: add Tx side tunneling type
> > against the mainline.
> 
> The first one can be applied by 3-way merge, git am -k -3, so I did not 
> rebase the first two.
> 
> Thomas, shall I do that?

Not mandatory.
  
Thomas Monjalon Oct. 9, 2016, 9:27 p.m. UTC | #6
2016-09-27 17:29, Ananyev, Konstantin:
> > > From: Tan, Jianfeng
> > > > Patch 1: mbuf: add Tx side tunneling type Patch 2: net/i40e: add TSO
> > > > support on tunneling packet Patch 3: app/testpmd: fix Tx offload on
> > > > tunneling packet
> 
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> I think you need to rebase your first one: mbuf: add Tx side tunneling type
> against the mainline.
> Also 3-rd one is v5 actually.

Applied (with 3/3 v5), thanks
  

Patch

diff --git a/app/test-pmd/csumonly.c b/app/test-pmd/csumonly.c
index ac4bd8f..d423c20 100644
--- a/app/test-pmd/csumonly.c
+++ b/app/test-pmd/csumonly.c
@@ -204,7 +204,8 @@  parse_ethernet(struct ether_hdr *eth_hdr, struct testpmd_offload_info *info)
 static void
 parse_vxlan(struct udp_hdr *udp_hdr,
 	    struct testpmd_offload_info *info,
-	    uint32_t pkt_type)
+	    uint32_t pkt_type,
+	    uint64_t *ol_flags)
 {
 	struct ether_hdr *eth_hdr;
 
@@ -215,6 +216,7 @@  parse_vxlan(struct udp_hdr *udp_hdr,
 		RTE_ETH_IS_TUNNEL_PKT(pkt_type) == 0)
 		return;
 
+	*ol_flags |= PKT_TX_TUNNEL_VXLAN;
 	info->is_tunnel = 1;
 	info->outer_ethertype = info->ethertype;
 	info->outer_l2_len = info->l2_len;
@@ -231,7 +233,9 @@  parse_vxlan(struct udp_hdr *udp_hdr,
 
 /* Parse a gre header */
 static void
-parse_gre(struct simple_gre_hdr *gre_hdr, struct testpmd_offload_info *info)
+parse_gre(struct simple_gre_hdr *gre_hdr,
+	  struct testpmd_offload_info *info,
+	  uint64_t *ol_flags)
 {
 	struct ether_hdr *eth_hdr;
 	struct ipv4_hdr *ipv4_hdr;
@@ -242,6 +246,8 @@  parse_gre(struct simple_gre_hdr *gre_hdr, struct testpmd_offload_info *info)
 	if ((gre_hdr->flags & _htons(~GRE_SUPPORTED_FIELDS)) != 0)
 		return;
 
+	*ol_flags |= PKT_TX_TUNNEL_GRE;
+
 	gre_len += sizeof(struct simple_gre_hdr);
 
 	if (gre_hdr->flags & _htons(GRE_KEY_PRESENT))
@@ -417,7 +423,7 @@  process_inner_cksums(void *l3_hdr, const struct testpmd_offload_info *info,
  * packet */
 static uint64_t
 process_outer_cksums(void *outer_l3_hdr, struct testpmd_offload_info *info,
-	uint16_t testpmd_ol_flags)
+	uint16_t testpmd_ol_flags, uint64_t orig_ol_flags)
 {
 	struct ipv4_hdr *ipv4_hdr = outer_l3_hdr;
 	struct ipv6_hdr *ipv6_hdr = outer_l3_hdr;
@@ -442,6 +448,9 @@  process_outer_cksums(void *outer_l3_hdr, struct testpmd_offload_info *info,
 	 * hardware supporting it today, and no API for it. */
 
 	udp_hdr = (struct udp_hdr *)((char *)outer_l3_hdr + info->outer_l3_len);
+	if ((orig_ol_flags & PKT_TX_TCP_SEG) &&
+	    ((orig_ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_VXLAN))
+		udp_hdr->dgram_cksum = 0;
 	/* do not recalculate udp cksum if it was 0 */
 	if (udp_hdr->dgram_cksum != 0) {
 		udp_hdr->dgram_cksum = 0;
@@ -705,15 +714,18 @@  pkt_burst_checksum_forward(struct fwd_stream *fs)
 			if (info.l4_proto == IPPROTO_UDP) {
 				struct udp_hdr *udp_hdr;
 				udp_hdr = (struct udp_hdr *)((char *)l3_hdr +
-					info.l3_len);
-				parse_vxlan(udp_hdr, &info, m->packet_type);
+					   info.l3_len);
+				parse_vxlan(udp_hdr, &info, m->packet_type,
+					    &ol_flags);
 			} else if (info.l4_proto == IPPROTO_GRE) {
 				struct simple_gre_hdr *gre_hdr;
 				gre_hdr = (struct simple_gre_hdr *)
 					((char *)l3_hdr + info.l3_len);
-				parse_gre(gre_hdr, &info);
+				parse_gre(gre_hdr, &info, &ol_flags);
 			} else if (info.l4_proto == IPPROTO_IPIP) {
 				void *encap_ip_hdr;
+
+				ol_flags |= PKT_TX_TUNNEL_IPIP;
 				encap_ip_hdr = (char *)l3_hdr + info.l3_len;
 				parse_encap_ip(encap_ip_hdr, &info);
 			}
@@ -745,7 +757,7 @@  pkt_burst_checksum_forward(struct fwd_stream *fs)
 		 * processed in hardware. */
 		if (info.is_tunnel == 1) {
 			ol_flags |= process_outer_cksums(outer_l3_hdr, &info,
-				testpmd_ol_flags);
+				testpmd_ol_flags, ol_flags);
 		}
 
 		/* step 4: fill the mbuf meta data (flags and header lengths) */
diff --git a/drivers/net/i40e/i40e_rxtx.c b/drivers/net/i40e/i40e_rxtx.c
index 049a813..272b04c 100644
--- a/drivers/net/i40e/i40e_rxtx.c
+++ b/drivers/net/i40e/i40e_rxtx.c
@@ -801,6 +801,12 @@  i40e_txd_enable_checksum(uint64_t ol_flags,
 			union i40e_tx_offload tx_offload,
 			uint32_t *cd_tunneling)
 {
+	/* Tx pkts tunnel type*/
+	if ((ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_VXLAN)
+		*cd_tunneling |= I40E_TXD_CTX_UDP_TUNNELING;
+	else if ((ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_GRE)
+		*cd_tunneling |= I40E_TXD_CTX_GRE_TUNNELING;
+
 	/* UDP tunneling packet TX checksum offload */
 	if (ol_flags & PKT_TX_OUTER_IP_CKSUM) {
 
@@ -1510,7 +1516,8 @@  i40e_calc_context_desc(uint64_t flags)
 
 /* set i40e TSO context descriptor */
 static inline uint64_t
-i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_offload)
+i40e_set_tso_ctx(struct rte_mbuf *mbuf,
+		 union i40e_tx_offload tx_offload)
 {
 	uint64_t ctx_desc = 0;
 	uint32_t cd_cmd, hdr_len, cd_tso_len;
@@ -1537,7 +1544,6 @@  i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_offload)
 		 I40E_TXD_CTX_QW1_TSO_LEN_SHIFT) |
 		((uint64_t)mbuf->tso_segsz <<
 		 I40E_TXD_CTX_QW1_MSS_SHIFT);
-
 	return ctx_desc;
 }
 
diff --git a/lib/librte_mbuf/rte_mbuf.h b/lib/librte_mbuf/rte_mbuf.h
index 15e3a10..847767c 100644
--- a/lib/librte_mbuf/rte_mbuf.h
+++ b/lib/librte_mbuf/rte_mbuf.h
@@ -133,6 +133,17 @@  extern "C" {
 /* add new TX flags here */
 
 /**
+ * Bits 45:48 used for the tunnel type.
+ * When doing Tx offload like TSO or checksum, the HW needs to configure the
+ * tunnel type into the HW descriptors.
+ */
+#define PKT_TX_TUNNEL_VXLAN   (1ULL << 45)
+#define PKT_TX_TUNNEL_GRE   (2ULL << 45)
+#define PKT_TX_TUNNEL_IPIP    (3ULL << 45)
+/* add new TX TUNNEL type here */
+#define PKT_TX_TUNNEL_MASK    (0xFULL << 45)
+
+/**
  * Second VLAN insertion (QinQ) flag.
  */
 #define PKT_TX_QINQ_PKT    (1ULL << 49)   /**< TX packet with double VLAN inserted. */