[dpdk-dev,v1] i40: fix the VXLAN TSO issue
Commit Message
Problem:
when using the TSO + VXLAM feature in i40e, the outer UDP len will
sometimes be a invalid value for the multiple UDP segements which
are TSOed by the i40e for the inner TCP.
Fix this problem by add the tunnel type field in the i40e descriptor
which is missed before.
Fixes: 77b8301733c3 ("i40e: VXLAN Tx checksum offload")
Signed-off-by: Zhe Tao <zhe.tao@intel.com>
---
app/test-pmd/csumonly.c | 26 +++++++++++++++++++-------
drivers/net/i40e/i40e_rxtx.c | 10 ++++++++--
lib/librte_mbuf/rte_mbuf.h | 11 +++++++++++
3 files changed, 38 insertions(+), 9 deletions(-)
Comments
> --- a/drivers/net/i40e/i40e_rxtx.c
> +++ b/drivers/net/i40e/i40e_rxtx.c
> @@ -801,6 +801,12 @@ i40e_txd_enable_checksum(uint64_t ol_flags,
> union i40e_tx_offload tx_offload,
> uint32_t *cd_tunneling)
> {
> + /* Tx pkts tunnel type*/
> + if ((ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_VXLAN)
> + *cd_tunneling |= I40E_TXD_CTX_UDP_TUNNELING;
> + else if ((ol_flags & PKT_TX_TUNNEL_MASK) ==
> PKT_TX_TUNNEL_GRE)
> + *cd_tunneling |= I40E_TXD_CTX_GRE_TUNNELING;
> +
> /* UDP tunneling packet TX checksum offload */
> if (ol_flags & PKT_TX_OUTER_IP_CKSUM) {
>
Besides the tunnel type programming, please also check the L4TUNLEN in tunnel parameters.
> @@ -1510,7 +1516,8 @@ i40e_calc_context_desc(uint64_t flags)
>
> /* set i40e TSO context descriptor */
> static inline uint64_t
> -i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_offload)
> +i40e_set_tso_ctx(struct rte_mbuf *mbuf,
> + union i40e_tx_offload tx_offload)
> {
> uint64_t ctx_desc = 0;
> uint32_t cd_cmd, hdr_len, cd_tso_len;
> @@ -1537,7 +1544,6 @@ i40e_set_tso_ctx(struct rte_mbuf *mbuf, union
> i40e_tx_offload tx_offload)
Please have a look at the calculation of tso length here. Only tunnel type added may not be enogh.
> I40E_TXD_CTX_QW1_TSO_LEN_SHIFT) |
> ((uint64_t)mbuf->tso_segsz <<
> I40E_TXD_CTX_QW1_MSS_SHIFT);
> -
> return ctx_desc;
> }
>
This patch contains changes on testpmd, driver and mbuf definition. It's better to split them and sent to the maintainers.
Patch 1: mbuf: add Tx side tunneling type
Patch 2: net/i40e: add TSO support on tunneling packet
Patch 3: app/testpmd: fix Tx offload on tunneling packet
v4:
- According to tunnel type flag to parse tunneling parameters.
- Add new capabilities to indicate support of TSO on tunneling packets.
- Add check to see if TSO on tunneling packets are supported for the
specified NIC.
- Add support for geneve (as i40e does not differentiate UDP tunneling.
- Split into three patches.
v3:
- added external IP offload flag when TSO is enabled for tunnelling packets
v2:
- edited the comments
Signed-off-by: Zhe Tao <zhe.tao@intel.com>
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Jianfeng Tan (3):
mbuf: add Tx side tunneling type
net/i40e: add TSO support on tunneling packet
app/testpmd: fix Tx offload on tunneling packet
app/test-pmd/cmdline.c | 42 +++++++++++++++++---
app/test-pmd/csumonly.c | 37 +++++++++++++----
drivers/net/i40e/i40e_ethdev.c | 6 ++-
drivers/net/i40e/i40e_rxtx.c | 90 +++++++++++++++++++++++++++++-------------
lib/librte_ether/rte_ethdev.h | 4 ++
lib/librte_mbuf/rte_mbuf.c | 4 ++
lib/librte_mbuf/rte_mbuf.h | 17 +++++++-
7 files changed, 157 insertions(+), 43 deletions(-)
> > > -----Original Message-----
> > > From: Tan, Jianfeng
> > > Sent: Monday, August 1, 2016 11:57 AM
> > > To: dev@dpdk.org
> > > Cc: thomas.monjalon@6wind.com; De Lara Guarch, Pablo; Ananyev,
> > > Konstantin; Wu, Jingjing; Zhang, Helin; Tan, Jianfeng; Tao, Zhe
> > > Subject: [PATCH v4 0/3] Add TSO on tunneling packet
> > >
> > > Patch 1: mbuf: add Tx side tunneling type Patch 2: net/i40e: add TSO
> > > support on tunneling packet Patch 3: app/testpmd: fix Tx offload on
> > > tunneling packet
> > >
> > > v4:
> > > - According to tunnel type flag to parse tunneling parameters.
> > > - Add new capabilities to indicate support of TSO on tunneling packets.
> > > - Add check to see if TSO on tunneling packets are supported for the
> > > specified NIC.
> > > - Add support for geneve (as i40e does not differentiate UDP tunneling.
> > > - Split into three patches.
> > >
> > > v3:
> > > - added external IP offload flag when TSO is enabled for
> > > tunnelling packets
> > > v2:
> > > - edited the comments
> > >
> > > Signed-off-by: Zhe Tao <zhe.tao@intel.com>
> > > Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
> > >
> > > Jianfeng Tan (3):
> > > mbuf: add Tx side tunneling type
> > > net/i40e: add TSO support on tunneling packet
> > > app/testpmd: fix Tx offload on tunneling packet
> > >
> > > app/test-pmd/cmdline.c | 42 +++++++++++++++++---
> > > app/test-pmd/csumonly.c | 37 +++++++++++++----
> > > drivers/net/i40e/i40e_ethdev.c | 6 ++-
> > > drivers/net/i40e/i40e_rxtx.c | 90 +++++++++++++++++++++++++++++-----
> > > --------
> > > lib/librte_ether/rte_ethdev.h | 4 ++
> > > lib/librte_mbuf/rte_mbuf.c | 4 ++
> > > lib/librte_mbuf/rte_mbuf.h | 17 +++++++-
> > > 7 files changed, 157 insertions(+), 43 deletions(-)
> > >
> > > --
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
I think you need to rebase your first one: mbuf: add Tx side tunneling type
against the mainline.
Also 3-rd one is v5 actually.
> > > 2.7.4
Hi Konstantin,
On 9/28/2016 1:29 AM, Ananyev, Konstantin wrote:
>
[...]
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> I think you need to rebase your first one: mbuf: add Tx side tunneling type
> against the mainline.
The first one can be applied by 3-way merge, git am -k -3, so I did not
rebase the first two.
Thomas, shall I do that?
Thanks,
Jianfeng
> Also 3-rd one is v5 actually.
>
>>>> 2.7.4
2016-09-28 01:52, Tan, Jianfeng:
> Hi Konstantin,
>
>
> On 9/28/2016 1:29 AM, Ananyev, Konstantin wrote:
> >
> [...]
> > Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> > I think you need to rebase your first one: mbuf: add Tx side tunneling type
> > against the mainline.
>
> The first one can be applied by 3-way merge, git am -k -3, so I did not
> rebase the first two.
>
> Thomas, shall I do that?
Not mandatory.
2016-09-27 17:29, Ananyev, Konstantin:
> > > From: Tan, Jianfeng
> > > > Patch 1: mbuf: add Tx side tunneling type Patch 2: net/i40e: add TSO
> > > > support on tunneling packet Patch 3: app/testpmd: fix Tx offload on
> > > > tunneling packet
>
> Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
> I think you need to rebase your first one: mbuf: add Tx side tunneling type
> against the mainline.
> Also 3-rd one is v5 actually.
Applied (with 3/3 v5), thanks
@@ -204,7 +204,8 @@ parse_ethernet(struct ether_hdr *eth_hdr, struct testpmd_offload_info *info)
static void
parse_vxlan(struct udp_hdr *udp_hdr,
struct testpmd_offload_info *info,
- uint32_t pkt_type)
+ uint32_t pkt_type,
+ uint64_t *ol_flags)
{
struct ether_hdr *eth_hdr;
@@ -215,6 +216,7 @@ parse_vxlan(struct udp_hdr *udp_hdr,
RTE_ETH_IS_TUNNEL_PKT(pkt_type) == 0)
return;
+ *ol_flags |= PKT_TX_TUNNEL_VXLAN;
info->is_tunnel = 1;
info->outer_ethertype = info->ethertype;
info->outer_l2_len = info->l2_len;
@@ -231,7 +233,9 @@ parse_vxlan(struct udp_hdr *udp_hdr,
/* Parse a gre header */
static void
-parse_gre(struct simple_gre_hdr *gre_hdr, struct testpmd_offload_info *info)
+parse_gre(struct simple_gre_hdr *gre_hdr,
+ struct testpmd_offload_info *info,
+ uint64_t *ol_flags)
{
struct ether_hdr *eth_hdr;
struct ipv4_hdr *ipv4_hdr;
@@ -242,6 +246,8 @@ parse_gre(struct simple_gre_hdr *gre_hdr, struct testpmd_offload_info *info)
if ((gre_hdr->flags & _htons(~GRE_SUPPORTED_FIELDS)) != 0)
return;
+ *ol_flags |= PKT_TX_TUNNEL_GRE;
+
gre_len += sizeof(struct simple_gre_hdr);
if (gre_hdr->flags & _htons(GRE_KEY_PRESENT))
@@ -417,7 +423,7 @@ process_inner_cksums(void *l3_hdr, const struct testpmd_offload_info *info,
* packet */
static uint64_t
process_outer_cksums(void *outer_l3_hdr, struct testpmd_offload_info *info,
- uint16_t testpmd_ol_flags)
+ uint16_t testpmd_ol_flags, uint64_t orig_ol_flags)
{
struct ipv4_hdr *ipv4_hdr = outer_l3_hdr;
struct ipv6_hdr *ipv6_hdr = outer_l3_hdr;
@@ -442,6 +448,9 @@ process_outer_cksums(void *outer_l3_hdr, struct testpmd_offload_info *info,
* hardware supporting it today, and no API for it. */
udp_hdr = (struct udp_hdr *)((char *)outer_l3_hdr + info->outer_l3_len);
+ if ((orig_ol_flags & PKT_TX_TCP_SEG) &&
+ ((orig_ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_VXLAN))
+ udp_hdr->dgram_cksum = 0;
/* do not recalculate udp cksum if it was 0 */
if (udp_hdr->dgram_cksum != 0) {
udp_hdr->dgram_cksum = 0;
@@ -705,15 +714,18 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
if (info.l4_proto == IPPROTO_UDP) {
struct udp_hdr *udp_hdr;
udp_hdr = (struct udp_hdr *)((char *)l3_hdr +
- info.l3_len);
- parse_vxlan(udp_hdr, &info, m->packet_type);
+ info.l3_len);
+ parse_vxlan(udp_hdr, &info, m->packet_type,
+ &ol_flags);
} else if (info.l4_proto == IPPROTO_GRE) {
struct simple_gre_hdr *gre_hdr;
gre_hdr = (struct simple_gre_hdr *)
((char *)l3_hdr + info.l3_len);
- parse_gre(gre_hdr, &info);
+ parse_gre(gre_hdr, &info, &ol_flags);
} else if (info.l4_proto == IPPROTO_IPIP) {
void *encap_ip_hdr;
+
+ ol_flags |= PKT_TX_TUNNEL_IPIP;
encap_ip_hdr = (char *)l3_hdr + info.l3_len;
parse_encap_ip(encap_ip_hdr, &info);
}
@@ -745,7 +757,7 @@ pkt_burst_checksum_forward(struct fwd_stream *fs)
* processed in hardware. */
if (info.is_tunnel == 1) {
ol_flags |= process_outer_cksums(outer_l3_hdr, &info,
- testpmd_ol_flags);
+ testpmd_ol_flags, ol_flags);
}
/* step 4: fill the mbuf meta data (flags and header lengths) */
@@ -801,6 +801,12 @@ i40e_txd_enable_checksum(uint64_t ol_flags,
union i40e_tx_offload tx_offload,
uint32_t *cd_tunneling)
{
+ /* Tx pkts tunnel type*/
+ if ((ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_VXLAN)
+ *cd_tunneling |= I40E_TXD_CTX_UDP_TUNNELING;
+ else if ((ol_flags & PKT_TX_TUNNEL_MASK) == PKT_TX_TUNNEL_GRE)
+ *cd_tunneling |= I40E_TXD_CTX_GRE_TUNNELING;
+
/* UDP tunneling packet TX checksum offload */
if (ol_flags & PKT_TX_OUTER_IP_CKSUM) {
@@ -1510,7 +1516,8 @@ i40e_calc_context_desc(uint64_t flags)
/* set i40e TSO context descriptor */
static inline uint64_t
-i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_offload)
+i40e_set_tso_ctx(struct rte_mbuf *mbuf,
+ union i40e_tx_offload tx_offload)
{
uint64_t ctx_desc = 0;
uint32_t cd_cmd, hdr_len, cd_tso_len;
@@ -1537,7 +1544,6 @@ i40e_set_tso_ctx(struct rte_mbuf *mbuf, union i40e_tx_offload tx_offload)
I40E_TXD_CTX_QW1_TSO_LEN_SHIFT) |
((uint64_t)mbuf->tso_segsz <<
I40E_TXD_CTX_QW1_MSS_SHIFT);
-
return ctx_desc;
}
@@ -133,6 +133,17 @@ extern "C" {
/* add new TX flags here */
/**
+ * Bits 45:48 used for the tunnel type.
+ * When doing Tx offload like TSO or checksum, the HW needs to configure the
+ * tunnel type into the HW descriptors.
+ */
+#define PKT_TX_TUNNEL_VXLAN (1ULL << 45)
+#define PKT_TX_TUNNEL_GRE (2ULL << 45)
+#define PKT_TX_TUNNEL_IPIP (3ULL << 45)
+/* add new TX TUNNEL type here */
+#define PKT_TX_TUNNEL_MASK (0xFULL << 45)
+
+/**
* Second VLAN insertion (QinQ) flag.
*/
#define PKT_TX_QINQ_PKT (1ULL << 49) /**< TX packet with double VLAN inserted. */