| Message ID | 20151022095730.5a3c742b@xeon-e3 (mailing list archive) |
|---|---|
| State | Not Applicable, archived |
| Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 8EBCD9A81; Thu, 22 Oct 2015 18:57:26 +0200 (CEST) Received: from mail-pa0-f48.google.com (mail-pa0-f48.google.com [209.85.220.48]) by dpdk.org (Postfix) with ESMTP id 488289655 for <dev@dpdk.org>; Thu, 22 Oct 2015 18:57:25 +0200 (CEST) Received: by pabrc13 with SMTP id rc13so91245975pab.0 for <dev@dpdk.org>; Thu, 22 Oct 2015 09:57:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=3Gwrn3C5VNrWuegraGcqCwzyW9jtNVjAlI0oG20Ro5U=; b=aTeTx6n7TkGKwio/cwmiwT+4Msp5kUixCVKL/1zt93P/JwnhXjJ1ciLidh0ur626UQ 6sak+21rXjsp9ffGBKNH7y+p3WfW4U0tyK/TUCG0JkwTfDrXRh04MpWN+TsJfWmHZbQI b++kOy0wbRXhwzuod28xU570A16rpgCt/7FYX0L1+iTpETpgENv5nj6oM7Rah2TPtnV4 mk0rC+yEKuFSHQ5zZDgUJKUaNSpZI7/nXi7qvcBMq4vAPZKomAqfln3cDERFO+potwsz 1KRfhA1pIqL7hwblxZl+b6K+o2rb0m7dASYdLZSvpAY7UFIZFc4RTKvYjzU9866g8d5S Hltg== X-Gm-Message-State: ALoCoQkdhB/iUiG8JJOjfZQqWqbCkokoU8K2sG8iycaQyfbW42haVbnBBWXg1h61HPhnozpDlNSC X-Received: by 10.67.15.100 with SMTP id fn4mr19093967pad.120.1445533044624; Thu, 22 Oct 2015 09:57:24 -0700 (PDT) Received: from xeon-e3 (static-50-53-82-155.bvtn.or.frontiernet.net. [50.53.82.155]) by smtp.gmail.com with ESMTPSA id pu5sm14717130pbc.58.2015.10.22.09.57.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 22 Oct 2015 09:57:24 -0700 (PDT) Date: Thu, 22 Oct 2015 09:57:30 -0700 From: Stephen Hemminger <stephen@networkplumber.org> To: Huawei Xie <huawei.xie@intel.com> Message-ID: <20151022095730.5a3c742b@xeon-e3> In-Reply-To: <1445515791-25909-7-git-send-email-huawei.xie@intel.com> References: <1443537953-23917-1-git-send-email-huawei.xie@intel.com> <1445515791-25909-1-git-send-email-huawei.xie@intel.com> <1445515791-25909-7-git-send-email-huawei.xie@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH v4 6/7] virtio: simple tx routine X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Stephen Hemminger
Oct. 22, 2015, 4:57 p.m. UTC
On Thu, 22 Oct 2015 20:09:50 +0800 Huawei Xie <huawei.xie@intel.com> wrote: > Changes in v4: > - move virtio_xmit_cleanup ahead to free descriptors earlier > > Changes in v3: > - Remove return at the end of void function > - Remove always_inline attribute for virtio_xmit_cleanup > bulk free of mbufs when clean used ring. > shift operation of idx could be saved if vq_free_cnt means > free slots rather than free descriptors. > > TODO: rearrange vq data structure, pack the stats var together so that we > could use one vec instruction to update all of them. > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > --- > drivers/net/virtio/virtio_ethdev.h | 3 ++ > drivers/net/virtio/virtio_rxtx_simple.c | 93 +++++++++++++++++++++++++++++++++ > 2 files changed, 96 insertions(+) > > diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h > index d7797ab..ae2d47d 100644 > --- a/drivers/net/virtio/virtio_ethdev.h > +++ b/drivers/net/virtio/virtio_ethdev.h > @@ -111,6 +111,9 @@ uint16_t virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, > uint16_t virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, > uint16_t nb_pkts); > > +uint16_t virtio_xmit_pkts_simple(void *tx_queue, struct rte_mbuf **tx_pkts, > + uint16_t nb_pkts); > + > /* > * The VIRTIO_NET_F_GUEST_TSO[46] features permit the host to send us > * frames larger than 1514 bytes. We do not yet support software LRO > diff --git a/drivers/net/virtio/virtio_rxtx_simple.c b/drivers/net/virtio/virtio_rxtx_simple.c > index ef17562..79b4f7f 100644 > --- a/drivers/net/virtio/virtio_rxtx_simple.c > +++ b/drivers/net/virtio/virtio_rxtx_simple.c > @@ -288,6 +288,99 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, > return nb_pkts_received; > } > > +#define VIRTIO_TX_FREE_THRESH 32 > +#define VIRTIO_TX_MAX_FREE_BUF_SZ 32 > +#define VIRTIO_TX_FREE_NR 32 > +/* TODO: vq->tx_free_cnt could mean num of free slots so we could avoid shift */ > +static inline void > +virtio_xmit_cleanup(struct virtqueue *vq) > +{ > + uint16_t i, desc_idx; > + int nb_free = 0; > + struct rte_mbuf *m, *free[VIRTIO_TX_MAX_FREE_BUF_SZ]; > + > + desc_idx = (uint16_t)(vq->vq_used_cons_idx & > + ((vq->vq_nentries >> 1) - 1)); > + free[0] = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; > + nb_free = 1; > + > + for (i = 1; i < VIRTIO_TX_FREE_NR; i++) { > + m = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; > + if (likely(m->pool == free[0]->pool)) > + free[nb_free++] = m; > + else { > + rte_mempool_put_bulk(free[0]->pool, (void **)free, > + nb_free); > + free[0] = m; > + nb_free = 1; > + } > + } > + > + rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free); > + vq->vq_used_cons_idx += VIRTIO_TX_FREE_NR; > + vq->vq_free_cnt += (VIRTIO_TX_FREE_NR << 1); > +} I think you need to handle refcount, here is a similar patch for ixgbe. Subject: ixgbe: speed up transmit Coalesce transmit buffers and put them back into the pool in one burst. Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Comments
On 10/23/2015 12:57 AM, Stephen Hemminger wrote: > On Thu, 22 Oct 2015 20:09:50 +0800 > Huawei Xie <huawei.xie@intel.com> wrote: > >> Changes in v4: >> - move virtio_xmit_cleanup ahead to free descriptors earlier >> >> Changes in v3: >> - Remove return at the end of void function >> - Remove always_inline attribute for virtio_xmit_cleanup >> bulk free of mbufs when clean used ring. >> shift operation of idx could be saved if vq_free_cnt means >> free slots rather than free descriptors. >> >> TODO: rearrange vq data structure, pack the stats var together so that we >> could use one vec instruction to update all of them. >> >> Signed-off-by: Huawei Xie <huawei.xie@intel.com> >> --- >> drivers/net/virtio/virtio_ethdev.h | 3 ++ >> drivers/net/virtio/virtio_rxtx_simple.c | 93 +++++++++++++++++++++++++++++++++ >> 2 files changed, 96 insertions(+) >> >> diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h >> index d7797ab..ae2d47d 100644 >> --- a/drivers/net/virtio/virtio_ethdev.h >> +++ b/drivers/net/virtio/virtio_ethdev.h >> @@ -111,6 +111,9 @@ uint16_t virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, >> uint16_t virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, >> uint16_t nb_pkts); >> >> +uint16_t virtio_xmit_pkts_simple(void *tx_queue, struct rte_mbuf **tx_pkts, >> + uint16_t nb_pkts); >> + >> /* >> * The VIRTIO_NET_F_GUEST_TSO[46] features permit the host to send us >> * frames larger than 1514 bytes. We do not yet support software LRO >> diff --git a/drivers/net/virtio/virtio_rxtx_simple.c b/drivers/net/virtio/virtio_rxtx_simple.c >> index ef17562..79b4f7f 100644 >> --- a/drivers/net/virtio/virtio_rxtx_simple.c >> +++ b/drivers/net/virtio/virtio_rxtx_simple.c >> @@ -288,6 +288,99 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, >> return nb_pkts_received; >> } >> >> +#define VIRTIO_TX_FREE_THRESH 32 >> +#define VIRTIO_TX_MAX_FREE_BUF_SZ 32 >> +#define VIRTIO_TX_FREE_NR 32 >> +/* TODO: vq->tx_free_cnt could mean num of free slots so we could avoid shift */ >> +static inline void >> +virtio_xmit_cleanup(struct virtqueue *vq) >> +{ >> + uint16_t i, desc_idx; >> + int nb_free = 0; >> + struct rte_mbuf *m, *free[VIRTIO_TX_MAX_FREE_BUF_SZ]; >> + >> + desc_idx = (uint16_t)(vq->vq_used_cons_idx & >> + ((vq->vq_nentries >> 1) - 1)); >> + free[0] = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; >> + nb_free = 1; >> + >> + for (i = 1; i < VIRTIO_TX_FREE_NR; i++) { >> + m = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; >> + if (likely(m->pool == free[0]->pool)) >> + free[nb_free++] = m; >> + else { >> + rte_mempool_put_bulk(free[0]->pool, (void **)free, >> + nb_free); >> + free[0] = m; >> + nb_free = 1; >> + } >> + } >> + >> + rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free); >> + vq->vq_used_cons_idx += VIRTIO_TX_FREE_NR; >> + vq->vq_free_cnt += (VIRTIO_TX_FREE_NR << 1); >> +} > I think you need to handle refcount, here is a similar patch > for ixgbe. ok, like this: m = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; if (likely(m != NULL)) { ... > > Subject: ixgbe: speed up transmit > > Coalesce transmit buffers and put them back into the pool > in one burst. > > Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> > > --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c > @@ -120,12 +120,16 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) > * Check for descriptors with their DD bit set and free mbufs. > * Return the total number of buffers freed. > */ > +#define TX_FREE_BULK 32 > + > static inline int __attribute__((always_inline)) > ixgbe_tx_free_bufs(struct ixgbe_tx_queue *txq) > { > struct ixgbe_tx_entry *txep; > uint32_t status; > - int i; > + int i, n = 0; > + struct rte_mempool *txpool = NULL; > + struct rte_mbuf *free_list[TX_FREE_BULK]; > > /* check DD bit on threshold descriptor */ > status = txq->tx_ring[txq->tx_next_dd].wb.status; > @@ -138,20 +142,26 @@ ixgbe_tx_free_bufs(struct ixgbe_tx_queue > */ > txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); > > - /* free buffers one at a time */ > - if ((txq->txq_flags & (uint32_t)ETH_TXQ_FLAGS_NOREFCOUNT) != 0) { > - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { > - txep->mbuf->next = NULL; > - rte_mempool_put(txep->mbuf->pool, txep->mbuf); > - txep->mbuf = NULL; > - } > - } else { > - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { > - rte_pktmbuf_free_seg(txep->mbuf); > - txep->mbuf = NULL; > + for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { > + struct rte_mbuf *m; > + > + /* free buffers one at a time */ > + m = __rte_pktmbuf_prefree_seg(txep->mbuf); > + txep->mbuf = NULL; > + > + if (n >= TX_FREE_BULK || check whether m is NULL here. > + (n > 0 && m->pool != txpool)) { > + rte_mempool_put_bulk(txpool, (void **)free_list, n); > + n = 0; > } > + > + txpool = m->pool; > + free_list[n++] = m; > } > > + if (n > 0) > + rte_mempool_put_bulk(txpool, (void **)free_list, n); > + > /* buffers were freed, update counters */ > txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh); > txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh); > >
On 10/23/2015 10:17 AM, Xie, Huawei wrote: > On 10/23/2015 12:57 AM, Stephen Hemminger wrote: >> On Thu, 22 Oct 2015 20:09:50 +0800 >> Huawei Xie <huawei.xie@intel.com> wrote: >> >>> Changes in v4: >>> - move virtio_xmit_cleanup ahead to free descriptors earlier >>> >>> Changes in v3: >>> - Remove return at the end of void function >>> - Remove always_inline attribute for virtio_xmit_cleanup >>> bulk free of mbufs when clean used ring. >>> shift operation of idx could be saved if vq_free_cnt means >>> free slots rather than free descriptors. >>> >>> TODO: rearrange vq data structure, pack the stats var together so that we >>> could use one vec instruction to update all of them. >>> >>> Signed-off-by: Huawei Xie <huawei.xie@intel.com> >>> --- >>> drivers/net/virtio/virtio_ethdev.h | 3 ++ >>> drivers/net/virtio/virtio_rxtx_simple.c | 93 +++++++++++++++++++++++++++++++++ >>> 2 files changed, 96 insertions(+) >>> >>> diff --git a/drivers/net/virtio/virtio_ethdev.h b/drivers/net/virtio/virtio_ethdev.h >>> index d7797ab..ae2d47d 100644 >>> --- a/drivers/net/virtio/virtio_ethdev.h >>> +++ b/drivers/net/virtio/virtio_ethdev.h >>> @@ -111,6 +111,9 @@ uint16_t virtio_xmit_pkts(void *tx_queue, struct rte_mbuf **tx_pkts, >>> uint16_t virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, >>> uint16_t nb_pkts); >>> >>> +uint16_t virtio_xmit_pkts_simple(void *tx_queue, struct rte_mbuf **tx_pkts, >>> + uint16_t nb_pkts); >>> + >>> /* >>> * The VIRTIO_NET_F_GUEST_TSO[46] features permit the host to send us >>> * frames larger than 1514 bytes. We do not yet support software LRO >>> diff --git a/drivers/net/virtio/virtio_rxtx_simple.c b/drivers/net/virtio/virtio_rxtx_simple.c >>> index ef17562..79b4f7f 100644 >>> --- a/drivers/net/virtio/virtio_rxtx_simple.c >>> +++ b/drivers/net/virtio/virtio_rxtx_simple.c >>> @@ -288,6 +288,99 @@ virtio_recv_pkts_vec(void *rx_queue, struct rte_mbuf **rx_pkts, >>> return nb_pkts_received; >>> } >>> >>> +#define VIRTIO_TX_FREE_THRESH 32 >>> +#define VIRTIO_TX_MAX_FREE_BUF_SZ 32 >>> +#define VIRTIO_TX_FREE_NR 32 >>> +/* TODO: vq->tx_free_cnt could mean num of free slots so we could avoid shift */ >>> +static inline void >>> +virtio_xmit_cleanup(struct virtqueue *vq) >>> +{ >>> + uint16_t i, desc_idx; >>> + int nb_free = 0; >>> + struct rte_mbuf *m, *free[VIRTIO_TX_MAX_FREE_BUF_SZ]; >>> + >>> + desc_idx = (uint16_t)(vq->vq_used_cons_idx & >>> + ((vq->vq_nentries >> 1) - 1)); >>> + free[0] = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; >>> + nb_free = 1; >>> + >>> + for (i = 1; i < VIRTIO_TX_FREE_NR; i++) { >>> + m = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; >>> + if (likely(m->pool == free[0]->pool)) >>> + free[nb_free++] = m; >>> + else { >>> + rte_mempool_put_bulk(free[0]->pool, (void **)free, >>> + nb_free); >>> + free[0] = m; >>> + nb_free = 1; >>> + } >>> + } >>> + >>> + rte_mempool_put_bulk(free[0]->pool, (void **)free, nb_free); >>> + vq->vq_used_cons_idx += VIRTIO_TX_FREE_NR; >>> + vq->vq_free_cnt += (VIRTIO_TX_FREE_NR << 1); >>> +} >> I think you need to handle refcount, here is a similar patch >> for ixgbe. > ok, like this: > > m = (struct rte_mbuf *)vq->vq_descx[desc_idx++].cookie; Missed a line m = __rte_pktmbuf_prefree_seg(m) > if (likely(m != NULL)) { > ... > >> Subject: ixgbe: speed up transmit >> >> Coalesce transmit buffers and put them back into the pool >> in one burst. >> >> Signed-off-by: Stephen Hemminger <stephen@networkplumber.org> >> >> --- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c >> +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c >> @@ -120,12 +120,16 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) >> * Check for descriptors with their DD bit set and free mbufs. >> * Return the total number of buffers freed. >> */ >> +#define TX_FREE_BULK 32 >> + >> static inline int __attribute__((always_inline)) >> ixgbe_tx_free_bufs(struct ixgbe_tx_queue *txq) >> { >> struct ixgbe_tx_entry *txep; >> uint32_t status; >> - int i; >> + int i, n = 0; >> + struct rte_mempool *txpool = NULL; >> + struct rte_mbuf *free_list[TX_FREE_BULK]; >> >> /* check DD bit on threshold descriptor */ >> status = txq->tx_ring[txq->tx_next_dd].wb.status; >> @@ -138,20 +142,26 @@ ixgbe_tx_free_bufs(struct ixgbe_tx_queue >> */ >> txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); >> >> - /* free buffers one at a time */ >> - if ((txq->txq_flags & (uint32_t)ETH_TXQ_FLAGS_NOREFCOUNT) != 0) { >> - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { >> - txep->mbuf->next = NULL; >> - rte_mempool_put(txep->mbuf->pool, txep->mbuf); >> - txep->mbuf = NULL; >> - } >> - } else { >> - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { >> - rte_pktmbuf_free_seg(txep->mbuf); >> - txep->mbuf = NULL; >> + for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { >> + struct rte_mbuf *m; >> + >> + /* free buffers one at a time */ >> + m = __rte_pktmbuf_prefree_seg(txep->mbuf); >> + txep->mbuf = NULL; >> + >> + if (n >= TX_FREE_BULK || > check whether m is NULL here. >> + (n > 0 && m->pool != txpool)) { >> + rte_mempool_put_bulk(txpool, (void **)free_list, n); >> + n = 0; >> } >> + >> + txpool = m->pool; >> + free_list[n++] = m; >> } >> >> + if (n > 0) >> + rte_mempool_put_bulk(txpool, (void **)free_list, n); >> + >> /* buffers were freed, update counters */ >> txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh); >> txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh); >> >> >
--- a/lib/librte_pmd_ixgbe/ixgbe_rxtx.c +++ b/lib/librte_pmd_ixgbe/ixgbe_rxtx.c @@ -120,12 +120,16 @@ rte_rxmbuf_alloc(struct rte_mempool *mp) * Check for descriptors with their DD bit set and free mbufs. * Return the total number of buffers freed. */ +#define TX_FREE_BULK 32 + static inline int __attribute__((always_inline)) ixgbe_tx_free_bufs(struct ixgbe_tx_queue *txq) { struct ixgbe_tx_entry *txep; uint32_t status; - int i; + int i, n = 0; + struct rte_mempool *txpool = NULL; + struct rte_mbuf *free_list[TX_FREE_BULK]; /* check DD bit on threshold descriptor */ status = txq->tx_ring[txq->tx_next_dd].wb.status; @@ -138,20 +142,26 @@ ixgbe_tx_free_bufs(struct ixgbe_tx_queue */ txep = &(txq->sw_ring[txq->tx_next_dd - (txq->tx_rs_thresh - 1)]); - /* free buffers one at a time */ - if ((txq->txq_flags & (uint32_t)ETH_TXQ_FLAGS_NOREFCOUNT) != 0) { - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { - txep->mbuf->next = NULL; - rte_mempool_put(txep->mbuf->pool, txep->mbuf); - txep->mbuf = NULL; - } - } else { - for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { - rte_pktmbuf_free_seg(txep->mbuf); - txep->mbuf = NULL; + for (i = 0; i < txq->tx_rs_thresh; ++i, ++txep) { + struct rte_mbuf *m; + + /* free buffers one at a time */ + m = __rte_pktmbuf_prefree_seg(txep->mbuf); + txep->mbuf = NULL; + + if (n >= TX_FREE_BULK || + (n > 0 && m->pool != txpool)) { + rte_mempool_put_bulk(txpool, (void **)free_list, n); + n = 0; } + + txpool = m->pool; + free_list[n++] = m; } + if (n > 0) + rte_mempool_put_bulk(txpool, (void **)free_list, n); + /* buffers were freed, update counters */ txq->nb_tx_free = (uint16_t)(txq->nb_tx_free + txq->tx_rs_thresh); txq->tx_next_dd = (uint16_t)(txq->tx_next_dd + txq->tx_rs_thresh);