[dpdk-dev,v2] virtio: fix rx ring descriptor starvation

Message ID 1457621051-17317-1-git-send-email-klarose@sandvine.com (mailing list archive)
State Accepted, archived
Delegated to: Bruce Richardson
Headers

Commit Message

Kyle Larose March 10, 2016, 2:44 p.m. UTC
  Virtio has an mbuf descriptor ring containing mbufs to be used for receiving
traffic. When the host queues traffic to be sent to the guest, it consumes
these descriptors. If none exist, it discards the packet.

The virtio pmd allocates mbufs to the descriptor ring every time it
succesfully receives a packet. However, it never does it if it does not
receive a valid packet. If the descriptor ring is exhausted, and the mbuf
mempool does not have any mbufs free (which can happen for various reasons,
such as queueing along the processing pipeline), then the receive call will
not allocate any mbufs to the descriptor ring, and when it finishes, the
descriptor ring will be empty. The ring being empty means that we will never
receive a packet again, which means we will never allocate mbufs to the ring:
we are stuck.

Ultimately, the problem arises because there is a dependency between receiving
packets and making the descriptor ring not be empty, and a dependency between
the descriptor ring not being empty, and receiving packets.

To fix the problem, this pakes makes virtio always try to allocate mbufs to the
descriptor ring, if necessary, when polling for packets. Do this by removing
the early exit if no packets were received. Since the packet loop later will do
nothing if there are no packets, this is fine.

I reproduced the problem by pushing packets through a pipelined systems (such
as the client_server sample application) after artificially decreasing the
size of the mbuf pool and introducing a delay in a secondary stage.

Without the fix, the process stops receiving packets fairly quicky. With the
fix, it continues to receive packets.

Signed-off-by: Kyle Larose <klarose@sandvine.com>
---
v2:
 * Added missing sign-off.
 * Cleaned up the commit message a bit.
---
 drivers/net/virtio/virtio_rxtx.c | 6 ------
 1 file changed, 6 deletions(-)
  

Comments

Huawei Xie March 22, 2016, 10:07 p.m. UTC | #1
Acked-by: Huawei Xie <huawei.xie@intel.com>
  
Bruce Richardson March 23, 2016, 2:21 p.m. UTC | #2
On Wed, Mar 23, 2016 at 06:07:33AM +0800, Huawei Xie wrote:
> Acked-by: Huawei Xie <huawei.xie@intel.com>

Applied to dpdk-next-net/rel_16_04

/Bruce
  

Patch

diff --git a/drivers/net/virtio/virtio_rxtx.c b/drivers/net/virtio/virtio_rxtx.c
index 41a1366..9d2f7d6 100644
--- a/drivers/net/virtio/virtio_rxtx.c
+++ b/drivers/net/virtio/virtio_rxtx.c
@@ -571,9 +571,6 @@  virtio_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
 	if (likely(num > DESC_PER_CACHELINE))
 		num = num - ((rxvq->vq_used_cons_idx + num) % DESC_PER_CACHELINE);
 
-	if (num == 0)
-		return 0;
-
 	num = virtqueue_dequeue_burst_rx(rxvq, rcv_pkts, len, num);
 	PMD_RX_LOG(DEBUG, "used:%d dequeue:%d", nb_used, num);
 
@@ -671,9 +668,6 @@  virtio_recv_mergeable_pkts(void *rx_queue,
 
 	virtio_rmb();
 
-	if (nb_used == 0)
-		return 0;
-
 	PMD_RX_LOG(DEBUG, "used:%d\n", nb_used);
 
 	hw = rxvq->hw;