Message ID | 1465741782-126976-1-git-send-email-jianfeng.tan@intel.com (mailing list archive) |
---|---|
State | Rejected, archived |
Delegated to: | Yuanhan Liu |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id CF8FA37AF; Sun, 12 Jun 2016 16:30:00 +0200 (CEST) Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by dpdk.org (Postfix) with ESMTP id 9EB7A37AA for <dev@dpdk.org>; Sun, 12 Jun 2016 16:29:57 +0200 (CEST) Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga103.fm.intel.com with ESMTP; 12 Jun 2016 07:29:52 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.26,462,1459839600"; d="scan'208"; a="1000356191" Received: from dpdk06.sh.intel.com ([10.239.128.225]) by fmsmga002.fm.intel.com with ESMTP; 12 Jun 2016 07:29:51 -0700 From: Jianfeng Tan <jianfeng.tan@intel.com> To: dev@dpdk.org Cc: yuanhan.liu@linux.intel.com, huawei.xie@intel.com, Jianfeng Tan <jianfeng.tan@intel.com> Date: Sun, 12 Jun 2016 14:29:42 +0000 Message-Id: <1465741782-126976-1-git-send-email-jianfeng.tan@intel.com> X-Mailer: git-send-email 2.1.4 Subject: [dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem aligned X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Jianfeng Tan
June 12, 2016, 2:29 p.m. UTC
Compile DPDK with clang, below line in virtio_rxtx.c could be
optimized with four "VMOVAPS ymm, m256".
memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf));
This instruction requires memory address is 32-byte aligned.
Or, it leads to segfault. Although only tested with Clang 3.6.0,
it can be reproduced in any compilers, which do aggressive
optimization, aka, change memset of known length to VMOVAPS.
The fact that struct rte_mbuf is cache line aligned, can only make
sure fake_mbuf is aligned compared to the start address of struct
virtnet_rx. Unfortunately, this address is not necessarily aligned
because it's allocated by:
rxvq = (struct virtnet_rx *)RTE_PTR_ADD(vq, sz_vq);
When sz_vq is not aligned, then rxvq cannot be allocated with an
aligned address, and then rxvq->fake_mbuf (addr of rxvq + cache line
size) is not an aligned address.
The fix is very simple that making sz_vq 32-byte aligned. Here we
make it cache line aligned for future optimization.
Fixes: a900472aedef ("virtio: split virtio Rx/Tx queue")
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
---
drivers/net/virtio/virtio_ethdev.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
Comments
On Sun, Jun 12, 2016 at 02:29:42PM +0000, Jianfeng Tan wrote: > Compile DPDK with clang, below line in virtio_rxtx.c could be > optimized with four "VMOVAPS ymm, m256". > memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > This instruction requires memory address is 32-byte aligned. > Or, it leads to segfault. That looks like a dangerous optimization to me. If that's the case, doesn't it mean we have to make sure the address is always aligned properly while calling memset? --yliu
On Mon, Jun 13, 2016 at 05:21:01PM +0800, Yuanhan Liu wrote: > On Sun, Jun 12, 2016 at 02:29:42PM +0000, Jianfeng Tan wrote: > > Compile DPDK with clang, below line in virtio_rxtx.c could be > > optimized with four "VMOVAPS ymm, m256". > > memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > > > This instruction requires memory address is 32-byte aligned. > > Or, it leads to segfault. > > That looks like a dangerous optimization to me. If that's the case, > doesn't it mean we have to make sure the address is always aligned > properly while calling memset? Above is just a side note. Anyway, I think making sure vq is cache aligned is good here. So, I will apply it. BTW, do you mind if I squash your 2 fixes into Huawei's Rx/Tx split commit? His commit is not pushed to upstream yet, therefore I can still do rebase: I'm thinking it's better to have one working commit other than one broken commit followed with several fixing commits. And of course, I will mention your contribution in the commit log. --yliu
> -----Original Message----- > From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com] > Sent: Monday, June 13, 2016 5:52 PM > To: Tan, Jianfeng > Cc: dev@dpdk.org; Xie, Huawei > Subject: Re: [dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem > aligned > > On Mon, Jun 13, 2016 at 05:21:01PM +0800, Yuanhan Liu wrote: > > On Sun, Jun 12, 2016 at 02:29:42PM +0000, Jianfeng Tan wrote: > > > Compile DPDK with clang, below line in virtio_rxtx.c could be > > > optimized with four "VMOVAPS ymm, m256". > > > memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > > > > > This instruction requires memory address is 32-byte aligned. > > > Or, it leads to segfault. > > > > That looks like a dangerous optimization to me. If that's the case, > > doesn't it mean we have to make sure the address is always aligned > > properly while calling memset? > > Above is just a side note. Anyway, I think making sure vq is cache > aligned is good here. So, I will apply it. BTW, do you mind if I > squash your 2 fixes into Huawei's Rx/Tx split commit? His commit is > not pushed to upstream yet, therefore I can still do rebase: I'm > thinking it's better to have one working commit other than one broken > commit followed with several fixing commits. And of course, I will > mention your contribution in the commit log. Not a problem from my side. But Huawei seems to have concerns on this fix. You can do that for the other fix firstly. Thanks, Jianfeng > > --yliu
On 6/13/2016 5:21 PM, Yuanhan Liu wrote: > On Sun, Jun 12, 2016 at 02:29:42PM +0000, Jianfeng Tan wrote: >> Compile DPDK with clang, below line in virtio_rxtx.c could be >> optimized with four "VMOVAPS ymm, m256". >> memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); >> >> This instruction requires memory address is 32-byte aligned. >> Or, it leads to segfault. > That looks like a dangerous optimization to me.If that's the case, > doesn't it mean we have to make sure the address is always aligned > properly while calling memset? I guess clang does such optimization when length is a 32-byte aligned immediate number. May need more information here. Thanks, Jianfeng > > --yliu
On Mon, Jun 13, 2016 at 10:06:22AM +0000, Tan, Jianfeng wrote: > > > > -----Original Message----- > > From: Yuanhan Liu [mailto:yuanhan.liu@linux.intel.com] > > Sent: Monday, June 13, 2016 5:52 PM > > To: Tan, Jianfeng > > Cc: dev@dpdk.org; Xie, Huawei > > Subject: Re: [dpdk-dev] [PATCH] virtio: fix allocating virtnet_rx not mem > > aligned > > > > On Mon, Jun 13, 2016 at 05:21:01PM +0800, Yuanhan Liu wrote: > > > On Sun, Jun 12, 2016 at 02:29:42PM +0000, Jianfeng Tan wrote: > > > > Compile DPDK with clang, below line in virtio_rxtx.c could be > > > > optimized with four "VMOVAPS ymm, m256". > > > > memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > > > > > > > This instruction requires memory address is 32-byte aligned. > > > > Or, it leads to segfault. > > > > > > That looks like a dangerous optimization to me. If that's the case, > > > doesn't it mean we have to make sure the address is always aligned > > > properly while calling memset? > > > > Above is just a side note. Anyway, I think making sure vq is cache > > aligned is good here. So, I will apply it. BTW, do you mind if I > > squash your 2 fixes into Huawei's Rx/Tx split commit? His commit is > > not pushed to upstream yet, therefore I can still do rebase: I'm > > thinking it's better to have one working commit other than one broken > > commit followed with several fixing commits. And of course, I will > > mention your contribution in the commit log. > > Not a problem from my side. But Huawei seems to have concerns on this fix. You can do that for the other fix firstly. What's the concern? Despite the clang issue (I still have no idea why clang would go that aggressively), I think making vq cache aligned is generically a good idea. --yliu
On Sun, Jun 12, 2016 at 02:29:42PM +0000, Jianfeng Tan wrote: > Compile DPDK with clang, below line in virtio_rxtx.c could be > optimized with four "VMOVAPS ymm, m256". > memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); > > This instruction requires memory address is 32-byte aligned. > Or, it leads to segfault. Although only tested with Clang 3.6.0, > it can be reproduced in any compilers, which do aggressive > optimization, aka, change memset of known length to VMOVAPS. > > The fact that struct rte_mbuf is cache line aligned, can only make > sure fake_mbuf is aligned compared to the start address of struct > virtnet_rx. Unfortunately, this address is not necessarily aligned > because it's allocated by: > rxvq = (struct virtnet_rx *)RTE_PTR_ADD(vq, sz_vq); > > When sz_vq is not aligned, then rxvq cannot be allocated with an > aligned address, and then rxvq->fake_mbuf (addr of rxvq + cache line > size) is not an aligned address. > > The fix is very simple that making sz_vq 32-byte aligned. Here we > make it cache line aligned for future optimization. > > Fixes: a900472aedef ("virtio: split virtio Rx/Tx queue") Folded this fix (and the other fix) to above commit, so that we could have a clean working tree. Thanks for good catch! --yliu
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index a995520..ad0f5a6 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -337,7 +337,10 @@ int virtio_dev_queue_setup(struct rte_eth_dev *dev, snprintf(vq_name, sizeof(vq_name), "port%d_%s%d", dev->data->port_id, queue_names[queue_type], queue_idx); - sz_vq = sizeof(*vq) + vq_size * sizeof(struct vq_desc_extra); + + sz_vq = RTE_ALIGN_CEIL(sizeof(*vq) + + vq_size * sizeof(struct vq_desc_extra), + RTE_CACHE_LINE_SIZE); if (queue_type == VTNET_RQ) { sz_q = sz_vq + sizeof(*rxvq); } else if (queue_type == VTNET_TQ) {