| Message ID | 1464106601-981-1-git-send-email-huawei.xie@intel.com (mailing list archive) |
|---|---|
| State | Superseded, archived |
| Delegated to: | Yuanhan Liu |
| Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id A50945683; Wed, 25 May 2016 10:11:32 +0200 (CEST) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 66C2C567F for <dev@dpdk.org>; Wed, 25 May 2016 10:11:30 +0200 (CEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga101.jf.intel.com with ESMTP; 25 May 2016 01:11:29 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.26,363,1459839600"; d="scan'208";a="974093362" Received: from dpdk15.sh.intel.com ([10.239.129.25]) by fmsmga001.fm.intel.com with ESMTP; 25 May 2016 01:11:28 -0700 From: Huawei Xie <huawei.xie@intel.com> To: dev@dpdk.org Cc: stephen@networkplumber.org, konstantin.ananyev@intel.com, thomas.monjalon@6wind.com, Huawei Xie <huawei.xie@intel.com> Date: Wed, 25 May 2016 00:16:41 +0800 Message-Id: <1464106601-981-1-git-send-email-huawei.xie@intel.com> X-Mailer: git-send-email 1.8.1.4 Subject: [dpdk-dev] [PATCH] virtio: use volatile to get used->idx in the loop X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Huawei Xie
May 24, 2016, 4:16 p.m. UTC
There is no external function call or any barrier in the loop,
the used->idx would only be retrieved once.
Signed-off-by: Huawei Xie <huawei.xie@intel.com>
---
drivers/net/virtio/virtio_ethdev.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
Comments
On 5/25/2016 4:12 PM, Xie, Huawei wrote: > There is no external function call or any barrier in the loop, > the used->idx would only be retrieved once. > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > --- > drivers/net/virtio/virtio_ethdev.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > index c3fb628..f6d6305 100644 > --- a/drivers/net/virtio/virtio_ethdev.c > +++ b/drivers/net/virtio/virtio_ethdev.c > @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > usleep(100); > } > > - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > + while (vq->vq_used_cons_idx != > + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > uint32_t idx, desc_idx, used_idx; > struct vring_used_elem *uep; > Find this issue when do the code rework of RX/TX queue. As in other places, we also have loop retrieving the value of avial->idx or used->idx, i prefer to declare the index in vq structure as volatile to avoid potential issue. Stephen: Another question is why we need a loop here? /huawei
On Wed, May 25, 2016 at 08:25:20AM +0000, Xie, Huawei wrote: > On 5/25/2016 4:12 PM, Xie, Huawei wrote: > > There is no external function call or any barrier in the loop, > > the used->idx would only be retrieved once. > > > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > > --- > > drivers/net/virtio/virtio_ethdev.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > > index c3fb628..f6d6305 100644 > > --- a/drivers/net/virtio/virtio_ethdev.c > > +++ b/drivers/net/virtio/virtio_ethdev.c > > @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > > usleep(100); > > } > > > > - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > > + while (vq->vq_used_cons_idx != > > + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > > uint32_t idx, desc_idx, used_idx; > > struct vring_used_elem *uep; > > > > Find this issue when do the code rework of RX/TX queue. > As in other places, we also have loop retrieving the value of avial->idx > or used->idx, i prefer to declare the index in vq structure as volatile > to avoid potential issue. It might be a good idea to wrap this in a macro similar to ACCESS_ONCE in Linux. > > Stephen: > Another question is why we need a loop here? > > /huawei
On Wed, May 25, 2016 at 11:34:24AM +0300, Michael S. Tsirkin wrote: > On Wed, May 25, 2016 at 08:25:20AM +0000, Xie, Huawei wrote: > > On 5/25/2016 4:12 PM, Xie, Huawei wrote: > > > There is no external function call or any barrier in the loop, > > > the used->idx would only be retrieved once. > > > > > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > > > --- > > > drivers/net/virtio/virtio_ethdev.c | 3 ++- > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > > > index c3fb628..f6d6305 100644 > > > --- a/drivers/net/virtio/virtio_ethdev.c > > > +++ b/drivers/net/virtio/virtio_ethdev.c > > > @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > > > usleep(100); > > > } > > > > > > - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > > > + while (vq->vq_used_cons_idx != > > > + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > > > uint32_t idx, desc_idx, used_idx; > > > struct vring_used_elem *uep; > > > > > > > Find this issue when do the code rework of RX/TX queue. > > As in other places, we also have loop retrieving the value of avial->idx > > or used->idx, i prefer to declare the index in vq structure as volatile > > to avoid potential issue. Is there a reason why the value is not always volatile? I would have thought it would be generally safer to mark the actual value as volatile inside the structure definition itself? In any cases where we do want to store the value locally and not re-access the structure, a local variable can be used. Regards, /Bruce > > It might be a good idea to wrap this in a macro > similar to ACCESS_ONCE in Linux. > > > > > Stephen: > > Another question is why we need a loop here? > > > > /huawei > > -- > MST
On Wed, May 25, 2016 at 10:47:30AM +0100, Bruce Richardson wrote: > On Wed, May 25, 2016 at 11:34:24AM +0300, Michael S. Tsirkin wrote: > > On Wed, May 25, 2016 at 08:25:20AM +0000, Xie, Huawei wrote: > > > On 5/25/2016 4:12 PM, Xie, Huawei wrote: > > > > There is no external function call or any barrier in the loop, > > > > the used->idx would only be retrieved once. > > > > > > > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > > > > --- > > > > drivers/net/virtio/virtio_ethdev.c | 3 ++- > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > > > > index c3fb628..f6d6305 100644 > > > > --- a/drivers/net/virtio/virtio_ethdev.c > > > > +++ b/drivers/net/virtio/virtio_ethdev.c > > > > @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > > > > usleep(100); > > > > } > > > > > > > > - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > > > > + while (vq->vq_used_cons_idx != > > > > + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > > > > uint32_t idx, desc_idx, used_idx; > > > > struct vring_used_elem *uep; > > > > > > > > > > Find this issue when do the code rework of RX/TX queue. > > > As in other places, we also have loop retrieving the value of avial->idx > > > or used->idx, i prefer to declare the index in vq structure as volatile > > > to avoid potential issue. > > Is there a reason why the value is not always volatile? I would have thought > it would be generally safer to mark the actual value as volatile inside the > structure definition itself? In any cases where we do want to store the value > locally and not re-access the structure, a local variable can be used. > > Regards, > /Bruce Linux generally discourages volatile as a general style guidance: https://www.kernel.org/doc/Documentation/volatile-considered-harmful.txt it doesn't have to apply to dpdk which has a different coding style but IIUC this structure is inherited from linux, deviating will make keeping things up to date harder. > > > > It might be a good idea to wrap this in a macro > > similar to ACCESS_ONCE in Linux. > > > > > > > > Stephen: > > > Another question is why we need a loop here? > > > > > > /huawei > > > > -- > > MST
On Wed, May 25, 2016 at 12:50:02PM +0300, Michael S. Tsirkin wrote: > On Wed, May 25, 2016 at 10:47:30AM +0100, Bruce Richardson wrote: > > On Wed, May 25, 2016 at 11:34:24AM +0300, Michael S. Tsirkin wrote: > > > On Wed, May 25, 2016 at 08:25:20AM +0000, Xie, Huawei wrote: > > > > On 5/25/2016 4:12 PM, Xie, Huawei wrote: > > > > > There is no external function call or any barrier in the loop, > > > > > the used->idx would only be retrieved once. > > > > > > > > > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > > > > > --- > > > > > drivers/net/virtio/virtio_ethdev.c | 3 ++- > > > > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > > > > > > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > > > > > index c3fb628..f6d6305 100644 > > > > > --- a/drivers/net/virtio/virtio_ethdev.c > > > > > +++ b/drivers/net/virtio/virtio_ethdev.c > > > > > @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > > > > > usleep(100); > > > > > } > > > > > > > > > > - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > > > > > + while (vq->vq_used_cons_idx != > > > > > + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > > > > > uint32_t idx, desc_idx, used_idx; > > > > > struct vring_used_elem *uep; > > > > > > > > > > > > > Find this issue when do the code rework of RX/TX queue. > > > > As in other places, we also have loop retrieving the value of avial->idx > > > > or used->idx, i prefer to declare the index in vq structure as volatile > > > > to avoid potential issue. > > > > Is there a reason why the value is not always volatile? I would have thought > > it would be generally safer to mark the actual value as volatile inside the > > structure definition itself? In any cases where we do want to store the value > > locally and not re-access the structure, a local variable can be used. > > > > Regards, > > /Bruce > > Linux generally discourages volatile as a general style guidance: > https://www.kernel.org/doc/Documentation/volatile-considered-harmful.txt > it doesn't have to apply to dpdk which has a different coding style > but IIUC this structure is inherited from linux, deviating > will make keeping things up to date harder. The prohibition on volatile indeed doesn't apply to DPDK, due to the fact that we so seldom use locks, and do a lot of direct register accesses in out PMDs. [I also still have the scars from previous issues where we had nice subtle bugs in our PMDs - which only occurred with specific subversions of gcc - all due to a missing "volatile" on one structure element.] However, in this case, I take your point about keeping things consistent with the kernel. :-) /Bruce > > > > > > > It might be a good idea to wrap this in a macro > > > similar to ACCESS_ONCE in Linux. > > > > > > > > > > > Stephen: > > > > Another question is why we need a loop here? > > > > > > > > /huawei > > > > > > -- > > > MST
On 5/25/2016 6:01 PM, Richardson, Bruce wrote: > On Wed, May 25, 2016 at 12:50:02PM +0300, Michael S. Tsirkin wrote: >> On Wed, May 25, 2016 at 10:47:30AM +0100, Bruce Richardson wrote: >>> On Wed, May 25, 2016 at 11:34:24AM +0300, Michael S. Tsirkin wrote: >>>> On Wed, May 25, 2016 at 08:25:20AM +0000, Xie, Huawei wrote: >>>>> On 5/25/2016 4:12 PM, Xie, Huawei wrote: >>>>>> There is no external function call or any barrier in the loop, >>>>>> the used->idx would only be retrieved once. >>>>>> >>>>>> Signed-off-by: Huawei Xie <huawei.xie@intel.com> >>>>>> --- >>>>>> drivers/net/virtio/virtio_ethdev.c | 3 ++- >>>>>> 1 file changed, 2 insertions(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c >>>>>> index c3fb628..f6d6305 100644 >>>>>> --- a/drivers/net/virtio/virtio_ethdev.c >>>>>> +++ b/drivers/net/virtio/virtio_ethdev.c >>>>>> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, >>>>>> usleep(100); >>>>>> } >>>>>> >>>>>> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { >>>>>> + while (vq->vq_used_cons_idx != >>>>>> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { >>>>>> uint32_t idx, desc_idx, used_idx; >>>>>> struct vring_used_elem *uep; >>>>>> >>>>> Find this issue when do the code rework of RX/TX queue. >>>>> As in other places, we also have loop retrieving the value of avial->idx >>>>> or used->idx, i prefer to declare the index in vq structure as volatile >>>>> to avoid potential issue. >>> Is there a reason why the value is not always volatile? I would have thought >>> it would be generally safer to mark the actual value as volatile inside the >>> structure definition itself? In any cases where we do want to store the value >>> locally and not re-access the structure, a local variable can be used. >>> >>> Regards, >>> /Bruce >> Linux generally discourages volatile as a general style guidance: >> https://www.kernel.org/doc/Documentation/volatile-considered-harmful.txt >> it doesn't have to apply to dpdk which has a different coding style >> but IIUC this structure is inherited from linux, deviating >> will make keeping things up to date harder. > The prohibition on volatile indeed doesn't apply to DPDK, due to the fact that > we so seldom use locks, and do a lot of direct register accesses in out PMDs. > [I also still have the scars from previous issues where we had nice subtle bugs > in our PMDs - which only occurred with specific subversions of gcc - all due > to a missing "volatile" on one structure element.] > > However, in this case, I take your point about keeping things consistent with > the kernel. :-) At least for virtio PMD, we have to support both Linux and FreeBSD, so DPDK defines its own vring structure instead of including linux header file. Two solutions for this volatile issue, 1) declare used->idx and avail->idx as volatile 2) define similar access_once/read_once/write_once macro. Would take the first one. In future, we could consider define access_once, and apply to all other data structures if we want to use the kernel style. One thing i am confusing is other DPDK components include Linux header files, do they compile on FreeBSD? > > /Bruce > >>>> It might be a good idea to wrap this in a macro >>>> similar to ACCESS_ONCE in Linux. >>>> >>>>> Stephen: >>>>> Another question is why we need a loop here? >>>>> >>>>> /huawei >>>> -- >>>> MST
On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: > There is no external function call or any barrier in the loop, > the used->idx would only be retrieved once. > > Signed-off-by: Huawei Xie <huawei.xie@intel.com> > --- > drivers/net/virtio/virtio_ethdev.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > index c3fb628..f6d6305 100644 > --- a/drivers/net/virtio/virtio_ethdev.c > +++ b/drivers/net/virtio/virtio_ethdev.c > @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > usleep(100); > } > > - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > + while (vq->vq_used_cons_idx != > + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such qualifier) and use this macro here? If you check the reference of that macro, you might find similar issues, say, it is also used inside the while-loop of virtio_recv_mergeable_pkts(). --yliu
On 5/30/2016 4:20 PM, Yuanhan Liu wrote: > On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: >> There is no external function call or any barrier in the loop, >> the used->idx would only be retrieved once. >> >> Signed-off-by: Huawei Xie <huawei.xie@intel.com> >> --- >> drivers/net/virtio/virtio_ethdev.c | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c >> index c3fb628..f6d6305 100644 >> --- a/drivers/net/virtio/virtio_ethdev.c >> +++ b/drivers/net/virtio/virtio_ethdev.c >> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, >> usleep(100); >> } >> >> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { >> + while (vq->vq_used_cons_idx != >> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such > qualifier) and use this macro here? > > If you check the reference of that macro, you might find similar > issues, say, it is also used inside the while-loop of > virtio_recv_mergeable_pkts(). > > --yliu > > Yes, seems it has same issue though haven't confirmed with asm code.
On Wed, Jun 01, 2016 at 05:40:08AM +0000, Xie, Huawei wrote: > On 5/30/2016 4:20 PM, Yuanhan Liu wrote: > > On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: > >> There is no external function call or any barrier in the loop, > >> the used->idx would only be retrieved once. > >> > >> Signed-off-by: Huawei Xie <huawei.xie@intel.com> > >> --- > >> drivers/net/virtio/virtio_ethdev.c | 3 ++- > >> 1 file changed, 2 insertions(+), 1 deletion(-) > >> > >> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > >> index c3fb628..f6d6305 100644 > >> --- a/drivers/net/virtio/virtio_ethdev.c > >> +++ b/drivers/net/virtio/virtio_ethdev.c > >> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > >> usleep(100); > >> } > >> > >> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > >> + while (vq->vq_used_cons_idx != > >> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > > I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such > > qualifier) and use this macro here? > > > > If you check the reference of that macro, you might find similar > > issues, say, it is also used inside the while-loop of > > virtio_recv_mergeable_pkts(). > > > > --yliu > > > > > > Yes, seems it has same issue though haven't confirmed with asm code. So, move the "volatile" qualifier to VIRTQUEUE_NUSED? --yliu
On 6/1/2016 2:03 PM, Yuanhan Liu wrote: > On Wed, Jun 01, 2016 at 05:40:08AM +0000, Xie, Huawei wrote: >> On 5/30/2016 4:20 PM, Yuanhan Liu wrote: >>> On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: >>>> There is no external function call or any barrier in the loop, >>>> the used->idx would only be retrieved once. >>>> >>>> Signed-off-by: Huawei Xie <huawei.xie@intel.com> >>>> --- >>>> drivers/net/virtio/virtio_ethdev.c | 3 ++- >>>> 1 file changed, 2 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c >>>> index c3fb628..f6d6305 100644 >>>> --- a/drivers/net/virtio/virtio_ethdev.c >>>> +++ b/drivers/net/virtio/virtio_ethdev.c >>>> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, >>>> usleep(100); >>>> } >>>> >>>> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { >>>> + while (vq->vq_used_cons_idx != >>>> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { >>> I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such >>> qualifier) and use this macro here? >>> >>> If you check the reference of that macro, you might find similar >>> issues, say, it is also used inside the while-loop of >>> virtio_recv_mergeable_pkts(). >>> >>> --yliu >>> >>> >> Yes, seems it has same issue though haven't confirmed with asm code. > So, move the "volatile" qualifier to VIRTQUEUE_NUSED? > > --yliu > Yes, anyway this is just intermediate fix. In next patch, will declare the idx as volatile, and remove the qualifier in the macro.
On Thu, Jun 02, 2016 at 08:39:36AM +0000, Xie, Huawei wrote: > On 6/1/2016 2:03 PM, Yuanhan Liu wrote: > > On Wed, Jun 01, 2016 at 05:40:08AM +0000, Xie, Huawei wrote: > >> On 5/30/2016 4:20 PM, Yuanhan Liu wrote: > >>> On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: > >>>> There is no external function call or any barrier in the loop, > >>>> the used->idx would only be retrieved once. > >>>> > >>>> Signed-off-by: Huawei Xie <huawei.xie@intel.com> > >>>> --- > >>>> drivers/net/virtio/virtio_ethdev.c | 3 ++- > >>>> 1 file changed, 2 insertions(+), 1 deletion(-) > >>>> > >>>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > >>>> index c3fb628..f6d6305 100644 > >>>> --- a/drivers/net/virtio/virtio_ethdev.c > >>>> +++ b/drivers/net/virtio/virtio_ethdev.c > >>>> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > >>>> usleep(100); > >>>> } > >>>> > >>>> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > >>>> + while (vq->vq_used_cons_idx != > >>>> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > >>> I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such > >>> qualifier) and use this macro here? > >>> > >>> If you check the reference of that macro, you might find similar > >>> issues, say, it is also used inside the while-loop of > >>> virtio_recv_mergeable_pkts(). > >>> > >>> --yliu > >>> > >>> > >> Yes, seems it has same issue though haven't confirmed with asm code. > > So, move the "volatile" qualifier to VIRTQUEUE_NUSED? > > > > --yliu > > > > Yes, anyway this is just intermediate fix. In next patch, will declare > the idx as volatile, and remove the qualifier in the macro. Hmm.., why we need an intermediate fix then, if we can come up with an ultimate fix very quickly? --yliu
On 6/2/2016 4:52 PM, Yuanhan Liu wrote: > On Thu, Jun 02, 2016 at 08:39:36AM +0000, Xie, Huawei wrote: >> On 6/1/2016 2:03 PM, Yuanhan Liu wrote: >>> On Wed, Jun 01, 2016 at 05:40:08AM +0000, Xie, Huawei wrote: >>>> On 5/30/2016 4:20 PM, Yuanhan Liu wrote: >>>>> On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: >>>>>> There is no external function call or any barrier in the loop, >>>>>> the used->idx would only be retrieved once. >>>>>> >>>>>> Signed-off-by: Huawei Xie <huawei.xie@intel.com> >>>>>> --- >>>>>> drivers/net/virtio/virtio_ethdev.c | 3 ++- >>>>>> 1 file changed, 2 insertions(+), 1 deletion(-) >>>>>> >>>>>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c >>>>>> index c3fb628..f6d6305 100644 >>>>>> --- a/drivers/net/virtio/virtio_ethdev.c >>>>>> +++ b/drivers/net/virtio/virtio_ethdev.c >>>>>> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, >>>>>> usleep(100); >>>>>> } >>>>>> >>>>>> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { >>>>>> + while (vq->vq_used_cons_idx != >>>>>> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { >>>>> I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such >>>>> qualifier) and use this macro here? >>>>> >>>>> If you check the reference of that macro, you might find similar >>>>> issues, say, it is also used inside the while-loop of >>>>> virtio_recv_mergeable_pkts(). >>>>> >>>>> --yliu >>>>> >>>>> >>>> Yes, seems it has same issue though haven't confirmed with asm code. >>> So, move the "volatile" qualifier to VIRTQUEUE_NUSED? >>> >>> --yliu >>> >> Yes, anyway this is just intermediate fix. In next patch, will declare >> the idx as volatile, and remove the qualifier in the macro. > Hmm.., why we need an intermediate fix then, if we can come up with an > ultimate fix very quickly? > > --yliu > ... Either is OK. I have no preference.
On Thu, Jun 02, 2016 at 08:54:38AM +0000, Xie, Huawei wrote: > On 6/2/2016 4:52 PM, Yuanhan Liu wrote: > > On Thu, Jun 02, 2016 at 08:39:36AM +0000, Xie, Huawei wrote: > >> On 6/1/2016 2:03 PM, Yuanhan Liu wrote: > >>> On Wed, Jun 01, 2016 at 05:40:08AM +0000, Xie, Huawei wrote: > >>>> On 5/30/2016 4:20 PM, Yuanhan Liu wrote: > >>>>> On Wed, May 25, 2016 at 12:16:41AM +0800, Huawei Xie wrote: > >>>>>> There is no external function call or any barrier in the loop, > >>>>>> the used->idx would only be retrieved once. > >>>>>> > >>>>>> Signed-off-by: Huawei Xie <huawei.xie@intel.com> > >>>>>> --- > >>>>>> drivers/net/virtio/virtio_ethdev.c | 3 ++- > >>>>>> 1 file changed, 2 insertions(+), 1 deletion(-) > >>>>>> > >>>>>> diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c > >>>>>> index c3fb628..f6d6305 100644 > >>>>>> --- a/drivers/net/virtio/virtio_ethdev.c > >>>>>> +++ b/drivers/net/virtio/virtio_ethdev.c > >>>>>> @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, > >>>>>> usleep(100); > >>>>>> } > >>>>>> > >>>>>> - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { > >>>>>> + while (vq->vq_used_cons_idx != > >>>>>> + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { > >>>>> I'm wondering maybe we could fix VIRTQUEUE_NUSED (which has no such > >>>>> qualifier) and use this macro here? > >>>>> > >>>>> If you check the reference of that macro, you might find similar > >>>>> issues, say, it is also used inside the while-loop of > >>>>> virtio_recv_mergeable_pkts(). > >>>>> > >>>>> --yliu > >>>>> > >>>>> > >>>> Yes, seems it has same issue though haven't confirmed with asm code. > >>> So, move the "volatile" qualifier to VIRTQUEUE_NUSED? > >>> > >>> --yliu > >>> > >> Yes, anyway this is just intermediate fix. In next patch, will declare > >> the idx as volatile, and remove the qualifier in the macro. > > Hmm.., why we need an intermediate fix then, if we can come up with an > > ultimate fix very quickly? > > > > --yliu > > > ... Either is OK. I have no preference. Mind to send an ultimate fix then? --yliu
diff --git a/drivers/net/virtio/virtio_ethdev.c b/drivers/net/virtio/virtio_ethdev.c index c3fb628..f6d6305 100644 --- a/drivers/net/virtio/virtio_ethdev.c +++ b/drivers/net/virtio/virtio_ethdev.c @@ -204,7 +204,8 @@ virtio_send_command(struct virtqueue *vq, struct virtio_pmd_ctrl *ctrl, usleep(100); } - while (vq->vq_used_cons_idx != vq->vq_ring.used->idx) { + while (vq->vq_used_cons_idx != + *((volatile uint16_t *)(&vq->vq_ring.used->idx))) { uint32_t idx, desc_idx, used_idx; struct vring_used_elem *uep;