Message ID | 1455863563-15751-5-git-send-email-i.maximets@samsung.com (mailing list archive) |
---|---|
State | Rejected, archived |
Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 1593CC4EA; Fri, 19 Feb 2016 07:33:02 +0100 (CET) Received: from mailout2.w1.samsung.com (mailout2.w1.samsung.com [210.118.77.12]) by dpdk.org (Postfix) with ESMTP id 664A5C4D2 for <dev@dpdk.org>; Fri, 19 Feb 2016 07:33:00 +0100 (CET) Received: from eucpsbgm2.samsung.com (unknown [203.254.199.245]) by mailout2.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0O2S00E3N7IZKO70@mailout2.w1.samsung.com> for dev@dpdk.org; Fri, 19 Feb 2016 06:32:59 +0000 (GMT) X-AuditID: cbfec7f5-f79b16d000005389-56-56c6b71a53eb Received: from eusync4.samsung.com ( [203.254.199.214]) by eucpsbgm2.samsung.com (EUCPMTA) with SMTP id FB.48.21385.A17B6C65; Fri, 19 Feb 2016 06:32:58 +0000 (GMT) Received: from imaximets.rnd.samsung.ru ([106.109.129.180]) by eusync4.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0O2S00G447IL8D70@eusync4.samsung.com>; Fri, 19 Feb 2016 06:32:58 +0000 (GMT) From: Ilya Maximets <i.maximets@samsung.com> To: dev@dpdk.org, Huawei Xie <huawei.xie@intel.com>, Yuanhan Liu <yuanhan.liu@linux.intel.com> Date: Fri, 19 Feb 2016 09:32:43 +0300 Message-id: <1455863563-15751-5-git-send-email-i.maximets@samsung.com> X-Mailer: git-send-email 2.5.0 In-reply-to: <1455863563-15751-1-git-send-email-i.maximets@samsung.com> References: <1455863563-15751-1-git-send-email-i.maximets@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprGLMWRmVeSWpSXmKPExsVy+t/xa7pS24+FGVz4a23x7tN2Jov2mWeZ LK60/2S3mDjJxGL60wiLybOlLK5PuMDqwO6x4UQ/q8evBUtZPRbvecnkcefaHjaPeScDPfq2 rGIMYIvisklJzcksSy3St0vgyrjb1M9asJSjYtW72WwNjDPZuxg5OSQETCTezVrPCGGLSVy4 t56ti5GLQ0hgKaPEvfNzWSCcViaJX28OgVWxCehInFp9BMwWEUiQOLL/NytIEbPAMkaJiycm M4EkhAVCJRbPeAu2gkVAVeLIvmYWEJtXwE3idssboBoOoHVyEgsupIOEOQXcJb5sagGbKQRU 0nd2F+MERt4FjAyrGEVTS5MLipPSc430ihNzi0vz0vWS83M3MUJC6+sOxqXHrA4xCnAwKvHw VugdCxNiTSwrrsw9xCjBwawkwqvrDxTiTUmsrEotyo8vKs1JLT7EKM3BoiTOO3PX+xAhgfTE ktTs1NSC1CKYLBMHp1QDo9XtmHnN0joWpfNT6l4d+Nmx2/vaF9+o0l5TwS/Jij2PK0LP1GnZ 3n9iu6VfbMHZL0JGXbN/vnkhGF6gvX7TpZKL787MWi/kZnzF8FPj3q9Ppxf9EP+x7yzjB8sP e1vertnTFWC9/zm3m+GaWX8Z1nvcWW5es+Xaol2NbySZQ1afjnHQkTlp3avEUpyRaKjFXFSc CADgoJzXKQIAAA== Cc: Ilya Maximets <i.maximets@samsung.com>, Dyasly Sergey <s.dyasly@samsung.com> Subject: [dpdk-dev] [PATCH RFC 4/4] doc: add note about rte_vhost_enqueue_burst thread safety. X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Ilya Maximets
Feb. 19, 2016, 6:32 a.m. UTC
Signed-off-by: Ilya Maximets <i.maximets@samsung.com>
---
doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 +
1 file changed, 1 insertion(+)
Comments
On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: > Signed-off-by: Ilya Maximets <i.maximets@samsung.com> > --- > doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst > index 403e5fc..13a6c89 100644 > --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst > +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst > @@ -67,6 +67,7 @@ then locking, or some other form of mutual exclusion, is necessary. > The ring library is based on a lockless ring-buffer algorithm that maintains its original design for thread safety. > Moreover, it provides high performance for either multi- or single-consumer/producer enqueue/dequeue operations. > The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. > +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, to aligh with the usage of rte_eth_tx_burst(). --yliu
On 2/19/2016 3:10 PM, Yuanhan Liu wrote: > On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: >> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> >> --- >> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + >> 1 file changed, 1 insertion(+) >> >> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >> index 403e5fc..13a6c89 100644 >> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >> @@ -67,6 +67,7 @@ then locking, or some other form of mutual exclusion, is necessary. >> The ring library is based on a lockless ring-buffer algorithm that maintains its original design for thread safety. >> Moreover, it provides high performance for either multi- or single-consumer/producer enqueue/dequeue operations. >> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. >> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. > FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, > to aligh with the usage of rte_eth_tx_burst(). > > --yliu I have a patch to remove the lockless enqueue. Unless there is strong reason, i prefer vhost PMD to behave like other PMDs, with no internal lockless algorithm. In future, for people who really need it, we could have dynamic/static switch to enable it.
On 19.02.2016 11:36, Xie, Huawei wrote: > On 2/19/2016 3:10 PM, Yuanhan Liu wrote: >> On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: >>> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> >>> --- >>> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + >>> 1 file changed, 1 insertion(+) >>> >>> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>> index 403e5fc..13a6c89 100644 >>> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>> @@ -67,6 +67,7 @@ then locking, or some other form of mutual exclusion, is necessary. >>> The ring library is based on a lockless ring-buffer algorithm that maintains its original design for thread safety. >>> Moreover, it provides high performance for either multi- or single-consumer/producer enqueue/dequeue operations. >>> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. >>> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. >> FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, >> to aligh with the usage of rte_eth_tx_burst(). >> >> --yliu > > I have a patch to remove the lockless enqueue. Unless there is strong > reason, i prefer vhost PMD to behave like other PMDs, with no internal > lockless algorithm. In future, for people who really need it, we could > have dynamic/static switch to enable it. OK, got it. So, I think, this documentation patch may be dropped. Other patches of series still may be merged to fix existing issues and keep code in consistent state for the future. Am I right? Best regards, Ilya Maximets.
On 2/19/2016 5:05 PM, Ilya Maximets wrote: > On 19.02.2016 11:36, Xie, Huawei wrote: >> On 2/19/2016 3:10 PM, Yuanhan Liu wrote: >>> On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: >>>> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> >>>> --- >>>> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + >>>> 1 file changed, 1 insertion(+) >>>> >>>> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>> index 403e5fc..13a6c89 100644 >>>> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>> @@ -67,6 +67,7 @@ then locking, or some other form of mutual exclusion, is necessary. >>>> The ring library is based on a lockless ring-buffer algorithm that maintains its original design for thread safety. >>>> Moreover, it provides high performance for either multi- or single-consumer/producer enqueue/dequeue operations. >>>> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. >>>> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. >>> FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, >>> to aligh with the usage of rte_eth_tx_burst(). >>> >>> --yliu >> I have a patch to remove the lockless enqueue. Unless there is strong >> reason, i prefer vhost PMD to behave like other PMDs, with no internal >> lockless algorithm. In future, for people who really need it, we could >> have dynamic/static switch to enable it. Thomas, what is your opinion on this and my patch removing lockless enqueue? > OK, got it. So, I think, this documentation patch may be dropped. > Other patches of series still may be merged to fix existing issues and > keep code in consistent state for the future. > Am I right? Yes. > Best regards, Ilya Maximets. >
2016-02-22 02:07, Xie, Huawei: > On 2/19/2016 5:05 PM, Ilya Maximets wrote: > > On 19.02.2016 11:36, Xie, Huawei wrote: > >> On 2/19/2016 3:10 PM, Yuanhan Liu wrote: > >>> On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: > >>>> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> > >>>> --- > >>>> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + > >>>> 1 file changed, 1 insertion(+) > >>>> > >>>> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst > >>>> index 403e5fc..13a6c89 100644 > >>>> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst > >>>> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst > >>>> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. > >>>> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. > >>> FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, > >>> to aligh with the usage of rte_eth_tx_burst(). > >>> > >>> --yliu > >> I have a patch to remove the lockless enqueue. Unless there is strong > >> reason, i prefer vhost PMD to behave like other PMDs, with no internal > >> lockless algorithm. In future, for people who really need it, we could > >> have dynamic/static switch to enable it. > > Thomas, what is your opinion on this and my patch removing lockless enqueue? The thread safety behaviour is part of the API specification. If we want to enable/disable such behaviour, it must be done with an API function. But it would introduce a conditional statement in the fast path. That's why the priority must be to keep a simple and consistent behaviour and try to build around. An API complexity may be considered only if there is a real (measured) gain.
On 2/22/2016 6:16 PM, Thomas Monjalon wrote: > 2016-02-22 02:07, Xie, Huawei: >> On 2/19/2016 5:05 PM, Ilya Maximets wrote: >>> On 19.02.2016 11:36, Xie, Huawei wrote: >>>> On 2/19/2016 3:10 PM, Yuanhan Liu wrote: >>>>> On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: >>>>>> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> >>>>>> --- >>>>>> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + >>>>>> 1 file changed, 1 insertion(+) >>>>>> >>>>>> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>> index 403e5fc..13a6c89 100644 >>>>>> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. >>>>>> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. >>>>> FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, >>>>> to aligh with the usage of rte_eth_tx_burst(). >>>>> >>>>> --yliu >>>> I have a patch to remove the lockless enqueue. Unless there is strong >>>> reason, i prefer vhost PMD to behave like other PMDs, with no internal >>>> lockless algorithm. In future, for people who really need it, we could >>>> have dynamic/static switch to enable it. >> Thomas, what is your opinion on this and my patch removing lockless enqueue? > The thread safety behaviour is part of the API specification. > If we want to enable/disable such behaviour, it must be done with an API > function. But it would introduce a conditional statement in the fast path. > That's why the priority must be to keep a simple and consistent behaviour > and try to build around. An API complexity may be considered only if there > is a real (measured) gain. Let us put the gain aside temporarily. I would do the measurement. Vhost is wrapped as a PMD in Tetsuya's patch. And also in DPDK OVS's case, it is wrapped as a vport like all other physical ports. The DPDK app/OVS will treat all ports equally. It will add complexity if the app needs to know that some supports concurrency while some not. Since all other PMDs doesn't support thread safety, it doesn't make sense for vhost PMD to support that. I believe the APP will not use that behavior. From the API's point of view, if we previously implemented it wrongly, we need to fix it as early as possible. > >
On 23.02.2016 08:56, Xie, Huawei wrote: > On 2/22/2016 6:16 PM, Thomas Monjalon wrote: >> 2016-02-22 02:07, Xie, Huawei: >>> On 2/19/2016 5:05 PM, Ilya Maximets wrote: >>>> On 19.02.2016 11:36, Xie, Huawei wrote: >>>>> On 2/19/2016 3:10 PM, Yuanhan Liu wrote: >>>>>> On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: >>>>>>> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> >>>>>>> --- >>>>>>> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + >>>>>>> 1 file changed, 1 insertion(+) >>>>>>> >>>>>>> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>>> index 403e5fc..13a6c89 100644 >>>>>>> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>>> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>>> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. >>>>>>> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. >>>>>> FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, >>>>>> to aligh with the usage of rte_eth_tx_burst(). >>>>>> >>>>>> --yliu >>>>> I have a patch to remove the lockless enqueue. Unless there is strong >>>>> reason, i prefer vhost PMD to behave like other PMDs, with no internal >>>>> lockless algorithm. In future, for people who really need it, we could >>>>> have dynamic/static switch to enable it. >>> Thomas, what is your opinion on this and my patch removing lockless enqueue? >> The thread safety behaviour is part of the API specification. >> If we want to enable/disable such behaviour, it must be done with an API >> function. But it would introduce a conditional statement in the fast path. >> That's why the priority must be to keep a simple and consistent behaviour >> and try to build around. An API complexity may be considered only if there >> is a real (measured) gain. > > Let us put the gain aside temporarily. I would do the measurement. > Vhost is wrapped as a PMD in Tetsuya's patch. And also in DPDK OVS's > case, it is wrapped as a vport like all other physical ports. The DPDK > app/OVS will treat all ports equally. That is not true. Currently vhost in Open vSwitch implemented as a separate netdev class. So, to use concurrency of vhost we just need to remove 2 lines (rte_spinlock_lock and rte_spinlock_unlock) from function __netdev_dpdk_vhost_send(). This will not change behaviour of other types of ports. > It will add complexity if the app > needs to know that some supports concurrency while some not. Since all > other PMDs doesn't support thread safety, it doesn't make sense for > vhost PMD to support that. I believe the APP will not use that behavior. >>From the API's point of view, if we previously implemented it wrongly, > we need to fix it as early as possible.
On 2/24/2016 1:07 PM, Ilya Maximets wrote: > On 23.02.2016 08:56, Xie, Huawei wrote: >> On 2/22/2016 6:16 PM, Thomas Monjalon wrote: >>> 2016-02-22 02:07, Xie, Huawei: >>>> On 2/19/2016 5:05 PM, Ilya Maximets wrote: >>>>> On 19.02.2016 11:36, Xie, Huawei wrote: >>>>>> On 2/19/2016 3:10 PM, Yuanhan Liu wrote: >>>>>>> On Fri, Feb 19, 2016 at 09:32:43AM +0300, Ilya Maximets wrote: >>>>>>>> Signed-off-by: Ilya Maximets <i.maximets@samsung.com> >>>>>>>> --- >>>>>>>> doc/guides/prog_guide/thread_safety_dpdk_functions.rst | 1 + >>>>>>>> 1 file changed, 1 insertion(+) >>>>>>>> >>>>>>>> diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>>>> index 403e5fc..13a6c89 100644 >>>>>>>> --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>>>> +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst >>>>>>>> The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. >>>>>>>> +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. >>>>>>> FYI, Huawei meant to make rte_vhost_enqueue_burst() not be thread-safe, >>>>>>> to aligh with the usage of rte_eth_tx_burst(). >>>>>>> >>>>>>> --yliu >>>>>> I have a patch to remove the lockless enqueue. Unless there is strong >>>>>> reason, i prefer vhost PMD to behave like other PMDs, with no internal >>>>>> lockless algorithm. In future, for people who really need it, we could >>>>>> have dynamic/static switch to enable it. >>>> Thomas, what is your opinion on this and my patch removing lockless enqueue? >>> The thread safety behaviour is part of the API specification. >>> If we want to enable/disable such behaviour, it must be done with an API >>> function. But it would introduce a conditional statement in the fast path. >>> That's why the priority must be to keep a simple and consistent behaviour >>> and try to build around. An API complexity may be considered only if there >>> is a real (measured) gain. >> Let us put the gain aside temporarily. I would do the measurement. >> Vhost is wrapped as a PMD in Tetsuya's patch. And also in DPDK OVS's >> case, it is wrapped as a vport like all other physical ports. The DPDK >> app/OVS will treat all ports equally. > That is not true. Currently vhost in Open vSwitch implemented as a separate > netdev class. So, to use concurrency of vhost we just need to remove > 2 lines (rte_spinlock_lock and rte_spinlock_unlock) from function > __netdev_dpdk_vhost_send(). This will not change behaviour of other types > of ports. I checked OVS implementation. It raised several concerns. For physical ports, it uses multiple queues to solve concurrent tx. For vhost ports, a) The thread safe behavior of vhost isn't used. rte_spinlock is used outside. Yes, it could be removed. b) If a packet is send to vhost, it is directly enqueued to guest without buffering. We could use thread safe ring to queue packets first and then enqueued to guest at appropriate time later, then vhost internal lockless isn't needed. Besides, IMO thread safe implementation adds the complexity of vhost implementation. >> It will add complexity if the app >> needs to know that some supports concurrency while some not. Since all >> other PMDs doesn't support thread safety, it doesn't make sense for >> vhost PMD to support that. I believe the APP will not use that behavior. >> >From the API's point of view, if we previously implemented it wrongly, >> we need to fix it as early as possible.
diff --git a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst index 403e5fc..13a6c89 100644 --- a/doc/guides/prog_guide/thread_safety_dpdk_functions.rst +++ b/doc/guides/prog_guide/thread_safety_dpdk_functions.rst @@ -67,6 +67,7 @@ then locking, or some other form of mutual exclusion, is necessary. The ring library is based on a lockless ring-buffer algorithm that maintains its original design for thread safety. Moreover, it provides high performance for either multi- or single-consumer/producer enqueue/dequeue operations. The mempool library is based on the DPDK lockless ring library and therefore is also multi-thread safe. +rte_vhost_enqueue_burst() is also thread safe because based on lockless ring-buffer algorithm like the ring library. Performance Insensitive API ---------------------------