| Message ID | 1458665888-4577-1-git-send-email-michael.frasca@oracle.com (mailing list archive) |
|---|---|
| State | Accepted, archived |
| Headers |
Return-Path: <dev-bounces@dpdk.org> X-Original-To: patchwork@dpdk.org Delivered-To: patchwork@dpdk.org Received: from [92.243.14.124] (localhost [IPv6:::1]) by dpdk.org (Postfix) with ESMTP id 37F5C2C1A; Tue, 22 Mar 2016 17:58:23 +0100 (CET) Received: from aserp1040.oracle.com (aserp1040.oracle.com [141.146.126.69]) by dpdk.org (Postfix) with ESMTP id 99B182BAC for <dev@dpdk.org>; Tue, 22 Mar 2016 17:58:21 +0100 (CET) Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by aserp1040.oracle.com (Sentrion-MTA-4.3.2/Sentrion-MTA-4.3.2) with ESMTP id u2MGwKtJ031545 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 22 Mar 2016 16:58:20 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by userv0022.oracle.com (8.14.4/8.13.8) with ESMTP id u2MGwJxE028512 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 22 Mar 2016 16:58:20 GMT Received: from abhmp0019.oracle.com (abhmp0019.oracle.com [141.146.116.25]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id u2MGwJNv025506; Tue, 22 Mar 2016 16:58:19 GMT Received: from mfrasca-ubuntu.oracle.com (/10.197.178.139) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 22 Mar 2016 09:58:19 -0700 From: Michael Frasca <michael.frasca@oracle.com> To: jing.d.chen@intel.com Cc: dev@dpdk.org, Michael Frasca <michael.frasca@oracle.com> Date: Tue, 22 Mar 2016 12:58:08 -0400 Message-Id: <1458665888-4577-1-git-send-email-michael.frasca@oracle.com> X-Mailer: git-send-email 2.5.0 X-Source-IP: userv0022.oracle.com [156.151.31.74] Subject: [dpdk-dev] [PATCH] fm10k: conditionally disable RSS during device initialization X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK <dev.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/dev>, <mailto:dev-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/dev/> List-Post: <mailto:dev@dpdk.org> List-Help: <mailto:dev-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/dev>, <mailto:dev-request@dpdk.org?subject=subscribe> Errors-To: dev-bounces@dpdk.org Sender: "dev" <dev-bounces@dpdk.org> |
Commit Message
Michael Frasca
March 22, 2016, 4:58 p.m. UTC
If the provided configuration does not call for RSS, then RSS is
explicitly disabled. Without this change, the device continues to
operate under the previous RSS configuration.
Fixes: 57033cdf8fdc ("fm10k: add PF RSS")
Signed-off-by: Michael Frasca <michael.frasca@oracle.com>
---
drivers/net/fm10k/fm10k_ethdev.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
Comments
Hi, > -----Original Message----- > From: Michael Frasca [mailto:michael.frasca@oracle.com] > Sent: Wednesday, March 23, 2016 12:58 AM > To: Chen, Jing D > Cc: dev@dpdk.org; Michael Frasca > Subject: [PATCH] fm10k: conditionally disable RSS during device initialization > > If the provided configuration does not call for RSS, then RSS is > explicitly disabled. Without this change, the device continues to > operate under the previous RSS configuration. > > Fixes: 57033cdf8fdc ("fm10k: add PF RSS") > > Signed-off-by: Michael Frasca <michael.frasca@oracle.com> Acked-by : Jing Chen <jing.d.chen@intel.com>
Hi Jing, I ran into this issue while trying to run experiments with different RSS configurations (no RSS being one cases). It is not clear to me that setting this register to zero is the best way to disable RSS. After digging further, I have a theory that I'm having this issues because I've only attached my DPDK application to SR-IOV ports. In fm10k_dev_dglort_map_configure(), I see that 'RSS Length' is set for the DGLORT decoder. However, it appears that this is only invoked for physical functions. Could this be my problem? Is it required that I bind to the physical function if I want to properly manipulate RSS? Thanks, Mike > On Mar 22, 2016, at 11:14 PM, Chen, Jing D <jing.d.chen@intel.com> wrote: > > Hi, > >> -----Original Message----- >> From: Michael Frasca [mailto:michael.frasca@oracle.com] >> Sent: Wednesday, March 23, 2016 12:58 AM >> To: Chen, Jing D >> Cc: dev@dpdk.org; Michael Frasca >> Subject: [PATCH] fm10k: conditionally disable RSS during device initialization >> >> If the provided configuration does not call for RSS, then RSS is >> explicitly disabled. Without this change, the device continues to >> operate under the previous RSS configuration. >> >> Fixes: 57033cdf8fdc ("fm10k: add PF RSS") >> >> Signed-off-by: Michael Frasca <michael.frasca@oracle.com> > Acked-by : Jing Chen <jing.d.chen@intel.com> >
Hi, Frasca, > -----Original Message----- > From: Michael Frasca [mailto:michael.frasca@oracle.com] > Sent: Wednesday, March 23, 2016 9:43 PM > To: Chen, Jing D > Cc: dev@dpdk.org > Subject: Re: [PATCH] fm10k: conditionally disable RSS during device > initialization > > Hi Jing, > > I ran into this issue while trying to run experiments with different RSS > configurations (no RSS being one cases). It is not clear to me that setting this > register to zero is the best way to disable RSS. > > After digging further, I have a theory that I'm having this issues because I've > only attached my DPDK application to SR-IOV ports. In > fm10k_dev_dglort_map_configure(), I see that 'RSS Length' is set for the > DGLORT > decoder. However, it appears that this is only invoked for physical functions. > > Could this be my problem? Is it required that I bind to the physical function > if I want to properly manipulate RSS? > > Thanks, > Mike > I don't know exactly what problem you ran into. I think we needn't worry about those DGLORT setting when using VF device. I've followed steps to use SRIOV device with RSS enabled and disabled, both are worked well from my side after applying your patch. Below is my setup. 1. PF with Linux driver "fm10k-next_0.19.3". 2. DPDK with latest code from master branch, apply your patch. 3. Use 1 VF device created by kernel driver. 4. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 --config="(0,0,2),(0,1,2),(0,2,3),(0,3,3)"" with RSS enabled. After sending packets, I can see all 4 queues received packets. 5. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 --config="(0,0,2)"" with RSS disabled. After sending packets, I can see queue 0 received packets. Can you explain what actual problem is? We can talk offline.
Jing, Thanks for your assistance. The experiment that you have built should allow you to observe the bug. In [5], I would expect that queue 0 receives roughly 1/4 of the packets that you sending, assuming the input packets have varied IP addresses. Can you measure what % of packets are actually being received in this single queue setup (after first running a 4-queue setup)? When trying to running with only one RX queue, the fm10k retains the same RSS hash function and redirection table that was configured from a previous run. As a result, some packets are still being directed to other receive queues. I have confirmed this by polling the queue specific stats, which I retrieved via rte_eth_xstats_get(). Looking at fm10k_dev_rss_configure(), one should see that there is no modification of fm10k registers when nb_rx_queues == 1. As far as I can tell, this is the reason that only a certain partition of packets are being receive in a single queue setup (after first running a multi-queue configuration). I am unable to access my development environment today, but if you need, I can later craft a patch to l3fwd that shows the measurement of packets received at each queue. Thanks, Mike > On Mar 24, 2016, at 2:40 AM, Chen, Jing D <jing.d.chen@intel.com> wrote: > > Hi, Frasca, > > > >> -----Original Message----- >> From: Michael Frasca [mailto:michael.frasca@oracle.com <mailto:michael.frasca@oracle.com>] >> Sent: Wednesday, March 23, 2016 9:43 PM >> To: Chen, Jing D >> Cc: dev@dpdk.org <mailto:dev@dpdk.org> >> Subject: Re: [PATCH] fm10k: conditionally disable RSS during device >> initialization >> >> Hi Jing, >> >> I ran into this issue while trying to run experiments with different RSS >> configurations (no RSS being one cases). It is not clear to me that setting this >> register to zero is the best way to disable RSS. >> >> After digging further, I have a theory that I'm having this issues because I've >> only attached my DPDK application to SR-IOV ports. In >> fm10k_dev_dglort_map_configure(), I see that 'RSS Length' is set for the >> DGLORT >> decoder. However, it appears that this is only invoked for physical functions. >> >> Could this be my problem? Is it required that I bind to the physical function >> if I want to properly manipulate RSS? >> >> Thanks, >> Mike >> > I don't know exactly what problem you ran into. I think we needn't worry about > those DGLORT setting when using VF device. > > I've followed steps to use SRIOV device with RSS enabled and disabled, both > are worked well from my side after applying your patch. Below is my setup. > > 1. PF with Linux driver "fm10k-next_0.19.3". > 2. DPDK with latest code from master branch, apply your patch. > 3. Use 1 VF device created by kernel driver. > 4. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 --config="(0,0,2),(0,1,2),(0,2,3),(0,3,3)"" > with RSS enabled. After sending packets, I can see all 4 queues received packets. > 5. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 --config="(0,0,2)"" > with RSS disabled. After sending packets, I can see queue 0 received packets. > > Can you explain what actual problem is? > We can talk offline.
Please, anyone to confirm that the patch is valid and must be applied? This discussion shows some doubts. 2016-03-24 13:35, Michael Frasca: > Jing, > > Thanks for your assistance. The experiment that you have built should allow you > to observe the bug. In [5], I would expect that queue 0 receives roughly 1/4 of > the packets that you sending, assuming the input packets have varied IP > addresses. Can you measure what % of packets are actually being received in this > single queue setup (after first running a 4-queue setup)? > > When trying to running with only one RX queue, the fm10k retains the same RSS > hash function and redirection table that was configured from a previous run. As > a result, some packets are still being directed to other receive queues. I have > confirmed this by polling the queue specific stats, which I retrieved via > rte_eth_xstats_get(). > > Looking at fm10k_dev_rss_configure(), one should see that there is no > modification of fm10k registers when nb_rx_queues == 1. As far as I can tell, > this is the reason that only a certain partition of packets are being receive in > a single queue setup (after first running a multi-queue configuration). > > I am unable to access my development environment today, but if you need, I can > later craft a patch to l3fwd that shows the measurement of packets received at > each queue. > > Thanks, > Mike > > > > On Mar 24, 2016, at 2:40 AM, Chen, Jing D <jing.d.chen@intel.com> wrote: > > > > Hi, Frasca, > > > > > > > >> -----Original Message----- > >> From: Michael Frasca [mailto:michael.frasca@oracle.com <mailto:michael.frasca@oracle.com>] > >> Sent: Wednesday, March 23, 2016 9:43 PM > >> To: Chen, Jing D > >> Cc: dev@dpdk.org <mailto:dev@dpdk.org> > >> Subject: Re: [PATCH] fm10k: conditionally disable RSS during device > >> initialization > >> > >> Hi Jing, > >> > >> I ran into this issue while trying to run experiments with different RSS > >> configurations (no RSS being one cases). It is not clear to me that setting this > >> register to zero is the best way to disable RSS. > >> > >> After digging further, I have a theory that I'm having this issues because I've > >> only attached my DPDK application to SR-IOV ports. In > >> fm10k_dev_dglort_map_configure(), I see that 'RSS Length' is set for the > >> DGLORT > >> decoder. However, it appears that this is only invoked for physical functions. > >> > >> Could this be my problem? Is it required that I bind to the physical function > >> if I want to properly manipulate RSS? > >> > >> Thanks, > >> Mike > >> > > I don't know exactly what problem you ran into. I think we needn't worry about > > those DGLORT setting when using VF device. > > > > I've followed steps to use SRIOV device with RSS enabled and disabled, both > > are worked well from my side after applying your patch. Below is my setup. > > > > 1. PF with Linux driver "fm10k-next_0.19.3". > > 2. DPDK with latest code from master branch, apply your patch. > > 3. Use 1 VF device created by kernel driver. > > 4. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 --config="(0,0,2),(0,1,2),(0,2,3),(0,3,3)"" > > with RSS enabled. After sending packets, I can see all 4 queues received packets. > > 5. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 --config="(0,0,2)"" > > with RSS disabled. After sending packets, I can see queue 0 received packets. > > > > Can you explain what actual problem is? > > We can talk offline. >
Thomas, We've agreed offline that the patch works without side effect. Please kindly apply if possible. > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Thursday, March 31, 2016 9:57 PM > To: dev@dpdk.org > Cc: Michael Frasca <michael.frasca@oracle.com>; Chen, Jing D > <jing.d.chen@intel.com> > Subject: Re: [dpdk-dev] [PATCH] fm10k: conditionally disable RSS during > device initialization > > Please, anyone to confirm that the patch is valid and must be applied? > This discussion shows some doubts. > > > 2016-03-24 13:35, Michael Frasca: > > Jing, > > > > Thanks for your assistance. The experiment that you have built should > > allow you to observe the bug. In [5], I would expect that queue 0 > > receives roughly 1/4 of the packets that you sending, assuming the > > input packets have varied IP addresses. Can you measure what % of > > packets are actually being received in this single queue setup (after first > running a 4-queue setup)? > > > > When trying to running with only one RX queue, the fm10k retains the > > same RSS hash function and redirection table that was configured from > > a previous run. As a result, some packets are still being directed to > > other receive queues. I have confirmed this by polling the queue > > specific stats, which I retrieved via rte_eth_xstats_get(). > > > > Looking at fm10k_dev_rss_configure(), one should see that there is no > > modification of fm10k registers when nb_rx_queues == 1. As far as I > > can tell, this is the reason that only a certain partition of packets > > are being receive in a single queue setup (after first running a multi-queue > configuration). > > > > I am unable to access my development environment today, but if you > > need, I can later craft a patch to l3fwd that shows the measurement of > > packets received at each queue. > > > > Thanks, > > Mike > > > > > > > On Mar 24, 2016, at 2:40 AM, Chen, Jing D <jing.d.chen@intel.com> wrote: > > > > > > Hi, Frasca, > > > > > > > > > > > >> -----Original Message----- > > >> From: Michael Frasca [mailto:michael.frasca@oracle.com > > >> <mailto:michael.frasca@oracle.com>] > > >> Sent: Wednesday, March 23, 2016 9:43 PM > > >> To: Chen, Jing D > > >> Cc: dev@dpdk.org <mailto:dev@dpdk.org> > > >> Subject: Re: [PATCH] fm10k: conditionally disable RSS during device > > >> initialization > > >> > > >> Hi Jing, > > >> > > >> I ran into this issue while trying to run experiments with > > >> different RSS configurations (no RSS being one cases). It is not > > >> clear to me that setting this register to zero is the best way to disable > RSS. > > >> > > >> After digging further, I have a theory that I'm having this issues > > >> because I've only attached my DPDK application to SR-IOV ports. In > > >> fm10k_dev_dglort_map_configure(), I see that 'RSS Length' is set > > >> for the DGLORT decoder. However, it appears that this is only > > >> invoked for physical functions. > > >> > > >> Could this be my problem? Is it required that I bind to the > > >> physical function if I want to properly manipulate RSS? > > >> > > >> Thanks, > > >> Mike > > >> > > > I don't know exactly what problem you ran into. I think we needn't > > > worry about those DGLORT setting when using VF device. > > > > > > I've followed steps to use SRIOV device with RSS enabled and > > > disabled, both are worked well from my side after applying your patch. > Below is my setup. > > > > > > 1. PF with Linux driver "fm10k-next_0.19.3". > > > 2. DPDK with latest code from master branch, apply your patch. > > > 3. Use 1 VF device created by kernel driver. > > > 4. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 -- > config="(0,0,2),(0,1,2),(0,2,3),(0,3,3)"" > > > with RSS enabled. After sending packets, I can see all 4 queues received > packets. > > > 5. use l3fwd with " ./examples/l3fwd/build/l3fwd -c fc -n 4 -- -p 0x1 -- > config="(0,0,2)"" > > > with RSS disabled. After sending packets, I can see queue 0 received > packets. > > > > > > Can you explain what actual problem is? > > > We can talk offline. > > >
> > If the provided configuration does not call for RSS, then RSS is > > explicitly disabled. Without this change, the device continues to > > operate under the previous RSS configuration. > > > > Fixes: 57033cdf8fdc ("fm10k: add PF RSS") > > > > Signed-off-by: Michael Frasca <michael.frasca@oracle.com> > Acked-by : Jing Chen <jing.d.chen@intel.com> Applied, thanks
diff --git a/drivers/net/fm10k/fm10k_ethdev.c b/drivers/net/fm10k/fm10k_ethdev.c index 4b07a8b..4d5365a 100644 --- a/drivers/net/fm10k/fm10k_ethdev.c +++ b/drivers/net/fm10k/fm10k_ethdev.c @@ -531,8 +531,10 @@ fm10k_dev_rss_configure(struct rte_eth_dev *dev) if (dev->data->nb_rx_queues == 1 || dev_conf->rxmode.mq_mode != ETH_MQ_RX_RSS || - dev_conf->rx_adv_conf.rss_conf.rss_hf == 0) + dev_conf->rx_adv_conf.rss_conf.rss_hf == 0) { + FM10K_WRITE_REG(hw, FM10K_MRQC(0), 0); return; + } /* random key is rss_intel_key (default) or user provided (rss_key) */ if (dev_conf->rx_adv_conf.rss_conf.rss_key == NULL)