diff mbox

[dpdk-dev,v4] eal: restrict cores auto detection

Message ID 1480700936-8450-1-git-send-email-jianfeng.tan@intel.com (mailing list archive)
State Accepted, archived
Delegated to: Thomas Monjalon
Headers

Checks

Context Check Description
checkpatch/checkpatch success coding style OK

Commit Message

Jianfeng Tan Dec. 2, 2016, 5:48 p.m. UTC 
  This patch uses pthread_getaffinity_np() to narrow down used
cores when none of below options is specified:
  * coremask (-c)
  * corelist (-l)
  * and coremap (--lcores)

The purpose of this patch is to leave out these core related options
when DPDK applications are deployed under container env, so that
users do not need decide the core related parameters when developing
applications. Instead, when applications are deployed in containers,
use cpu-set to constrain which cores can be used inside this container
instance. And DPDK application inside containers just rely on this
auto detect mechanism to start polling threads.

Note: previously, some users are using isolated CPUs, which could
be excluded by default. Please add commands like taskset to use
those cores.

Test example:
$ taskset 0xc0000 ./examples/helloworld/build/helloworld -m 1024

Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
---
v4:
  - Address Bruce's comment: only enable this auto detection
    mechanism when none of core options is specified.
  - More detailed use case on how it helps in containers.
v3:
  - Choose a more descriptive variable name, and remove comments
    as suggested by Stephen Hemminger.
v2:
  - Make it as default instead of adding the new options.
---
 lib/librte_eal/common/eal_common_options.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)

Comments

Thomas Monjalon Dec. 8, 2016, 6:19 p.m. UTC | #1 
2016-12-02 17:48, Jianfeng Tan:
> This patch uses pthread_getaffinity_np() to narrow down used
> cores when none of below options is specified:
>   * coremask (-c)
>   * corelist (-l)
>   * and coremap (--lcores)
> 
> The purpose of this patch is to leave out these core related options
> when DPDK applications are deployed under container env, so that
> users do not need decide the core related parameters when developing
> applications. Instead, when applications are deployed in containers,
> use cpu-set to constrain which cores can be used inside this container
> instance. And DPDK application inside containers just rely on this
> auto detect mechanism to start polling threads.
> 
> Note: previously, some users are using isolated CPUs, which could
> be excluded by default. Please add commands like taskset to use
> those cores.
> 
> Test example:
> $ taskset 0xc0000 ./examples/helloworld/build/helloworld -m 1024

Bruce, what do you think of this version?
It requires taskset only if -c, -l and --lcores are not used.

>  static int master_lcore_parsed;
>  static int mem_parsed;
> +static int core_specified;

I think it's better to keep the word "parsed" as others.
Bruce Richardson Dec. 9, 2016, 3:14 p.m. UTC | #2 
On Thu, Dec 08, 2016 at 07:19:41PM +0100, Thomas Monjalon wrote:
> 2016-12-02 17:48, Jianfeng Tan:
> > This patch uses pthread_getaffinity_np() to narrow down used
> > cores when none of below options is specified:
> >   * coremask (-c)
> >   * corelist (-l)
> >   * and coremap (--lcores)
> > 
> > The purpose of this patch is to leave out these core related options
> > when DPDK applications are deployed under container env, so that
> > users do not need decide the core related parameters when developing
> > applications. Instead, when applications are deployed in containers,
> > use cpu-set to constrain which cores can be used inside this container
> > instance. And DPDK application inside containers just rely on this
> > auto detect mechanism to start polling threads.
> > 
> > Note: previously, some users are using isolated CPUs, which could
> > be excluded by default. Please add commands like taskset to use
> > those cores.
> > 
> > Test example:
> > $ taskset 0xc0000 ./examples/helloworld/build/helloworld -m 1024
> 
> Bruce, what do you think of this version?
> It requires taskset only if -c, -l and --lcores are not used.
> 
I'm fine with that since it maintains backward compatibilty for those
options.

Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Thomas Monjalon Dec. 21, 2016, 2:31 p.m. UTC | #3 
2016-12-09 15:14, Bruce Richardson:
> On Thu, Dec 08, 2016 at 07:19:41PM +0100, Thomas Monjalon wrote:
> > 2016-12-02 17:48, Jianfeng Tan:
> > > This patch uses pthread_getaffinity_np() to narrow down used
> > > cores when none of below options is specified:
> > >   * coremask (-c)
> > >   * corelist (-l)
> > >   * and coremap (--lcores)
> > > 
> > > The purpose of this patch is to leave out these core related options
> > > when DPDK applications are deployed under container env, so that
> > > users do not need decide the core related parameters when developing
> > > applications. Instead, when applications are deployed in containers,
> > > use cpu-set to constrain which cores can be used inside this container
> > > instance. And DPDK application inside containers just rely on this
> > > auto detect mechanism to start polling threads.
> > > 
> > > Note: previously, some users are using isolated CPUs, which could
> > > be excluded by default. Please add commands like taskset to use
> > > those cores.
> > > 
> > > Test example:
> > > $ taskset 0xc0000 ./examples/helloworld/build/helloworld -m 1024
> > 
> > Bruce, what do you think of this version?
> > It requires taskset only if -c, -l and --lcores are not used.
> > 
> I'm fine with that since it maintains backward compatibilty for those
> options.
> 
> Acked-by: Bruce Richardson <bruce.richardson@intel.com>

Applied with "s/specified/parsed/", thanks
diff mbox

Patch

diff --git a/lib/librte_eal/common/eal_common_options.c b/lib/librte_eal/common/eal_common_options.c
index 6ca8af1..d192de1 100644
--- a/lib/librte_eal/common/eal_common_options.c
+++ b/lib/librte_eal/common/eal_common_options.c
@@ -126,6 +126,7 @@  static const char dpdk_solib_path[] __attribute__((used)) =
 
 static int master_lcore_parsed;
 static int mem_parsed;
+static int core_specified;
 
 void
 eal_reset_internal_config(struct internal_config *internal_cfg)
@@ -797,6 +798,7 @@  eal_parse_common_option(int opt, const char *optarg,
 			RTE_LOG(ERR, EAL, "invalid coremask\n");
 			return -1;
 		}
+		core_specified = 1;
 		break;
 	/* corelist */
 	case 'l':
@@ -804,6 +806,7 @@  eal_parse_common_option(int opt, const char *optarg,
 			RTE_LOG(ERR, EAL, "invalid core list\n");
 			return -1;
 		}
+		core_specified = 1;
 		break;
 	/* size of memory */
 	case 'm':
@@ -912,6 +915,7 @@  eal_parse_common_option(int opt, const char *optarg,
 				OPT_LCORES "\n");
 			return -1;
 		}
+		core_specified = 1;
 		break;
 
 	/* don't know what to do, leave this to caller */
@@ -923,12 +927,38 @@  eal_parse_common_option(int opt, const char *optarg,
 	return 0;
 }
 
+static void
+eal_auto_detect_cores(struct rte_config *cfg)
+{
+	unsigned int lcore_id;
+	unsigned int removed = 0;
+	rte_cpuset_t affinity_set;
+	pthread_t tid = pthread_self();
+
+	if (pthread_getaffinity_np(tid, sizeof(rte_cpuset_t),
+				&affinity_set) < 0)
+		CPU_ZERO(&affinity_set);
+
+	for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
+		if (cfg->lcore_role[lcore_id] == ROLE_RTE &&
+		    !CPU_ISSET(lcore_id, &affinity_set)) {
+			cfg->lcore_role[lcore_id] = ROLE_OFF;
+			removed++;
+		}
+	}
+
+	cfg->lcore_count -= removed;
+}
+
 int
 eal_adjust_config(struct internal_config *internal_cfg)
 {
 	int i;
 	struct rte_config *cfg = rte_eal_get_configuration();
 
+	if (!core_specified)
+		eal_auto_detect_cores(cfg);
+
 	if (internal_config.process_type == RTE_PROC_AUTO)
 		internal_config.process_type = eal_proc_type_detect();