diff mbox

[dpdk-dev] mempool: avoid memory waste with large pagesize

Message ID 1457490571-24429-1-git-send-email-stephen@networkplumber.org (mailing list archive)
State Superseded, archived
Delegated to: Thomas Monjalon
Headers

Commit Message

Stephen Hemminger March 9, 2016, 2:29 a.m. UTC 
  If page size is large (like 64K on ARM) and object size is small
then don't waste lots of memory by rounding up to page size.
Instead, round up so that 1 or more objects all fit in a page.

This preserves the requirement that an object must not a page
or virt2phys would break, and makes sure 62K is not wasted per mbuf.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 lib/librte_mempool/rte_mempool.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

Comments

Olivier Matz March 9, 2016, 5:49 p.m. UTC | #1 
On 03/09/2016 03:29 AM, Stephen Hemminger wrote:
> If page size is large (like 64K on ARM) and object size is small
> then don't waste lots of memory by rounding up to page size.
> Instead, round up so that 1 or more objects all fit in a page.
> 
> This preserves the requirement that an object must not a page
> or virt2phys would break, and makes sure 62K is not wasted per mbuf.

You should specify that it only affects runs with "--no-huge".


> --- a/lib/librte_mempool/rte_mempool.c
> +++ b/lib/librte_mempool/rte_mempool.c
> @@ -300,18 +300,24 @@ rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
>  	if (! rte_eal_has_hugepages()) {
>  		/*
>  		 * compute trailer size so that pool elements fit exactly in
> -		 * a standard page
> +		 * a standard page. If elements are smaller than a page
> +		 * then allow multiple elements per page
>  		 */
> -		int page_size = getpagesize();
> -		int new_size = page_size - sz->header_size - sz->elt_size;
> -		if (new_size < 0 || (unsigned int)new_size < sz->trailer_size) {
> +		unsigned page_size = getpagesize();
> +		uint32_t orig_size, new_size;
> +
> +		orig_size = sz->header_size + sz->elt_size;
> +		new_size = rte_align32pow2(orig_size);
> +		if (new_size > page_size) {
>  			printf("When hugepages are disabled, pool objects "
>  			       "can't exceed PAGE_SIZE: %d + %d + %d > %d\n",
>  			       sz->header_size, sz->elt_size, sz->trailer_size,
>  			       page_size);
>  			return 0;
>  		}
> -		sz->trailer_size = new_size;
> +
> +		sz->trailer_size = (new_size - orig_size)
> +			/ (page_size / new_size);
>  	}

Looks it does not work, did I miss something?

Examples:

# start with --no-huge
mp = rte_mempool_create("test", 128,
	35, 0, 0, NULL, NULL, NULL, NULL, SOCKET_ID_ANY, 0);
rte_mempool_dump(stdout, mp);

shows:
  header_size=64
  elt_size=40
  trailer_size=0
  total_obj_size=104       <<<<< should be 128?


# start with --no-huge
mp = rte_mempool_create("test", 128,
	191, 0, 0, NULL, NULL, NULL, NULL, SOCKET_ID_ANY,
	MEMPOOL_F_NO_CACHE_ALIGN);
rte_mempool_dump(stdout, mp);

shows:
  header_size=8
  elt_size=192
  trailer_size=3
  total_obj_size=203       <<<<< should be 256?


The RFC I've just submitted also aims to fix this issue
(but differently).

Regards,
Olivier
diff mbox

Patch

diff --git a/lib/librte_mempool/rte_mempool.c b/lib/librte_mempool/rte_mempool.c
index f8781e1..8fa855a 100644
--- a/lib/librte_mempool/rte_mempool.c
+++ b/lib/librte_mempool/rte_mempool.c
@@ -300,18 +300,24 @@  rte_mempool_calc_obj_size(uint32_t elt_size, uint32_t flags,
 	if (! rte_eal_has_hugepages()) {
 		/*
 		 * compute trailer size so that pool elements fit exactly in
-		 * a standard page
+		 * a standard page. If elements are smaller than a page
+		 * then allow multiple elements per page
 		 */
-		int page_size = getpagesize();
-		int new_size = page_size - sz->header_size - sz->elt_size;
-		if (new_size < 0 || (unsigned int)new_size < sz->trailer_size) {
+		unsigned page_size = getpagesize();
+		uint32_t orig_size, new_size;
+
+		orig_size = sz->header_size + sz->elt_size;
+		new_size = rte_align32pow2(orig_size);
+		if (new_size > page_size) {
 			printf("When hugepages are disabled, pool objects "
 			       "can't exceed PAGE_SIZE: %d + %d + %d > %d\n",
 			       sz->header_size, sz->elt_size, sz->trailer_size,
 			       page_size);
 			return 0;
 		}
-		sz->trailer_size = new_size;
+
+		sz->trailer_size = (new_size - orig_size)
+			/ (page_size / new_size);
 	}
 
 	/* this is the size of an object, including header and trailer */