[dpdk-dev,v2] mempool: adjust name string size in related data types
Commit Message
A recent patch brought up an issue about the size of the 'name' fields:
85cf0079 mem: avoid memzone/mempool/ring name truncation
These relations should be observed:
1. Each ring creates a memzone with a prefixed name:
RTE_RING_NAMESIZE <= RTE_MEMZONE_NAMESIZE - strlen(RTE_RING_MZ_PREFIX)
2. There are some mempool handlers which create a ring with a prefixed
name:
RTE_MEMPOOL_NAMESIZE <= RTE_RING_NAMESIZE - strlen(RTE_MEMPOOL_MZ_PREFIX)
3. A mempool can create up to RTE_MAX_MEMZONE pre and postfixed memzones:
sprintf(postfix, "_%d", RTE_MAX_MEMZONE)
RTE_MEMPOOL_NAMESIZE <= RTE_MEMZONE_NAMESIZE -
strlen(RTE_MEMPOOL_MZ_PREFIX) - strlen(postfix)
Setting all of them to 32 hides this restriction from the application.
This patch decreases the mempool and ring string size to accommodate for
these prefixes, but it doesn't apply the 3rd constraint. Applications
relying on these constants need to be recompiled, otherwise they'll run
into ENAMETOOLONG issues.
The size of the arrays are kept 32 for ABI compatibility, it can be
decreased next time the ABI changes.
Signed-off-by: Zoltan Kiss <zoltan.kiss@schaman.hu>
---
Notes:
v2: keep arrays 32 bytes and decrease the max sizes to maintain ABI
compatibility
lib/librte_mempool/rte_mempool.h | 11 +++++++++--
lib/librte_ring/rte_ring.h | 12 ++++++++++--
2 files changed, 19 insertions(+), 4 deletions(-)
Comments
Hi Zoltan,
On 07/20/2016 07:16 PM, Zoltan Kiss wrote:
> A recent patch brought up an issue about the size of the 'name' fields:
>
> 85cf0079 mem: avoid memzone/mempool/ring name truncation
>
> These relations should be observed:
>
> 1. Each ring creates a memzone with a prefixed name:
> RTE_RING_NAMESIZE <= RTE_MEMZONE_NAMESIZE - strlen(RTE_RING_MZ_PREFIX)
>
> 2. There are some mempool handlers which create a ring with a prefixed
> name:
> RTE_MEMPOOL_NAMESIZE <= RTE_RING_NAMESIZE - strlen(RTE_MEMPOOL_MZ_PREFIX)
>
> 3. A mempool can create up to RTE_MAX_MEMZONE pre and postfixed memzones:
> sprintf(postfix, "_%d", RTE_MAX_MEMZONE)
> RTE_MEMPOOL_NAMESIZE <= RTE_MEMZONE_NAMESIZE -
> strlen(RTE_MEMPOOL_MZ_PREFIX) - strlen(postfix)
>
> Setting all of them to 32 hides this restriction from the application.
> This patch decreases the mempool and ring string size to accommodate for
> these prefixes, but it doesn't apply the 3rd constraint. Applications
> relying on these constants need to be recompiled, otherwise they'll run
> into ENAMETOOLONG issues.
> The size of the arrays are kept 32 for ABI compatibility, it can be
> decreased next time the ABI changes.
>
> Signed-off-by: Zoltan Kiss <zoltan.kiss@schaman.hu>
Looks like to be a good compromise for the 16.07 release. One question
however: why not taking in account the 3rd constraint? Because it may
not completly fix the issue if the mempool is fragmented.
We could define RTE_MEMPOOL_NAMESIZE to 24
= 32 - len('mp_') - len('_0123'))
Thanks,
Olivier
On 21/07/16 14:40, Olivier Matz wrote:
> Hi Zoltan,
>
>
> On 07/20/2016 07:16 PM, Zoltan Kiss wrote:
>> A recent patch brought up an issue about the size of the 'name' fields:
>>
>> 85cf0079 mem: avoid memzone/mempool/ring name truncation
>>
>> These relations should be observed:
>>
>> 1. Each ring creates a memzone with a prefixed name:
>> RTE_RING_NAMESIZE <= RTE_MEMZONE_NAMESIZE - strlen(RTE_RING_MZ_PREFIX)
>>
>> 2. There are some mempool handlers which create a ring with a prefixed
>> name:
>> RTE_MEMPOOL_NAMESIZE <= RTE_RING_NAMESIZE - strlen(RTE_MEMPOOL_MZ_PREFIX)
>>
>> 3. A mempool can create up to RTE_MAX_MEMZONE pre and postfixed memzones:
>> sprintf(postfix, "_%d", RTE_MAX_MEMZONE)
>> RTE_MEMPOOL_NAMESIZE <= RTE_MEMZONE_NAMESIZE -
>> strlen(RTE_MEMPOOL_MZ_PREFIX) - strlen(postfix)
>>
>> Setting all of them to 32 hides this restriction from the application.
>> This patch decreases the mempool and ring string size to accommodate for
>> these prefixes, but it doesn't apply the 3rd constraint. Applications
>> relying on these constants need to be recompiled, otherwise they'll run
>> into ENAMETOOLONG issues.
>> The size of the arrays are kept 32 for ABI compatibility, it can be
>> decreased next time the ABI changes.
>>
>> Signed-off-by: Zoltan Kiss <zoltan.kiss@schaman.hu>
>
> Looks like to be a good compromise for the 16.07 release. One question
> however: why not taking in account the 3rd constraint? Because it may
> not completly fix the issue if the mempool is fragmented.
>
> We could define RTE_MEMPOOL_NAMESIZE to 24
> = 32 - len('mp_') - len('_0123'))
I was trying to figure out a compile time macro for strlen(postfix), but
I could not. Your suggestion would work only until someone increases
RTE_MAX_MEMZONE above 9999. As the likelihood of fragmenting a pool over
99 memzones seems small, I did not bother to fix this with an ugly hack,
but if you think we should include it, let me know!
>
> Thanks,
> Olivier
>
On 07/21/2016 03:47 PM, Zoltan Kiss wrote:
>
>
> On 21/07/16 14:40, Olivier Matz wrote:
>> Hi Zoltan,
>>
>>
>> On 07/20/2016 07:16 PM, Zoltan Kiss wrote:
>>> A recent patch brought up an issue about the size of the 'name' fields:
>>>
>>> 85cf0079 mem: avoid memzone/mempool/ring name truncation
>>>
>>> These relations should be observed:
>>>
>>> 1. Each ring creates a memzone with a prefixed name:
>>> RTE_RING_NAMESIZE <= RTE_MEMZONE_NAMESIZE - strlen(RTE_RING_MZ_PREFIX)
>>>
>>> 2. There are some mempool handlers which create a ring with a prefixed
>>> name:
>>> RTE_MEMPOOL_NAMESIZE <= RTE_RING_NAMESIZE -
>>> strlen(RTE_MEMPOOL_MZ_PREFIX)
>>>
>>> 3. A mempool can create up to RTE_MAX_MEMZONE pre and postfixed
>>> memzones:
>>> sprintf(postfix, "_%d", RTE_MAX_MEMZONE)
>>> RTE_MEMPOOL_NAMESIZE <= RTE_MEMZONE_NAMESIZE -
>>> strlen(RTE_MEMPOOL_MZ_PREFIX) - strlen(postfix)
>>>
>>> Setting all of them to 32 hides this restriction from the application.
>>> This patch decreases the mempool and ring string size to accommodate for
>>> these prefixes, but it doesn't apply the 3rd constraint. Applications
>>> relying on these constants need to be recompiled, otherwise they'll run
>>> into ENAMETOOLONG issues.
>>> The size of the arrays are kept 32 for ABI compatibility, it can be
>>> decreased next time the ABI changes.
>>>
>>> Signed-off-by: Zoltan Kiss <zoltan.kiss@schaman.hu>
>>
>> Looks like to be a good compromise for the 16.07 release. One question
>> however: why not taking in account the 3rd constraint? Because it may
>> not completly fix the issue if the mempool is fragmented.
>>
>> We could define RTE_MEMPOOL_NAMESIZE to 24
>> = 32 - len('mp_') - len('_0123'))
>
> I was trying to figure out a compile time macro for strlen(postfix), but
> I could not. Your suggestion would work only until someone increases
> RTE_MAX_MEMZONE above 9999. As the likelihood of fragmenting a pool over
> 99 memzones seems small, I did not bother to fix this with an ugly hack,
> but if you think we should include it, let me know!
Ok, looks fair, thanks for the clarification.
Acked-by: Olivier Matz <olivier.matz@6wind.com>
2016-07-21 16:25, Olivier Matz:
> On 07/21/2016 03:47 PM, Zoltan Kiss wrote:
> > On 21/07/16 14:40, Olivier Matz wrote:
> >> On 07/20/2016 07:16 PM, Zoltan Kiss wrote:
> >>> A recent patch brought up an issue about the size of the 'name' fields:
> >>>
> >>> 85cf0079 mem: avoid memzone/mempool/ring name truncation
> >>>
> >>> These relations should be observed:
> >>>
> >>> 1. Each ring creates a memzone with a prefixed name:
> >>> RTE_RING_NAMESIZE <= RTE_MEMZONE_NAMESIZE - strlen(RTE_RING_MZ_PREFIX)
> >>>
> >>> 2. There are some mempool handlers which create a ring with a prefixed
> >>> name:
> >>> RTE_MEMPOOL_NAMESIZE <= RTE_RING_NAMESIZE -
> >>> strlen(RTE_MEMPOOL_MZ_PREFIX)
> >>>
> >>> 3. A mempool can create up to RTE_MAX_MEMZONE pre and postfixed
> >>> memzones:
> >>> sprintf(postfix, "_%d", RTE_MAX_MEMZONE)
> >>> RTE_MEMPOOL_NAMESIZE <= RTE_MEMZONE_NAMESIZE -
> >>> strlen(RTE_MEMPOOL_MZ_PREFIX) - strlen(postfix)
> >>>
> >>> Setting all of them to 32 hides this restriction from the application.
> >>> This patch decreases the mempool and ring string size to accommodate for
> >>> these prefixes, but it doesn't apply the 3rd constraint. Applications
> >>> relying on these constants need to be recompiled, otherwise they'll run
> >>> into ENAMETOOLONG issues.
> >>> The size of the arrays are kept 32 for ABI compatibility, it can be
> >>> decreased next time the ABI changes.
> >>>
> >>> Signed-off-by: Zoltan Kiss <zoltan.kiss@schaman.hu>
> >>
> >> Looks like to be a good compromise for the 16.07 release. One question
> >> however: why not taking in account the 3rd constraint? Because it may
> >> not completly fix the issue if the mempool is fragmented.
> >>
> >> We could define RTE_MEMPOOL_NAMESIZE to 24
> >> = 32 - len('mp_') - len('_0123'))
> >
> > I was trying to figure out a compile time macro for strlen(postfix), but
> > I could not. Your suggestion would work only until someone increases
> > RTE_MAX_MEMZONE above 9999. As the likelihood of fragmenting a pool over
> > 99 memzones seems small, I did not bother to fix this with an ugly hack,
> > but if you think we should include it, let me know!
>
> Ok, looks fair, thanks for the clarification.
>
> Acked-by: Olivier Matz <olivier.matz@6wind.com>
Applied, thanks
@@ -123,7 +123,9 @@ struct rte_mempool_objsz {
/**< Total size of an object (header + elt + trailer). */
};
-#define RTE_MEMPOOL_NAMESIZE 32 /**< Maximum length of a memory pool. */
+/**< Maximum length of a memory pool's name. */
+#define RTE_MEMPOOL_NAMESIZE (RTE_RING_NAMESIZE - \
+ sizeof(RTE_MEMPOOL_MZ_PREFIX) + 1)
#define RTE_MEMPOOL_MZ_PREFIX "MP_"
/* "MP_<name>" */
@@ -208,7 +210,12 @@ struct rte_mempool_memhdr {
* The RTE mempool structure.
*/
struct rte_mempool {
- char name[RTE_MEMPOOL_NAMESIZE]; /**< Name of mempool. */
+ /*
+ * Note: this field kept the RTE_MEMZONE_NAMESIZE size due to ABI
+ * compatibility requirements, it could be changed to
+ * RTE_MEMPOOL_NAMESIZE next time the ABI changes
+ */
+ char name[RTE_MEMZONE_NAMESIZE]; /**< Name of mempool. */
union {
void *pool_data; /**< Ring or pool to store objects. */
uint64_t pool_id; /**< External mempool identifier. */
@@ -100,6 +100,7 @@ extern "C" {
#include <rte_lcore.h>
#include <rte_atomic.h>
#include <rte_branch_prediction.h>
+#include <rte_memzone.h>
#define RTE_TAILQ_RING_NAME "RTE_RING"
@@ -126,8 +127,10 @@ struct rte_ring_debug_stats {
} __rte_cache_aligned;
#endif
-#define RTE_RING_NAMESIZE 32 /**< The maximum length of a ring name. */
#define RTE_RING_MZ_PREFIX "RG_"
+/**< The maximum length of a ring name. */
+#define RTE_RING_NAMESIZE (RTE_MEMZONE_NAMESIZE - \
+ sizeof(RTE_RING_MZ_PREFIX) + 1)
#ifndef RTE_RING_PAUSE_REP_COUNT
#define RTE_RING_PAUSE_REP_COUNT 0 /**< Yield after pause num of times, no yield
@@ -147,7 +150,12 @@ struct rte_memzone; /* forward declaration, so as not to require memzone.h */
* a problem.
*/
struct rte_ring {
- char name[RTE_RING_NAMESIZE]; /**< Name of the ring. */
+ /*
+ * Note: this field kept the RTE_MEMZONE_NAMESIZE size due to ABI
+ * compatibility requirements, it could be changed to RTE_RING_NAMESIZE
+ * next time the ABI changes
+ */
+ char name[RTE_MEMZONE_NAMESIZE]; /**< Name of the ring. */
int flags; /**< Flags supplied at creation. */
const struct rte_memzone *memzone;
/**< Memzone, if any, containing the rte_ring */