DPDK Summit 2025 Registration and CFP are Open!
Skip to main content
All Posts By

benthomas

DPDK Dispatch Jan 2025

By Monthly Newsletter

1. Main Announcements

2. Blogs, User Stories and Developer Spotlights

3. DPDK & Technologies in the news:

4. Performance Reports & Meeting Minutes

This newsletter is sent out to thousands of DPDK developers, it’s a collaborative effort. If you have a project release, pull request, community event, and/or relevant article you would like to be considered as a highlight for next month, please reply to marketing@dpdk.org

Thank you for your continued support.

Grout: A Community-Driven Blueprint for High-Performance Networking with DPDK

By User Stories

Introduction: 

Grout is an emerging open source project hosted under the DPDK community that aims to simplify high-performance networking and provide a reference application for building applications using DPDK. Spearheaded by Robin Jarry from Red Hat, the project was born out of the need for a vendor-neutral, practical tool to validate server setups optimized for DPDK performance.

This user story dives into the origins, challenges, and aspirations of the Grout project.

Building Grout to Simplify High-Performance Networking

Robin Jarry, a software engineer at Red Hat and member of the OpenStack NFV Integration team, has been at the forefront of tackling the intricate challenges of deploying high-performance networking solutions. Based in France, Robin works primarily with telco customers, focusing on tasks like power tuning, CPU isolation, and configuring servers to maximize the performance of DPDK.

While these optimizations are critical for achieving the desired results, validating the setups proved to be a recurring issue in his work. The tools available, such as testpmd, were limited in scope. Although testpmd is a valuable application for testing DPDK drivers, it lacks networking functionality and cannot fully verify the correctness of server configurations. 

Recognizing this gap, Robin and his colleagues decided to create a solution that could act as a vendor-neutral application for validating DPDK-based server setups. This effort gave rise to Grout, a project designed to simplify and standardize the use of DPDK in high-performance environments.

Like any new project, the early stages of Grout were challenging, with foundational work requiring significant effort. However, once the initial framework was laid out, the team found it increasingly easy to add new features and protocols. Their hard work culminated in a presentation Grout # a Graph Router Based on DPDK at the DPDK Summit North America 2024 event, where Robin and key contributor David Marchand received overwhelmingly positive feedback.

“I was thrilled by how well Grout was received, the feedback during and after the talk confirmed that we’re solving a real problem, and it’s exciting to see the community’s enthusiasm for this project.” 

Robin credits the project’s success not only to his efforts but also to the collaboration with key contributors, including David Marchand, a long-time DPDK maintainer, and Christophe Fontaine, a colleague at Red Hat.

Christophe focuses on OpenStack engineering for Telco use cases, leading the design of Network Functions Virtualization (NFV) within the OpenStack team to optimize high-performance connectivity for Virtual Network Functions (VNFs) and Cloud-Native Functions (CNFs). Before joining Red Hat, he worked at Qosmos (ENEA), where he contributed to FD.io and OpenDaylight, enhancing dynamic service function chaining for OpenStack on x86 and ARM servers.

Additionally, Christophe has provided detailed documentation, engaged with the community, and shared insights on performance tuning and Grout integration at DPDK summits, making advanced features more accessible to both new and experienced users.

Together, they are pushing Grout forward as a reference application for DPDK, designed to simplify its adoption and showcase best practices for achieving high performance in networking workloads.

A Journey into Open Source: From Sound Waves to Network Packets

Robin’s path to becoming a key contributor in the DPDK community is anything but conventional. Before writing code and configuring high-performance servers, Robin was immersed in a completely different world: the music industry. Working as a sound engineer in Brussels, Robin honed his technical skills in an environment that demanded precision and creativity. However, financial challenges forced him to make a career change—a decision that ultimately led him to software engineering and the open source world.

“I had to start from scratch,” Robin shared. “I went to night school, earned my engineering degree, and began exploring what I could do in the world of technology.” 

The Connection Between Music and Code

Interestingly, Robin sees parallels between his past career as a sound engineer and his work as a software engineer. Both disciplines require problem-solving, creativity, and a focus on building something that works seamlessly. “There’s a certain creative energy in both fields. Coding feels like crafting something, almost like creating music, where every piece fits together to form a complete whole.”

“There’s a certain creative energy in both fields. Coding feels like crafting something, almost like creating music, where every piece fits together to form a complete whole.”

Robin’s unique perspective isn’t uncommon in the tech world. A recent poll by another Linux Foundation project revealed that nearly 38% of developers are also musicians—a convergence that highlights the shared skills of discipline, creativity, and focus. For Robin, this duality reinforces the idea that diverse backgrounds can bring fresh perspectives to technical challenges.

A recent poll by another Linux Foundation project revealed that nearly 38% of developers are also musicians—a convergence that highlights the shared skills of discipline, creativity, and focus.

Discovering Open Source and the DPDK Community

Robin’s first exposure to open source came from an unexpected place: side projects in video game development. While exploring these projects, he became captivated by the ethos of open source. “What struck me was how these developers were building software for free, just to help others and share knowledge. I wanted to understand their mindset, and it resonated deeply with me,” Robin explained.

“What struck me was how these developers were building software for free, just to help others and share knowledge. I wanted to understand their mindset, and it resonated deeply with me,” 

This philosophy of collaboration and giving back to the community would become a cornerstone of Robin’s career. His professional journey into open source began in earnest when he joined 6WIND, the company responsible for establishing the open source DPDK community in 2013. Here, he worked on projects that blended his growing technical expertise with his newfound passion for open source development.

Over the years, Robin’s involvement with DPDK became both a professional responsibility and a personal conviction. 

“I don’t just do open source work because it’s part of my job—I genuinely believe in the values of open source, it fits with my character. I enjoy the process of contributing, maintaining projects, and seeing the positive impact on the community.” 

Giving Back to the Open Source Community

Robin’s commitment to open source extends beyond DPDK. In his spare time, he maintains personal projects and contributes to other open-source initiatives. He describes this as a “virtuous cycle” of giving and receiving energy: “People are often grateful when you contribute something useful, and that gratitude motivates you to keep going. It’s a positive feedback loop.”

This passion for open source is also evident in Grout, where Robin and his team have made contributions not just to their own project, but back to the DPDK ecosystem. By identifying and fixing issues within DPDK’s core libraries, they’re embodying the principles of open source collaboration.

A Welcoming Community

Reflecting on his experience with DPDK, Robin praised the community’s inclusiveness and support for newcomers. “When I started contributing to DPDK about 10 years ago, I found it much more welcoming compared to other projects, which have a steeper entry barrier,” he said. “The DPDK community is patient, helpful, and understanding—even with people who are just getting started.”

 “The DPDK community is patient, helpful, and understanding—even with people who are just getting started.”

This collaborative spirit has been a key factor in Robin’s long-term involvement with DPDK, as well as his enthusiasm for helping others onboard into the ecosystem.

Grout at the Crossroads: A Pivotal Shift for DPDK

As the DPDK project evolves, it finds itself in a transitional phase—moving from years of foundational development into a focus on real-world applications and use cases. Grout, a new addition to the DPDK ecosystem, is emerging as a key project in this transformation. 

The origins of Grout lie in the complexity of getting started with DPDK. While DPDK has been a pioneering project for high-performance networking, its setup process can be daunting. “You have too many things to configure,” Robin explained, “from enabling VFIO and allocating huge pages to binding devices to the right drivers. It’s complex, and people have told us that this is one of the biggest hurdles.”

“You have too many things to configure, from enabling VFIO and allocating huge pages to binding devices to the right drivers. It’s complex, and people have told us that this is one of the biggest hurdles.”

Grout is designed to simplify this experience. It aims to serve as a reference example—a guide to using DPDK libraries in a clean, best-practice-driven way. This includes demonstrating how to configure networking for high performance or even integrate AI applications using DPDK.

Robin highlighted that the DPDK community is keenly aware of this need. “We’ve received a lot of feedback asking for either best practices or an example of how things are intended to be used. Grout could become that example—a model of how DPDK can be leveraged effectively and efficiently.”

Laying the Foundation with the Graph Library

Grout isn’t starting from scratch. One of its core building blocks is the DPDK Graph Library, which provides a structured approach for building applications. According to Robin, the Graph Library already delivers roughly “a third of the structure needed to build a DPDK application.”

This modularity makes Grout not only a great starting point but also a potential framework for future DPDK projects. By leveraging existing tools and adding layers of usability, Grout is poised to streamline development for other contributors and users.

“Part of this foundation is already in DPDK itself,” Robin said, emphasizing that Grout’s architecture is tightly integrated with existing libraries, making it an ideal reference point for those entering the ecosystem.”

Moving Toward Applications and Use Cases

The feedback from the DPDK community underscores the importance of Grout’s focus on usability and practical application. As DPDK transitions from being a toolkit to supporting more application-level use cases, projects like Grout play an essential role in bridging the gap.

This shift is especially relevant as the networking world increasingly embraces cloud-native technologies. Grout’s primary use case targets container-native and cloud-native network functions (CNFs), making it a valuable asset for telcos and enterprises deploying high-performance networking in OpenStack, Kubernetes, and OpenShift environments.

“Grout isn’t a replacement for other tools like OVS, but it’s designed to run in VMs or containers, rather than on a host server. Its focus is entirely different—Grout is a router, not a virtual switch.”

A Catalyst for the Future of DPDK

Grout is still in its early stages, but its potential is already clear. By addressing pain points in the DPDK ecosystem and providing a practical, scalable reference architecture, it paves the way for easier adoption and innovation. Whether it’s serving as a blueprint for building high-performance CNFs or simplifying the onboarding process for new contributors, Grout is an important part of the next chapter of DPDK.

With its foundations rooted in the community and a focus on usability and real-world applications, Grout exemplifies the values of open source development. As Robin aptly put it, 

“Grout is here to simplify, to guide, and to inspire what’s possible with DPDK.”

Target Use Case

Grout’s target use case is clear: cloud-native network functions (CNFs) and container-native network functions. These applications are vital for telcos and high-performance networking environments but represent a niche within the broader world of cloud computing.

Robin contrasted this with the dominant web application model in the cloud. “A web service won’t really be affected by Grout,” he noted. “But for telco applications or high-performance networking, where the requirements are far more specialized, Grout provides the extra care needed.”

While this niche focus may seem limiting at first glance, it’s precisely this specialization that makes Grout so valuable. By addressing the unique challenges of high-performance networking, Grout fills a critical gap in the DPDK ecosystem.

“A web service won’t really be affected by Grout,” he noted. “But for telco applications or high-performance networking, where the requirements are far more specialized, Grout provides the extra care needed.”

The Future of Grout: Usability, Automation, and Niche Applications

As Grout evolves, its focus remains on bridging gaps in the DPDK ecosystem and addressing specific use cases that demand high-performance networking. While not directly tied to AI or machine learning, Grout contributes to the infrastructure that supports automation and efficiency, helping simplify workflows and creating a foundation for innovation.

For instance, Grout could play a role in automating aspects of testing or continuous integration (CI) processes in environments requiring high-performance networking. By providing a standardized reference for DPDK applications, it could ensure seamless interactions between hardware and software, laying the groundwork for advanced automation tasks.

Impact on High-Performance Applications

In the short term, Grout’s primary contribution is its role as a reference for building containerized, high-performance network functions. Long-term, its automation and usability features could influence broader workflows, including testing, CI pipelines, and operational efficiency in telco and enterprise environments.

By simplifying the complexities of DPDK setups and offering clear examples of best practices, Grout is poised to make high-performance networking more accessible—not just for seasoned developers, but for newcomers to the ecosystem as well.

The Road Ahead: Plans for Usability Improvements

Having reached its minimum viable product (MVP) stage, Grout’s next steps focus on usability. Robin outlined several key areas for improvement, emphasizing the need to simplify tasks that currently require manual intervention.

One prominent feature on the roadmap is the automatic assignment of port RX queues to CPUs. “Right now, this is something you need to configure manually, but automating this will make the setup process much easier,” Robin said. This improvement would make Grout more accessible to users deploying DPDK applications in cloud environments, whether in OpenStack, OpenShift, or Kubernetes.

Another goal is to refine Grout’s role as a reference application—a good example of how DPDK libraries and tools should be used. By demonstrating best practices, Grout aims to set a standard for developers, helping them build applications without grappling with unnecessary complexity.

Evolving Organically

As for the future, Robin believes that Grout will evolve based on community needs and feedback. “Maybe it will serve as an example, as a reference architecture,” he said. “I don’t want to overstate its impact—it’s going to evolve organically based on how the community uses it.”

“Maybe it will serve as an example, as a reference architecture,” he said. “I don’t want to overstate its impact—it’s going to evolve organically based on how the community uses it.”

This flexible approach reflects the open source mindset at the heart of Grout’s development. Rather than dictating its direction, Robin and his team are focused on providing a solid foundation and letting the project grow in response to real-world use cases.

A Tool Built for Collaboration

As Grout continues to evolve, it stands as a testament to the power of open source collaboration. Whether serving as a blueprint for DPDK applications, simplifying cloud-native networking setups, or enabling automation in niche use cases, Grout demonstrates how focused, community-driven efforts can lead to impactful tools that solve real problems.

In Robin’s words, “Grout isn’t trying to be everything, but by focusing on what it does best—providing clarity and structure—it has the potential to make a lasting difference in the DPDK ecosystem.”

“Grout isn’t trying to be everything, but by focusing on what it does best—providing clarity and structure—it has the potential to make a lasting difference in the DPDK ecosystem.”

Red Hat’s Strategy: Leveraging DPDK and Grout

At Red Hat, DPDK plays a critical role, particularly in conjunction with Open vSwitch (OVS). However, Grout occupies a distinct niche in this ecosystem. “OVS runs on the host, while Grout is designed to run in VMs or containers, It’s not meant to replace OVS; it’s a router, not a virtual switch. The scopes are entirely different.”

Grout aligns with Red Hat’s strategy to develop examples of virtual network functions (VNFs) and cloud-native network functions (CNFs) that demonstrate DPDK’s capabilities. While Grout is still in its early stages, its development reflects an experimental approach: build something, see if it gains traction, and adapt based on feedback.

This iterative mindset mirrors the ethos of open source—collaborate, experiment, and refine based on community needs.

Cross-Project Collaboration: Building Bridges Across Ecosystems

The DPDK community has long emphasized the importance of integrating its technology with other open-source projects, and Grout exemplifies this philosophy.

Robin highlighted Kubernetes as a natural integration point for DPDK, given the growing adoption of containerized environments in networking. “Kubernetes is an obvious pathway for collaboration, especially for projects like Grout that target container-native applications,” he noted.

“Kubernetes is an obvious pathway for collaboration, especially for projects like Grout that target container-native applications”.

Additionally, Grout has facilitated a reverse flow of contributions—where challenges encountered during Grout’s development led to improvements in the DPDK core itself. This reciprocal relationship underscores the project’s commitment to the principles of open source: contribute back to what you use.

“Instead of just consuming DPDK, we’ve raised issues, fixed them, and sent patches back to the community,” Robin said. “This approach isn’t just rewarding; it’s essential for maintaining the ecosystem’s vitality.”

Raising Awareness and Building Momentum

Looking ahead, Robin sees opportunities to raise awareness about Grout and its role in the DPDK ecosystem. Collaborating with organizations like the Linux Foundation and participating in events can help showcase the project’s potential. As Robin summed it up, “Grout is as much about helping others as it is about building something new. It’s a shared effort, and that’s what makes it special.”

“Grout is as much about helping others as it is about building something new. It’s a shared effort, and that’s what makes it special.”

To get involved, join the mailing list at grout@dpdk.org, explore the project on GitHub at https://github.com/DPDK/grout, and share your ideas for how you’d like to use Grout!

Don’t forget to catch the Grout presentation at FOSDEM 2025 on February 2.

DPDK Dispatch December

By Monthly Newsletter

1. Main Announcements

2. Blogs, User Stories and Developer Spotlights

  • Share your latest updates here
  • Start your developer spotlight here

3. DPDK & Technologies in the news:

4. Performance Reports & Meeting Minutes

This newsletter is sent out to thousands of DPDK developers, it’s a collaborative effort. If you have a project release, pull request, community event, and/or relevant article you would like to be considered as a highlight for next month, please reply to marketing@dpdk.org

Thank you for your continued support and enthusiasm.

DPDK Team.

DPDK 24.11: Another Step Forward for Performance Networking

By Blog

DPDK has unveiled its latest release, DPDK 24.11, marking a significant step forward in performance, flexibility, and hardware integration for packet processing. Here’s what you need to know about the latest advancements, new features, and what they mean for developers and system integrators.


A Busy Release Cycle: By the Numbers

This release was the product of an impressive collaborative effort:

  • 1329 commits by 196 authors.
  • Changes across 2557 files, with 376,587 insertions and 177,108 deletions.
  • Contributions from a wide range of organizations, ensuring a diverse and robust codebase.

The new release introduces some API/ABI compatibility breakages, marking a clear departure from previous versions. The new ABI version (25) means that while 24.11 will be supported for three years, its successors (25.03 and 25.07) will maintain ABI compatibility with this release. This makes 24.11 an excellent foundation for long-term system integration and deployment.


Key Highlights of DPDK 24.11

The new features and improvements span various areas, from power management and IPv6 handling to enhanced cryptography and logging. Below are some of the most notable updates:

1. Performance and Resource Management

  • Lcore Variable Allocation: Improved CPU core management allows for more dynamic and efficient resource utilization.
  • Bit Set and Atomic Bit Manipulation: Streamlined bit operations enhance concurrency and reduce overhead.
  • Power Management Enhancements:
    • AMD uncore power management improves energy efficiency.
    • Per-CPU power management QoS for resume latency enables finer control of power-performance trade-offs.

2. Networking Features

  • IPv6 Address API: Simplifies the handling of IPv6 addresses in applications.
  • RSS Hash Key Generation: Automates generation of RSS keys, enhancing load balancing capabilities.
  • Ethernet Link Lanes: Improves support for high-speed Ethernet configurations.
  • Flow Table Index Action: Adds more control for flow-based operations, aiding advanced packet processing.

3. Hardware and Driver Support

New drivers and enhancements to existing ones extend DPDK’s hardware compatibility:

  • New Hardware Integrations:
    • Cisco ENIC VF, Marvell CN20K, Napatech NTN flow engine.
    • Realtek R8169 and ZTE drivers expand support for a broader range of network interfaces.
  • Enhanced Cryptography Support:
    • Symmetric Crypto SM4 and Asymmetric Crypto EdDSA bolster security capabilities.

4. Event and Logging Improvements

  • Event Device Features: Pre-scheduling and independent enqueue capabilities enhance event-driven processing.
  • Logging Rework: Revamped logging adds timestamps, color-coded outputs, syslog, and journal support, improving debugging and operational insights.

Welcoming New Contributors

The DPDK community grew stronger with 50 new contributors, spanning roles such as authors, reviewers, and testers. This diversity reflects the open source ethos and ensures continuous innovation.

Key contributors include individuals from Intel, Marvell, NVIDIA, Red Hat, and many others, with Intel leading in the number of commits.

A special thanks to the top reviewers who played a vital role in maintaining code quality and collaboration. Their efforts highlight the often-underappreciated task of reviewing contributions.


Looking Ahead: What’s Next?

The next release, DPDK 25.03, is slated for March 2025. Development for this version is already underway, with feature submissions open through December. This cadence ensures a predictable and collaborative development process.

For those planning deployments, DPDK 24.11’s three-year support period makes it the recommended version for stability and long-term integration.

Download DPDK 24.11 here

Elevating Network Security Performance: Suricata’s Integration with DPDK

By User Stories

Introduction

The demand for high-performance network security solutions is at an all-time high, as organizations constantly seek faster and more efficient ways to handle traffic, detect threats, and ensure real-time response capabilities. 

Suricata as an open-source high-performance network security engine has long been at the forefront of these efforts. Network security professionals appreciate Suricata for its capabilities to act as an IDS (Intrusion Detection System), IPS (Intrusion Prevention System), and as an NSM (Network Security Monitoring) system.

But it’s the integration of the Data Plane Development Kit (DPDK) into Suricata that has allowed it to reach unprecedented performance levels, providing a vital boost for packet processing at high speeds. 

This story explores the journey of Suricata’s DPDK integration, the technical challenges and solutions, and the ongoing impact on Suricata’s functionality and performance.

Origins of Suricata: A Security Solution with Community at Its Core

In 2008, a group of security-focused professionals came together with a vision to improve open-source network security. 

Victor Julien, who was working as a contractor in the network security field, joined forces with Matt Jonkman, who led an early threat intelligence project (known as Emerging Threats), and Will Metcalf, who was involved in developing an inline version of Snort—a popular intrusion detection and prevention system (IDS/IPS). 

Their collaborative work in network security sparked the idea to create something new that would address gaps in existing solutions.

The journey truly began when Victor experimented with code on his own in 2007, without expecting much traction. However, after meeting Matt and Will at a conference in the U.S. and sharing his prototype with them, the project gained momentum. 

By 2008, they secured initial seed funding from the Department of Homeland Security (DHS), allowing them to pursue their vision formally. This funding was instrumental in establishing the Open Information Security Foundation (OISF), a nonprofit entity designed to ensure that the project would remain community-oriented and free from corporate control.

From the start, they were committed to making Suricata an open-source, community-driven project. With the OSF foundation’s setup, they chose the GPLv2 license, reflecting their belief in open collaboration and safeguarding the project from being absorbed by larger corporations. DHS funding, while crucial, was temporary, so they developed a sustainable model that allowed vendors to join OSF as members, offering a more flexible licensing option.

This foundational approach set the stage for what has now been a 15-year journey of innovation and collaboration in the network security field.

“We wanted to establish an organization that would make Suricata safe from acquisition, which we’d seen happen to other open-source projects at the time.” 

– Victor Julien – Suricata IDS/IPS Lead Developer

Since then, Suricata has gained adoption from large enterprises, including AWS, which integrates Suricata in its network firewall services.

A Perfect Fit: The Role of DPDK in Suricata’s Development

With increasing demand for high-performance network security tools, Suricata’s team saw an opportunity to leverage DPDK. DPDK provides a set of libraries and drivers for fast packet processing, bypassing traditional kernel limitations. 

This high-performance potential caught the attention of users and developers alike, many of whom were eager to see DPDK integration in Suricata. Lukas Sismis, a contributor who led Suricata’s DPDK integration, explained that several teams had previously worked on integrating DPDK with Suricata. 

However, most of these efforts were specific to unique use cases and lacked general applicability, which is why they hadn’t been contributed back to the Suricata codebase.

Lukas initially engaged with Suricata’s architecture through a master’s thesis, where his primary goal was to expand Suricata’s packet capture capabilities using DPDK. He explains, “Suricata’s architecture, with its separate capture logic, made it easy to add a new capture method.” 

His work, later incorporated into Suricata’s main codebase, helped create a general-purpose DPDK integration, ensuring Suricata’s compatibility with multiple DPDK-supported network interface cards (NICs) and enabling seamless configuration.

“Suricata’s architecture, with its separate capture logic, made it easy to add a new capture method.”

 – Lukas Sismis, Software Engineer at Suricata & Cesnet

Suricata’s Architectural Evolution and DPDK Integration

Suricata’s multi-threaded, modular design made it an ideal candidate for integration with DPDK. Suricata supports packet-capturing methods through its modular “capture interface,” which allows users to swap out packet capture techniques. 

DPDK, as an input method, fits naturally within this design and supports Suricata’s scalability goal: Suricata aims to run effectively across small, low-power deployments to high-speed data centers.

Lukas’ integration efforts involved setting up DPDK within Suricata as an alternative capture method, making it possible to directly interface with high-speed NICs while bypassing kernel overhead. Some of the major steps in this integration included:

  • Creating a New Capture Method: Lukas established DPDK as a new capture method within Suricata’s architecture, mapping configuration options for different NICs.
  • Testing Different NICs: Through this process, Lukas tested various NICs supported by DPDK, noting disparities in how each handled DPDK configurations.
  • Traffic Distribution Strategies: To manage packet distribution effectively, Lukas leveraged DPDK to configure hash-based packet distribution, efficiently balancing traffic load across CPU cores.

While most initial optimizations focused on enabling basic packet capture, this work laid the foundation for further enhancements. Testing showed a notable 10-15% performance gain, an exciting outcome that validated the decision to integrate DPDK as a core feature of Suricata’s capture options.

Tackling Hardware Offloading for Enhanced Performance

Beyond standard packet capture, the Suricata team recognized a significant opportunity in DPDK’s hardware offloading capabilities. Suricata’s high-speed packet processing can greatly benefit from the offloading of repetitive tasks to hardware, potentially bypassing certain types of network traffic. 

Lukas and his team began exploring offload capabilities that would allow Suricata to selectively filter traffic in hardware.

The primary focus of Suricata’s hardware offloading research has been on:

  • Flow Bypass: Allowing Suricata to ignore certain flows after initial inspection, reducing the processing load on uninteresting traffic.
  • Packet Filter: Discarding unwanted traffic from the start helps Suricata use more resources on the important part.
  • Decapsulation and Encapsulation Offloads: Offloading these operations can reduce overhead in packet analysis, freeing up CPU resources for other tasks.

Although full offload implementation is still underway, initial testing shows promising potential. DPDK’s RegEx accelerator API, supported by NVIDIA BlueField and Marvell NICs, is an example of hardware that could handle pattern-matching offloads. This ongoing work has been presented at Suricon 2024.

Since Suricata’s detection engine performs extensive pattern matching, a hardware-based solution could significantly reduce CPU load.

Challenges and Technical Hurdles in DPDK Integration

Lukas encountered several challenges while working with DPDK, primarily related to hardware compatibility and traffic distribution. While DPDK offers a standardized API, not all NICs perform identically, which led to variations in performance during testing. 

One challenge was to cover and unify the different configurations of the load balancing hash function (RSS) in the NICs. This required NIC-specific experimentation and testing with different configuration mechanisms.

Lukas also had to modify Suricata’s configuration parsing to map settings to DPDK-compatible options, ensuring a more user-friendly experience. 

This testing phase highlighted the need for adaptable configurations to support a wide range of DPDK-enabled hardware.

Despite these challenges, Lukas’ integration work has laid a strong foundation for Suricata’s use of DPDK, making Suricata more adaptable to high-performance environments.

Leveraging Community and Industry Feedback

Suricata’s community engagement plays a vital role in its development. Lukas worked closely with the CESNET team, a network research institution with deep experience in DPDK. 

This collaboration allowed him to troubleshoot issues in real time without relying solely on online forums. In addition, Victor and Lukas sought feedback from DPDK maintainers like Thomas Monjalon and David Marchand, whose insights were invaluable in refining Suricata’s integration.

Suricata’s developers also participate in community channels, including a Discourse forum, Redmine, and a Discord server. While direct communication with the DPDK team has been limited, Suricata’s community-driven model allows users to share feedback directly with developers, accelerating improvements and ensuring the tool meets evolving needs.

Real-World Impact: Enhanced Packet Processing for Modern Network Demands

DPDK’s integration has brought measurable performance gains to Suricata, providing faster packet processing for users. Major security vendors are already leveraging Suricata with the DPDK integration in their products, attesting to its reliability and scalability.

DPDK’s impact is particularly evident in high-speed environments where packet capture bottlenecks could otherwise lead to packet drops or latency. The integration allows Suricata to handle higher packet rates efficiently, extending its utility in demanding, real-time network security use cases.

Looking Forward: New Horizons with AI and Machine Learning

As artificial intelligence and machine learning applications expand across technology sectors, Suricata’s team remains open to exploring AI-driven enhancements. 

Victor explained that AI’s most promising role would likely be in post-processing. Suricata currently exports JSON-formatted data, which can be fed into AI models for insights beyond immediate packet inspection. 

Many current machine learning models operate at a macro level, analyzing data patterns over time rather than in real time, which aligns well with Suricata’s current functionality as a data generator for other analytics tools.

Real-time AI inference for packet processing, however, remains a challenge. Victor elaborated, “Most AI models require milliseconds for inference, which is too slow for packet-level detection in real-time.” Still, the team is ready to adopt AI models once hardware advances make real-time AI feasible.

“Most AI models require milliseconds for inference, which is too slow for packet-level detection in real-time.”

– Victor Julien – Suricata IDS/IPS Lead Developer

Future Development: Suricata as a Library for Broader Integration

A major long-term goal for Suricata is to establish a core API, effectively transforming Suricata’s detection engine into a library that other tools can leverage. 

This approach could enable seamless integration of Suricata’s capabilities with other applications, such as proxy servers, endpoint security products, and cloud-based services. 

While the foundational work for this API exists, achieving a fully developed API will take time. Victor noted that this goal, motivated by growing encryption in network traffic, could broaden Suricata’s utility in increasingly secure environments.

This library initiative would allow third-party developers to incorporate Suricata’s detection features in novel ways, creating a flexible, modular ecosystem where Suricata is part of larger, more complex security infrastructures.

Expanding Community Engagement Through Events

Suricata’s annual conference, Suricon, exemplifies the project’s community-centric approach. Suricon gathers developers, users, and industry professionals to share insights, discuss roadmap goals, and showcase new features. 

With a mix of training sessions and talks, Suricon provides a valuable opportunity for knowledge exchange and collaboration. DPDK community members have shown interest in attending future events, strengthening cross-community relationships, and fostering a shared development approach.

Suricata’s collaboration model has proven instrumental in its growth. This strong community foundation ensures that Suricata can keep pace with rapidly changing security demands.

Conclusion: Pushing Network Security Boundaries

Suricata’s integration with DPDK marks a significant milestone in its evolution, empowering it to achieve higher performance, greater adaptability, and better hardware compatibility. 

From initial testing to real-world deployments, DPDK’s impact has been transformative, enabling Suricata to meet the demands of today’s high-speed, security-focused networks. 

Through community feedback, industry collaboration, and a forward-looking approach to hardware offloading and AI, Suricata continues to redefine what’s possible in open-source network security.

As Suricata looks ahead, its development team remains committed to innovation and community-driven progress. With a roadmap that includes expanded hardware offloading, AI-driven enhancements, and new API integrations, Suricata is well-positioned to lead the next generation of network security solutions. 

This DPDK integration story exemplifies how open-source collaboration can drive meaningful advancements, pushing technology forward in response to real-world needs.

Learn more about contributing to DPDK here

Kamalakshitha’s Journey from Noise-Canceling Algorithms to Open Source Networking

By Community Spotlight

In the fast-paced world of high-performance networking, DPDK (Data Plane Development Kit) stands as a powerful tool, and its success is due in no small part to the dedication of its community members. 

One such contributor is Kamalakshitha, a talented developer whose journey took her from studying electronics and communications in India to making a notable impact in open source software at Arm. 

This developer spotlight explores Kamalakshitha’s journey into tech, her entry into open source, and her contributions to the DPDK project.

From Academia to First Tech Role

Kamalakshitha’s journey in tech began with an academic foundation in Electronics and Communications, where she completed an integrated Bachelor’s and Master’s program in India. This program sparked her interest in technology, laying the groundwork for her career.

After graduation, she accepted a role as a software engineer at a startup in India. This company focused on developing noise cancellation algorithms, and her work centered on researching and implementing solutions that would filter background noise, allowing only the target sound to pass through. 

This experience gave her a strong foundation in software development, algorithm design, and research-based problem-solving—skills that would later contribute to her open source career.

Master’s in Computer Engineering and Exposure to DPDK

Seeking to deepen her technical expertise, Kamalakshitha pursued a Master’s in Computer Engineering at Texas A&M University, where her interests broadened toward network performance and user-space networking.

It was during this period that she secured an internship with Arm, a leading semiconductor company. In this role, she joined Arm’s Open Source Software team, where she was introduced to DPDK and VPP (Vector Packet Processor).

Her internship with Arm marked a pivotal moment in her career. Kamalakshitha delved into the performance analysis of VPP, by collecting and analyzing PMU(Performance monitoring unit) counters to dissect code hotspots and optimize their performance

This work familiarized her with the fundamentals of user-space networking and performance optimization, opening her eyes to DPDK’s potential to improve data processing speeds in network applications by bypassing the traditional kernel-based networking stack.

Full-Time Role at Arm and Entry into Open Source

After her internship, Kamalakshitha was interested in a full-time role with Arm’s Open Source Software team. Although no positions were available in that team at the time, she secured a role in a different team, where she focused on performance analysis for networking applications. 

This role concentrated on identifying performance bottlenecks without direct code contributions. However, her desire to contribute to the codebase led her to eventually rejoin the Open Source Software team when a position became available.

This transition was significant for Kamalakshitha, as it allowed her to fulfill her aspiration of not only identifying performance bottlenecks but also addressing them through code contributions. Joining the open-source team allowed her to actively engage with the DPDK community, sharing her insights and participating in open discussions.

Key Technical Contributions to DPDK

Kamalakshitha’s contributions to DPDK have been multifaceted. Here’s a look at some of the highlights of her work:

  1. First Patch – Driver Fix
    Her initial contribution to DPDK was a small but crucial driver fix. This patch taught her about the processes involved in open-source contributions and helped her become familiar with DPDK’s mailing list and review system.
  2. Zero Copy API
    Kamalakshitha volunteered to write test cases for DPDK’s Zero Copy API after spotting its development in the community mailing list. Her proactive approach and dedication to improvement led her to create test cases that enhanced the API’s robustness.
  3. Cache-Aware Mempool Project and Blog Post
    Recognizing that users could benefit from understanding the importance of buffer and cache line sizes for performance, Kamalakshitha undertook a project to create a blog post on cache-aware memory pooling. This piece provided an in-depth look at how certain buffer sizes and cache allocations could impact DPDK performance, transforming complex technical details into accessible knowledge for the community.
  4. Multi-Packet Receive Queue (MPRQ)
    Currently, Kamalakshitha is focused on improving the performance of DPDK’s multi-packet receive queue (MPRQ) on Arm systems. This project, centered on the Mellanox NIC, involves analyzing MPRQ configurations and their impact on packet processing, demonstrating her skill in both hardware-specific optimizations and cross-platform performance improvements.

Read Kamalakshitha’s blog Cache Awareness in DPDK Mempool for a detailed understanding of how buffer size and cache awareness influence packet processing in DPDK, including practical insights into buffer allocation strategies, cache utilization and performance benchmarks. 

Engaging with the DPDK Community

Kamalakshitha’s involvement in the DPDK community extends beyond code contributions. She first engaged through DPDK’s mailing list, where she reviewed patches and learned the nuances of community contribution. 

She credits her former manager, Honnappa Nagarahalli, with encouraging her to join Arm’s open source team, which further facilitated her entry into the DPDK community.

In addition to her formal role, Kamalakshitha has found that the open source environment allows her to connect with diverse experts across different companies. This experience has not only expanded her technical expertise but has also developed her soft skills, such as presenting technical ideas, discussing optimization methods, and building consensus on improvements with a global audience.

A Methodical Approach to Programming

Kamalakshitha’s approach to programming is systematic and meticulous. She prefers to analyze code performance and identify hotspots, using tools like perf to monitor program execution. Before implementing a fix, she visualizes potential solutions and maps out her strategy on paper. 

This analog approach allows her to organize her ideas clearly, giving her a structured plan for tackling complex coding tasks. She then methodically tests her solutions, adding only incremental changes to optimize performance.

DPDK’s Future in High-Speed Networking

Kamalakshitha is particularly excited about DPDK’s future role in emerging technologies. With the expansion of 5G networks and the rise of AI-driven applications, she sees DPDK as a foundational technology enabling high-speed data processing and low-latency communication. 

Whether it’s supporting 5G’s data transfer needs or facilitating communication between distributed AI nodes, DPDK provides a versatile, high-performance toolkit. 

She also envisions DPDK as a vital component in building data-plane stacks on top of smart NICs and other accelerators, thus broadening its applications in cutting-edge technology.

Championing Diversity in Open Source

As a woman in a traditionally male-dominated field, Kamalakshitha is aware of the challenges and opportunities that come with increasing diversity in tech. 

She notes that while progress has been made, there’s still work to be done in creating an inclusive environment. She encourages women to explore open source as a platform for professional growth, as it provides unique opportunities for visibility, learning, and collaboration.

Reflecting on her experiences, Kamalakshitha emphasizes the importance of community and role models in motivating women in tech. Seeing other female engineers in open source helps create a sense of belonging and inspires more women to participate and contribute.

Work-Life Balance and Personal Interests

Balancing a demanding career with personal life is essential for Kamalakshitha, especially after recently becoming a mother. She manages her responsibilities by setting clear priorities and taking regular breaks to recharge. 

Small rituals, like preparing healthy meals or going for short walks, help her unwind. The arrival of her child has further sharpened her time management skills, as she carefully balances work and family responsibilities.

Essential Tools and Advice for New Developers

For Kamalakshitha, perf is an indispensable tool, enabling her to perform in-depth performance analysis for her projects. On a personal level, her phone is an essential device, with applications like slack keeping her connected to her team and enabling her to join meetings on the go when necessary. (join the DPDK slack channel here)

One piece of career advice that has resonated with her is the importance of understanding the basics of any project before diving in. 

She believes that a clear grasp of the fundamentals not only accelerates learning but also enables more impactful contributions. She recommends that new developers have a quick chat with mentors to clarify the broader picture before delving into details.

Final Thoughts

Kamalakshitha’s journey from noise-canceling algorithms to open-source networking at Arm illustrates the power of perseverance, curiosity, and community. 

Through her contributions to DPDK, she is helping shape the future of high-performance networking. 

Her story is an inspiration for other developers, particularly women in tech, highlighting the benefits of open-source collaboration and the exciting opportunities it offers.

As she continues her journey in DPDK, Kamalakshitha looks forward to new projects, deeper community engagement, and expanding DPDK’s role in supporting next-generation networking technologies.

Start contributing to DPDK here.

DPDK Dispatch November

By Monthly Newsletter

1. Main Announcements

  • Read the DPDK Summit overview here.

2. Blogs, User Stories and Developer Spotlights

3. DPDK & Technologies in the news:

4. Performance Reports & Meeting Minutes

This newsletter is sent out to thousands of DPDK developers, it’s a collaborative effort. If you have a project release, pull request, community event, and/or relevant article you would like to be considered as a highlight for next month, please reply to marketing@dpdk.org

Thank you for your continued support and enthusiasm.

DPDK Team.

Qian Xu’s Leadership in High-Performance Networking at NVIDIA

By Community Spotlight

Welcome to another edition of our DPDK Developer Spotlight, where we explore the careers and insights of key figures who have shaped the DPDK community. In this spotlight, we share the journey of Qian Xu, a Software Validation Manager at NVIDIA, who has been deeply involved in the DPDK from its early days.

Early Days and Introduction to Open Source  

Qian’s introduction to open source tech happened somewhat serendipitously. While working at Intel, she was assigned to the DPDK project, which Intel initiated in 2010 and released under a permissive open source license. The open source community for DPDK was established at dpdk.org in 2013 by 6WIND, and the project continued to grow until it was successfully transferred to the Linux Foundation in 2017.

In the project’s early days, Qian became fascinated by its community-driven aspects. The difference between corporate-led initiatives and the inclusive, collaborative nature of DPDK captivated her, leading her to fully engage with the community.

First Experiences in the Community

When Qian Xu first joined the DPDK community, her entry was marked by interactions that would shape her entire experience. Initially, she was primarily engaged in testing, not frequently submitting patches, which allowed her to observe the dynamics of the community from a unique vantage point. 

Her initial period with the project was guided by Thomas Monjalon, one of the original maintainers of DPDK, whose reputation was well-known among developers in Shanghai and beyond. Contrary to the daunting tales of his strictness, Qian found Thomas to be incredibly helpful and supportive, a disposition that greatly eased her integration into the community.

Thomas and the other maintainers’ approachability was crucial as Qian navigated the new environment. He was not only receptive to feedback but actively encouraged Qian to voice her concerns about continuous integration practices or any other aspects of the project that might require improvement. This open dialogue fostered a collaborative relationship, proving that the community was responsive and genuinely interested in evolving based on its members’ input.

Qian’s interactions weren’t limited to informal discussions; they extended into more structured community engagements. She actively participated in various forums, detailed email threads about release testing statuses, to the DPDK Summit APAC and other international meetings, often communicating through emails and occasionally through IRC for more immediate concerns. 

Technical Contributions

Qian Xu’s technical contributions have been vital to the growth of the DPDK community, especially in Continuous Integration (CI) processes, testing frameworks, and performance testing methodologies. Starting as a validation engineer, she played a key role in developing the DPDK Test Suite (DTS), one of her most notable achievements. The DTS has since become a cornerstone of the DPDK open source automation framework.

Development and Evolution of DTS

Under Qian’s guidance DTS marked a significant advancement in how DPDK could be tested and validated across different environments. Her leadership in this project involved designing the architecture of the test suite, ensuring it was robust and flexible enough to handle a variety of networking scenarios. This tool allowed for the automated testing of DPDK components, significantly reducing manual testing efforts and accelerating the developmental feedback loop.

Launching the First Performance Report

Qian was also instrumental in publishing the first DPDK performance report. This initiative set a precedent within the community, providing a transparent and replicable method to benchmark the performance of DPDK implementations. Her work laid the groundwork for subsequent reports, which have become crucial resources for developers seeking to understand and optimize DPDK performance.

Collaboration and Lab Development

Beyond software development, Qian played a pivotal role in establishing a physical testing lab for DPDK. This lab, set up in collaboration with the University of New Hampshire, provided a shared resource for developers around the world to conduct more sophisticated testing scenarios that could not be easily replicated locally. She facilitated remote debugging and testing procedures, which were critical during the integration and continuous delivery phases of the CI/CD pipeline.

DPDK into the Future

Qian envisions a dynamic and evolving future for DPDK, particularly as it integrates more deeply with AI, machine learning, 5G, and cloud computing. Over the past decade, DPDK has undergone transformative changes, significantly enhancing networking performance and becoming a fundamental component across various infrastructure domains

Integration with AI and Machine Learning 

As DPDK ventures into AI, Qian foresees the framework enhancing its capabilities to support AI operations, particularly in managing and optimizing data flow at incredible speeds. This integration aims to reduce latency in AI data processing, making real-time analytics and decision-making more efficient.

Evolving Continuous Integration (CI) Practices 

Qian also highlights the potential evolution of DPDK’s CI practices. Initially, the CI systems were less mature, but they have grown more robust over time, reducing false alarms and improving stability. With the infusion of AI into CI, Qian believes that testing processes can be further automated and refined, allowing for quicker identification and resolution of critical issues before they affect the main branches. This would involve less human intervention, making the testing process more efficient and less prone to error.

The Importance of a Diverse Community 

Qian’s experience as a woman in tech highlights the evolving diversity within the DPDK community and NVIDIA. She has witnessed a positive shift towards greater gender equality, supported by inclusive practices and the Linux Foundation’s collaborative approach. Her advice to aspiring female engineers is to engage deeply, contribute meaningfully, and leverage the supportive environment increasingly present in open source tech communities.

In essence, Qian Xu sees a positive trajectory for diversity in tech, especially in the DPDK Asia Pacific community, driven by systemic changes in the region and a community commitment to equality. Her experience underlines the importance of inclusive practices and the continuous effort needed to maintain and expand these gains within the tech industry.

Best Advice Ever Given

Qian often reflects on a piece of advice that has influenced her professional trajectory and personal growth. Throughout her career, from her early days in networking to her current role at NVIDIA, where her responsibilities are more focused on AI, Networking and DPDK integration, she has adhered to this guiding principle: focus deeply on your chosen domain. This advice, garnered from numerous interviews with industry leaders and her own mentors, encouraged her to dedicate herself fully to mastering the intricacies of her field.

The Growth of the APAC Community and Getting Involved

Qian emphasizes the importance of active participation in the DPDK community, especially in the APAC region. She encourages newcomers to engage in discussions, attend regional summits, and contribute code. The APAC region has seen increased participation, with diverse representation from countries like China, Japan, and India, contributing significantly to the global DPDK landscape.

Forging the Future at NVIDIA

Qian Xu envisions a future where DPDK (Data Plane Development Kit) continues to be a pivotal element in the evolution of networking and computational technologies, particularly as these fields intersect with AI and cloud computing. At NVIDIA, her focus on enhancing solution-level testing allows her to channel DPDK’s capabilities into broader technological advancements, even if her work no longer revolves directly around DPDK’s core components.

This is particularly relevant as NVIDIA pushes the boundaries of AI computing and network performance, areas where efficient data handling and processing are critical. 

As highlighted in NVIDIA’s recent initiatives, such as the development of AI-driven platforms and enhancements in GPU rendering, the underlying principles of high-speed data processing championed by DPDK are integral to these advancements.

By marrying DPDK’s network processing strengths with NVIDIA’s leadership in AI and GPU technology, Qian aims to cultivate a technology landscape that not only advances NVIDIA’s commercial goals but also propels the industry forward. 

Balancing Work and Life

Qian values family time and enjoys bicycling and traveling with her family to maintain a healthy work-life balance. These activities help her stay grounded and focused.

Indispensable Technology

AI and networking technologies are indispensable to Qian. She highlights AI advancements, like ChatGPT, as crucial in both her professional and personal life.

Closing Thoughts

Qian champions the contributions of female engineers in China, expressing gratitude for their increasing recognition. Her journey offers valuable lessons in cultural adaptability and proactive community engagement, inspiring both current and aspiring DPDK developers.

We hope Qian’s story motivates you to engage with the DPDK community and contribute to the future of networking and infrastructure technologies.

Get involved by joining the mailing lists here: https://www.dpdk.org/contribute/#Mailing-Lists

DPDK Dispatch October

By Monthly Newsletter

1. Main Announcements

  • Discover how DPDK transcends typical network performance software for routers and firewalls, integrating deeply into global infrastructure and powering bleeding-edge applications in particle processors and radio telescopes! Read the 2024 North America Summit highlights.
  • DPDK Summit talks are now online – catch some of the top presentations.

2. Blogs, User Stories and Developer Spotlights

  • Share your latest updates here
  • Start your developer spotlight here

3. DPDK & Technologies in the news:

4. Performance Reports & Meeting Minutes

This newsletter is sent out to thousands of DPDK developers, it’s a collaborative effort. If you have a project release, pull request, community event, and/or relevant article you would like to be considered as a highlight for next month, please reply to marketing@dpdk.org

Thank you for your continued support and enthusiasm.

DPDK Team.

Explore the latest Innovations and Insights from this year’s DPDK North America Summit

By Blog

This year’s DPDK North America Summit highlighted the projects ongoing technical excellence and innovation in high-performance networking. The event gathered experts and enthusiasts from around the globe, including project pioneers and new community contributors. They engaged in discussions and demonstrations focused on recent code developments, the technical board’s plans, and notably, exciting use cases and applications.

Over the past 14 years, DPDK has methodically developed an open stack that meets a broad variety of user requirements. The project’s open development approach and adaptability to community needs have been invaluable. They showcase the project’s commitment to open source principles. However, these practices have also led to a more expansive and less streamlined code base. Nevertheless, the technical board skillfully manages the necessary compromises for both core developers and end users, highlighting some exciting developments at the summit.

The review of new technology integrations and organic implementation alongside the community’s evolution has been impressive. The project’s development, impact, and extensive application across a global infrastructure have been significant. This includes not just data centers, enterprise cloud services, and network equipment, but also transportation networks, telecom systems, financial trading platforms, industrial control systems, and even particle processors, and astronomical data processing!

The project has reached a pivotal stage of maturity, with use cases and applications expanding dynamically. This evolution presents an opportune moment to explore and showcase real-world applications, extending far beyond its conventional roles in routers and firewalls.

One highlight of the summit was the presentation by Robin Jarry and David Marchand from Red Hat, who introduced “Grout,” a graph router based on DPDK. This tool is designed to simulate network functions and physical routers to replicate the behavior of typically closed-source VNFs and CNFs using an open source tool. They provided a detailed explanation of the rte_graph library’s role in data path processing and showcased Grout’s capabilities.

Another notable session was led by Dr. John Romein from the Netherlands Institute for Radio Astronomy (ASTRON). He discussed how advanced GPU technologies, specifically the NVIDIA Grace Hopper Superchip, are being utilized to process vast amounts of data from radio telescopes. This session not only emphasized the integration of DPDK with GPU technology but also demonstrated its real-world applications in astronomical data processing, pushing the boundaries of modern hardware capabilities.

Each session, from discussions on the challenges of implementing DPDK on non-cache coherent platforms by Hemant Agrawal and Gagandeep Singh from NXP, to insights into machine learning inference within network processing by Srikanth Yalavarthi from Marvell, highlighted the versatility and robustness of DPDK. These discussions underscored its ability to meet the increasingly complex demands of network performance solutions.

In summary the latest DPDK summit provided a platform for learning and sharing, reinforcing the community’s commitment to driving innovation in network performance through open development and governance, as highlighted by the ongoing initiatives of the project.

Watch all the presentations on the DPDK youtube here

Have a use case you’d like to share and feature on the website? Email marketing@dpdk.org