Apply to Speak at the DPDK Summit Montreal by July 31.
Skip to main content
Monthly Archives

May 2024

DPDK Dispatch May

By Monthly Newsletter

1. Main Announcements

3. Blogs, User Stories and Developer Spotlights

4. DPDK & Technologies in the news:

5. Performance Reports & Meeting Minutes

This newsletter is sent out to thousands of DPDK developers, it’s a collaborative effort. If you have a project release, pull request, community event, and/or relevant article you would like to be considered as a highlight for next month, please reply to marketing@dpdk.org

Thank you for your continued support and enthusiasm.

DPDK Team.

DPDK Long Term Stable (LTS) Release 22.11.05

By Blog

The latest DPDK Long Term Stable (LTS) Release 22.11.05 includes several updates and enhancements across various components of the DPDK framework. Significant changes in this release involve numerous file modifications, which are indicated by a large number of insertions and deletions across the codebase. The release notes document extensive additions, suggesting improvements and new features in the areas of network interface controllers (NICs), cryptographic devices, event devices, and baseband processing.

Notable adjustments were made to the build system, documentation, and driver support for various hardware. Improvements in error handling, memory management, and device operation stability are also reflected in the release notes. The release also addresses various bug fixes and performance enhancements to ensure better stability and efficiency.

Contributors

The update involved 246 files with a total of 3,235 insertions and 2,053 deletions. A big shout out to all the contributors to this release including:

Ajit Khaparde, Akhil Goyal, Akshay Dorwat, Alan Elder, Alex Vesker, Ali Alnubani, Andrew Boyer, Anoob Joseph, Arkadiusz Kusztal, Bing Zhao, Bruce Richardson, Chaoyong He, Chengwen Feng, Ciara Power, Dariusz Sosnowski, David Marchand, Dengdui Huang, Edwin Brossette, Eli Britstein, Emi Aoki, Erez Shitrit, Ferruh Yigit, Fidel Castro, Flore Norceide, Ganapati Kundapura, Gregory Etelson, Hamdan Igbaria, Hanumanth Pothula, Hao Chen, Harman Kalra, Hernan Vargas, Holly Nichols, Huisong Li, Jie Hai, Jonathan Erb, Joyce Kong, Kaiwen Deng, Kalesh AP, Kevin Traynor, Kiran Kumar K, Kishore Padmanabha, Kommula Shiva Shankar, Konstantin Ananyev, Kumara Parameshwaran, Long Li, Luca Boccassi, Maayan Kashani, Masoumeh Farhadi Nia, Maxime Coquelin, Michael Baum, Mingjin Ye, Morten Brørup, Mário Kuka, Neel Patel, Nithin Dabilpuram, Pavan Nikhilesh, Pengfei Sun, Qi Zhang, Qian Hao, Radu Nicolau, Rahul Bhansali, Rakesh Kudurumalla, Robin Jarry, Rongwei Liu, Satheesh Paul, Shai Brandes, Shaowei Sun, Shihong Wang, Shun Hao, Simei Su, Sivaprasad Tummala, Sivaramakrishnan Venkat, Stephen Hemminger, Suanming Mou, Sunil Kumar Kori, Sunyang Wu, Tom Jones, Viacheslav Ovsiienko, Wathsala Vithanage, Weiguo Li, Yajun Wu, and Yunjian Wang.

These contributors addressed various aspects from software fixes and performance enhancements to security improvements across multiple components of the system.

Download it here: DPDK 22.11.5

The git tree for this version can be accessed here: DPDK Stable 22.11

DPDK’s Role in Hyperscaling

By Blog

In the rapidly evolving digital landscape, hyperscaling in the cloud has emerged as a critical strategy for businesses aiming to scale their operations efficiently. The webinar, “Hyperscaling in the Cloud,” hosted by Honnappa Nagarahalli (Arm), from the DPDK Tech Board, brings together industry experts to discuss how the Data Plane Development Kit (DPDK) is revolutionizing hyperscale cloud environments.

The Webinar Panelists

The webinar featured three distinguished panelists:

1. Brian Denton: A Senior Program Manager at Microsoft Azure, Brian brings a wealth of experience in Azure’s host networking. He shared insights into Azure’s implementation of DPDK, emphasizing its use in enhancing Ethernet, and overall network performance.

2. Rushil Gupta: As a Senior Software Engineer at Google, Rushil highlighted the critical role of DPDK in financial technology (Fintech) applications on Google Cloud Platform (GCP). His discussion focused on achieving consistency, performance, and reliability in high-frequency trading platforms.

3. Jim Thompson: Co-founder of Netgate, Jim delved into the use of DPDK in networking applications outside the traditional cloud domain. His contribution illuminated the versatility of DPDK across different cloud environments and its impact on virtual private networks (VPNs).

Insights from the Webinar

DPDK in Azure’s Cloud Networking

Brian Denton’s presentation offered a glimpse into how Microsoft Azure leverages DPDK to offload packet processing from the CPU to dedicated hardware. This approach significantly reduces latency and improves throughput, enabling Azure to offer enhanced performance for virtual machines (VMs) and networking services.

Brian shared valuable insights into how DPDK has been instrumental in Azure’s network infrastructure, particularly highlighting its impact on Azure’s host networking and the broader ecosystem of partners and customers. He explained that Azure has integrated DPDK to address the need for high-speed packet processing, which is crucial for a wide range of applications, from basic web services to complex, latency-sensitive tasks like real-time analytics and high-frequency trading.

One of the key points Brian made was about the technical architecture that enables Azure to leverage DPDK’s capabilities. He detailed how DPDK is used in conjunction with Azure’s hardware, such as SmartNICs, to offload and accelerate network functions traditionally handled by software. This hardware-software synergy, as Denton explained, not only reduces CPU overhead but also significantly decreases latency, providing Azure customers with improved network performance and efficiency.

Furthermore, Brian highlighted real-world applications of DPDK in Azure, illustrating how partners and customers utilize DPDK for scenarios that require minimal latency and maximum throughput. He also discussed the continuous evolution of Azure’s networking stack, underscored by the introduction of new hardware and the ongoing optimization of DPDK to meet the growing demands of cloud computing.

Some examples: 

Clearent by Xplor used Azure SQL Database Hyperscale to revamp its merchant transaction reporting system. Previously operating on in-house systems, Clearent, which handles over 500 million transactions annually, shifted to a cloud-based setup. This move significantly boosted their ability to process and report data.

Protocall Services, a provider of telephonic crisis and behavioral health digital tools, embarked on a cloud migration journey to enhance the reliability, security, and scalability of its IT infrastructure. 

DPDK’s Impact on Fintech Applications

Rushil Gupta’s discussion on the use of DPDK in financial technology (Fintech) applications, particularly in the realm of high-frequency trading (HFT) platforms on Google Cloud Platform (GCP), sheds light on how bleeding-edge network processing technologies are evolving financial markets. 

In the fast-paced world of HFT, where milliseconds can equate to millions of dollars, the need for ultra-low latency and high throughput is paramount. Traditional cloud networking approaches may falter under such demanding requirements due to the involvement of kernel-based networking stacks that introduce additional latency. Here, DPDK’s bypass of the kernel networking stack, allowing direct access to network hardware, presents a compelling solution. This direct path significantly reduces latency and increases packet processing speed, enabling HFT platforms to operate at the speed required to capitalize on fleeting market opportunities.

Rushil illustrates how Google leverages DPDK to empower fintech customers on GCP, providing them with the infrastructure necessary to achieve the high throughput and low-latency communication essential for HFT platforms. One notable application is in the construction of complex event processing (CEP) systems, which are at the heart of many trading platforms. These systems analyze and act upon market data in real-time, necessitating the rapid processing capabilities that DPDK facilitates.

Rushil discusses the role of DPDK in enhancing data replication and recovery processes within fintech applications. In an industry where data integrity and availability are critical, DPDK’s efficiency in handling large volumes of data packets ensures that financial institutions can maintain robust data replication frameworks. This capability not only supports the high availability demands of trading platforms but also aids in achieving regulatory compliance related to data persistence and recovery.

Rushil explained how DPDK’s application in fintech on GCP demonstrates the technology’s pivotal role in enabling HFT and other financial services to meet their stringent performance and reliability criteria. With DPDK, Google provides a competitive edge to fintech applications, facilitating new levels of speed and efficiency in financial markets. 

Some examples: 

1. CME Group. As one of the world’s leading derivatives marketplaces, CME Group leverages GCP and DPDK for enhanced market data analytics and to facilitate high-speed trading. Their partnership aims to accelerate CME Group’s move to the cloud, transforming the global markets ecosystem with cloud-based innovation and scaling capacity dynamically to meet market demands.

2. Talos. Specializing in digital asset trading technology, Talos utilizes GCP’s infrastructure to support its trading platform. With DPDK, Talos benefits from reduced latency and increased throughput, essential for executing trades and managing orders across multiple exchanges and liquidity pools efficiently.

3. Clowd9. This cloud-based trading technology provider uses GCP to offer a scalable and secure platform for trading firms and financial institutions. DPDK supports Clowd9’s need for high performance and low latency in executing trades, managing risk, and processing real-time market data.

4. Freetrade. Freetrade, an investment platform, leverages GCP to power its app, offering users commission-free trading. GCP’s global infrastructure and DPDK’s network optimization capabilities ensure that Freetrade can manage high volumes of transactions and data analysis. 

5. TD Securities Automated Trading (TDSAT): TDSAT uses GCP for trading fixed-income bonds, benefiting from DPDK’s high-performance packet processing capabilities. This enables TDSAT to execute trades at high speed and with precision, critical for maintaining competitiveness in the fixed income market.

These customers and use cases underscore the importance of DPDK in enhancing network performance on GCP, making it an ideal platform for capital market applications that demand high throughput, low latency, and scalability. By leveraging GCP and DPDK, capital market firms innovate and adapt quickly to market changes, manage risks more effectively, and unlock new opportunities for growth.

Broadening DPDK’s Application Scope in VPNs and Software Routers

Jim Thompson’s insights during the DPDK webinar shed light on how DPDK is leveraged in cloud networking through the lens of Netgate’s product, TNSR (pronounced ‘Tensor’). This serves as a case study of DPDK’s implementation outside its traditional use cases. TNSR, a virtual router developed by Netgate, underscores the adaptability and robustness of DPDK in addressing specific cloud networking challenges.

In cloud environments, networking demands can quickly escalate due to the sheer volume of data transfer and the need for secure connections. Traditional VPN solutions often fall short due to bandwidth limitations and the number of tunnels they can support. Jim highlighted how these constraints could hinder the scalability and performance of cloud-based services. This scenario is particularly relevant for large organizations that require extensive interconnectivity across various cloud environments.

The introduction of TNSR as a DPDK-powered solution exemplifies how DPDK’s high-performance packet processing capabilities can be extended beyond typical use cases to solve complex cloud networking problems. By utilizing DPDK’s efficient polling mode drivers (PMDs) for network and cryptography offload, TNSR significantly enhances throughput and reduces latency in VPN connections. 

Jim explained how TNSR facilitates seamless connectivity between on-premise networks and cloud regions, highlighting the importance of VPN connections for secure data transfer. He underscored the limitations of existing cloud VPN solutions, such as bandwidth caps and tunnel number restrictions, which can significantly hamper large organizations’ networking needs. By leveraging DPDK, TNSR bypasses these limitations, providing a more flexible and scalable solution for cloud-based networking.

Take a look at Netgate’s customer stories here

Conclusion

The webinar underscored DPDK’s pivotal role in enabling hyperscaling in the cloud. By providing a high-performance packet processing framework, DPDK not only enhances network efficiency but also opens new avenues for application development across various industries. As cloud architectures continue to evolve, the collaboration between cloud providers, technology firms, and the open source community will be vital in harnessing the full potential of DPDK.

Join in the Hyperscaling discussion and the community on slack here.

Unleashing Network Performance with Microsoft Azure MANA and DPDK

By User Stories

Introduction

In the modern cloud computing era, network performance and efficiency are paramount. Microsoft Azure has been at the forefront of this revolution, introducing innovative solutions like the Microsoft Azure Network Adapter (MANA) and integrating the Data Plane Development Kit (DPDK) to enhance the network capabilities of Azure virtual machines.

In this user story we interview Brian Denton, and Matt Reat, Senior Program Managers for Azure Core. Brian’s role has been pivotal, focusing on engaging with all network virtual appliance partners to ensure they are prepared and supported for the introduction of a new Network Interface Card (NIC) into Azure. 

Matt’s journey at Microsoft began primarily within the networking domain. His career commenced with network monitoring before transitioning, about four years ago, into what is referred to as the host networking space. This area encompasses the SDN software stack and hardware acceleration efforts aimed at enhancing customers’ ability to utilize an open virtual network (OVN) and improve their overall experience on Azure. 

A natural progression of his work has involved spearheading innovations in software and the development of hardware, which have recently been introduced to the public as Azure Boost. Additionally, his contributions include the development of the MANA NIC, a product developed in-house at Microsoft. 

The Genesis of Azure MANA

Azure MANA represents a leap in network interface technology, designed to provide higher throughput and reliability for Azure virtual machines. As the demand for faster and more reliable cloud services grows, Azure’s response with MANA smartNICs marks a significant milestone, aiming to match and surpass AWS Nitro-like functions in network and storage speed acceleration. 

Microsoft’s strategy encompasses a comprehensive approach, with a primary focus on hardware acceleration from top to bottom. This effort involves current work being conducted on the host and in the hypervisor (Hyper-V), aiming to advance hardware capabilities. Such initiatives are also being pursued by competitors, including AWS with its Nitro system and Google with a similar project, marking Microsoft’s contribution to this competitive field.

Behind the scenes, the team implemented several enhancements that remained undisclosed until the announcement of Azure Boost last July. This development compelled them to reveal their progress, especially with the introduction of the MANA NIC, which had been concealed from customer view until then.

The introduction of the new MANA NIC, boasting ratings of up to 200 Gbps in networking throughput, represents a significant enhancement of the current Azure offerings, in-line with Microsoft’s competition. The reliance on off-the-shelf solutions proved to be cost-prohibitive, prompting a shift to a fully proprietary, in-house solution integrated with their Field-Programmable Gate Array (FPGA).

DPDK’s Role in Azure’s Network Evolution

DPDK offers a set of libraries and drivers that accelerate packet processing on a wide array of CPU architectures. Microsoft Azure’s integration of DPDK into its Linux Virtual Machines (VMs) is specifically designed to address the needs of applications that demand high throughput and low latency, making Azure a compelling choice for deploying network functions virtualization (NFV), real-time analytics, and other network-intensive workloads.

The technical essence of DPDK’s acceleration capabilities lies in its bypass of the traditional Linux kernel network stack. By operating in user space, DPDK enables direct access to network interface cards (NICs), allowing for faster data plane operations. This is achieved through techniques such as polling for packets instead of relying on interrupts, batch processing of packets, and extensive use of CPU cache to avoid unnecessary memory access. Additionally, DPDK supports a wide range of cryptographic algorithms and protocols for secure data processing, further enhancing its utility in cloud environments.

Azure enhances DPDK’s capabilities by offering support for a variety of NICs optimized for use within Azure’s infrastructure, including those that support SR-IOV (Single Root I/O Virtualization), providing direct VM access to physical NICs for even lower latency and higher throughput. Azure’s implementation also includes provisions for dynamically managing resources such as CPU cores and memory, ensuring optimal performance based on workload demands.

Microsoft’s commitment to DPDK within Azure Linux VMs underscores a broader strategy to empower developers and organizations with the tools and platforms necessary to build and deploy high-performance applications at scale. By leveraging DPDK’s packet processing acceleration in conjunction with Azure’s global infrastructure and services, users can achieve the highest possible performance on Azure. 

Enhancing Cloud Networking with Azure MANA and DPDK

Azure MANA and DPDK work in tandem to push the boundaries of cloud networking. MANA’s introduction into Azure’s ecosystem not only enhances VM throughput but also supports DPDK, enabling network-focused Azure partners and customers to access hardware-level functionalities. When introducing a new Network Interface Card (NIC), it is essential to have support for the Data Plane Development Kit (DPDK). The primary concern is that Azure customers will begin to encounter Mana NICs across various Virtual Machine (VM) sizes, necessitating support for these devices. This situation highlights a notable challenge.

The scenario involves three NICs and two Mellanox drivers requiring support, indicating a significant transition. The introduction of this new NIC and its drivers is intended for long-term use. The goal is for the MANA driver to be forward-compatible, ensuring that the same driver remains functional many yearsfrom now, without the need to introduce new drivers for new NICs with future revisions, as previously experienced with ConnectX and Mellanox.

The objective is a long-term support driver that abstracts hardware changes in Azure and the cloud affecting guest VMs, offering a steadfast solution for network I/O. Although the future specifics remain somewhat to be determined, the overarching aim is to support the features available on Azure, focusing on those needs rather than the broader spectrum of Mellanox’s customer requirements. Some features necessary for Azure may not be provided by Mellanox, and vice versa. Thus, the ultimate goal is to support Azure customers with tailored features, ensuring compatibility and functionality for the long term.

Microsoft offers a wide array of networking appliances that are essential to their customers’ architectures in Azure. Therefore, part of their effort and emphasis on supporting DPDK is to ensure our customers receive the support they need to operate their tools effectively and achieve optimal performance.

Supporting DPDK is essential to accommodate those toolsets. Indeed, maximizing the use of our hardware is also crucial. This is an important point because there’s potential for greater adoption of DPDK.

Matt Reat, Senior Program Manager at Microsoft

Typically, Microsoft’s users, mainly those utilizing network virtual appliances, leverage DPDK, and they are observing increased adoption not only among Microsoft’s Virtual Academy’s but also among customers who express intentions to use DPDK. It’s not limited to virtual appliance products alone. They also have large customers with significant performance requirements who seek to maximize their Azure performance. To achieve this, leveraging DPDK is absolutely essential.

The Technicals of MANA and DPDK

The MANA poll mode driver library (librte_net_mana) is a critical component in enabling high-performance network operations within Microsoft Azure environments. It provides specialized support for the Azure Network Adapter Virtual Function (VF) in a Single Root I/O Virtualization (SR-IOV) context. This integration facilitates direct and efficient access to network hardware, bypassing the traditional networking stack of the host operating system to minimize latency and maximize throughput.

By leveraging the DPDK (Data Plane Development Kit) framework, the MANA poll mode driver enhances packet processing capabilities, allowing applications to process network packets more efficiently. This efficiency is paramount in environments where high data rates and low latency are crucial, such as in cloud computing, high-performance computing, and real-time data processing applications.

The inclusion of SR-IOV support means that virtual functions of the Azure Network Adapter can be directly assigned to virtual machines or containers. This direct assignment provides each VM or container with its dedicated portion of the network adapter’s resources, ensuring isolated, near-native performance. It allows for scalable deployment of network-intensive applications without the overhead typically associated with virtualized networking.

Overall, the technical sophistication of the MANA poll mode driver library underscores Microsoft Azure’s commitment to providing advanced networking features that cater to the demanding requirements of modern applications. Through this library, Azure ensures that its cloud infrastructure can support a wide range of use cases, from web services to complex distributed systems, by optimizing network performance and resource utilization.

“The MANA poll mode driver library, coupled with DPDK’s efficient packet processing, allows us to optimize network traffic at a level we couldn’t before. It’s about enabling our customers to achieve more with their Azure-based applications.”

Matt Reat, Senior Program Manager at Microsoft

The setup procedure for MANA DPDK outlined in Microsoft’s documentation provides a practical foundation for these advancements, ensuring that users can leverage these enhancements with confidence. Furthermore, the support for Microsoft Azure Network Adapter VF in an SR-IOV context, as implemented in the MANA poll mode driver library, is a testament to the technical prowess underlying this integration.

Performance Evaluation and Use Cases

Evaluating the performance impact of MANA and DPDK on Linux VMs highlights significant improvements in networking performance. Azure’s documentation provides insights into setting up DPDK for Linux VMs, emphasizing the practical benefits and scenarios where the combination of MANA and DPDK can dramatically improve application responsiveness and data throughput. 

Microsoft effectively utilizes the Data Plane Development Kit (DPDK) on the host side to optimize network performance across its Azure services. This approach not only supports customer applications by enhancing the speed and efficiency of data processing on virtual machines but also strengthens Microsoft’s own infrastructure. 

By leveraging DPDK, Azure can handle higher data loads more effectively, which is crucial for performance-intensive applications. For a deeper understanding of how DPDK facilitates these improvements in cloud computing, view the latest webinar, “Hyperscaling in the Cloud,” which discusses the scale and scope of DPDK’s impact on Azure’s network architecture. 

“We’re aiming to push the boundaries of network performance within Azure, leveraging MANA alongside DPDK to achieve unprecedented throughput and reliability for our virtual machines.” 

Brian Denton, Senior Program Manager, Microsoft Azure Core

Significant emphasis is placed on the first 200 gig NIC, highlighting a substantial focus on achieving high throughput. Additionally, the necessity to support a high packet rate stands as a corollary to this objective. To comprehend and benchmark their throughput across various packet sizes, extensive work is undertaken. DPDK serves as the primary method for testing their hardware in this regard.

Microsoft’s engineering counterparts focus on the overall testing methodology for developing a DPDK driver set, as well as testing the hardware itself and the VM performance on that hardware. This includes client-side involvement in testing. Currently, only Linux is officially supported for DPDK, although there have been attempts to use Windows and FreeBSD. Various host configurations also play a crucial role in qualifying their hardware.

Future Directions and Community Engagement

As Azure continues to evolve, the collaboration between Microsoft’s engineering teams and the open-source community remains vital. The development of MANA and its integration with DPDK reflects a broader commitment to open innovation and community-driven improvements in cloud networking.

Conclusion

As Microsoft Azure continues to evolve, the partnership between Microsoft’s engineering teams and the DPDK open-source community is poised to play a crucial role in shaping the future of cloud networking. The development of the Microsoft Azure Network Adapter (MANA) and its integration with the Data Plane Development Kit (DPDK) underscore a commitment to leveraging open innovation and fostering community-driven advancements.

The future role of Azure MANA, in conjunction with the DPDK community, is expected to focus on breaking new technical limits in cloud networking. This collaboration could lead to significant enhancements in network performance, including higher throughput, reduced latency, and greater efficiency in packet processing. By leveraging DPDK’s efficient packet processing capabilities alongside the hardware acceleration offered by MANA, Azure aims to provide an optimized networking stack that can meet the demanding requirements of modern applications and services.

Moreover, this is likely to drive the development of new features and capabilities that are specifically tailored to the needs of Azure’s diverse user base. This could include advancements in virtual network functions (VNFs), network function virtualization (NFV), and software-defined networking (SDN), which are essential components in a cloud-native networking landscape.

The open-source nature of DPDK also ensures that the broader community can contribute to and benefit from these developments, promoting a cycle of continuous improvement and innovation. This collaborative approach not only enhances the capabilities of Azure’s networking services but also contributes to the evolution of global cloud networking standards and practices.

Ultimately, the future of Microsoft Azure MANA and the DPDK open-source community is likely to be characterized by the breaking of current technical barriers, the introduction of groundbreaking networking solutions, and the establishment of Azure as a leading platform for high-performance, cloud-based networking services.

Check out the summary and additional use cases on Hyperscaling in the Cloud here.

Join the community on slack here