A Q&A with Rashid Khan, DPDK Board Member and Director, Networking at Red Hat
Rashid Khan is the Director of Networking development at Red Hat. His team’s responsibilities include upstream and downstream development of Linux kernel networking, DPDK, OVS OVN, NFV, MPTCP, ebpf XDP. Before joining Red Hat, Rashid managed the Linux team at Broadcom consumer products division. There he was in charge of board support drivers team, silicon bring up, and application deployment. Prior to that Rashid has approximately 20 years of networking, telco, voice over IP, signal processing experience at various startups and huge conglomerates.
What is Red Hat’s history with DPDK? How long have you been involved in the community?
Red Hat is an open-source Linux company that participates in the development of the Linux operating system, and provides support to its customers via Red Hat’s Linux distribution (RHEL). About four years ago, Red Hat noticed a lot of traction for the Data Plane Development Kit (DPDK) project to get traffic from network interface cards into virtual machines (VMs) and containers, with better packet speeds and development, resulting in faster time-to-market. Allowing new features and new NICs to work with Linux kernel is a bit hard, because Linux maintains the principle of not breaking things that are already working. DPDK offered a new approach; it did not come with the same constraints, and it offered an easier way to move network traffic, add new drivers for new cards, and add more features quickly.
DPDK provided a method for getting packets faster to VMs, and by helping accelerate networking for its virtualization products.
What was the primary use case that got you started with DPDK?
Red Hat primarily uses DPDK with Open vSwitch to get packets routed within servers from Network Interface Cards (NIC), to VMs (we leverage DPDK as the foundation for our NFV stack).
Can you talk a little more about some of these (and other) use cases?
Sure, there are a few I can touch on:
- Telcos are used to custom SW/HW builds from vendors, but because of increasing consumer demand for new features without a willingness to pay more, there is a strong need to go with commodity hardware and open-source software, and to put applications on top through virtual machines and containers. So right now, we’re investing heavily in solving those issues and getting network traffic to VMs, containers and pods as efficiently as possible.
- We’re also investing time and resources into the hardening of DPDK itself, taking it to the next level if you will. We’ve worked really hard to get our CI tests working at the University of New Hampshire’s Interoperability Lab (UNH-IOL). We can detect regressions in DPDK-OVS and alert the submitters of patches. HW vendors collaborating very closely with us on this include Intel, Mellanox, Netronome, and Broadcom.
So what are some of the challenges DPDK is experiencing, and how do you think they can be addressed?
- DPDK is being used extensively everywhere, and that is great. Now we need to collaborate to take it to the next level of maturity. For example, the community needs to collaborate to find a way to continue adding advanced features without compromising the stability of existing HW and existing features. Stability of Long Term Releases (LTS) has to be solidified. DPDK will have to adapt to cater to that — number of releases, duration of releases, integration with other products and projects will have to be evaluated. But the community is up for the challenge! There are enough senior/experienced people who are putting the right steps in place to solve for this.
- Another challenge is that there are a lot of consumers of DPDK, but not very many active participants. Red Hat, Mellanox, and Intel are really stepping up and putting in resources, but we’d love to see more organizations participating. There are folks working full time on LTSs, and Red Hat has actually hired head count just for DPDK: we have about a half-dozen people focused on DPDK upstream, plus 8-9 working with DPDK for product integration.
- There’s also the challenge of debuggability. The more we add to DPDK the harder it is to debug systems running DPDK. And larger customers demand bug fixes plus an absolute root cause for each bug. That requires debuggability of every layer and complete predictability. In some cases adding debug code moves the problem, or may even alleviate the problem. Unfortunately, that is not enough for large telco customers. They need to know the root cause of the problem, and how a patch fixes that problem directly. More work is needed to get DPDK to that level. As products mature, more of these “Day 2” problems need to be addressed. The same is the case with DPDK; the time has come to get the project to the next level.
- The next challenge for DPDK is to integrate with smart(er) NICs to enable HW offloads so main motherboard CPUs inside the servers are not doing the heavy lifting of the packet movement. We are working with multiple HW vendors to address that challenge.
How can someone get involved in DPDK?
There are so many ways to get involved! There are no impediments in joining — it’s easy to participate. You can start by making DPDK work for your specific hardware, enhance CI activities, collaborate on UNH lab, patch review, or just start by participating in online meetings. There are no barriers to entry – joining mailing lists and conference calls is a good first step. Does not have to be a large commitment in the beginning, you can start with just a few hours a week.
And again – we see a lot more consumers of DPDK than we do participants. Some of the biggest benefits of participation are to (selfishly) safeguard your own hardware and products, but to also have a voice in the overall direction of the project. And the more diverse the community, the better – that’s the spirit of open source.