The Netdev 0x18 Conference, held from July 15th to 19th, 2024, in Santa Clara, California, brought together leading minds in Linux networking for a week of insightful presentations, technical sessions, and networking opportunities. Hosted at Intel’s Santa Clara campus, the event was a hub of innovation and collaboration, focusing on the latest advancements in Linux kernel networking and user-space applications.
Highlights of Netdev 0x18
TC Workshop
The Traffic Control (TC) Workshop, chaired by J Hadi Salim, facilitated face-to-face discussions on various TC topics. Participants continued their biweekly discussions on P4TC hardware offload, providing a platform for in-depth technical exploration and collaborative problem-solving. The workshop focused on practical applications and enhancements in traffic control mechanisms, fostering a robust exchange of ideas and strategies among attendees.
IPsec Workshop
Steffen Klassert chaired the IPsec Workshop, which gathered specialists from the Linux and IETF communities. This session, traditionally invitation-only, was opened to Netdev attendees, encouraging broader participation. The workshop covered a wide range of IPsec-related topics, including crypto, network security, and the IPsec protocol suite. Attendees contributed through presentations and discussions, promoting a collaborative environment aimed at advancing IPsec technologies.
Introduction to PTP on Linux – APIs
Maciek Machnikowski led a tutorial on Precision Time Protocol (PTP) APIs within the Linux Kernel. This session provided a comprehensive overview of key components essential for synchronization, including POSIX clock APIs, IOCTLs, and netdev socket options. Attendees learned about the lifecycle of hardware timestamps and explored extensions to improve the robustness of PTP APIs, addressing current limitations in time synchronization accuracy.
Tutorial: PTP from Scratch
Instructors Milena Olech and Maciek Machnikowski offered a hands-on tutorial on creating and implementing a driver for a PTP Hardware Clock (PHC). Using the netdevsim driver as a base, participants were guided through exposing clock and timestamping functionality. The session provided practical experience in enabling PHC and implementing PTP-related kernel APIs, equipping attendees with the skills to tackle common PTP-related challenges.
Workshop on Extension Headers
Chaired by Tom Herbert, this workshop focused on IPv6 (and potentially IPv4) Extension Headers (EH). Topics included the creation and processing of EH in eBPF, mitigating packet drops due to EH, and changes to the Linux kernel to enhance EH usability. The workshop also explored use cases for Hop-by-Hop Options in host-to-network and network-to-host signaling, highlighting the importance of EH in making IPv6 a truly extensible protocol.
Real-World Tips, Tricks, and Notes of Using Epoll-Based Busy Polling to Reduce Latency
Joe Damato presented a tutorial on epoll-based busy polling, a technique to reduce latency in network applications. He shared insights from modifying an open-source application to use busy polling, covering multi-NIC configurations, helpful NIC features, and key metrics for production environments. The talk also touched on future kernel enhancements to further optimize busy polling, providing practical advice for developers.
Devmem TCP & Io_uring Zero Copy
Willem de Bruijn, Jakub Kicinski, and their team discussed advancements in devmem TCP and io_uring zero copy. They introduced the ‘network memory’ abstraction (netmem), enabling support for non-struct paged memory in the net stack. The session included performance results and challenges faced in real-world applications, as well as recent developments in queue management ndos to enhance the efficiency of devmem TCP and io_uring zero copy.
Introduction to Falcon Reliable Transport
Speakers Yadong Li, Jay Bhat, and others introduced Falcon, a hardware-offloaded reliable transport designed by Google. Falcon addresses the needs of high-bandwidth, low-latency workloads, such as AI/ML training and HPC. The talk covered Falcon’s software enablement on the Intel IPU E2000, connection setup, key exchange, and integration with rdma-core/ibverbs. Proposed extensions for advanced features were also discussed, highlighting Falcon’s potential to transform high-performance networking.
Introduction to Ultra Ethernet Consortium 1.0
Uri Elzur and Shrijeet Mukherjee introduced the Ultra Ethernet Consortium 1.0. This session provided an overview of the consortium’s objectives, the technologies being developed, and the potential impact on the future of Ethernet. The presentation aimed to engage the community and foster collaboration on Ethernet advancements.
Driver and H/W APIs Workshop
Chaired by David Ahern and Alexander Duyck, this workshop addressed issues in driver development and hardware APIs. Topics included devlink extensions, memory pools, new queue management APIs, and challenges of netdev/IB co-existence. The session provided a collaborative environment for discussing proposals and solutions to improve driver and hardware API development.
Netfilter Mini Workshop
Chaired by Pablo Neira and the core team, the Netfilter Mini Workshop covered various aspects of Netfilter, the packet filtering framework in Linux. Although specific topics were to be announced, the workshop aimed to provide a platform for in-depth technical discussions and collaborative problem-solving in Netfilter development.
FRRouting Workshop
David Lamparter and Chris Hopps chaired the FRRouting workshop, covering topics such as mgmtd, configuration dispatch, and YANGification status. The session also addressed the debug and tracing framework, providing insights into the current state and future direction of FRRouting development. Although the topic of next-hop group adaptation did not make it into the workshop, the session provided valuable updates and discussions on FRRouting.
Netdev Testing & Automation
Jakub Kicinski presented an overview of the efforts to improve testing and automation within Netdev over the past year. The talk covered the testing infrastructure, helper libraries, and development statistics since the previous conference. This session highlighted the continuous improvement in testing methodologies and the importance of automation in maintaining high-quality code in the Netdev community.
Netdev Processes, Community, and Respect/Reputation
Andrew Lunn discussed the operational aspects of Netdev, focusing on patch submission, maintainership, and review processes. He emphasized the role of community, respect, and reputation in the effective functioning of Netdev. The talk aimed to trigger discussions on improving processes and fostering a positive and collaborative community environment.
vDPA Live Migration Downtime Improvements for Net Devices
Speakers Eugenio Perez Martin, Dragos Tatulea, and Si-Wei Liu explored efforts to reduce downtime in vDPA-net live migration. They discussed concepts like VirtIO, vDPA, and the challenges of live migration. The session provided solutions to minimize downtime, sharing techniques such as reducing memory map handling time and pre-warming IOMMU configurations. The talk concluded with a round table discussion on future improvements and vendor contributions.
Domain Specific Accelerators for Networking
Tom Herbert and Felipe Almeida presented on Domain Specific Accelerators (DSA) for networking. They discussed offloads, acceleration instructions, and accelerator engines, proposing solutions to integrate DSA into Linux for robust and correct offloads. The talk highlighted the need for ease-of-use in DSA, showcasing compiler innovations that simplify the use of hardware accelerators in networking.
Compilers and Linkers for Network Optimization
Felipe Magno de Almeida and Tom Herbert demonstrated how compilers can optimize network code. They introduced techniques like dynamic constant instantiation and pattern matching to convert general-purpose code into optimized code for specific architectures. The session showcased custom C++ extensions and the integration of DSAs into the compiler stack, providing insights into advanced network code optimization.
Advancing Device Migration for Virtio PCI Hardware Devices
Speakers Parav Pandit, Yishai Hadas, and others detailed the journey of advancing live migration for Virtio PCI hardware devices. They discussed design considerations, implementation challenges, and performance benchmarks. The session highlighted differences between their approach and other vendor-based methods, sharing lessons learned and progress in Virtio specification development.
How Linux Tracing Can Use the Net/ Subsystem
Alexander Aring presented a novel idea of integrating Linux tracing with the network subsystem. He proposed using NIC tx/rx ring buffers for tracing, enabling classification, filtering, and offloading tracing data to hardware. The talk explored the potential of networkifying Linux tracing, aiming to make tracing faster and more intuitive for networking developers.
Scheduling HTTP Streams
In the talk by Evgeny Mekhanik, Konstantin Tatar, and Alexander Krizhanovsky, the complexities of HTTP stream prioritization were explored. As web servers juggle millions of TCP connections and numerous HTTP/2 or HTTP/3 streams, efficient scheduling becomes critical. The speakers highlighted fair and fast scheduling algorithms, underlying data structures, and server-side optimizations in platforms like Nginx and Cloudflare. They also discussed how modern browsers manage stream priorities and the known vulnerabilities in HTTP/2 prioritization mechanisms.
The Future of AI Networks: Advancing TCP with Device Memory and Collective Communication
Anjali Singhai, Shaopeng He, and Sridhar Samudrala discussed extending TCP to support Collective Communication (CC) semantics, initially introduced by Google as Devmem TCP. These advancements are pivotal for AI networks, simplifying CC semantics and enhancing performance for applications in AI, HPC, and storage solutions. The talk detailed practical deployments on Intel’s NICs, performance data showcasing TCP’s effectiveness compared to RDMA, and the broader implications for the industry.
Machine Learning Optimization Algorithms in HTTP Latency Tuning on Nginx
Jasmine Mou and Krz Sywula presented a proof of concept leveraging machine learning to optimize HTTP latency for Nginx servers. By automating kernel parameter tuning, they demonstrated significant improvements in performance over manual tuning methods. Their session covered experimental design, data pipeline setups, and the comparative analysis of manual versus automated tuning, highlighting the potential of machine learning in system optimization.
Machine Learning Practices in Network Traffic across Data Centers
Jasmine Mou’s solo session focused on using machine learning, statistical profiling, and visualization techniques to manage network traffic across data centers. The talk emphasized discovering trends, setting up alerting systems, and projecting capacity needs, all aimed at optimizing resource planning and reducing costs. These practices promise smoother product experiences and more efficient network management.
AI-Enhanced Reviews for Linux Networking
Jesse Brandeburg and Kamel Ayari introduced an AI-based solution to streamline patch reviews in open-source projects. Utilizing large language models (LLMs), their approach aims to reduce the burden on human reviewers by automating the detection of common issues and enforcing guidelines consistently. The session highlighted the benefits of AI in improving review efficiency, adaptability, and semantic understanding, proposing a future integration with zero-day bots for enhanced effectiveness.
Linux Networking on s390 Architecture
Alexandra Winter and Niklas Schnelle explored the unique aspects of networking on the s390 architecture, also known as IBM Z. They discussed its I/O virtualization, multi-level hypervisors, and the special network interfaces that distinguish it from other architectures. Their insights into the hardware, firmware, and hypervisor software provided a comprehensive look at the challenges and solutions specific to Linux networking on mainframes.
Fireside Chat with Martin Casado
In a captivating keynote, Martin Casado shared his journey from co-founding Nicira to leading Andreessen Horowitz’s infrastructure practice. He discussed the evolution of software-defined networking (SDN) and his vision for the future of networking technology. Casado’s experiences and insights into scaling businesses and driving innovation provided valuable inspiration for attendees.
Achieving Linear CPU Scaling in WireGuard with an Efficient Multi-Tunnel Architecture
Mirco Barone, Federico Parola, Fulvio Risso, and Davide Miola tackled the limitations of WireGuard in providing high-speed connectivity through a single tunnel setup. Their proposed multi-tunnel architecture demonstrated significant performance improvements by efficiently utilizing multiple CPU cores. The session detailed the architectural changes, performance data, and the potential for further enhancements in WireGuard’s scalability.
My Fu is Better Than Yours!
PJ Waskiewicz’s Birds of a Feather (BoF) session was a lively, open forum for sharing hacks, development flows, and testing methodologies. This collaborative environment encouraged participants to showcase their innovative solutions and learn from each other’s experiences, fostering a spirit of community and continuous improvement.
Characterizing IOTLB Wall for Multi-100-Gbps Linux-based Networking
Alireza Farshin and Luigi Rizzo discussed the performance bottlenecks in high-speed NICs, focusing on the IOTLB (I/O Translation Lookaside Buffer) wall. They modeled the problem, presented experimental data, and proposed mitigation techniques to improve throughput. Their research highlighted the need for rethinking I/O management in Linux to cope with increasing network speeds.
A New Lightweight Zero-Copy Notification Mechanism in Linux
Zijian Zhang and Xiaochun Lu introduced a novel zero-copy mechanism that simplifies application architecture changes and reduces notification overhead. Their approach, embedding notifications within user arguments, achieved a 7% performance increase for TCP compared to the existing MSG_ZEROCOPY flag. The session delved into the technical implementation and future implications for zero-copy network applications.
Fine-grained TCP Tuning
Amery Hung and Xiaochun Lu’s session on Fine-grained TCP Tuning addressed the complexities of flow-based traffic management in modern data centers. They highlighted how next-generation NICs, with their increased throughput, necessitate advanced TCP parameter tuning to avoid under-utilization of networks. The discussion revolved around leveraging per-path flow information and employing a receiver-side flow scheduler to detect and mitigate congestion proactively. The use of bpf qdisc for precise tuning was also explored, promising enhanced network performance by accommodating diverse application requirements.
Drinking From The Host Packet Fire Hose
Nabil Bitar, Jamal Hadi Salim, and Pedro Tammela tackled the challenge of bridging the gap between IO capabilities and CPU processing in their talk, Drinking From The Host Packet Fire Hose. They illustrated the dramatic increase in Ethernet port capacities compared to stagnant CPU and memory bandwidth advancements. The team presented a multi-phase approach, initially offloading network tasks to XPUs and later examining various network processing techniques to optimize resource utilization. Their future plans include integrating XPU acceleration with these techniques to significantly reduce host CPU load, enhancing efficiency in handling high-volume data transfers.
Is it OK to Hijack TCP?
John Ousterhout’s provocative talk, Is it OK to Hijack TCP?, proposed utilizing Homa-over-TCP to leverage decades of TCP hardware support. By transmitting Homa packets with the TCP protocol number, this approach aims to integrate Homa’s benefits seamlessly with existing TCP infrastructure. The session sought audience feedback on the feasibility and potential implications of this innovative method, highlighting its promise in enhancing network performance without extensive hardware changes.
State of the Union in TCP Land
Eric Dumazet provided a comprehensive overview of the latest changes in the Linux kernel and user space TCP implementations in his talk, State of the Union in TCP Land. He discussed recent advancements, optimizations, and future directions in TCP technology, ensuring the audience was up-to-date with the most current developments in this critical area of networking.
A Linux Implementation of SEARCH
Maryam Ataei Kachooei and her team introduced SEARCH, a new TCP slow start algorithm, designed to improve TCP performance over various link types, including WiFi and satellite links. SEARCH aims to exit the slow start phase at the right congestion point, balancing between premature and delayed exits. Their Linux kernel implementation demonstrated significant performance improvements, promising better network utilization and reduced packet loss across different network environments.
Multi-PF Single Netdev
Tariq Toukan’s session on Multi-PF Single Netdev explored the benefits of Multi-PF NIC technology, which allows multiple CPUs in a server to connect directly to the network. This approach reduces latency and CPU utilization while boosting network throughput. The talk covered the design, implementation, and performance metrics of combining multiple PFs under one netdev instance, emphasizing its potential for modern data centers.
Interrupt Moderation Approaches for Virtio Device
Parav Pandit and his team presented innovative interrupt moderation techniques for virtio devices to reduce CPU utilization. They introduced a novel approach leveraging the hypervisor’s dynamic interrupt moderation library, which simplifies implementation and improves performance. Their results demonstrated significant CPU resource savings and enhanced network efficiency, particularly in virtualized environments.
Shared Memory Pool for Representors
William Tu, Michal Swiatkowski, and Yossi Kuperman discussed a new shared memory pool mechanism for representor ports in switchdev mode. This approach optimizes memory usage by allowing multiple receive queues to share descriptors from a common pool, significantly reducing idle memory consumption. The session highlighted the design decisions, performance benefits, and future enhancements of this mechanism, which promises more efficient network resource management.
Architectural Considerations for Interplanetary Space Networking
Tom Herbert, Fred Templin, and Rachel Dudukovich delved into the unique challenges of interplanetary networking. They emphasized the need for delay-tolerant networking (DTN) protocols, like the Licklider Transmission Protocol (LTP), to handle the significant delays and disruptions inherent in space communications. Their talk covered DTN implementation architectures, performance optimization strategies, and the role of service differentiation in enhancing interplanetary data transmission.
Innovation from Within: The Story of Meta’s Host Network Interface and fbnic Driver
Alexander Duyck’s presentation on Meta’s Host Network Interface (HNI) and fbnic driver revealed Meta’s efforts to innovate within their data center networks. By developing a lightweight NIC and driver, Meta aims to overcome the limitations of traditional NIC designs, achieving better network visibility, reliability, and performance. The session covered the motivations, design challenges, and benefits of this in-house development, showcasing Meta’s commitment to cutting-edge networking solutions.
PSP Crypto Protocol and HW Offload
Anjali Singhai Jain and her team introduced the PSP crypto protocol, developed by Google, and discussed its hardware offload implementation by Intel. PSP is designed for scalable encryption in transit, leveraging modern NICs to offload cryptographic operations. The talk covered the protocol’s architecture, offload mechanisms, and performance advantages, highlighting its potential for enhancing data security in high-throughput environments.
AMP to Reduce Network Jitter
Satish Kumar and Fam Zheng’s talk on AMP (Asymmetric Multi-Processing) strategies aimed at reducing network jitter in data center applications. They presented findings on how reserving CPUs for the kernel network stack and isolating applications to other CPUs improved cache efficiency and reduced context switches. Their case studies demonstrated significant performance gains, offering a compelling approach to optimizing network performance in distributed applications.
Tailoring eBPF Maps for DDoS Protection
Ivan Koveshnikov discussed enhancements to eBPF maps for more effective DDoS protection. He covered strategies to handle aggressive insertions, preallocated space exhaustion, and race conditions during attacks. The session detailed the modifications made to improve scalability and performance of eBPF maps under attack conditions, ensuring robust protection against DDoS threats.
Transparent Shared Memory Communications with eBPF
Cong Wang and A K M Fazla Mehrab proposed a novel approach using shared memory with eBPF to optimize communication between co-resident VMs and containers. By bypassing the TCP/IP stack, their method reduces overhead and enhances performance. The session highlighted the use of eBPF maps and sockops infrastructures, providing a transparent and efficient solution for inter-VM and inter-container communications.
Benefits and Drawbacks of Syscall Hooks
Hajime Tazaki and Kenichi Yasukata’s talk on syscall hooks examined their historical development, current capabilities, and limitations. Syscall hooks allow alternate kernel functions to be executed, providing benefits for kernel emulators and other applications. The session discussed the challenges of achieving low overhead and complete hookability, offering insights into future improvements and applications of this technology.
Conclusion
The Netdev 0x18 conference in California concluded with a wealth of insights and innovative solutions that promise to shape the future of Linux networking. Key sessions highlighted the significant strides being made in TCP tuning, efficient network processing techniques, and the development of new protocols and algorithms. From tackling the challenges of high throughput and diverse network behaviors to exploring architectural considerations for interplanetary networking, the event underscored the community’s dedication to overcoming complex networking issues. Discussions demonstrated the depth of research and collaboration, but also a deeper understanding of the cutting-edge developments poised to enhance network performance, reliability, and scalability.
RELIANOID’s shown their commitment and continuous support to advancing Linux networking by sponsoring one more year the Netdev Conference. Moving forward, RELIANOID’s ongoing involvement will be crucial in implementing the breakthroughs discussed at the conference, ensuring that the Linux networking ecosystem continues to evolve and thrive.
Related Blogs
