Over the years, people have talked about the potential of remote direct memory access (RDMA) to greatly accelerate application performance by bypassing the CPU and enabling direct access to memory. But there was a notable roadblock in this route to low-latency networking: slow storage media.
More specifically, with the slow speeds of widely used spinning disk and the relatively high cost of DRAM, there wasn’t a compelling reason for application developers to use RDMA for general purpose, distributed storage. Storage was basically a bottleneck in the I/O pipeline, and that bottleneck had the effect of negating the need for RDMA.
Now fast forward to 2015 and the arrival of a new generation of lightning-fast non-volatile memory (NVM) technologies, such as the upcoming Intel® Optane™ technology based on 3D XPoint™ memory. These new technologies are going to obliterate the storage bottlenecks of the past.
Consider these metrics from a fact sheet (PDF) from Intel and Micron, the joint developers of 3D XPoint technology:
- HDD latency is measured in milliseconds, NAND latency is measured in microseconds, and 3D XPoint technology latency is measured in nanoseconds (one-billionth of a second)
- 3D XPoint technology is up to 1,000x faster than NAND
- In the time it takes an HDD to sprint the length of a basketball court, NAND could finish a marathon, and 3D XPoint technology could nearly circle the globe.
So how do we make use of these revolutionary storage innovations?
As a first step, we need to remove the bottlenecks in storage software that was written for the era of spinning disk. The assumptions about storage speeds and memory access built into legacy code no longer apply.
After that problem is fixed, we need to move on to the networking side of the equation. With the new generation of NVM technologies, storage performance has leapt ahead of networking performance—at least when using common networking technologies. This evolutionary change in storage creates the need for the speed of RDMA, which does network processing much more efficiently by enabling direct access to memory.
Removing the imbalance between NVM and RDMA isn’t an untested proposition. One big cloud service provider— Microsoft Azure—is already there. They prove the concept every day. They scale workloads out over distributed cores and exploit RDMA to offload cycles related to network processing. RDMA is one of their keys to achieving low latency and high message rates in bandwidth-hungry cloud applications.
If you are attending the SNIA Storage Developer Conference in Santa Clara this week, you will have the opportunity to explore these topics at various levels in presentations from Intel and Microsoft, among others. To learn more about RDMA, check out my pre-conference presentation where will explore RDMA and Four Trends in the Modern Data Center as well as presentations from Chet Douglas and Tom Talpey. I also recommend Bev Crair’s keynote on Next Generation Storage and Andy Rudoff’s talk exploring the Next Decade of NVM Programming.
Meanwhile, for a closer look at today’s new non-volatile memory technologies, including those based of 3D XPoint technology, visit https://www.intel.com/nvm.