December 11, 2019
Using the NVMe standard in storage infrastructure requires three things: internal connections between storage systems, internal connections between storage systems, and interconnections to physical servers.The first point of adoption will be in the storage system itself.As vendors update the hardware on which they run their storage software, part of the update will be to leverage the internal NVMe.Using NVMe in a storage system is the most seamless of the three and can provide the most direct return.
Why use an NVMe on a storage server ?
Even with an all-flash system, the storage server eventually becomes a bottleneck.The number of servers can be expanded, and virtual machines on these servers can increase the density to create additional IO.Each server can have multiple network connections, and switches can be extended to support all these connections.However, at the other end of the network switch is the storage system, whose network resources are limited.
The network shortage, while not a network connection from the storage system to the switch, is internal to the storage system itself.Traditionally, this network is based on serial connection SCSI (SAS), which has a limited set of commands and queues that can be supported.
SAS - based all-flash arrays basically have to process each IO sequentially.Although flash allows this serial IO processing to occur very quickly, it can still only be performed once at a time, meaning that even adding SAS bandwidth won't have much of an impact.On the other hand, NVMe can handle thousands of IO simultaneously, which means that with the same bandwidth, it can handle more IO requests from multiple hosts faster than SAS.
Because most data centers have an increasingly mixed IO workload and a large number of parallel IO streams, they are well suited to NVMe.When most all-flash environments are refreshed to an nvme-based system, performance per workload may increase if the workload is IO intensive.More importantly, these environments will see their ability to further expand the data center with the same number of storage systems.
How do I put an NVMe into a storage server?
Most all-flash vendors are moving from SAS based all-flash to nvme-based pure systems.Because of the high price of NVMe drives and components, they will continue to keep SAS in their product line so that they can support customers who do not require extremely high performance NVMe.As a result, many data centers may have two systems, nvme-based all-flash for IO intensive workloads and SAS -based all-flash arrays.
Another approach is to create a new hybrid array.Hybrid arrays were popular in the early days of flash memory.They integrate flash-based storage with hardware-based storage, automatically moving data between the two storage types and IO requirements as needed.
A modern implementation of hybrid design is the combination of nvme-based flash memory with SAS -based flash memory.The advantage is that the vendor can provide a single system that efficiently meets the needs of all workloads.They can also load SAS based storage with more capacity than is supported by SCSI connections, because most of the actual IO will be on the NVMe, while SAS SSDS are used for less active data.
The concern with flash /HDD hybrid designs is that if the requested data is not on flash, but on the hard drive, it can affect performance.The possibility of significant performance changes is the reason why many organizations have adopted all-flash.The NVMe/SAS hybrid design has no such concerns because not only is the media type flash, SAS is not busy because it is no longer responsible for all IO.This means that requests from the SAS layer almost have private access to the SAS bus, and because it is a flash medium, it immediately responds to IO requests.
The NVMe in the storage server will cause most data centers to encounter an NVMe for the first time.While these organizations may not need to add NVMe performance on a per-workload basis, nvme-based systems should allow them to extend the storage system to accommodate more workloads by mixing more IO types.Although it is important that all-flash vendors design these systems, they are now affordable and suitable for the entire data center.Organizations don't need niche products that solve only one problem.