Way back in early 2014 we had a blog on this topic but it could use some updating, so let’s tackle that.
If you’re just joining us, take a step back and review the first installment of our series on types of flash storage about the different drive types.
What’s fast, and what’s slow, has always had some generalizations associated with it. For example, SAS is fast, SATA is slow, flash is fast regardless, and nowadays NVME is always “the fastest thing ever.” All that isn’t really true — these are just the interconnects and performance and speed can vary.
SAS vs SATA
Serial Attached SCSI [Small Computer System Interface] (SAS) is nearly identical to the SCSI drives that we’ve used for over 30 years in servers; the only difference is that the same commands are now passed serially over a different, higher bandwidth medium 12 Gbits is the current standard, and 24 Gbits is already in the wild. Serial attached ATA (SATA) is nearly identical to the ATA drives (IDE) that we’ve used for over 30 years in desktops, except, again, it’s over a different medium. As opposed to SAS, SATA topped out at 6 Gbits, and there isn’t any plan to create a higher bandwidth option. Long term, SATA is essentially dead.
These two mediums are inter-operable, which causes people to think that the two technologies are basically the same when they aren’t. SCSI was made to support multiple devices on a single bus, whereas ATA was made for very few devices on a bus. Because of this, ATA does not play well with others, and does not accept commands from multiple places very well.
There have been advances to optimize things over the years, but it is only glossing over the issue that the ATA protocol is inferior to the SCSI protocol in multiple device environments. If there are only two or three spinning or low-performance flash drives, the issue may be negligible. If there are five, 10, or 100 drives, then the issue can have a HUGE impact on performance. Additionally, with SATA topping out at 6 Gbits, it can create some real bottlenecks in your environment when inter-operating with 12/24 Gbit SAS.
NVMe
NVMe is a newer technology made explicitly for flash storage. It connects directly over the PCIe bus at high bandwidth and has a different signaling method that is significantly lower latency than either SATA or SAS, which increases overall performance. This is largely due to the fact that NVMe assumes it always has an extremely fast, always-on connection that won’t be disrupted. This allows it to skip a lot of checks and balances of shared interfaces, and a lot of the wait timers associated with slower connections. The same drive on NVMe will often be much more performant than that drive on the other interfaces. However, a slow QLC drive with very little cache will still be slow over NVMe, even though the interface is faster. Make sure that you’re looking at the whole drive AND the connectivity, not just one or the other.
One of the big things to keep in mind with NVMe is that only specially designed arrays/controllers can take advantage of the NVMe spec. You can’t simply put NVMe drives into a server and expect to put them in a RAID 5. By default, being on the PCIe bus, NVMe drives talk directly to the CPU instead of to a RAID controller like SATA/SAS disks. Dell and a few others have recently announced hardware RAID cards for NVME, but they are not widely distributed (more on this in a later installment). Pay careful attention to how the configuration of those drives will be handled. This takes us directly into managing drives themselves.
That’s the basics on storage interfaces! Next week we’ll talk about how drives are managed and how that impacts their resiliency and performance.