SSD: Storage Caching and Storage Tiering

More and more SSD and flash memory storage solutions are coming to the storage market. Some of vendors use the fast storage as cache, some – as a tiered storage. Let’s look at the difference between these two approaches.

Storage Tiering

In Storage Tiering all data are distributed between different storage tiers:

• Tier 1 – SSD
• Tier 2 – FC or SAS HDD
• Tier 3 – NL SAS or SATA HDD

The “hot” data blocks are placed on the fast storage (SSD), all other data blocks are placed on HDDs. After the learning process which identifies what data is “hot”, the data blocks are redistributed between storage tiers. This process is running periodically by the administrator defined schedule.

Examples: HP 3PAR Adaptive Optimization, IBM Easy Tier, NetApp Flash Pool, EMC FAST

  

Storage Cache

What’s the storage cache? The cache is the interim fast device that transparently stores data so that future requests for that data can be served faster. After data is written to the cache it must also be written to the main storage (HDDs). The caching device keeps data temporarily.

There are two types of cache:

• Read-Only
• Read-Write

Read-Only – cache is not used for writes or is used only in Write-Through mode, where data must be written to hard drives before the storage system will send data write confirmation to the host. This type of caching is often used with MLC SSDs which provide excellent read performance but are not so good for write workloads.

Read-Only caching also can be used on the host side.

Examples: NetApp Flex Cache, Nimble Storage.

Read-Write – cache is used in Write-Back mode and the storage system sends the data write confirmation to the host before data is written to the main storage (HDDs). Data can be modified within the cache even without going to the main storage. This type of cache processes write intensive workloads and the SLC or flash memory should be used as caching device. As the Read-Write cache can contain data which is not written to the HDDs it requires the RAID or other protection.

Typically the storage controller placed memory is used as Read-Write Level 1 cache. And it’s protected by the mirroring between the storage controllers and the battery devices.

What’s the difference between caching and tiering approaches?

First of all, the cache contains a copy of data already placed on HDDs. And in case of the storage tiering the data is distributed between different storage tiers (SDDs and HDDs). The data redistribution between the storage tiers is resource-demanding process, that’s why data should remain on the defined storage tiers for quite a long time (usually hours or even days).

What’s better?

The Storage Tiering is good for the relatively sustained workloads, when the workload pattern is being changing not too often (remember that data blocks redistribution is a resource-demanding process). It works well for both read and write workloads.

The Storage Caching is good for unpredictable fast changing workloads. The cache provides much faster response for the workload pattern change. The Read-Only caching solutions are getting more and more popular because of effective usage of relatively inexpensive MLC SSDs.

The main idea is to understand how the proposed solution works and how it meets your requirements.

And also be careful, some solutions named as "caching" realize the tiered storage approach, and vice versa.

Mt. Rainer - the server-side caching solution from Qlogic

Qlogic announced today the Mt. Rainier project, the server-side caching solution leveraging internal server SSD and PCI-e flash memory.

There are three possible Mt. Rainer cards configurations:

• An adapter with a separate flash card and PCIe link using an x4 cable drawing 25W
• An Adapter with a SAS IO port daughter card which links to internal server SSD
• An Adapter with an SSD daughter card with all power coming from the PCIe slot - 50W

From the OS point of view, the Mt. Rainer adapter is installed as a single adapter which needs only one driver, no additional drivers for PCIe cards or SSD devices will be required.

Another interesting feature is the Shared Caching feature which makes the shared cache available across multiple servers, all transparent to the host.  The Qlogic storage router technology is used for this feature. Each Mt. Rainier adapter can see what the other cards in the SAN zone are doing and know which LUNs have been cached. The SAN protocols are used to synchronize this information.

For the high availability purposes the Cache Mirroring feature can be used. The Cache Mirroring provides the synchronous peer-to-peer mirroring with write-back and write-through modes support.  

One of the most important things is that cache memory management, Cache Mirroring and Shared Caching don’t require host CPU resources – all operations are performed on the adapter’s ASIC level.

The first Mount Rainier products should be available in the first half of 2013.

Now NetApp performs server-side caching too

In addition to existing caching technologies, such as Flex Cache and Flex Pool, August 21^st, 2012 NetApp announced the Flash Accel solution which speeds application performance using server-side cache for storing frequently accessed data working with NetApp storage arrays.

The Flash Accel is the software leveraging the existing SSD and PCIe flash memory hardware installed into the servers. Will be supported the SSD and PCI-e flash memory products of Server Partner Caching Alliance members: Fusion-io, LSI, Micron, QLogic, SanDisk (FlashSoft), STEC and Virident.  Also the first version of Flash Accel will support Microsoft Windows Server 2003 and 2008 and VMware vSphere  5. (including DRS, HA and vMotion support).

The Flash Accel provides the following features:

• End-to-End Intelligent Data Coherency: if data blocks are changed on the backend array the Flash Accel will update the local cache accordingly. 
• Cache Persistency and Durability: Flash Accel preserves cache content consistently with applications in the event of server and VM reboot and crashes. After the server restart the cache content is checked and updated as the server is back on, what allows to have a warm cache after restart.

It’s also interesting that competing with other server-side caching solutions such as HP flash cache in Gen8 servers and EMC VFcache the Flash Accel is looking more open allowing to use hardware from different SSD and PCI-e flash memory vendors. 

But anyway, we shouldn’t also forget about other server-side caching solutions which are external storage system independent, e.g. FlashSoft from SANDisk , Nytro XD from LSI or ioTurbine from Fusion IO.