Tuesday, September 24, 2013

vSphere Flash Read Cache (vFRC)

Flash Reads Cash...
Pretty awesome alterations and additions are implemented in VMware's latest vSphere release, vSphere 5.5. Amongst the additions are items like a newly revised and much easier to deploy version of SSO, VMware vSAN, improved vCloud services and a lot of other features. A couple stood out to me and one in particular, vSphere Flash Read Cache, was an awesome and much needed addition. I spent some time researching and got into the nitty gritty details of how it works and what someone using vFRC could expect.

vFRC is actually a hypervisor-like layer that sits on top of physical caching devices installed in ESXi hosts, devices like SSDs or PCI caching cards like FusionIO. Whether you have one caching device or mulitple, vFRC aggregates these devices and creates a cache pool that then can be allocated to Virtual Machines on the ESXi hosts (devices are not required to be placed in the cache pool). This in essence presents a situation similar to a pass-thru device and passes thru the caching capabilities to the VMs VMDKs (virtual hard disks).

The vFRC pooled resources are assigned on a per VMDK basis and each assignment is granular to the point of assigning a cache size space and cache block size per VMDK, this allows for portional assignment of cache resources per VMDK if you have limited caching hardware in your hosts. Some items to keep in mind:
  • Space allocated in the cache pool for VMDKs is thick provisioned and reserved, so allocate what you know your VM/VMDK will actively use.
  • Sizing your cache block size is extremely important, its obvious to say but size your cache blocks to match your workload block size. For instance if you were allocating a portion of the cache pool to a SQL Server you possibly be setting your cache block size to 64K.
  • vFRC is best used and the most performance increase is gained with data that is high re-read (i.e. databases) 
  • Know your workload, some performance monitoring and evaluation will go a long ways when trying to fairly and accurately size the cache amount allocated and block size.
  • Your cache block size affects your system memory too, vFRC keeps cache metadata in memory for quick access. The smaller the cache block size, the more metadata is read in and kept in memory.
  • By default VMs that have vFRC assigned to any of their VMDKs are set to vMotion cache with the system memory too. This will increase vMotion times but will keep the cache "warm" instead of evicting all cached data to simply rebuild once it moves. During warm ups, performance will potentially be degraded compared to a full warm cache.

Sizing your cache and cache block size is also important to acheive more hits (reads from cache), less misses (reads that had to go to disk because the data wasnt in cache) and less evictions (when data is pushed from cache to allow for other data to be read into cache). If you have your block size configured improperly, large blocks of data stored in 64K cache blocks could potentially be evicted so smaller 4 or 8K blocks blocks of data can take that space in cache. When a smaller data block fills a larger cache block the remainder of the space not used in the cache block by data is marked as invalid and cannot be used until the smaller amount of data is eventually evicted. This type fo scenario can kill performance and cause misses (bypassing cache to go to disk for data). During every of data if it is not found in cache, a miss occurs, and the data is then asyncronously read into cache then delivered to the requesting party.

Two points that are really cool: vFRC can (and most likely will) improve performance for VMs that are not actually utilizing vFRC, this occurs because vFRC is alleviating I/O requests from the storage system where other VMS share the storage, which allows them to find more "time" getting I/O requests filled. The other cool thing is the algorithm used to to evict data from cache. Some caching systems use a round robin effect and evict data on a systematic "You're next because your next in line" system, vFRC is intelligent enough (as are some other caching systems) to instead evict data based on a least used model, so data in cache that was accessed the least is evicted when space is needed.

Finishing up here, I recommend reading up on types of SSDs and Caching systems, SLC (Single Level-Cell) is faster than MLC (Multi Level-Cell), and PCIe based caching systems are normally faster than drive based SSDs. VMware recommends PCIe based caching cards with SLC flash and a deep queue depth, 256 or above. This is an awesome new feature to add to vSphere, I am all sorts of excited about it! Good caching all!