FioSynth I/O benchmark and knowledge visualizer
What is it:
FioSynth is a benchmarking tool that can be used to automate the execution of storage workload suites and analyze results. It contains a basic set of block-level storage workloads synthesized from production I / O traces that simulate a variety of Facebook production services. This is useful for predicting the performance of a storage device in realistic production environments and to help optimize performance.
Traditionally, synthetic I / O benchmarking tools have been used to push flash drives and hard drives to their limits, with the drive with the highest I / O operations per second (IOPS) or throughput winning. Facebook applications rarely push the peak IOPS or throughput limits of our drives, but they are very sensitive to latency outliers. Because of this, the majority of FioSynth workload suites are limited to I / O patterns that typically run on Facebook hardware. The drives with the lowest outlier latency (P99, P99.99, P99.9999, etc.) while maintaining the expected I / O rates are considered the best performing option regardless of their maximum IOPS and throughput capabilities.
What it does:
FioSynth used fio, a flexible I / O workload generator. We designed FioSynth to be easy to deploy and run on a wide variety of storage devices and configurations, and to be easy to view results. FioSynth workloads are scalable and the results can be used to directly calculate the relative performance of direct attached storage with disaggregated storage (iSCSI, AOE, etc.), small capacity drives to large capacity devices, non-RAID to RAID etc. to compare.
Some of the specific improvements that FioSynth offers include the following:
- It contains dozens of flash drive and hard drive workload suites that represent the I / O profiles of a wide variety of services.
- Workloads can be scaled based on the available storage capacity and can be scaled per terabyte. This allows the technician to directly compare drives with different capacities.
- FioSynth analyzes the results of each workload suite and combines them into a single CSV file for easy viewing.
- In client / server mode, all workload suites can be run in parallel on multiple hosts. Results from individual servers are saved in their own CSV files, while the combined summary results are saved in a separate CSV file.
- Optionally, the tool can be used to collect device health logs before and after each workload. This is useful for calculating write gain (a value that represents the data physically written to flash media in relation to the data written by the host) for various workloads.
- In the Workload Suite definition files, there is a section for workloads that precondition the drives and a section for workloads that are used to measure drive performance. By default, the flash drives are preconditioned before measuring drive performance. However, preconditioning can be skipped through a command line option.
- The number of execution cycles can be defined by default in the workload suite definition file or via the command line to ensure repeatability of results.
Why it matters:
FioSynth has been used internally on Facebook for years to iFix performance inefficiencies in flash devices before they go live in the data center and reproduce functional problems previously observed in production. Something Flash device vendors have used this benchmark tool to optimize the performance of their own drives and to meet the performance goals set in the Open Compute projects NVMe Cloud SSD specification.
We made FioSynth available to the open source community to make it easier for storage device manufacturers to optimize their drives for hyperscale applications, to simplify the standardization of the flash preconditioning method, and to make it easier to run and visualize storage results in benchmarks.
Get it on GitHub: