Caching plays an important role in helping people access their information efficiently. For example, when an email app loads, some messages are temporarily cached so that the user can refresh the page without the app getting the same messages. However, large caching has long been a complex technical challenge. Organizations need to balance the speedy experience people expect from caching with maintaining high performance and cost-effectiveness in their systems. Traditionally, each cache implementation is created and managed independently by different development teams. This approach is inefficient because it ignores the common challenges of different caching systems from deployment to maintenance.
As traditional dynamic random access memory (DRAM) becomes more expensive and requires more power to scale, companies like Facebook are examining hardware options such as non-volatile memory (NVM) drives to expand their caching systems. This hybrid DRAM and NVM model is an advancement, but innovative caching designs are required to realize the full potential of the hybrid cache. This also includes new caching heuristic research that needs to push the boundaries of conventional systems by caching the relevant content for the correct duration. We have consolidated these innovations and taken them one step further through collaborations and open source work.
Today we announce the release of. known CacheLib, a pluggable in-process caching engine to Build and scale powerful services collaborative. CacheLib’s C ++ library enables developers to create and customize scalable and concurrent caches through its simple API. We are also open sourcing CacheBench, a benchmarking tool for assessing caching performance for different production workloads.
CacheLib is used as an in-process cache in more than 70 major systems on Facebook, including the Social Graph, Content Delivery Network, Storage and Look-Aside-Key-Value-Caches. This existing size and the potential for the introduction of open source make CacheLib an aggregation point for optimization and CacheBench an effective benchmarking tool for evaluating new ideas for various caching applications.
Enable innovation through partnerships
As an open source platform, CacheLib and CacheBench have the potential to become an industry standard for caching innovation and benchmarking. To date, our collaborations with research universities, hardware manufacturers and software companies have produced significant results that show the value of this toolkit.
Over the past two years, we’ve partnered with many well-known companies to push the boundaries of caching innovation. Today we are working with Twitter on the integration of CacheLib into Pelikan.io to activate SSDs for caching objects within the Twitter infrastructure. Pinterest is evaluating the introduction of CacheLib into its machine learning infrastructure systems to improve predictive performance and system stability.
In academia, researchers at Carnegie Mellon University, Princeton University, and Yale University use CacheLib and CacheBench to prototypical research ideas. By evaluating their prototypes against the industry’s caching workloads, these researchers can replicate their projects much faster and more accurately than before.
We have also worked with hardware industry partners such as Intel, KIOXIA, Samsung and Western Digital Standardize and improve SSD technologies that enable improved caching solutions. This work is now part of the Open Compute Project (OCP) NVMe Cloud SSD specificationthat we discussed in this webinar. This specification, along with CacheLib, will help adapt future NVM technologies for caching workloads across the industry.
CacheLib and CacheBench have enormous potential to shape the future of caching thanks to their developer-friendly API, access to many benchmark workloads across the industry, and the collaborative nature of open source. We are grateful for the support and input from our partners in using the platform to drive innovation in such an important and complex area. We are offering this work as open sourcing to make the future of caching a more collaborative and open space for sharing across the industry. Read more about how you can contribute to the project here.
Comments are closed.