Database researchers focus on analysis award alternative in next-generation knowledge infrastructure

On April 19, 2021, Facebook launched a call for proposals (RFP) on the next generation data infrastructure. With this RFP, which ends June 2, 2021, the Facebook Core Data and Data Infra teams hope to deepen their connections with the academic research community by finding innovative solutions to the challenges that remain in the data management community. For an inside look at the teams behind the RFP, we reached out to Stavros Harizopoulos and Shrikanth Shankar, who are leading the effort in their respective teams.
View RFPShankar is Director of Engineering on the Core Data team that creates and supports the online data serving stack for Facebook, and provides the databases, caches, and global distribution for Facebook, Instagram, WhatsApp, and more. Harizopoulos is a software engineer within the data infrastructure providing efficient platforms and end-user tools for the collection, management and analysis of Facebook data. In these questions and answers, Shankar and Harizopoulos contextualize the RFP by providing more background information on database research on Facebook. They also discuss what inspired this tender and where people can stay up to date on their teams’ activities.

Q: What is the database research like on Facebook and how has it developed over the years?

A: Facebook has a long history of contributing to the database space – Hive, Presto, RocksDB, and MyRocks are examples of innovative work that has begun within the company. The size we work at and the unique constraints of our workloads make many existing solutions impossible and offer a perspective that leads to new ideas. This has become increasingly true over the years as the company has grown and new challenges related to this scale have emerged. We aim to continue our tradition of building new, innovative database technologies.

Q: What is the goal of this RFP?

A: As businesses and organizations become increasingly data-driven and products and services continue to rely on information extracted from data, the need for a highly reliable, flexible, and efficient data infrastructure becomes even more important. Modern data infrastructure architectures stem from decades of database research, but recent trends and developments, such as the decoupling of computers and storage and the need to work efficiently on a global level, as well as the emergence of new use cases such as data science and machine workload learning, pose new challenges and opportunities .

With this RFP we are looking for innovative approaches to a range of problems that have the potential to define the defining characteristics of the next generation of data infrastructure. A lot of these issues aren’t unique to Facebook, and we’d like to learn more about the great research in the field and strengthen our relationships with science.

Q: How does this RFP fit into the overall picture of database research on Facebook?

A: The definition of the fundamentals of a data infrastructure that is reliable, resilient, flexible, efficient and powerful on a global level forms the core of database research at Facebook. However, our research efforts extend in different directions along the modeling, management and visualization of various data types, from structured data to machine-generated logs to time series data. We are innovative in areas such as data storage and indexing, query processing, data modeling, transaction processing and distributed systems as well as novel approaches to data protection and security in data management.

Q: What inspired this RFP?

A: While we share our experience by writing and publishing papers and, in turn, benefiting from any innovations in the database space, we’ve seen some ways we can improve this. Specifically, we have seen that certain areas may not be perceived as effective or important from the outside, even if they are critical for us. On our part, we recognize that the solutions we have or are considering may be limited by our specific systems and the history behind them. We started this RFP process as a way to collaborate with academia, highlighting specific issues and looking for innovative approaches to address these issues.

Q: Where can people stay up to date and learn more?

A: We actively participate in major database conferences such as ICDE, SIGMOD, VLDB and CIDR every year. This is where the academic community can turn to us with questions and ideas. We also do a large part of our work through open source. Here are some examples:

– –

Applications for the RFP for Next Generation Data Infrastructure will close on June 2, 2021. The winners will be announced the following month. Subscribe to our RFP email list to be informed of new research pricing and appointment notification opportunities.