Rsys: A smaller, quicker video calling library

  • We’re introducing a new video call library for all relevant products in our apps and services, including Instagram, Messenger, Portal, Workplace Chat, etc.
  • In order to create a library generic enough to support all of these different use cases, we had to rewrite our existing library from scratch with the latest version of the open source WebRTC library. This was an incredibly rare endeavor that involved multiple engineers from across the company.
  • Compared to the previous library, rsys can support multiple platforms including Android, iOS, MacOS, Windows and Linux. It’s about 20 percent smaller, which makes it easier to integrate with platform-constrained platforms like Messenger Lite. Rsys offers approximately 90 percent unit test coverage and a thorough integration testing framework that covers all of our major call scenarios.
  • This was achieved by optimizing libraries and architectures to binary size where possible, by dividing the parts required for the call into isolated, stand-alone modules and using cross-platform solutions that are not dependent on the operating system and the environment.

Facebook’s first version of video calling was written on a seven year old fork of WebRTC and specifically to enable native audio calls in Messenger. At the time, our goal was to give our users the most feature-rich experience possible. Since then, we’ve added video calling, group calling, video chat heads, and interactive AR effects. With millions of people using video calling every month, the library had become far more complex behind the scenes with all the features that looked simple at first glance. We had a large amount of Messenger-specific code, which made it difficult to support apps like Portal and Instagram. We had separate signaling protocols for group calls and peer-to-peer calls that required us to write features twice and created a huge gap in the code base. We also spent a lot more time keeping our WebRTC version up to date with the latest improvements from open source. Ultimately, we wanted to provide a reliable service for devices with low power consumption and in low bandwidth scenarios.

When we started thinking about how we could scale video calls more efficiently and easily for the future, we found that our best route was to redesign the library from the ground up and rewrite the entire library. The result is rsys, a video calling library that has enabled us to take advantage of significant advances in video calling since the original library was written in 2014. Compared to the previous version, rsys is around 20 percent smaller and available for all development platforms. With this new iteration, we’re rethinking our take on our video calling platform, starting from the ground up with a new client core and extension framework. This helps us to further develop our own cutting-edge technologies. The new code base is designed to be sustainable and scalable for the next decade. It lays the foundation for remote presence and interoperability between apps.

Smaller and faster

A smaller library will load, update, and start faster for the person using it regardless of device type or network conditions. A small library is also easier to manage, update, test, and optimize. When we thought about this new version, our maximum binary size had reached up to 20MB. We were able to get some reduction by shrinking some sections of code, but in order to get where we wanted to, we realized we had to rewrite the entire library from scratch.

The easiest way to get a smaller library would have been to remove many of the features we’ve added over the years. However, it was important to us to keep all of the most commonly used features like AR effects. So we stepped back and considered how to apply what we have learned over the past decade and what we know about the needs of the people who use our apps today. After exploring our options, we decided to look beyond the user interface and get into the infrastructure of the library itself.

We made various architectural decisions to optimize the size and implemented a plug-and-play framework Choose Selectively compile features into apps that need them and introduced a generic framework for writing new features based on flow architecture. We’ve also moved away from generic libraries with strong templates like folly and towards optimal libraries like Boost.SML Realize size increases in all apps.

In the end, we reduced the core binary size by about 20 percent from about 9MB to about 7MB. We achieved this by redesigning our functions to fit a simplified architecture and design. While we’ve kept most of the features, we’ll be adding more pluggable features over time. Fewer lines of code make the library lighter, faster, and more reliable. Thanks to an optimized code base, engineers can innovate faster.

One of our main goals in this work was to minimize code complexity and eliminate redundancies. We knew that a unified architecture would enable global optimization (instead of focusing every feature on local optimizations) and enable intelligent code reuse. To create this unified architecture, we made a few important changes:

  • Signaling: We have developed a state machine architecture for the signaling stack that can unify the protocol semantics for both peer-to-peer and group calls. We were able to abstract all protocol-specific details from the rest of the library and provide a signal component that is solely responsible for negotiating the common status between the call participants. By cutting duplicate code, we can write functions once, easily make protocol changes, and provide a unified user experience for peer-to-peer and group calls.
  • Media: We decided to reuse our state machine architecture and apply it to the media stack as well. This time, however, we’ve captured the semantics of open source WebRTC APIs. In parallel, we’ve been working on replacing our forked version of WebRTC with the latest, while keeping all product-specific optimizations. This gave us the ability to change the WebRTC version under the state machine as long as the semantics of the APIs themselves didn’t change significantly and we were able to set up regular pulls from the open source code base. That way, we can easily update to the latest features without any downtime or delays.
  • SDK: To get function-specific states, we used the Flux architecture to manage data and provide an API for calling products that worked similarly to the React JS-based applications that web developers are familiar with. Every API call leads to certain actions being forwarded via a central dispatcher. These actions are then processed by specific reducer classes and output model objects based on the type of action. These model objects are sent to bridges that contain all of the function-specific business logic and result in subsequent actions to change the model. Finally, all model updates are sent to the user interface, where they are converted into platform-specific view objects for rendering. This way we can clearly define a function that includes a reducer, bridge, actions and models, which allows us to make functions configurable for different apps at runtime.
  • THE: In order to make our platform generic and scalable for different products, we decided to abstract all functions that depend directly on the operating system. We knew that platform specific code for Android, iOS etc would be required for certain functions like building HW encoders, decoders, threading abstractions etc, but we tried to create generic interfaces for these so that platforms like macOS and Windows could easily be connected by providing various implementations through proxy objects. We also used heavily cxx_library in Buck to easily configure platform specific libraries for compiler flags, linker arguments, etc.

Rsys architecture for calling.

What’s next

Today, our calling platform is significantly smaller and can be scaled to accommodate many different use cases and platforms. We support calls that millions of people use every day. Our library is part of all major calling apps, including Messenger, Instagram, Portal and Workplace Chat. Building rsys has been a long way, and yet it won’t look or feel much different to the users of these apps. It will continue to be the same great calling experience that people expect and love. However, this is just the beginning.

The work we’ve put into rsys will enable us to keep innovating and scaling our calling experiences as we move into the future. In addition to building a library that will be sustainable for the next decade or more, this work laid the foundation for it Cross-app calls across all of our apps. It has also laid the foundation for an environment that focuses on the presence in the distance.

This work was made possible in collaboration with the customer platform teams. We’d like to thank everyone who contributed to rsys, especially Ed Munoz, Hani Atassi, Alice Meng, Shelly Willard, Val Kozina, Adam Hill, Matt Hammerly, Ondrej Lehecka, Eugene Agafonov, Michael Stella, Cameron Pickett, Ian Peterson and Mudit Goel for help with implementation and continued guidance and support.

Comments are closed.