What video infrastructure analysis appears to be like like at Fb
Research tells about everything we do on Facebook, from improving real-time augmented reality experiences to keeping people safe on our platforms. In the video infrastructure space, Facebook’s video quality and research team is working on technological challenges arising from implementing video on a large scale.
Researchers in video infrastructure are working on two fundamental and related problems: (1) how can video encoding efficiency be improved or how fewer bits can be output to compress a given video at a given quality, and (2) how can video quality be improved accurately measured, or how to predict the perception of video quality by a viewer through automated algorithms. An important limitation in their work is that given the billions of Facebook and Instagram videos, both tasks must be performed with the greatest possible computational efficiency.
We have a top team of video infrastructure engineers and researchers, some from industry and some directly from PhD programs in video processing at leading research universities. To learn more about what the video quality and research team has made so far and where to find their research, we spoke to Ioannis Katsavounidis, Research Scientist for Video Infrastructure at Facebook.
Video @Scale 2019 and SPIE 2020
With the quality keynote from Katsavounidis at the Video @Scale 2019 event, the team presented the concept of the convex hull with computational efficiency / compression efficiency. This way, different encoders with even different coding standards like AVC, VP9 and AV1 can be compared and prioritized for videos of different popularity.
At the latest SPIE conference on digital image processing applications, which took place online in August 2020, Ioannis invited researchers from industry and science to contribute to a special session on energy-efficient video processing. There were a total of 18 articles covering all aspects of video processing, including video quality metrics, video encoders, and software and hardware architectures. Among those special meeting papers were four posts from two different teams on Facebook:
- Efficient measurement of quality in the Facebook video ecosystem by Shankar Regunathan, Haixiong Wang, Yun Zhang, Yu (Ryan) Liu, David Wolstencroft, Srinath Reddy, Cosmin Stejerean, Sonal Gandhi, Minchuan Chen, Pankaj Sethi, Amit Puntambekar, Mike Coward, and Ioannis Katsavounidis (video infra)
- Quick selection of coding parameters for video coding with convex hull by Nick Wu, Volodymyr Kondratenko and Ioannis Katsavounidis (Video Infra)
- Optimizations for the cross codec coding for video transcoding by Gaurang Chaudhari, Hariharan Lalgudi and Harikrishna Reddy (our sister team, HW ASIC)
- Hardware acceleration of video quality metrics by Deepa Palamadai Sundar, Visala Vaduganathan and Cindy Chen (HW ASIC)
“All of these articles, as well as others presented at SPIE, represent important steps towards our goal of ensuring the highest possible video quality for Facebook videos within the performance and physical capacity of our data center,” says Katsavounidis. “Participants responded positively to the taped video presentations of each paper, and we received a lot of engagement that continues to this day.”
ICIP, Video @ Scale and more
The team continues to conduct research in this area and share the results with the academic community. A current example is the IEEE International Conference on Image Processing (ICIP) 2020. “We are proud to be platinum sponsors of this flagship conference,” says Katsavounidis. “At the conference, we held an industry workshop on efficient video compression and quality measurement on Facebook where we talked about the types of problems and the solutions we were looking for.” Check out the entire workshop below.
Katsavounidis and others announced brief updates and research highlights at the recent Video @ Scale event on October 22nd and will follow up their energy efficiency research during the Picture Coding Symposium 2021. “I am a general co-chair and the team is working to have a number of contributions made to this conference that we are very excited to receive,” said Katsavounidis.
Stay up to date
Facebook’s video quality and research team plans to continue its commitment to the Picture Coding Symposium 2021 and the SPIE and ICIP 2021. All updates will be published on our blog. Information on all Facebook sponsored conferences can be found on our events page.