Meet the Rustaceans: Daniel Xu

For today’s interview we have Daniel Xu, a production engineer on the Linux userspace team at Facebook. The Linux userspace team works on a variety of system tools, libraries, and services, all of which deal directly with the Linux kernel. While working on this team, Daniel has used Rust as one of the main development languages. Most of his work these days focuses on observability, especially dynamic tracers and debuggers. Let’s hear from him about his experience with Rust and learn more about his work.

In what capacity did you use Rust?

I have used Rust for the past year and a half writing services, libraries, and standalone tools to solve low-level problems, e.g. projects have high requirements for correctness and any errors would cause problems that would be very difficult to debug or require special knowledge for the end user.

Why did you / your Facebook team prefer Rust to other languages?

I’d heard all the fanfare surrounding Rust and of course I was hesitant – choosing a programming language for a non-trivial task is an important design decision. Wrong choice can lead to frustration, technical debt, or worse. Fortunately for me, my colleague, who is a long-time systems expert and C ++ programmer, was also a fan of Rust and encouraged me to try it out. On her advice, I took a leap of faith and started writing Below, an interactive tool for viewing and recording historical system data, along with a few other enthusiastic colleagues.

On paper, Rust seemed like a natural choice. Downstairs had to parse a lot of text data, had high uptime requirements and had to be efficient. These factors play a role in Rust’s strengths. Rust’s focus on memory security by default and correctness-when-compiled allows us to move faster and be more confident that we are writing correct and safe code. The deal is made even more sweeter by the fact that Rust compiles to native code and has a relatively light runtime compared to Python / Go / Java – every bit of efficiency counts when Below is running on millions of hosts.

What projects have you been working on on Facebook that use Rust?

Below is one of the most important projects I’ve worked on in Rust. Below is a system monitor: Think of it as an interactive process viewer like top or htop, except that it records historical data to disk and is also cgroup capable. Engineers across the company rely on Below to troubleshoot system problems on each of the millions of bare metal hosts the company manages.

Another project I wrote in Rust is libbpf-rs, which provides secure and idiomatic bindings to libbpf (a C library) that allow Rust developers to work with the extended Berkeley Packet Filter (eBPF). This project was developed outdoors and is currently being used by a number of outside developers. To learn more about libbpf visit my blog where I have written in more detail on the subject.

Two other fairly niche projects I’ve worked on are btrfs-fuzz and btrd. Btrfs-fuzz is an unattended, coverage-controlled fuzzer for btrfs, a next-generation Linux file system, while brtd is a file system debugger that allows file system developers to view the data on disk directly without looking at literal ones and zeros.

With all of the above projects, I was incredibly surprised at the care with which the Rust developers went to develop the language. I have used Rust as both an application developer and a library developer. As an application developer, I loved how to write relatively powerful code by writing idiomatic rust. that idiomatic Rust looks and feels pretty much like a scripting language; and the fact that when your code is compiled you are in pretty good shape means system programmers can have really nice things too.

As a library developer, I loved how Rust provided powerful ways to create abstractions that resist abuse. Ownership and lending are concepts system programmers are very familiar with. Getting them into the language and enforcing them at compile time is a huge step forward in writing correct code.

What do you think of Rust’s growth path on Facebook?

I feel pretty good about Rust’s trajectory. Facebook owns a lot of physical hosts and our engineers face all sorts of problems related to systems. Rust is being received pretty well internally and is getting more and more attention. I believe there is and will remain enough in-house expertise for other teams to confidently choose Rust for their next project.

How do you think Rust will grow as a language in 2021?

I think Rust is nearing the height of its hype cycle or has passed it. Personally, I think that’s a good thing. There is an old saying, “There are only two kinds of languages: those that people complain about and those that no one uses.” I don’t necessarily think complaining is a good sign, but I think it means people are starting to look more critically at Rust. I think that’s a good thing: nothing is perfect and something can only be improved through feedback.

Some people who have used Rust really like it. Why do you think that is the case and what is your favorite feature of Rust?

It depends a lot on what a person is doing and what room they are working in. For me, I love the compile time guarantees. Coming from a C ++ background, I’m used to (and love) using (and loving) ASAN (AddressSanitizer) / TSAN (ThreadSanitizer) and all the other disinfectants. What Rust really sold me was when someone told me that Rust doesn’t have disinfectants (yet) because everything is forced by the compiler at compile time.

Where can people find out more about Rust and how can they start contributing to it?

I think the best way to learn a programming language is to start writing code. Write some non-trivial code and just search Google for all of your questions.

As far as collaboration is concerned, you don’t necessarily have to be familiar with programming languages. Almost all open source projects urgently need bug reports and testers. The easiest way to get started is to try libraries, nightly builds, and tools, and report any bugs you see.

We thank Daniel for taking the time to do this interview. It was very interesting to see how Rust is used as the primary language for building a wide variety of systems, tools, libraries, and services, and how we learn from all of the things we build here and contribute to the Rust community. We hope you found this interview helpful and it gave you some insight into how and where Rust is used on Facebook. Look out for more interview blogs where we meet with many more engineers and hear their views on the subject.

Check out our previous blogs from the Meet the Rustaceans series:
Meet the Rustaceans: Chris Konstad
Meet the Rustaceans: Pedro Rittner
Meet the Rustaceans: Neil Mitchell

About the Meet the Rustaceans series

Rust has been ranked the “Most Popular” language consistently for the past 5 years and we at Facebook believe Rust is a standout language that shines and is widely used on critical issues like storage security, performance and reliability by projects here. We joined the Rust Foundation to help improve and grow Rust, which not only strengthens our commitment to the Rust language, but also to the sustainable development of open source technologies and developer communities around the world.

This blog is part of our Meet the Rustaceans series where we invite engineers and developers who use Rust regularly to share their experiences and tell us about the amazing products they are building with Rust here on Facebook. Keep an eye out for more interview blogs where we meet many more engineers and hear their thoughts on the subject.

To learn more about Facebook Open Source, visit our open source site, subscribe to our YouTube channel, or follow us on Twitter and Facebook.

Interested in working in the field of production technology on Facebook? Here you will find our job advertisements on our career page Production Technology.

Comments are closed.