The Rust Starlark library

We are pleased to introduce you to our open source Rust library for working with the Starlark programming language, including parser, evaluator, linter and debugger.

Starlark as the configuration language

Starlark is a programming language for configuring applications. It sits between a full programming language (like Python) and a configuration language (like JSON or YAML) and offers expressiveness and abstraction, but with simplicity and determinism. As an example of using Starlark, consider the buck-build system. A simple buck build file written in Starlark looks like this:

android_library (name = ‘activity’, srcs = glob ([‘*.java’]), Visibility = [ ‘PUBLIC’ ],)

It’s not hard to imagine what that might look like in JSON. However, once you have 20 rules, you may want to abstract out general details, such as: For example, imply visibility or add some additional standard arguments. You may want to use some boolean arguments or read configuration options. While formats like YAML / HOCON add limited abstraction, sometimes you need to move on to something a little more powerful. Starlark offers functions and modules in addition to a rich expression language.

But why not switch entirely to a programming language like Python? For a build system, we want three properties that Python does not offer:

• We want to evaluate all imports in parallel so that we can get answers more quickly.
• We just want to reevaluate changed imports so that less has to be recalculated.
• We want determinism so that all users see the same configuration every time, which enables caching and predictability.

As an example of a potential problem with Python, consider a module that, when evaluated, increments a global variable – which would corrupt all of the above properties. Starlark’s elegant solution is that after a module has been rated, its values ​​are frozen so that they are never mutated again. When you create a global variable, the execution of the module that defines it is frozen – and therefore immutable and no longer problematic. This freezing trick, combined with some simplifications compared to normal Python, gives us Starlark.

Starlark in Rust

Google uses Starlark in its Bazel build system and has written a specification for the language. Google engineers have also written three implementations in Go, Java and Rust. Facebook is starting to use more Rust, and configuration is always an important part of any project. So, of course, the Rust Starlark library was very attractive. Thanks to the beauty of open source, we were able to split up and improve on Google’s Starlark Rust library by sharing our branch of the library. Working with Google and the Starlark Rust authors, Facebook is now the manager of the Starlark Rust library, so Rust authors can easily embed Starlark into their projects.

For the Starlark Rust library, starting from the open source base, we focused on four dimensions:

• Ease of use that provides handy features such as procedural macros to define Rust code that Starlark can call. We’ve also thoroughly documented and tested the library to help new users get the ropes in hand.
• Compatibility with the aim of following the Starlark standard where possible and working with the upstream specification where not possible (e.g. Rust’s strings do not allow invalid UTF8, but Starlark does). We have significantly improved compatibility and would like to support new features such as floats and byte strings in the near future.
• Performance to keep configuration costs down. We’ve invested in a faster interpreter, a faster lexer, garbage collected values, and lots of performance optimizations for critical operations like manipulating strings and creating dictionaries.
• Features, with the hope of making Starlark a delightful experience for users. We added an IDE integration mode (with LSP), a debugger (with DAP) and a linter. We’re still experimenting with new features, like guys in Starlark.

This project was only possible thanks to the open source availability of the Starlark specification and implementations. In particular, we owe our thanks to Damien Martin-Guillerez, who wrote the first implementation. We’re excited to share our Starlark implementation with the Rust community and welcome feedback, users, and contributors.

Developer infrastructure information

Facebook’s Developer Infrastructure team (DevInfra) owns most of the coding lifecycle, from the time code an engineer has in mind to the people who use our apps. Our mission is to increase developer efficiency so we can continue to deliver fantastic products quickly. We lead the industry by creating innovative developer tools and automation infrastructures that are reliable, fast, and ensure that every second of engineering time is spent on the things that matter.

Would you like to work in the infrastructure on Facebook? Take a look at our vacancies on our career page for infrastructure.

To learn more about Facebook Open Source, visit our Open Source website, subscribe to our YouTube channel, or follow us on Twitter and Facebook.