rust-lld: How It Can Give You Faster Linking Times

rust-lld: How It Can Give You Faster Linking Times

Estimated Reading Time: 7 minutes

• rust-lld is now the default linker for x86_64-unknown-linux-gnu on Rust nightly builds, significantly reducing linking times.

• It’s an optimized version of LLVM’s lld, packaged directly with rustc, built for modern multi-core architectures.

• Expect dramatic performance gains: up to 7x faster linking for projects like ripgrep, resulting in a 40% reduction in total compilation time.

• While largely seamless, developers can easily revert to the system linker using -Zlinker-features=-lld if any specific compatibility issues arise.

• This rollout on nightly, starting with nightly-2024-05-18, aims to gather crucial real-world feedback for broader adoption.

• The Hidden Bottleneck: Understanding Linking Times
• Enter rust-lld: Rust’s In-House Solution for Speed
• Unlocking Unprecedented Speed: The Benefits of rust-lld
• Navigating Potential Hurdles and Taking Control
  • Actionable Steps to Manage rust-lld:
• Summary
• Share Your Feedback!
• Frequently Asked Questions

In the world of Rust development, compilation speed is a frequent topic of discussion. While the Rust compiler, rustc, has seen tremendous performance improvements over the years, there’s often one phase that remains a significant bottleneck: linking. This crucial step, where compiled object files are combined into a final executable or library, can disproportionately inflate build times, especially for larger projects.

Good news is on the horizon for Rust developers. A substantial leap in compilation efficiency is being rolled out thanks to rust-lld. This powerful new default promises to drastically cut down on those frustrating waiting periods during development.

TL;DR: rustc will use rust-lld by default on x86_64-unknown-linux-gnu on nightly to significantly reduce linking times.

The Hidden Bottleneck: Understanding Linking Times

When you compile a Rust project, rustc doesn’t just turn your source code into a runnable program directly. Instead, it generates intermediate object files, and then these object files need to be “linked” together. This linking process involves resolving symbol references, arranging code and data, and preparing the final executable or shared library. It’s an indispensable part of the build chain.

Traditionally, rustc would invoke the system’s default linker. These linkers, often integral parts of popular operating systems, perform critical functions, ensuring stability and backward compatibility. However, many were designed in an era where single-core processors were the norm, leading to less-than-optimal performance on modern, multi-core machines. This legacy design often means they struggle to leverage today’s parallel processing capabilities, becoming a significant time sink.

Consider a practical example: building ripgrep 13 in debug mode on Linux. Roughly half of the total compilation time for this substantial project was spent solely in the linker. This illustrates just how much of an impact the linking phase can have on overall development cycles and productivity. Developers have long sought ways to circumvent this, often resorting to alternative, newer linkers like LLVM’s lld or Rui Ueyama’s mold, which are built with modern system architectures in mind.

Enter rust-lld: Rust’s In-House Solution for Speed

Recognizing the substantial performance gains offered by modern linkers, the Rust compiler team has taken proactive steps. rust-lld isn’t an entirely new linker from scratch; it’s a version of LLVM’s highly efficient lld linker, carefully packaged and shipped alongside rustc when you install it via rustup. It’s named rust-lld specifically to prevent any conflicts with a system-wide lld installation you might already have.

The concept of integrating lld isn’t new to Rust. Some of Rust’s specialized targets, such as those for WebAssembly (wasm) and AArch64 architectures, already utilize lld by default. This existing deployment has provided valuable experience and validation of its benefits. For years, the community has discussed and desired broader adoption of lld for general-purpose targets, reflected in long-standing GitHub issues like #39915 and #71515. The compiler team has diligently conducted extensive internal testing, spanning CI, crater (Rust’s regression testing tool), and dedicated benchmarking infrastructure. This rigorous vetting has paved the way for a more widespread rollout.

The time has come for broader real-world validation. To gather crucial feedback and usage scenarios, rust-lld is now being enabled as the default linker for x86_64-unknown-linux-gnu on nightly builds. This strategic move aims to bring the proven speed enhancements to a significant portion of the Rust development ecosystem.

Unlocking Unprecedented Speed: The Benefits of rust-lld

The most immediate and tangible benefit of rust-lld is a dramatic reduction in linking times. This isn’t just a minor tweak; it’s a game-changer for overall compilation speed. While this also opens doors for the compiler to leverage more advanced linker features in the future, the primary focus is on current performance gains.

Let’s revisit the ripgrep example mentioned earlier. With rust-lld, the linking time for ripgrep is slashed by an astonishing 7x. This single improvement translates to a significant 40% reduction in the total end-to-end compilation time. Imagine nearly halving your build times for a complex project – that’s the kind of impact rust-lld delivers.

While most Rust binaries will see some level of improvement, the benefits are particularly pronounced in scenarios where linking is typically a bottleneck. This includes compiling larger binaries or projects that involve extensive debug information. These are precisely the cases where traditional linkers struggle the most, making rust-lld an invaluable asset. Its modern design, leveraging parallelism and optimized algorithms, is the core reason for these substantial speedups. For a comprehensive overview of the performance improvements, you can refer to the detailed benchmark results published by the compiler performance working group.

Navigating Potential Hurdles and Taking Control

The transition to rust-lld is designed to be as seamless as possible. From extensive prior testing, the compiler team anticipates that it will serve as a drop-in replacement for the vast majority of use cases. However, it’s important to acknowledge that lld is not perfectly bug-for-bug compatible with the legacy GNU ld linker. While minor, these differences could, in rare instances, lead to unexpected behavior for certain projects.

The good news is that you, as a developer, retain full control. If you encounter any issues or regressions, Rust provides simple mechanisms to revert to the system’s default linker or to address specific linking requirements. This flexibility ensures that you can harness the benefits of rust-lld while maintaining the stability of your existing projects.

Actionable Steps to Manage rust-lld:

1. Embrace Rust Nightly: To experience the immediate benefits of rust-lld, ensure you are using a Rust nightly build (specifically nightly-2024-05-18 or later) on an x86_64-unknown-linux-gnu system. It’s enabled by default, so just update your toolchain!

2. Address Specific Linking Issues: In rare cases, some crates that rely on specific behaviors of older linkers might fail to link. For example, issues related to different default encapsulation symbols might require adding -Clink-arg=-Wl,-z,nostart-stop-gc to your RUSTFLAGS or Cargo configuration to match the legacy GNU ld behavior.

3. Revert if Necessary: Should you experience any unexpected problems, you can easily disable rust-lld and revert to your system’s default linker. Use the -Z linker-features=-lld flag. You can apply this either by setting it in the RUSTFLAGS environment variable, or by adding it to your project’s .cargo/config.toml configuration file, like so:

   [target.x86_64-unknown-linux-gnu]
   rustflags = ["-Zlinker-features=-lld"]
   

It’s also worth noting that some of the significant performance gains come from rust-lld‘s parallelism. While beneficial for most, this could potentially be undesirable in extremely resource-constrained environments where every CPU cycle and memory allocation is critical.

Summary

The integration of rust-lld as the default linker for x86_64-unknown-linux-gnu nightly builds marks a pivotal moment for Rust compilation performance. Starting with nightly-2024-05-18, Rust developers will experience significantly improved linking times, leading to faster development cycles and enhanced productivity. This move, spearheaded by Rémy Rakic on behalf of the compiler performance working group, reflects years of diligent testing and community input.

While the transition is expected to be smooth, the Rust team values real-world feedback. Your experience is crucial to ensuring this powerful enhancement becomes a stable feature for everyone.

Photo by Antoine Gravier on Unsplash

Frequently Asked Questions

What is rust-lld?

rust-lld is a version of LLVM’s high-performance lld linker, specifically packaged and shipped alongside the Rust compiler (rustc). It’s designed to significantly speed up the linking phase of Rust compilation by leveraging modern system architectures, including parallelism.

Which Rust versions and targets benefit from rust-lld by default?

Starting with nightly-2024-05-18, rust-lld is enabled by default for x86_64-unknown-linux-gnu targets on Rust nightly builds. Some specialized targets like WebAssembly (wasm) and AArch64 already use lld by default.

How much faster is rust-lld compared to traditional linkers?

rust-lld offers dramatic speed improvements. For example, it can make linking times up to 7x faster for complex projects like ripgrep, leading to an overall 40% reduction in total compilation time. The benefits are most pronounced for larger binaries or projects with extensive debug information.

What should I do if I encounter issues with rust-lld?

If you experience unexpected behavior or linking failures, you can easily disable rust-lld and revert to your system’s default linker. This can be done by adding -Zlinker-features=-lld to your RUSTFLAGS environment variable or your project’s .cargo/config.toml file. For specific issues, certain link-arg flags might also help.

Who spearheaded the integration of rust-lld?

The integration of rust-lld as the default linker was spearheaded by Rémy Rakic on behalf of the Rust compiler performance working group, following years of testing and community discussions.