Mold: A Massively Parallel Linker
A new paper introduces Mold, a massively parallel linker designed to tackle the long-standing bottleneck of link times in software development. By systematically applying data parallelism across the entire linking pipeline, Mold dramatically accelerates the build process, especially for large C++ programs. This technical deep dive showcases how a clean-slate design can yield exceptional performance gains, garnering significant interest from developers tired of sluggish compile cycles.
The Lowdown
Linking is a crucial, often overlooked, step in the software build process that frequently becomes a significant bottleneck, especially for large C++ projects. The paper introduces Mold, a groundbreaking Unix/Linux linker that rethinks traditional approaches by employing massively parallel data processing to drastically reduce link times. Unlike existing linkers that leave CPU cores largely idle, Mold leverages all available parallelism.
- The Problem: Traditional linkers struggle with speed due to architectural constraints like entangled symbol resolution and inefficient archive processing, making link times a major hurdle in the edit-compile-debug cycle.
- Mold's Solution: It employs a clean-slate design that decouples these interdependent processes, systematically applying data parallelism across every stage of the linking pipeline.
- Performance: On real-world, multi-gigabyte debug binaries, Mold completes linking in seconds, often under one second. It demonstrates a speedup of 2.4-16.1 times compared to the state-of-the-art LLD linker and an astonishing 112 times faster than the traditional GNU ld.
- Key Insight: An ablation study reveals that this impressive speedup isn't from a single magic bullet but the cumulative effect of parallelizing all passes within the linking process.
Mold's systematic approach to parallelism offers a substantial improvement to developer productivity by turning a significant wait time into a near-instantaneous operation, fundamentally changing the experience of building large software projects.
The Gossip
Mold's Meritorious Momentum
Commenters are quick to laud Mold's real-world impact, with one user noting it 'shaved hours off our full tree build time' for their Linux distribution. There's appreciation for Rui Ueyama's continuous optimization, even improving Mold's speed further while writing the paper. The discussion also touches on the desire for broader platform support (Windows, Mac) to enable enterprise adoption, suggesting a willingness to fund such development.
Competing Linker Contenders
The conversation naturally drifts to comparisons with other modern linkers. `wild` is highlighted as a potentially faster alternative in some benchmarks, notably offering incremental linking, a feature Mold's author reportedly does not plan to implement. The historical context of `GNU gold` is also brought up, acknowledging its past significance in optimizing linking, despite its eventual deprecation.
Rui's Radical Rationale
Many commenters express admiration for Rui Ueyama's open-source philosophy. His approach, articulated as 'increasing the value of commons by, say, 100 and get 1 as a return,' resonates deeply. This strategy emphasizes providing free, high-quality tools for the community, with the belief that personal and project sustainability will follow, fostering a positive open-source ecosystem.
Innovation's Inherent Advantage
A meta-observation from the paper's abstract, echoed by commenters, suggests that 'drastic improvements tend to come from new linkers rather than existing ones.' This is attributed to the freedom new projects have to take risks and implement clean-slate designs, unburdened by legacy constraints or backward compatibility requirements that often hinder established solutions. This theme encourages rethinking conventional wisdom in software development.