How Our Rust-to-Zig Rewrite Is Going

Roc's compiler team has completed a year-and-a-half journey, rewriting 300,000 lines of Rust into Zig, now reaching feature parity. This deep dive meticulously compares the two systems languages through the lens of a complex compiler project, highlighting Zig's benefits in build times, memory control, and its approach to memory-unsafe code. The post sparks lively debate on language trade-offs and the perennial quest for the 'perfect' systems programming environment.

Score

Comments

Highest Rank

on Front Page

First Seen

Jul 16, 2:00 PM

Last Seen

Jul 16, 7:00 PM

Rank Over Time

The Lowdown

The Roc compiler project recently achieved a significant milestone: feature parity after rewriting its 300,000-line codebase from Rust to Zig over 18 months. This comprehensive report details the motivations, challenges, and outcomes of this substantial engineering effort, offering a candid comparison between Rust and Zig from the perspective of a compiler development team.

Motivation for the Rewrite: The original Rust compiler faced architectural issues, particularly with polymorphic defunctionalization, leading to intractable bugs. A full rewrite was deemed more effective than incremental fixes, aligning with the common practice of compiler rewrites and the goal of significant architectural changes.
Why Zig? The team chose Zig primarily for its anticipated faster build times, superior granular memory control (essential for arenas and struct-of-arrays layouts), and a more relevant ecosystem for specific compiler needs, such as a custom LLVM bitcode serializer. They also sought better assistance for inherently memory-unsafe code, which is prevalent in compilers.
Memory Safety Comparison: The article presents a nuanced view on memory safety, distinguishing between memory corruption in the compiler itself and miscompilations in its output. While Rust's borrow checker would have prevented two use-after-free bugs in Zig's error reporting, the overall impact on the project was minimal. For Roc, memory safety in compiled output was a greater concern than internal compiler memory safety, a problem outside the borrow checker's scope.
Build Time Gains: Zig delivered on its promise of vastly superior build times, with incremental rebuilds as fast as 35ms (pending a stable Zig release fix). This significantly outperforms Rust's already improved 3.4s incremental builds for comparable codebases.
Memory Control in Practice: Zig's memory model enabled techniques like zero-parse deserialization for compiler caches, loading data structures directly from disk at memcpy speed. This approach, which relies on 'programming without pointers' and indices, is more challenging to implement safely in Rust due to its borrow checker design assumptions.
Rust Features Missed: The team missed Rust's automatic allocation/deallocation in tests, parametric and ad hoc polymorphism, private struct fields, and its robust backwards compatibility and trivial upgrade process.
Zig Features Enjoyed: Praised were Zig's subtractive nature (no macros), fine-grained control over data layouts (e.g., custom integer sizes, packed structs), its powerful build toolchain (used even by non-Zig projects like Uber), and its explicit error-handling strategy, especially for failed allocations.

In conclusion, the Roc team found their decision to rewrite in Zig to be highly beneficial, aligning perfectly with the project's unique requirements and yielding significant improvements in areas like build speed and memory management. The project is now moving towards its first numbered release later this year.

The Gossip

Safety Squabbles & Unsafe Usage

Commenters debated the article's claims about memory safety, particularly regarding Zig's `ReleaseSafe` guarantees and the necessity of `unsafe` code in compilers. Some questioned if emitting machine code inherently requires `unsafe` in the way described, suggesting that compilers primarily dealing with well-formed programs might not need it for their core logic. There's an ongoing discussion about whether a borrow checker could be retrofitted into Zig as a static analysis tool.

Build Time Bravado

The blazing-fast 35ms incremental rebuild time touted by Zig emerged as a major point of interest and desire among commenters. Many viewed this as a 'killer feature' and a compelling reason for the rewrite, expressing hope that Rust might eventually achieve similar performance. This highlighted the continuous trade-off between compile-time speed and runtime safety guarantees in systems languages.

Language Legacy & Future Fabrications

The discussion touched on the historical context of compilers (e.g., Rust's origins in OCaml, OCaml's role as a prototype language) and speculated about future language trends. Some anticipated a 'wave of rewriting Rust to X,' reflecting a broader community desire for a language that combines Rust's safety, Zig's build speed, and Go's runtime efficiency (without its GC). This theme underscores the continuous evolution and aspiration for ideal language designs in systems programming.