WAL-RUS: a Rust Rewrite of WAL-G for PostgreSQL Backups
ClickHouse engineered WAL-RUS, an open-source Rust rewrite of the PostgreSQL backup tool WAL-G, to tackle memory predictability issues inherent in Go's garbage-collected runtime. This deep dive reveals how Rust's explicit memory management significantly reduces virtual memory consumption by over 70% while maintaining full compatibility, offering a compelling solution for resource-constrained environments. It's a perfect example of how targeted systems-level optimization can yield substantial operational benefits for critical infrastructure.
The Lowdown
ClickHouse has introduced WAL-RUS, an open-source tool for PostgreSQL backup and Write-Ahead Log (WAL) archival, developed as a memory-efficient alternative to the popular Go-based WAL-G. The project was born out of a need to address the unpredictable memory consumption of garbage-collected languages like Go in critical, long-running services.
- Motivation for Rewrite: While WAL-G is a robust tool, its Go runtime's garbage collection led to unpredictable memory usage, particularly virtual memory, which complicated capacity planning and resource allocation in resource-constrained PostgreSQL environments. This unpredictability forced operators to over-provision memory, diverting resources that could otherwise benefit the database itself.
- Rust's Advantage: WAL-RUS leverages Rust's explicit memory management, bounded worker pools, and controlled concurrency to achieve a significantly more predictable and lower memory footprint. It minimizes unnecessary buffering and data copies, crucial for streaming workloads like WAL archival.
- Key Features: The tool is designed for continuous WAL archival using a daemon architecture, maintaining persistent object storage connections. Crucially, it ensures full compatibility with WAL-G, using the same configuration variables and allowing archives to be read interchangeably, simplifying migration for existing deployments.
- Performance Benchmarking: Benchmarks demonstrate WAL-RUS achieving a peak virtual memory usage below 1GB, a 70% reduction compared to WAL-G's nearly 2.8GB, while maintaining comparable archival throughput and CPU utilization. PgBackRest, being C-based, also showed good memory control but accumulated a larger backlog during intense WAL activity.
- Future Plans: ClickHouse plans to integrate WAL-RUS as the default backup and WAL archival mechanism for its managed PostgreSQL offering in ClickHouse Cloud and welcomes community contributions.
WAL-RUS successfully addresses the operational challenge of predictable resource usage in PostgreSQL backups, demonstrating that a carefully engineered Rust implementation can provide substantial memory efficiency without compromising functionality or compatibility with the established WAL-G ecosystem.