Show HN: ZeroFS – A log-structured filesystem for S3
ZeroFS presents a log-structured filesystem that makes S3 buckets available via POSIX interfaces like NFS and 9P, or as raw NBD block devices. This deep dive offers a compelling technical solution to a persistent problem: how to use cheap object storage for traditional filesystem workloads. Its extensive testing suite and robust feature set resonate with HN's infrastructure-minded audience, sparking debate on the fundamental approach of abstracting S3.
The Lowdown
ZeroFS is an innovative project that transforms S3-compatible object storage into a usable, performant filesystem. Designed with a log-structured architecture, it treats S3 buckets as its primary backend, providing a POSIX-compliant interface over NFS, 9P, or as raw NBD block devices. This allows developers and sysadmins to leverage the scalability and cost-effectiveness of object storage while maintaining familiar filesystem semantics.
- Log-Structured Engine: Writes are handled as immutable objects, with a compaction process reclaiming space from deleted data. This design facilitates features like snapshots and efficient data handling.
- Extensive Protocol Support: ZeroFS offers compatibility with NFS, 9P, and NBD, allowing various operating systems and applications to mount S3 buckets natively.
- Robust Feature Set: Includes always-on encryption (XChaCha20-Poly1305), zstd/lz4 compression, configurable local caching for warm reads, point-in-time checkpoints, read replicas, TRIM support for cost optimization, and high availability with leader/standby configurations.
- Rigorous Verification: The project boasts an impressive public CI pipeline, running
pjdfstest,xfstests, ZFS scrubs, kernel builds,stress-ng, and Jepsen's local-fs and failover suites to ensure data integrity and POSIX compliance. - Geo-Distribution Capabilities: Demonstrates the ability to create ZFS mirrors across multiple S3 regions using its NBD block device feature, ensuring data availability even if a region fails.
- Web UI: An optional web console provides a file manager, a dashboard for streaming stats, and a terminal for booting a Linux VM rooted on the mount.
By packaging object storage with these advanced filesystem features and extensive testing, ZeroFS aims to bridge the gap between traditional storage paradigms and the cloud-native world, offering a practical solution for complex data management needs.
The Gossip
S3 Abstraction: Practicality and Performance Pitfalls
A significant portion of the discussion centered on the inherent challenges and practicality of abstracting object storage like S3 to mimic a traditional filesystem. Commenters highlighted that S3's cost model, which charges per operation rather than just storage, often makes direct application integration more efficient than adding a filesystem layer. Alternatives such as JuiceFS, S3FS, and AutoMQ (which leverages object storage for Kafka without HDDs) were mentioned as different approaches. Despite these concerns, the technical ingenuity of ZeroFS's log-structured design and rigorous testing was acknowledged.
Marketing & 'Vibe-Coded' Perceptions
Several users found the marketing language on the ZeroFS landing page to be overly explicit or unnecessary, leading to the term 'vibe coded' being used. Specific examples included the emphasis on 'public CI' for an open-source project and the detailed explanation of asciinema recordings. The author actively engaged with these comments, clarifying that the intent was to highlight verifiable claims and acknowledging the feedback for improving the phrasing.
Website & Technical Scrutiny
Some commenters delved into specific technical details of the ZeroFS website and project. One user questioned the presence of a script from 'merklemap.com' on the landing page. The author promptly clarified that it was a self-hosted Plausible analytics instance, with another commenter noting that both projects share the same author, highlighting the HN community's tendency to scrutinize technical implementations closely.