Celld: Self-hosted, distributed Durable Objects

Celld is Deno's open-source daemon enabling self-hosted, distributed Durable Objects, bringing Cloudflare's powerful primitive to your infrastructure. It ingeniously uses S3-compatible storage as its sole coordination mechanism, making each object a tiny, replicated SQLite database without a complex control plane. This innovative approach appeals to HN for its technical elegance, promise of vendor independence, and potential for resilient, sharded applications.

107

Score

Comments

Highest Rank

on Front Page

First Seen

Aug 5, 6:00 PM

Last Seen

Aug 6, 12:00 AM

Rank Over Time

The Lowdown

Celld is an open-source daemon from Deno that allows developers to self-host and distribute Cloudflare Workers and Durable Objects on their own infrastructure. It presents an innovative, minimalist architecture for achieving high availability and scalability for stateful serverless functions, sidestepping the complexities of traditional distributed systems. The project is led by Ry, the creator of Node.js and Deno.

Core Architecture: Each 'Durable Object' within Celld is essentially its own SQLite database. This database is automatically replicated to an S3-compatible bucket, which serves as the durable source of truth.
Decentralized Coordination: Nodes in a Celld fleet coordinate exclusively through this S3 bucket, leveraging object-storage compare-and-swap for ownership. This design eliminates the need for a separate control plane, membership protocol, or consensus service.
Runtime Environment: Celld nodes embed the V8 JavaScript engine and execute Wrangler bundles, offering compatibility with the Cloudflare Workers ecosystem.
Resilience and Durability: In case of node failure or migration, a new owner can restore the object's state from the S3 bucket, ensuring data persistence and application continuity. Nodes are considered entirely replaceable.
Resource Management: The system is designed for efficiency, with idle objects 'hibernating' to consume minimal resources. It also supports optional pressure shedding to manage resource utilization on individual nodes.
Deployment and Operation: Installation is via a simple shell script or Docker. Deployment involves specifying the S3 bucket, and a diagnose tool helps monitor fleet health and peer status.
Security Considerations: Peer-to-peer HTTP communication is not TLS-terminated by default, requiring trusted private networks or encryption overlays. Fleet authentication relies on a shared secret stored in the S3 bucket.
Contribution Model: The project explicitly disables Pull Requests, preferring git format-patch submissions to ry@deno.com, emphasizing thoughtful, focused contributions.

In essence, Celld offers a compelling vision for running distributed, stateful serverless components with an S3 bucket acting as the foundational layer for both durability and coordination, simplifying operational overhead while maintaining a high degree of resilience.

The Gossip

Cloudflare Comparisons & Self-Hosting Appeal

Many commenters immediately drew comparisons between Celld and Cloudflare's existing offerings, particularly `workerd` and Durable Objects. There was significant enthusiasm for the ability to run Durable Objects—a 'valuable abstraction'—outside of Cloudflare's ecosystem, praising the promise of vendor independence. While some acknowledged that Celld doesn't replicate Cloudflare's global geo-distribution, it's seen as a strong alternative for scenarios where Cloudflare's full suite might be 'overkill' or for those preferring to self-host within a single region.

S3's Sovereignty: The Bucket as Control Plane

The core architectural claim that Celld nodes coordinate 'through that bucket alone, with no control plane or consensus' sparked considerable discussion. Some argued that S3 effectively *is* the control plane and consensus layer, merely shifting that responsibility to the S3 provider. Others praised this design choice as an ingenious simplification, leveraging a highly available and robust service for critical distributed systems primitives like compare-and-swap, making it a 'fantastic common-mode infra requirement' and a 'mindblowing' approach to distributed coordination.

Operational Outlook: Local Dev, Spot Instances, and Geo-Challenges

Commenters raised practical concerns and hopes for Celld's operational aspects. Desires included easier local development and prototyping without requiring immediate S3 configuration. The idea of running Celld's Durable Objects on cost-effective 'spot instances' was also suggested, implying the need for resilient failure handling. Conversely, the significant challenge of self-hosting a system that could genuinely compete with Cloudflare's global geo-synchronization and low-latency network reach was highlighted, with Durable Objects often being tied to specific datacenters.