What Async Promised and What It Delivered
This article dissects the evolution of asynchronous programming, from callbacks to promises and async/await, tracing their development as solutions to the 'C10K problem'. While each paradigm aimed to simplify concurrent I/O locally, the author argues they collectively introduced significant systemic costs like 'function coloring' and ecosystem fragmentation. It's a critical, retrospective look at widely adopted programming constructs, showcasing their hidden complexities and making it a deep dive for the HN crowd.
The Lowdown
The article meticulously traces the evolution of asynchronous programming paradigms, from callbacks to promises and finally async/await, all in response to the 'C10K problem'—the challenge of efficiently handling thousands of concurrent connections without the prohibitive overhead of one operating system thread per connection. It posits that while each successive wave offered ergonomic improvements for writing concurrent code, it simultaneously introduced new, often systemic, complexities.
- Callbacks: These were the initial answer, allowing non-blocking I/O by registering functions to be called upon completion. While solving the resource problem (e.g., Node.js, Nginx), they introduced 'callback hell' due to inverted control flow, fragmented error handling, and a lack of cancellation mechanisms.
- Promises/Futures: This wave introduced objects representing an operation's eventual result. They improved upon callbacks with composable
.then()chains and consolidated.catch()error handling, offering a first-class value for future results. However, promises proved one-shot (unsuitable for streams), had clunky composition for complex parallel operations (e.g., requiringPromise.all), initially suffered from silent error swallowing, and introduced a mild 'type split' where functions either returned a value or a promise. - Async/Await: Pioneered by C# and widely adopted (JavaScript, Python, Rust), this syntax made asynchronous code appear sequential, greatly improving readability and allowing traditional
try/catchblocks. Despite its ergonomic wins for linear async sequences, it brought forth significant challenges. - Function Coloring Tax: Async/await made the 'function coloring' problem (sync vs. async functions) viral. An async function cannot be called from a synchronous one without special handling, forcing the 'red' (async) color to propagate throughout a codebase. This led to ecosystem fragmentation (e.g., competing Rust async runtimes like Tokio), increased library maintenance burdens (requiring both sync and async versions), and complex refactoring efforts.
- New Bugs and Sequential Trap: Async/await also introduced new categories of bugs, such as 'futurelocks' in Rust, where futures holding resources can stop being polled, leading to deadlocks. Furthermore, its greatest strength—making async code look sequential—became a 'cognitive trap,' as it obscured opportunities for parallel execution, requiring explicit, often clunky, patterns like
Promise.allto achieve concurrency.
Despite these accumulating costs, async abstractions did bring improvements, particularly in making linear sequences of I/O more readable and debuggable than their callback-based predecessors. The article notes that later language designs (Go's goroutines, Java's Project Loom, Zig's Io interface) learned from these challenges, often opting for different approaches to avoid function coloring altogether.
The article concludes that while each technological wave successfully addressed the immediate shortcomings of its predecessor, it did so by introducing new, structural complexities that permeate entire codebases and ecosystems. This continuous cycle highlights a fundamental tension in managing concurrent execution, where successive fixes for symptoms often lead to new, underlying problems that accumulate over time.