A Preview of DuckDB v2.0

DuckDB v2.0, "Cyanoptera," is a landmark release that extends the popular in-process analytical database into new territory, notably embracing server-side operations and significantly enhancing its core capabilities. With over 10,000 commits since v1.5, this update is more than just a version bump; it's a strategic evolution that brings both carefully chosen breaking changes and a wealth of new features designed to make DuckDB more powerful, versatile, and performant for a wider array of workloads.

Key highlights and new features of DuckDB v2.0 include:

• DuckDB as a Server: The new quack extension and CONNECT statement enable a native client/server mode, allowing any DuckDB process to serve databases over the network. This expands its utility for multi-tenant, long-running transactional deployments and facilitates query pushdown to other databases like PostgreSQL.
• VARIANT Type as First-Class Citizen: The "JSON on steroids" VARIANT type receives significant enhancements, including shredded execution directly from storage, extraction pushdown, and VARIANT reading/writing for Parquet, making it highly efficient for semi-structured data.
• Triggers: A long-requested feature is fully implemented, supporting BEFORE and AFTER triggers, FOR EACH ROW and FOR EACH STATEMENT, and transition tables, useful for audit logging and internal system logic.
• SQL Dialect Additions: Introduces NEAREST joins for similarity search, DML within CTEs, nested schemas, a new variable syntax ($x), and JSON mutation functions, significantly expanding SQL expressiveness and flexibility.
• Asynchronous I/O: The engine now features asynchronous I/O, particularly for object stores like S3, leading to substantially faster remote reads and greater parallelism for queries on network storage.
• Faster Queries Across the Board: Major performance gains include partial aggregate pushdown, a rewritten recursive CTE engine (up to 40x faster for certain queries), memory-spilling aggregations, and expanded row-group pruning for various data types and function predicates.
• Storage Format v2.0: Updates the default storage format with buffer-managed ART indexes (no longer pinned in memory), lazy column metadata loading, improved string compression, and stronger corruption validation, resulting in faster opening of large databases and reduced memory usage.
• Brand New SQL Parser: Replaces the PostgreSQL-derived parser with a modern, extensible PEG-based parser, offering better error messages, support for dialect compatibility modes (e.g., 'spark'), and allowing extensions to hook into the grammar.
• Timezones, Calendars, and Collations Without ICU: Removes the ICU library dependency by implementing these features natively, resulting in a smaller binary, easier updates, and faster execution (e.g., 2.2x to 2.6x speedup for timezone conversions and collations).
• Stable C API and Self-Hostable Extensions: Broadens the stable C API for extensions, allowing them to be written once and work across DuckDB versions. It also introduces custom extension repositories for self-hosting and signing extensions, enhancing the ecosystem's robustness.
• Advisory Board: The DuckDB Foundation will form a stakeholder advisory board to provide input on the development roadmap for DuckDB, DuckLake, and Quack.

This release not only strengthens DuckDB's existing analytical prowess but also strategically positions it for new use cases, from becoming a backend for sophisticated data applications to a foundational component in cloud-native data architectures. The continuous engagement with the community and transparent development process remain central to DuckDB's rapid evolution and widespread adoption.