RIP, vector database
Turbopuffer is killing its vector-primary index, a foundational pivot for its database architecture. The article details the technical journey and the painful but necessary trade-offs made to scale a specialized vector database into a general-purpose search engine, offering a candid look at infrastructure evolution for HN's database wonks.
The Lowdown
Turbopuffer is undergoing a fundamental architectural shift, deprecating its long-standing vector-primary index in favor of a more flexible, generalized approach. This change, dubbed "turbopuffer v3," aims to significantly enhance scalability and support a broader array of query plans beyond vector search, addressing critical limitations encountered in their earlier designs.
- v1: The Vector-First Era: Initially, Turbopuffer focused on cheap, fast vector searches using a hierarchical clustering index (SPANN, then SPFresh) where documents were simple ID-vector pairs. The entire system revolved around the Approximate Nearest Neighbor (ANN) address as the primary key.
- v2: Expanding Horizons: As customer demands grew, Turbopuffer introduced attribute filtering and full-text search capabilities. These were implemented using inverted indexes that mapped to the existing ANN addresses, alongside storing attributes with vectors. Despite these additions, the core ANN-primary storage architecture remained intact.
- The Downfall of the Vector Primary Index: This vector-centric design, while excellent for ANN searches, became a bottleneck for other query types due to three main issues:
- Storage Amplification: Duplicating document content for multi-vector representations.
- Write Amplification: Rebalancing vectors required moving entire document contents and associated indexes, severely impacting write efficiency.
- Limited Vectorization: The ANN's optimal block size constrained other query engines, preventing them from leveraging larger, more efficient blocks for vectorized operations.
The solution, Turbopuffer v3, will decouple the primary index from the ANN address, transforming ANN into just another secondary index. This ambitious undertaking has already seen its first milestone with 100% CI passes, though initial performance regressions are expected as the team now focuses on optimization. Turbopuffer promises more detailed updates on the new architecture and performance tuning as they progress, aiming for a faster, more versatile search engine.