Executable Is a SQLite Database
The author proposes replacing the standard ELF executable format with a SQLite database, demonstrating how a SQLite file can be directly chmod +x and run. This radical idea, developed from PhD research, leverages SQLite's robust database features to manage program headers, symbols, and dynamic linking, offering powerful query capabilities over executable metadata. Critically, packing an entire userland into a single SQLite database results in a smaller footprint than traditional ELF files, hinting at a transformative approach to system packaging and dynamic linking.
The Lowdown
The article introduces a groundbreaking concept: using a SQLite database as the actual executable and linkable format (SELF) for programs, moving beyond the traditional ELF format. The author, driven by a long-standing fascination and enabled by tools like Nix, posits that ELF is inherently a poorly implemented database and that a robust, self-describing format like SQLite could offer significant advantages. This exploration culminates in a working prototype that reimagines how programs are structured, loaded, and dynamically linked.
- ELF's Limitations: The author argues that ELF acts like a database but implements many primitives manually, leading to repetitive parsing, difficult modifications due to its terse, tightly packed nature, and lack of a self-describing schema.
- SQLite as a Replacement: SQLite is proposed as a superior alternative due to its stability, extensibility, and self-describing nature. The SELF format uses SQLite tables to store program segments, symbol tables, and other metadata traditionally found in ELF.
- Implementation Details: A SELF file is identified by a special
application_idin its SQLite header. The kernel'sbinfmt_miscsystem is configured to recognize and execute these files via a customself-execinterpreter, which reads executable data from the SQLite database. - Dynamic Linking with SQL: Dynamic linking can be achieved either by using
glibc'srtld-auditinterface to intercept symbol lookups with SQL queries or by a completely custom SQL-driven dynamic linker (self-ld). - Size and Performance: Initial SELF files are larger than ELF, but stripping debug information brings them within 1% of ELF size. More notably, a single SQLite database containing an entire userland (multiple executables and their shared libraries) can be smaller than the aggregated ELF files due to data deduplication. Execution incurs a fixed ~5ms SQLite overhead, plus time proportional to image size, and currently lacks ELF's
mmap-based text page sharing. - "System as a Closure": The concept extends to packaging an executable and all its transitive dependencies into a single SQLite database closure. This eliminates dynamic linking ambiguities, turning
lddfunctionality into a simple SQLJOIN. Furthermore, an entire userland can be packed into one database, enabling transactional operations likeLD_PRELOAD.
The project, dubbed SELF (Structured Executable & Linkable Format), demonstrates the feasibility and surprising benefits of this approach. It highlights how rethinking fundamental system components, especially with enabling technologies like Nix, can lead to innovative and potentially more efficient and manageable software distribution and execution paradigms.