Extensible Software in the Age of LLMs

The traditional web software model often caters to common user needs, leaving a significant 'long tail' of specialized requirements unmet due to the complexity and overhead of custom development. The author posits that the advent of Large Language Models (LLMs) fundamentally changes this dynamic, enabling the creation of 'Software for One' or 'Small Software' that is precisely tailored to individual workflows and easily extensible.

• Existing software is often static and feature-bloated, failing to address unique user demands.
• LLMs drastically lower the barrier to creating custom, personal applications, fostering a new category of 'LLM-native software' that is extensible by natural language prompts.
• Potential applications span AI agents, internal corporate tools, customer support platforms, and observability systems, where users can build bespoke automations and views.
• A significant hurdle is ensuring security and robust isolation when allowing arbitrary user-generated code to run within a web application, preventing issues like data exfiltration, DoS attacks, or service crashes.
• Salesforce's Apex platform is highlighted as a historical example of a successfully extensible, multi-tenant programmable platform that addressed similar challenges using a custom language and runtime.
• The article outlines key technical requirements for such extensibility: economical execution, fast cold starts, fine-grained control over resource limits, strong isolation boundaries, and a safe mechanism for user code to interact with the system, ideally through a 'capability model' rather than broad API access.
• Various technologies like interpreters, V8 Isolates, MicroVMs, and WASM+WASI are discussed as potential underlying primitives for building such platforms.
• Cloudflare's Dynamic Workers are presented as a cutting-edge, production-ready framework that meets these requirements, offering integrated solutions for observability, multi-tenant data storage, durable execution, source control, and hosted LLMs.

Ultimately, building highly extensible web platforms in the age of LLMs is a challenging endeavor, requiring careful design around security and operational complexity. However, the author concludes that the creative potential unlocked for users makes the effort well worth it.