Making your own programming language is easier than you think (but also harder)
The author shares their journey of building a new programming language, detailing its features and the intricacies of compiler design, motivated by both personal interest and the practical need for a sandboxed, high-performance modding language for their game. This deep dive into programming language internals, from parsing to JIT compilation and IR, resonates with Hacker News's appreciation for technical craftsmanship and the 'build your own X' ethos. It's a candid look at the challenges and triumphs of such an ambitious undertaking.
The Lowdown
The article chronicles the author's ambitious project: creating their own programming language, pslang, a venture driven by a long-standing personal interest, the specific requirements for a game modding language, and a newfound fascination with assembly and compiler optimizations. The author emphasizes that while it's a significant undertaking, certain aspects proved easier than expected, while others presented considerable technical hurdles, particularly at the low-level architecture specific stages.
- Motivation & Goals: The language project stemmed from a desire to create a "real" language beyond toy interpreters, fueled by the need for a robust, sandboxable, and performant modding solution for a simulation-heavy game. Key design goals included seamless C interop, low-level control, ease of sandboxing, a small and fast compiler, cross-platform compatibility, reasonable execution speed, and an attempt to diverge from C++ paradigms.
- Language Design:
pslangis an imperative, eager-evaluated, call-by-value language with a static, strict, and nominal type system. It features indentation-based scoping (like Python), a comprehensive set of primitive numeric types (signed/unsigned integers, floats from f16 to f64, plusunit), first-class arrays, explicit pointers, and structs with guaranteed memory layout for C interop and GPU programming. Control flow is standard, andforeignfunctions facilitate integration with C libraries. - Compiler Architecture: The project is modular, comprising libraries for types, AST, parser (using Bison, with a clever linear pass for indentation-based scoping), an identifier resolver, and a type checker. An early tree-walking interpreter served for debugging. The initial JIT compiler targeted Aarch64, featuring hand-written instruction generation and navigating the quirks of the architecture like constant loading and stack alignment.
- Intermediate Representation (IR): To improve code sharing, optimization, and target multiple architectures, the author introduced an SSA-like IR, though it deviates by allowing reassignments. This IR handles complex data structures like nested structs with specialized copy and assign nodes to simplify optimizations.
- Future Vision: The roadmap is extensive, including a register allocator, various IR optimizations (constant propagation, dead code elimination, inlining), an IR-based interpreter, generating standalone executables, and debugging support. Language feature plans span polymorphism (leaning towards C++-style templates), operator overloading, enhanced control flow (for-loops), automatic resource management (exploring linear types), polymorphic literals, and compile-time evaluation. A modular library system with a prelude, string literals, and a standard library is also planned.
Ultimately, the author positions pslang as an ongoing passion project, acknowledging its ambitious scope and the likelihood that it will evolve at a pace dictated by personal interest and the demands of their primary game development, rather than a strict timeline for production readiness.