Show HN: MacMind – A transformer neural network in HyperCard on a 1989 Macintosh
MacMind is a transformer neural network, complete with embeddings, self-attention, and backpropagation, meticulously implemented in HyperTalk on a 1989 Macintosh. This ambitious project aims to demystify modern AI, demonstrating that its core mathematical principles are universal and not dependent on vast computational resources. It resonates with Hacker News for its blend of retrocomputing ingenuity, deep technical exploration, and philosophical statement about AI accessibility.
The Lowdown
MacMind is an extraordinary project that successfully implements a complete transformer neural network, including all its fundamental components, using HyperTalk on a vintage 1989 Macintosh. This remarkable feat serves as a tangible demonstration that the underlying mathematics of artificial intelligence, even for advanced models, is accessible and understandable.
- Comprehensive Transformer Implementation: The project includes token embeddings, positional encoding, self-attention with scaled dot-product scores, cross-entropy loss, full backpropagation, and stochastic gradient descent, all coded line-by-line in HyperTalk without external libraries or compiled code.
- Demystifying AI Principles: The primary motivation behind MacMind is to illustrate that the process training large language models is identical in kind, differing only in scale. It emphasizes that concepts like backpropagation and attention are mathematical processes, not magic.
- Learning the Bit-Reversal Permutation: MacMind is trained to learn the bit-reversal permutation, the opening step of the Fast Fourier Transform. The model discovers this positional pattern purely through self-attention and gradient descent, mirroring how larger models learn complex relationships.
- Inspectable and Modifiable: Every line of code and the mathematical operations are fully inspectable within HyperCard's script editor. Users can modify parameters like learning rate or training tasks directly, offering a transparent view into the model's workings.
- Vintage Hardware Constraints: Designed for systems like the Macintosh SE/30 running HyperCard 2.0+ and System 7, each training step takes several seconds, with full convergence requiring hours of overnight processing on an 8 MHz 68030 processor.
- Persistent Model State: The trained model's 1,216 parameters are stored in hidden HyperCard fields, ensuring the learned intelligence persists even after quitting and reopening the application.
- Validation with Python: A Python/NumPy reference implementation is provided to validate the mathematical accuracy of the HyperCard version, confirming the integrity of the core algorithms.
Ultimately, MacMind stands as a powerful educational tool and a testament to the universality of mathematical principles in AI, proving that computational complexity is a matter of scale, not an intrinsic barrier to understanding.