On The Need For Understanding
This essay delves into the evolution of software engineering, contrasting Gerald Sussman's mid-90s observation that programming shifted from building with known parts to 'basic science on foreign libraries.' The author shares a personal journey from seeking easy, surface-level solutions to embracing deep understanding, arguing that despite modern complexity, tools like open source make true comprehension more accessible and crucial than ever.
The Lowdown
The author begins by referencing a Mastodon post by Andy Wingo, recalling Gerald Sussman's lament from 16 years prior that programming had devolved into 'basic science on the functionality of foreign libraries.' This sentiment, born from the increasing complexity of software, suggests that engineers no longer build from fully understood components but instead poke and prod at black boxes. The author recounts his own journey, initially agreeing with the difficulty of understanding but ultimately arriving at a different conclusion.
Key takeaways from the author's narrative include:
- His early programming experiences, starting with BASIC and struggling with the overwhelming complexity as technology advanced, wishing for tools that simply 'did things' without requiring deep understanding.
- The 'Easymik' anecdote, where he proudly built a convoluted, file-writing wrapper to avoid comprehending the underlying audio library's flexible API, only to later recognize its fundamental flaw.
- A turning point arrived with a job working with QNX, which, unlike the opaque MS-DOS and Windows, demonstrated that well-designed systems could be inherently understandable.
- A subsequent attempt to build an 'infinitely configurable via XML' toolkit to avoid code duplication ultimately fell victim to Greenspun's Tenth Rule, highlighting the pitfalls of abstraction without underlying comprehension.
- The ultimate realization came during a prolonged struggle with a tree-merging algorithm: true productivity and problem-solving stem from confronting the fear of complexity and committing to understanding how things actually work.
- He argues that today's programming landscape, though complex, is better than the 90s because open-source components allow developers to inspect source code and truly understand behavior, unlike the 'secret' inner workings of proprietary systems like Windows 95.
- An example from Android UI programming illustrates how digging into the source code of
requestLayoutandforceLayoutdemystified their behavior and resolved persistent bugs.
In conclusion, the author asserts that while the complexity Sussman noted in the 90s was real, the modern era offers more avenues for understanding. By committing to digging in, reading source code, and asking fundamental questions (like those in Polya's 'How To Solve It'), programmers can transform overwhelming tasks into manageable problems, making everything 'easier' once the light of understanding is turned on.