AI handles incidents, engineers lose touch with their systems
AI is increasingly automating incident response, a welcome relief for engineers facing routine alerts. However, this automation risks engineers losing critical hands-on experience and intuition, leaving them unprepared for complex, never-before-seen outages. The article argues for mandatory, hands-on simulation and training to prevent this 'comprehension debt' and keep human expertise sharp, drawing parallels to pilot training in aviation.
The Lowdown
The growing sophistication of AI in incident response is a double-edged sword, offering significant efficiency benefits while simultaneously threatening human engineers' practical skills and intuition. This paradox, described as 'The Ironies of Automation,' suggests that as AI handles more routine incidents, human responders become less practiced and potentially less capable when confronted with ambiguous, high-severity events that automation cannot resolve.
- AI SRE tools are becoming adept at inspecting alerts, forming hypotheses, querying telemetry, correlating deployments, and even implementing fixes, particularly for common issues.
- This automation, while convenient for avoiding late-night callouts, deprives engineers of the 'safe' practice opportunities routine incidents provide for developing a deep understanding of system behavior and failure modes.
- The author predicts a future where average Mean Time To Resolution (MTTR) for simple incidents decreases due to AI, but resolution times for complex incidents skyrocket as human engineers struggle with diminished hands-on experience.
- An analogy is drawn to the aviation industry, where pilots rely on automation for much of flying but must rigorously train in simulators for rare emergencies, such as engine failures, to avoid catastrophic outcomes like TransAsia Airways Flight 235.
- To counteract this, the article advocates for solutions like realistic incident simulations (e.g., Rootly's partnership with Uptime Labs) where engineers practice investigation, coordination, and communication under pressure.
- While AI can explain its actions, the author stresses that explanation and observation are not substitutes for hands-on practice, comparing it to learning tennis by playing, not just watching Serena Williams.
- The concept of 'comprehension debt' is introduced: a widening gap between how systems work and engineers' understanding, necessitating continued hands-on interaction, chaos engineering, and tabletop exercises.
Ultimately, as AI takes on more operational tasks, maintaining human expertise through proactive, hands-on training and simulation becomes paramount to ensure readiness for the inevitable moments when automation fails or reaches its limits, preventing a critical skill decay in the engineering workforce.