Going Dark, and the era of law enforcement hacking
Matthew Green provocatively argues that AI will make software too secure for traditional law enforcement hacking methods. This newfound impermeability, he warns, will inevitably lead to governments aggressively demanding mandatory backdoors in software. The post resonates deeply on HN for its prescient analysis of privacy, state surveillance, and the unforeseen consequences of advanced AI on societal security.
The Lowdown
In a thought-provoking post, cryptographer Matthew Green posits that artificial intelligence, specifically its advanced vulnerability-finding capabilities, is on the verge of making major software practically unhackable. This unprecedented level of security, he argues, will inadvertently lead to a new and dangerous era for law enforcement and intelligence agencies.
- Historical Context: Green traces the evolution of digital surveillance from the simple wiretaps depicted in The Wire (2002) to the widespread adoption of encrypted smartphones and end-to-end messaging by the mid-2010s.
- "Going Dark" 1.0: The rise of strong encryption led to the FBI's initial "Going Dark" initiative, culminating in the Apple-FBI dispute over an unlocked iPhone. This period saw law enforcement resorting to purchasing targeted hacking tools from private vendors to bypass encryption.
- AI's Game-Changing Role: The emergence of AI models, like Anthropic's Mythos, that excel at finding software vulnerabilities, promises to rapidly eliminate exploitable bugs. Green predicts that within two years, major software will largely run out of remotely exploitable flaws.
- The Unintended Consequence: For law enforcement and intelligence agencies, this means losing their primary means of digital access, forcing them into a truly "dark" state for the first time since the pre-smartphone era. This, Green warns, will cause immense pressure to pivot from exploiting vulnerabilities to demanding intentional backdoors.
- Risk of Self-Sabotage: Such mandated backdoors, Green contends, would create systemic weaknesses, potentially allowing foreign adversaries to compromise U.S. systems, thus undermining national security rather than bolstering it.
Green concludes by expressing uncertainty about how to navigate this impending shift, emphasizing that critical, principled choices will be necessary to avoid a "long greasy slide" into an unpredictable future.
The Gossip
AI's Security Paradox
Many commenters express skepticism regarding the article's premise that AI will make software *too* secure. Some argue that while AI might find existing bugs, the rapid rate of new, often sloppily written, AI-generated code will introduce an equal or greater number of new vulnerabilities, preventing a net reduction. Others question AI's current capability to find complex, multi-component exploits akin to those developed by sophisticated firms.
The 'Going Dark' Delusion
A significant portion of the discussion criticizes law enforcement's recurring 'going dark' narrative as disingenuous or overblown. Commenters highlight the vast array of surveillance methods still available, from extensive metadata collection to ubiquitous cameras, questioning how agencies can claim to be 'dark.' Many express satisfaction at the prospect of government surveillance capabilities being curbed and advocate for stronger privacy rights over expanded state access.
Backdoor Blowback
Commenters largely agree with the author's prediction that governments will likely push for mandated backdoors if exploits become scarce. There's concern about the potential for self-sabotage, where intentional vulnerabilities designed for law enforcement could be exploited by adversaries, weakening overall security. The conversation extends to the concept of 'digital sovereignty' and the possibility of countries developing their own, potentially backdoored, software ecosystems.
Historical and Practical Surveillance
Discussions delve into the historical evolution of surveillance, from physical wiretaps to modern digital methods, underscoring law enforcement's continuous desire for more access. Practical considerations for implementing backdoors are also debated, with some suggesting they might appear as 'superuser APIs' rather than traditional bugs. Concerns are raised about how such backdoors could be implemented (e.g., through executive pressure on company leadership) and the challenges of detecting them, even with AI.