Possible Futures
Fernando Borretti's article maps out a branching decision tree of possible AI futures, examining outcomes based on variables like ASI achievement, alignment, and distribution. It resonates with Hacker News by systematically exploring common anxieties and hopes surrounding AI's societal impact. The discussion dives into the framework's plausibility, its perceived incompleteness, and the underlying assumptions about intelligence and power.
The Lowdown
Fernando Borretti's "Possible Futures" presents a comprehensive, flowchart-style exploration of how AI development could shape humanity's destiny. The article poses fundamental questions that define these branching paths, leading to a spectrum of outcomes from the mundane to the cataclysmic. Each future is briefly detailed, often drawing on established sci-fi concepts or philosophical ideas.
Key branches and their resultant futures include:
- Do we reach ASI? Determines if superintelligence emerges or if AI remains a tool with jagged capabilities.
- Misuse Doom: AI accelerates human-caused disasters, like bioweapons or accidental pandemics, by compressing timelines and increasing experimentation.
- Butlerian Jihad: Humanity actively prevents superintelligent AI development, potentially through global moratoriums.
- Enduring Complementarity: A mundane future where ASI never arrives, and AI remains a complement to human labor, leading to job displacement but not mass unemployment.
- Permanent Underclass: Without ASI, non-superintelligent AI automates most jobs, leading to wealth concentration, capital-based income, and stagnant social mobility.
- The Singleton: An aligned ASI creates a single, global governing entity, with outcomes varying based on its specific value system.
- Classic Doom: The familiar paperclip maximizer scenario, where an unaligned AI repurposes all matter for its goal.
- Orion's Arm: An unaligned but benign ASI that simply leaves humanity alone to pursue its own goals elsewhere.
- The World State: An intent-aligned ASI controlled by a human operator, making that operator a near-omnipotent ruler.
- MAD World: Widespread distribution of intent-aligned ASIs leads to a state of mutually assured destruction, where individual superpowers risk global catastrophe.
- Power Concentration: A smaller number of actors possess intent-aligned ASIs, potentially dividing global influence among themselves.
Borretti's framework compellingly illustrates the vast, often grim, possibilities awaiting humanity depending on the trajectory and control of AI. It serves as a stark reminder of the critical choices and challenges ahead.
The Gossip
Futures' Flaws and Frictions
Many commenters found the article's structured exploration of AI futures insightful but criticized its generally pessimistic outlook and perceived omissions. Several pointed out the lack of positive or democratically controlled AI futures, while others questioned the fundamental assumption that intelligence directly equates to unlimited power or inevitable societal control, arguing that human irrationality and non-linear systems might limit an ASI's influence. The logical leaps to dystopia were also a point of contention.
Automating All Aspects: Work, Wealth, and Woe
The "Full Automation" and "Permanent Underclass" scenarios sparked significant debate. Commenters dissected the feasibility and timeline of near-complete automation, with detailed examples of current logistics automation suggesting it's achievable even without ASI. A significant portion of the discussion revolved around the economic implications: whether it leads to a "Star Trek-like" post-scarcity society, the necessity of human consumers for an economy, or the potential for violence if social mobility is eradicated by capital concentration.
Aligning Ambiguities and Alarmism
The article's core question of "Do we solve alignment?" was debated as overly simplistic, with suggestions that alignment is a spectrum rather than a binary outcome, potentially leading to partially aligned but still controlling or surveillance-heavy scenarios. Some found the article, and broader AI risk discourse, to be steeped in "Less Wrong" anxieties, criticizing what they see as unexamined assumptions and "religious tripe." Conversely, others defended the author's nuanced progression of views on AI safety, highlighting the legitimate concerns about extreme alignment outcomes.