GLM-5.1: Towards Long-Horizon Tasks
GLM-5.1, a new flagship AI model, touts impressive capabilities in tackling long-horizon software engineering tasks, demonstrating sustained optimization over hundreds of iterations and the autonomous generation of complex systems. However, Hacker News commenters are quick to point out significant real-world issues, including loss of coherence over longer contexts and poor service reliability, sparking debate about the practical utility versus benchmark claims. The discussion also touches on potential astroturfing, casting a shadow on the model's reported triumphs.
The Lowdown
GLM-5.1 is introduced as Z.AI's next-generation flagship model, specifically designed for agentic engineering with significantly enhanced coding capabilities. It claims state-of-the-art performance on various benchmarks, particularly distinguishing itself through its ability to sustain effectiveness on complex tasks over extended periods, breaking problems down, and iteratively refining solutions. This marks a notable shift from previous models that tended to plateau after initial gains.
- Complex Software Engineering: GLM-5.1 achieves leading results on SWE-Bench Pro, NL2Repo (repo generation), and Terminal-Bench 2.0 (real-world terminal tasks).
- Long-Horizon Optimization: The model excels at sustained optimization, exemplified by a vector database optimization task where it made meaningful improvements over 600+ iterations and 6,000+ tool calls, boosting QPS by roughly 6x and demonstrating structural changes in its approach.
- Machine Learning Workload Optimization: In optimizing GPU kernels, GLM-5.1 delivered a 3.6x speedup over 1,000+ turns, showing continued progress longer than its predecessor, though still behind Claude Opus 4.6 in this specific scenario.
- Open-Ended Task Execution: For less structured problems, GLM-5.1 autonomously built a complete, visually consistent web-based Linux desktop environment over 8 hours, continuously identifying and addressing areas for improvement without explicit metrics.
- Availability: GLM-5.1 is released as open source under the MIT License and is available on Z.AI developer platforms, HuggingFace, and ModelScope for local deployment.
While GLM-5.1 represents a significant advancement in long-horizon agentic AI, the authors acknowledge ongoing challenges such as escaping local optima, maintaining coherence over vast execution traces, and developing reliable self-evaluation for subjective tasks.
The Gossip
Coherency Catastrophes
Despite the article's focus on long-horizon coherence, many users reported that GLM-5.1 struggles significantly with maintaining context and producing sensible output over extended interactions. Commenters frequently describe the model devolving into 'schizo mode' or spouting 'gibberish' after reaching a certain token count (often around 100k-128k tokens), directly contradicting the core claim of the model's improvement. Some theorize this might be an infrastructure or hosting issue rather than an inherent model limitation, citing inconsistent context window performance over time.
Service Stability Scrutiny
Hacker News users expressed widespread dissatisfaction with the practical usability and cost-effectiveness of Z.AI's GLM-5.1 service. Complaints included severe performance issues like quantization problems, endless loops, excessively long response times (e.g., 50 minutes for a minor CSS change), frequent timeouts, and a generally 'unusable' experience. Several users also noted a recent price hike, making the service significantly more expensive than competitors like ChatGPT Plus, leading them to cancel subscriptions.
Astroturfing Accusations
A notable theme in the comments was the suspicion of astroturfing or 'booster comments' promoting the GLM-5.1 article. Several users, including a moderator (`dang`), pointed out newly created accounts posting highly positive or 'spammy' comments shortly after the article's release. This led to a broader discussion about the rising trend of AI SaaS companies employing marketing firms for comment spam and engagement bait on platforms like Reddit, questioning the authenticity of some initial engagement.