Qwen 3.8 27B available on Cerebras at 1500 tokens/s
Cerebras is making the Qwen 3.8 27B model available on its public inference endpoints, boasting an impressive 1500 tokens/s output speed. While the raw speed is lauded as 'wild,' the Hacker News crowd quickly dissects the practical limitations, including a restrictive 128k context window and complex pricing structures, which often negate the benefits for real-world agentic coding tasks. The discussion highlights the perennial tension between raw performance metrics and the actual user experience and cost-effectiveness in AI deployment.
The Lowdown
Cerebras has announced the availability of the Qwen 3.8 27B model on its public inference endpoints, emphasizing an exceptional output speed of 1500 tokens/second. This offering comes with a clear stance on model integrity and optimization practices.
- High-Speed Inference: The Qwen 3.8 27B model is designed for rapid output, achieving 1500 tokens/second, which Cerebras positions as a significant performance benchmark.
- Model Integrity: Cerebras commits to serving original, unpruned model architectures on its public endpoints. This ensures that users interact with the base model without architectural modifications.
- Quantization Strategy: While models are unpruned, Cerebras employs selective weight-only quantization (e.g., partial 16-bit/8-bit/4-bit) during storage. Crucially, sensitive layers maintain full precision, and activations, attention, and KV cache remain unquantized and in full precision to preserve quality.
- Pruning Research Separation: Cerebras conducts research into pruning techniques like REAP (Router-weighted Expert Activation Pruning), but these pruned models are shared on Hugging Face for research and experimentation, not via their production API. Future compressed models would be offered as distinct, transparent endpoints.
This release showcases Cerebras's focus on delivering high-throughput inference for specific models, coupled with a transparent approach to how those models are managed and optimized on their hardware.
The Gossip
Speedy Synthesis, Slow Situations
Commenters are initially amazed by the advertised 1500 tokens/second speed, calling it 'wild' and noting the 'awesome, super fast' output. Many describe the user experience as 'genuinely difficult to keep up with' due to the rapid generation, creating a 'very different feeling and UX'. However, this excitement is tempered by observations that input processing can still be slow, and other bottlenecks like shell commands or tool calls can negate the output speed advantage. Some argue that for coding, a more moderate speed (100-200 tok/s) might be the actual 'sweet spot' if it allows for better overall workflow, questioning if pure output speed is the only or primary bottleneck.
Contextual Conundrums & Pricing Puzzles
A significant point of contention revolves around the model's context window, with many finding the 128k context size for Qwen 3.8 to be a 'disappointing' limitation, insufficient for complex 'one shot xhigh requests' or 'long tasks'. The discussion also delves into pricing and rate limits, with a 150k TPM limit on public endpoints making the service 'unusable for many coding tasks'. Users express frustration over Cerebras's prompt caching policy, noting that while caching is supported, it 'doesn't reduce the price' – a policy some describe as 'flipping the narrative' or 'marketing spin' rather than a true cost benefit. Comparisons to other providers like OpenRouter reveal varying pricing structures that make Cerebras potentially competitive on output but often more expensive on input, particularly impacting 'agentic tasks' due to lack of cost-effective caching.
Cerebras's Customer Concerns & Core Competencies
The comment section frequently criticizes Cerebras's customer experience, with users reporting issues ranging from billing access restrictions and 'model not found' errors that are actually billing-related, to difficulties with customer support primarily residing on Discord, which some find inaccessible or frustrating. There's also debate over Cerebras's model strategy; some lament the removal of models like Gemma4 31B, suggesting it points to a focus on code generation over other 'agentic' use cases. Underlying these discussions is a curiosity about Cerebras's unique hardware – its giant inference chips and wafer-scale integration – leading to speculation on why it primarily hosts smaller models and whether its inference service serves more as an advertisement for its core hardware value proposition.