IBM CEO says open weight models are now good and cheap on-prem
IBM's CEO declares open-weight AI models are now 90% as effective as frontier models for 10% of the cost, making on-prem enterprise AI a compelling, yet complex, proposition. This heralds a potential shift from cloud-first AI to local deployments, creating a fertile ground for IBM's hardware and services, and sparking debate among HN users about cost, control, and corporate motivations.
The Lowdown
During IBM's Q3 earnings call, CEO Arvind Krishna announced a significant shift in the AI landscape: open-weight models have rapidly closed the performance gap with proprietary 'frontier' models. This development makes on-premise AI deployments incredibly attractive for enterprises, boosting IBM's prospects in hardware and services.
- Krishna highlighted that open-weight models, once significantly inferior, now offer about 90% of the effectiveness of top frontier models but at a mere 10-25% of the token cost.
- He believes that for the 'vast majority' of enterprise tasks—like email summarization, contract analysis, and customer service—the slightly less effective but dramatically cheaper open models are the clear choice.
- This shift is driving 'booming interest' in on-prem AI infrastructure, which IBM is keen to capitalize on.
- However, Krishna acknowledged the practical challenges for businesses, such as a shortage of 'kernel engineers' and expertise in network configuration for efficient GPU clustering and caching.
- IBM plans to address these challenges with new software solutions and anticipates increased demand for its Spyre mainframe accelerator cards.
- The company reported 5% growth in infrastructure revenue and 9% in software, with Red Hat leading the charge, particularly in virtualization.
In essence, IBM is positioning itself as a key enabler for enterprises looking to leverage powerful, cost-effective AI solutions locally, aiming to solve the implementation hurdles and drive sales of its hardware and software.
The Gossip
Corporate Commentary Critique
Many commenters expressed skepticism regarding IBM's CEO's pronouncement, noting that his advocacy for on-premise, hardware-intensive AI perfectly aligns with IBM's business model. They highlighted the inherent bias in such statements made during earnings calls, questioning whether it's a neutral assessment or merely 'talking their book' to boost sales of mainframes and related infrastructure. The sentiment often boiled down to 'he would say that, wouldn't he?'
Local LLM Liberation
Conversely, a significant portion of the discussion celebrated the growing viability of running open-weight AI models locally. Users detailed their personal experiences with on-prem setups, emphasizing the benefits of cost savings, enhanced privacy (not sharing data with external providers), and freedom from API restrictions or censorship. Many felt empowered by the ability to customize, control, and scale their AI usage without reliance on cloud giants, predicting a future where local AI becomes the standard for many applications.
IBM's Identity and Opportunity
The comments also delved into IBM's historical and current market position, questioning whether the company can truly capitalize on this shift. Some highlighted IBM's legacy of selling expensive solutions and its perceived corporate dysfunction, casting doubt on its ability to offer competitive, affordable hardware or innovative software. Others, however, acknowledged IBM's deep pockets, existing hardware capabilities (like the Spyre accelerator), and extensive enterprise connections, suggesting a potential resurgence if they can overcome internal inertia and deliver user-friendly, open solutions.