Tell HN: Litellm 1.82.7 and 1.82.8 on PyPI are compromised
The Litellm PyPI package versions 1.82.7 and 1.82.8 were compromised, containing a stealthy credential-stealing malware that activates on Python interpreter startup. This sophisticated supply chain attack, linked to the notorious TeamPCP group, exfiltrated a wide array of sensitive data, from SSH keys to crypto wallets. The incident has sparked a critical discussion on dependency trust, sandboxing, and the vulnerabilities inherent in modern software ecosystems, particularly in the age of AI.
The Lowdown
A critical security alert was raised after malicious code was discovered in recent versions of the Litellm Python package distributed via PyPI. Specifically, litellm==1.82.8 included a malicious .pth file, and 1.82.7 contained a payload in proxy_server.py, both designed to automatically execute upon Python interpreter launch or package import, respectively.
- Compromise Mechanism: The
.pthfile in1.82.8ensures execution without an explicitimport litellmstatement, leveraging a Python interpreter feature. The payload itself was double base64-encoded to evade detection. - Extensive Data Collection: The malware systematically collected a vast amount of sensitive information, including system details, environment variables (often containing API keys and secrets), SSH keys, various cloud provider credentials (AWS, GCP, Azure, Kubernetes), Git configurations, Docker configs, package manager credentials, shell histories, crypto wallet data, SSL/TLS private keys, and CI/CD secrets.
- Exfiltration: Collected data was encrypted using a randomly generated AES key, which was then encrypted with a hardcoded RSA public key. The encrypted bundle was packed into a
tpcp.tar.gzarchive and exfiltrated to an attacker-controlled server athttps://models.litellm.cloud/. - Widespread Impact: Any system—from local development machines and CI/CD pipelines to Docker containers and production servers—that installed
litellm==1.82.8(or1.82.7) is potentially compromised, necessitating immediate rotation of all credentials. - Attribution & Root Cause: This incident is strongly linked to the TeamPCP supply chain attacks, which previously compromised Trivy. The Litellm team confirmed their CI/CD pipeline, which used Trivy, was likely the initial point of compromise, leading to their PyPI publishing credentials and GitHub accounts being breached.
The incident serves as a stark reminder of the escalating risks within software supply chains and the profound implications of trusting third-party dependencies.
The Gossip
Supply Chain Catastrophe
Commenters expressed deep concern about the inherent fragility of modern software supply chains and the pervasive trust crisis in dependencies. Many echoed the sentiment that developers can no longer implicitly trust packages, highlighting the need for fundamental changes in how software components are managed and isolated. The recurring nature of such attacks, following incidents like XZ Utils and Trivy, left many feeling that current practices are unsustainable.
TeamPCP Takedown Tactics
The discussion quickly linked the Litellm compromise to the TeamPCP threat actor, responsible for recent attacks on Trivy. Details emerged about their modus operandi, including the use of bot accounts to flood GitHub issue threads with generic, positive comments to stifle genuine discussion and investigation. The connection to the Trivy compromise through Litellm's CI/CD pipeline was a key revelation, showing a chain of attacks rather than isolated incidents.
Mitigation Measures & Mindset Shifts
A significant portion of the conversation focused on practical and philosophical solutions. Recommendations ranged from implementing stricter sandboxing for development and production environments to rethinking dependency management—pinning versions, auditing packages, and even 'vibe coding' replacements for common dependencies. Proposals for ecosystem-level changes included multi-factor authentication for package releases, better build artifacts, and an ecosystem-wide credential reset to combat the widespread compromise.
Maintainer's Measured Response
The Litellm maintainer, Krrish, actively participated in the discussion, providing timely updates, confirming the compromise, and outlining steps taken (e.g., yanking versions, rotating credentials). Their candid and human response, including an apology, was widely appreciated by the community, contrasting sharply with typical corporate statements. The maintainer also sought assistance from security experts to audit their systems further.