Mythos Finds a Curl Vulnerability

The story begins by discussing Anthropic's new AI model, Mythos, which was touted as being exceptionally skilled at finding security vulnerabilities in code. Due to its perceived "dangerous" efficacy, Anthropic initially restricted its release, offering access only to select companies and open-source projects, including curl. The author, lead developer of curl, was granted access, albeit indirectly, to have Mythos scan curl's extensively audited codebase.

• Anthropic's Mythos AI generated significant media buzz for its supposed prowess in discovering security flaws, with Anthropic limiting public release.
• The curl project, already deeply committed to security and using multiple AI code analysis tools (like AISLE, Zeropath, OpenAI's Codex Security, GitHub Copilot, Augment Code), received a scan report from Mythos.
• Prior AI tools had led to 200-300 bugfixes in curl over the past 8-10 months, including over a dozen CVEs, demonstrating the existing value of AI in their security process.
• Mythos's initial report claimed five "confirmed security vulnerabilities" after scanning 178K lines of curl's C code.
• Upon review by the curl security team, four of these findings were dismissed as false positives or minor bugs, leaving only one low-severity vulnerability.
• This single confirmed vulnerability will be published as a CVE with the upcoming curl release 8.21.0 in late June.
• The report also included about twenty non-vulnerability bugs, which the curl team is investigating and fixing, noting the high quality and low false-positive rate of these non-critical findings.
• The author notes that curl is an exceptionally secure and well-audited codebase, having published 188 CVEs to date and running on over twenty billion instances across 110 operating systems.
• Despite the low number of new findings from Mythos, the author reaffirms that AI-powered code analyzers are significantly better than traditional tools and are now indispensable for finding security flaws.
• AI analyzers excel at tasks like spotting discrepancies between comments and code, checking configurations beyond traditional environments, understanding third-party APIs, questioning protocol implementations, and generating clear explanations or even patch suggestions.
• Critically, AI tools, including Mythos, primarily find existing types of errors, not novel or groundbreaking vulnerabilities, but they do so with greater volume and efficiency.

While Mythos did not deliver the "dangerously good" breakthrough promised by its initial hype, leading the author to conclude the buzz was largely marketing, it still contributed to curl's ongoing security improvements. The story underscores that while curl's established security practices and previous AI scans had already tackled many "easier" bugs, AI code analysis remains a crucial, evolving component of modern software security, finding new instances of known error types and continually enhancing code quality.