Black Hat Asia Open source models can find bugs as effectively as Anthropic’s Mythos, according to Ari Herbert-Voss, CEO of AI-powered security startup RunSybil and OpenAI’s first security hire.<\/p>\n
Speaking at the Black Hat Asia conference in Singapore today, Herbert-Voss said Mythos excels at finding both “shallow” bugs – well-described flaws that are and easy to validate – and more complex vulnerabilities.<\/p>\n
In his talk, he attributed this to “supralinear scaling”: where researchers assumed LLM capability would improve linearly, evidence now suggests a model trained on twice the data, compute, and time produces something four times more capable.<\/p>\n
He hinted supralinear scaling might produce even better multipliers but could not say more due to a non-disclosure agreement.<\/p>\n
Anthropic has kept access to Mythos tghtly restricted, citing fears of misuse.<\/p>\n
However Herbert-Voss argues attackers and defenders alike can achieve comparable results with open source models by building “scaffolding” to run several of them in harness. That approach also improves defense in depth, as different models tend to catch different flaws \u2014 a useful hedge against any single model’s blind spots.<\/p>\n
Cost is another driver. Mythos is expensive to build and run, and may never be publicly available, making open source alternatives not just viable but necessary for many organizations.<\/p>\n
Herbert-Voss feels human expertise is still needed to orchestrate open source models so they together deliver Mythos-grade performance, and to assess the bug reports AI generates.<\/p>\n