A new AI coding challenge has revealed its first winner \u2014 and set a new bar for AI-powered software engineers.\u00a0<\/p>\n
On Wednesday at 5pm PST, the nonprofit Laude Institute announced the first winner of the K Prize, a multi-round AI coding challenge launched by Databricks and Perplexity co-founder Andy Konwinski. The winner was a Brazilian prompt engineer named Eduardo Rocha de Andrade, who will receive $50,000 for the prize. But more surprising than the win was his final score: he won with correct answers to just 7.5% of the questions on the test.<\/p>\n
\u201cWe\u2019re glad we built a benchmark that is actually hard,\u201d said Konwinski. \u201cBenchmarks should be hard if they\u2019re going to matter,\u201d he continued, adding: \u201cScores would be different if the big labs had entered with their biggest models. But that\u2019s kind of the point. K Prize runs offline with limited compute, so it favors smaller and open models. I love that. It levels the playing field.\u201d<\/p>\n
Konwinski has pledged $1 million to the first open-source model that can score higher than 90% on the test.<\/p>\n
Similar to the well-known SWE-Bench system, the K Prize tests models against flagged issues from GitHub as a test of how well models can deal with real-world programming problems. But while SWE-Bench is based on a fixed set of problems that models can train against, the K Prize is designed as a \u201ccontamination-free version of SWE-Bench,\u201d using a timed entry system to guard against any benchmark-specific training. For round one, models were due by March 12th. The K Prize organizers then built the test using only GitHub issues flagged after that date.<\/p>\n
The 7.5% top score stands in marked contrast to SWE-Bench itself, which currently shows a 75% top score on its easier \u2018Verified\u2019 test and 34% on its harder \u2018Full\u2019 test. Konwinski still isn\u2019t sure whether the disparity is due to contamination on SWE-Bench or just the challenge of collecting new issues from GitHub, but he expects the K Prize project to answer the question soon.<\/p>\n
\u201cAs we get more runs of the thing, we\u2019ll have a better sense,\u201d he told TechCrunch, \u201cbecause we expect people to adapt to the dynamics of competing on this every few months.\u201d<\/p>\n
Techcrunch event<\/p>\n
\n\t\t\t\t\t\t\t\t\tSan Francisco
\n\t\t\t\t\t\t\t\t\t\t\t\t\t|
\n\t\t\t\t\t\t\t\t\t\t\t\t\tOctober 27-29, 2025\n\t\t\t\t\t\t\t<\/p>\n
It might seem like an odd place to fall short, given the wide range of AI coding tools already publicly available \u2013 but with benchmarks becoming too easy, many critics see projects like the K Prize as a necessary step toward solving AI\u2019s growing evaluation problem<\/a>.<\/p>\n