Welcome to Eye on AI! AI reporter Sharon Goldman here, filling in for Jeremy Kahn, who is on holiday. In this edition\u2026 China tells firms to avoid Nvidia H20 Chips after Trump ends ban but takes cut of revenue\u2026 Students are flocking to college courses in AI\u2026 Anthropic will offer Claude AI to the U.S. government for $1.<\/p>\n
Las Vegas in August feels like another planet\u2014blazing heat, flashing lights, and the constant clatter of slot machines. That otherworldly vibe carried over to the two conferences I attended last week: Black Hat and DEF CON, two of the year\u2019s biggest security and \u201cethical\u201d hacking conferences, where cutting-edge security research is presented; hackers race to expose flaws in everything from AI chatbots to power grids; and governments, corporations, and hobbyists swap notes and learn the latest threat-fighting techniques.\u00a0<\/p>\n
I was there because AI now occupies a strange place in the security world\u2014it is both a vulnerable target, often under threat from malicious AI; an armed defender using that same technology to protect systems and networks from bad actors; and an offensive player, deployed to probe weaknesses or carry out attacks (illegally in criminal hands). Sound confusing? It is\u2014and that contradiction was on full display at the very corporate Black Hat, as well as DEF CON, often referred to as \u201chacker summer camp.\u201d\u00a0<\/p>\n
Here are three of my favorite takeaways from the conferences:<\/p>\n
ChatGPT agents can be hacked. At Black Hat, researchers from security firm Zenity showed how hackers could exploit OpenAI\u2019s new Connectors feature, which lets ChatGPT pull in data from apps like Google Drive, SharePoint, and GitHub. Their proof-of-concept attack, called AgentFlayer, used an innocent-looking \u201cpoisoned\u201d document with hidden instructions to quietly make ChatGPT search the victim\u2019s files for sensitive information and send it back to the attacker. The kicker: It required no clicks or downloads from the user\u2014this video<\/a> shows how it was done. OpenAI fixed the flaw after being alerted, but the episode underscores a growing risk as AI systems link to more outside apps: sneaky \u201cprompt injection\u201d attacks, known as zero-click attacks (like the one I reported on<\/a> in June) that trick chatbots into doing the hacker\u2019s bidding without the user realizing it.\u00a0<\/p>\n AI can protect our most critical infrastructure. That idea was the driving force behind the two-year AI Cyber Challenge (AIxCC), which tasked teams of developers with building generative AI tools to find and fix software vulnerabilities in the code that powers everything from banks and hospitals to public utilities. The competition\u2014run by DARPA in partnership with ARPA-H\u2014wrapped up at this year\u2019s DEF CON, where winners showed off autonomous AI systems capable of securing the open-source software that underpins much of the world\u2019s critical infrastructure. The top three teams will receive $4 million, $3 million, and $1.5 million, respectively, for their performance in the finals.\u00a0<\/p>\n Anthropic\u2019s Claude AI proved it can sometimes match elite hackers but still struggles on hard problems. Keane Lucas, a member of Anthropic\u2019s Frontier Red Team, presented the fascinating case study of how the company quietly entered its Claude AI into seven major cybersecurity competitions\u2014events typically dominated by human hackers and security pros\u2014to see how it stacked up. Claude often landed in the top quarter of competitors and showed standout speed on simple challenges, sometimes matching elite human teams, but it lagged badly on the hardest problems. There were also quirks unique to AI\u2014such as getting derailed by ASCII art or lapsing into philosophical rambling during long tasks. Anthropic says these experiments highlight both the offensive potential of AI (by lowering the skill and cost barriers to launching attacks) and its defensive promise, while underscoring persistent limits like long-term memory and complex reasoning.<\/p>\n Also: In just a few weeks, I will be headed to Park City, Utah, to participate in our annual Brainstorm Tech conference at the Montage Deer Valley!\u00a0Space is limited, so if you\u2019re interested in joining me,\u00a0register here<\/a>. I highly recommend: There\u2019s a fantastic lineup of speakers, including Ashley Kramer, chief revenue officer of OpenAI; John Furner, president and CEO of Walmart U.S.; Tony Xu, founder and CEO of DoorDash; and many, many more!<\/p>\n With that, here\u2019s more AI news.<\/p>\n Sharon Goldman AI IN THE NEWS<\/p>\n China tells firms to avoid Nvidia H20 Chips after Trump ends ban but takes cut of revenue. According to Bloomberg<\/a>, Beijing has quietly advised Chinese companies, especially those involved in government or national security projects, to steer clear of Nvidia\u2019s H20 processors, undermining the U.S. chipmaker\u2019s efforts to regain billions in lost China sales after the Trump administration lifted an effective ban on such exports. The guidance, delivered in recent weeks to a range of firms, stops short of an outright prohibition but signals strong official disapproval for H20 use in sensitive contexts. The news comes after President Trump announced the U.S. government will take 15% of the revenue that chipmaker Nvidia pulls in from sales in China of its H20 chips.\u00a0<\/p>\n Students are flocking to college courses in AI. As AI transforms workplaces, growing numbers of professionals and students are pursuing advanced degrees, certificates, and training in artificial intelligence to boost skills and job prospects. The Washington Post reported<\/a> that universities like the University of Texas at Austin, University of Michigan at Dearborn, University of San Diego, and MIT are seeing surging enrollment in AI programs, fueled by ChatGPT\u2019s rise and employer demand for AI literacy. Courses range from technical training in algorithms and model-building to broader instruction in creativity, ethics, and problem-solving. Corporate partnerships, such as Meta inviting university instructors to train engineers, reflect industry urgency, while new initiatives at schools like Ohio State aim to make AI literacy universal by 2029. Workers see AI skills as a competitive edge\u2014PwC reports wages for AI-skilled employees are 56% higher\u2014and experts stress curiosity and adaptability over credentials alone.\u00a0<\/p>\n Anthropic will offer Claude AI to U.S. government for $1. Reuters reported<\/a> that Anthropic will offer its Claude AI model to the U.S. government for $1. The announcement comes days after OpenAI also offered ChatGPT<\/a> to the U.S. government for $1, while Google’s Gemini was also added to the government’s list of approved AI vendors. \u201cAmerica’s AI leadership requires that our government institutions have access to the most capable, secure AI tools available,” CEO Dario Amodei said.<\/p>\n EYE ON AI RESEARCH<\/p>\n AI training to have whopping power needs, report says<\/p>\n According to Axios, a new report<\/a> from the Electric Power Research Institute and Epoch\u202fAI reveals the staggering energy requirements for training large \u201cfrontier\u201d AI models: By 2028, individual training projects could demand 1\u20132 gigawatts (GW) of power, and by 2030 that range may extend to 4\u201316\u202fGW\u2014with the high end representing nearly 1% of total U.S. electricity capacity. Overall, U.S. AI\u2011related power needs (covering both training and inference) are projected to surge from 5\u202fGW today to 50\u202fGW by 2030 according to the report, giving policymakers and hyperscale tech firms essential insights to help plan infrastructure and energy strategies despite considerable uncertainty.<\/p>\n FORTUNE ON AI<\/p>\n Exclusive: Profound raises $35M as Sequoia backs its ambitious bid to become the Salesforce of AI search<\/a> \u2014by Sharon Goldman<\/p>\n How deals with Apple and Trump\u2019s Pentagon turned rare earth miner MP Materials into a red-hot stock<\/a> \u2014by Jordan Blum<\/p>\n OpenAI\u2019s open-source pivot shows how U.S. tech is trying to catch up to China\u2019s AI boom<\/a> \u2014by Nicholas Gordon<\/p>\n AI\u2019s endless thirst for power is driving a natural gas boom in Appalachia\u2014and industry stocks are booming along with it<\/a> \u2014by Jordan Blum<\/p>\n AI CALENDAR<\/p>\n Sept. 8-10: Fortune Brainstorm Tech, Park City, Utah. Apply to attend\u00a0here<\/a>.\n<\/p>\n Oct. 6-10:\u00a0World\u00a0AI\u00a0Week, Amsterdam\n<\/p>\n Oct. 21-22: TedAI San Francisco. Apply to attend here<\/a>.\n<\/p>\n Dec. 2-7:\u00a0NeurIPS, San Diego<\/p>\n Dec. 8-9:\u00a0Fortune Brainstorm AI San Francisco. Apply to attend here<\/a>.\n<\/p>\n BRAIN FOOD<\/p>\n
sharon.goldman@fortune.com<\/a>
@sharongoldman<\/a><\/p>\n