\u201cAI is being used by cyber actors at every chain of the attack right now,\u201d said Jacob Klein, who leads the threat intelligence team at AI start-up Anthropic.\u00a0<\/p>\n
AI groups are using a variety of techniques, including hiring external testers and using AI-powered tools, to detect and reduce malicious uses<\/a> of their powerful technologies. But experts warned the industry had not yet solved how to stop indirect prompt injection attacks. <\/p>\n Part of the problem is LLMs are designed to follow instructions, and currently do not distinguish between legitimate commands from users and input that should not be trusted. This is also the reason why AI models are prone to jailbreaking, where users can prompt LLMs to disregard their safeguards. <\/p>\n Klein said Anthropic works with external testers to make its Claude model<\/a> more resistant to indirect prompt injection attacks. They also have AI tools to detect when they might be happening.\u00a0<\/p>\n \u201cWhen we find a malicious use, depending on confidence levels, we may automatically trigger some intervention or it may send it to human review,\u201d he added.\u00a0<\/p>\n Google DeepMind uses a technique called automated red teaming, where the company\u2019s internal researchers constantly attack its Gemini model in a realistic way to uncover potential security weaknesses.\u00a0<\/p>\n In May, the UK\u2019s National Cyber Security Centre warned that this flaw posed an increased threat, as it risks exposing millions of companies and individuals that use LLMs and chatbots to sophisticated phishing attacks and scams<\/a>.\u00a0<\/p>\n LLMs also have another major vulnerability, where outsiders can create back doors and cause the models to misbehave by inserting malicious material into data that is then used in AI training. <\/p>\n These so-called \u201cdata poisoning attacks\u201d are easier to conduct than scientists previously believed, according to new research<\/a> published last month by AI start-up Anthropic, the UK\u2019s AI Security Institute and the Alan Turing Institute. <\/p>\n While these vulnerabilities pose big risks, experts argue that AI is also helping to boost company\u2019s defences against cyber attacks. <\/p>\n For years, attackers have had a slight advantage, in that they only needed to find one weakness, while defenders had to protect everything, said Microsoft\u2019s corporate vice-president and deputy chief information security officer Ann Johnson. <\/p>\n \u201cDefensive systems are learning faster, adapting faster, and moving from reactive to proactive,\u201d she added. <\/p>\n The race to solve flaws in AI models comes as cyber security is emerging as one of the top concerns for companies seeking to adopt AI tools into their business.<\/p>\n A recent Financial Times analysis<\/a> of hundreds of corporate filings and executive transcripts at S&P 500 companies last year found the most commonly cited worry was cyber security, which was mentioned as a risk by more than half of the S&P 500 in 2024.<\/p>\n Hacking experts said the advancement of AI in recent years has already boosted the multibillion-dollar cyber crime industry. It has provided amateur hackers with cheap tools to write harmful software, as well as systems for professional criminals to better automate and scale up their operations.\u00a0<\/p>\n LLMs allow hackers to quickly generate new malicious code that has not been detected yet, which makes it harder to defend against, said Jake Moore, global cyber security adviser at cyber security group ESET. <\/p>\n A recent study<\/a> by researchers at MIT found that 80 per cent of ransomware attacks they examined used AI, and in 2024, phishing scams and deepfake-related fraud linked to the technology saw a 60 per cent increase.<\/p>\n
![\"An](\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2025\/11\/https:\/\/images.ft.com\/v3\/image\/raw\/https%3A%2F%2Fd1e00ek4ebabms.cloudfront.net%2Fproduction%2Fd69475.jpeg\")
<\/a><\/p>\n