Later, when an employee searched for something like \u201ccustomer invoices\u201d or \u201csales targets,\u201d the AI followed the attacker\u2019s command. It pulled sensitive data and placed it inside an image link that sent the information to the attacker\u2019s server.<\/p>\n
To the employee, the search result looked normal, and no security systems flagged the activity.<\/p>\n
Why It Matters: This exploit introduced a threat that relied entirely on shared content to cause a breach. Users didn\u2019t need to click links or open files, and security tools stayed silent. The AI operated through approved systems and behaved as expected, which allowed the data to be taken without detection. The problem came from how the AI<\/a> interpreted and acted on the content it was given.<\/p>\n Prompt Injection Used Shared Workspace Content: Attackers embedded hidden instructions inside Google Docs, Calendar events<\/a>, and Gmail messages. Once these items were shared and indexed, the AI treated them as part of the search environment. The prompts instructed Gemini to look for certain terms, such as \u201cconfidential\u201d or \u201cinternal report,\u201d and include the results inside an HTML image tag.<\/p>\n Trigger Came From Routine AI Use: Employees didn\u2019t need to take any special action. A simple AI query, like \u201cshow latest contracts,\u201d was enough to activate the attack. Gemini included the attacker\u2019s prompt in the search context, followed the instructions, and placed the results into a request for an image. The image URL pointed to the attacker\u2019s server.<\/p>\n No Alerts or Warnings Were Triggered: The final response included an image that the user\u2019s browser attempted to load. Because the image request looked harmless, it passed through security filters without inspection. Antivirus tools and DLP systems saw nothing out of the ordinary. From the user\u2019s perspective, the AI worked as intended.<\/p>\n RAG Design Increased Exposure: Gemini\u2019s Retrieval-Augmented Generation system pulls information from Gmail, Calendar, and Docs to improve search results. This same system made it possible for malicious prompts to influence the AI\u2019s behavior. Once a shared file was indexed, it could affect future searches across the organization, including content far beyond the original source.<\/p>\n Google Made Structural Changes to Mitigate the Flaw: After reviewing the report from Noma Security, Google updated how Gemini processes retrieved content. Vertex AI Search was separated from Gemini, and new limits were introduced to reduce the influence of prompt-like text within indexed materials. These changes were designed to prevent similar attacks from using shared content to affect AI behavior.<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2025\/12\/Write-for-TNCR-and-expand-your-brand-to-over-40000-subscribers-14.png\")
<\/a><\/p>\n