AI first, security later: As GenAI tools make their way into mainstream apps and workflows, serious concerns are mounting about their real-world safety. Far from boosting productivity, these systems are increasingly being exploited – benefiting cybercriminals and cost-cutting executives far more than end users. Researchers this week uncovered how Google's Gemini model used in Gmail can be subverted in an incredibly simple way, making phishing campaigns easier than ever.
Mozilla recently unveiled a new prompt injection attack against Google Gemini for Workspace, which can be abused to turn AI summaries in Gmail messages into an effective phishing operation. Researcher Marco Figueroa described the attack on 0din, Mozilla's bug bounty program for generative AI services.
We strongly recommend reading the full report if you still think GenAI technology is ready for deployment in production or live, customer-facing products.
Like many other Gemini-powered services, the AI summary feature was recently forced onto Gmail users as a supposedly powerful new workflow enhancement. The "summarize this email" option is meant to provide a quick overview of selected messages – though its behavior depends heavily on Gemini's whims. Originally introduced as an optional feature, the summary tool is now baked into the Gmail mobile app and functions without user intervention.
The newly disclosed prompt injection attack exploits the autonomous nature of these summaries – and the fact that Gemini will "faithfully" follow any hidden prompt-based instructions. Attackers can use simple HTML and CSS to hide malicious prompts in email bodies by setting them to zero font size and white text color, rendering them essentially invisible to users. This is somewhat similar to a story we reported on this week, about researchers hiding prompts in academic papers to manipulate AI peer reviews.
Using this method, researchers crafted an apparently legitimate warning about a compromised Gmail account, urging the user to call a phone number and provide a reference code.
According to 0din's analysis, this type of attack is considered "moderate" risk, as it still requires active user interaction. However, a successful phishing campaign could lead to serious consequences by harvesting credentials through voice-phishing.
Even more concerning, the same technique can be applied to exploit Gemini's AI in Docs, Slides, and Drive search. Newsletters, automated ticketing emails, and other mass-distributed messages could turn a single compromised SaaS account into thousands of phishing beacons, the researchers warn.
Figueroa described prompt injections as "the new email macros," noting that the perceived trustworthiness of AI-generated summaries only makes the threat more severe.
In response to the disclosure, Google said it is currently implementing a multi-layered security approach to address this type of prompt injection across Gemini's infrastructure.
Google Gemini vulnerable to a stupidly easy prompt injection attack in Gmail AI summaries
