Facepalm: Prompt injection attacks are emerging as a significant threat to generative AI services and AI-enabled web browsers. Researchers have now uncovered an even more insidious method – one that could quietly turn AI agents into powerful tools for manipulation and cyberattack.
A new report by NeuralTrust highlights the immature state of today's AI browsers. The company found that ChatGPT Atlas, the agentic browser recently launched by OpenAI for macOS, is vulnerable to a novel prompt injection attack capable of "jailbreaking" the browser's omnibox. In Atlas and other Chromium-based browsers, the omnibox serves as a unified field for both web addresses and search queries.
The attack hides malicious instructions within a deliberately malformed URL that begins with the "https:" prefix and appears to contain domain-like text. However, Atlas interprets the remaining portion of the URL as natural-language instructions for its AI agent.
Atlas treats instructions embedded in the maliciously crafted URL as genuine commands from a trusted user and executes them with elevated privileges. Attackers can exploit the flaw to bypass the browser's safety policies, risking user privacy and data security.
The researchers demonstrated the real-world disruption that omnibox-based prompt-injection attacks can inflict on the web ecosystem. For example, attackers could trick users into pasting a malicious URL into the omnibox, which might open fake Google pages or other phishing content. They could also embed destructive instructions that abuse the AI agent to delete a user's files in Google Drive.
Chromium treats omnibox prompts as trusted user input, allowing attackers to bypass additional security checks on external webpages. The new attack vector opens the door for criminal actors to subvert a user's intent and perform cross-domain actions.
NeuralTrust said this type of "boundary" error is now common among agentic AI browsers. These new Chrome clones should adopt stricter rules for parsing URLs. They should also explicitly ask users whether they intend to use the omnibox for web navigation or to issue instructions to the chatbot. The researchers are now testing additional AI browsers to determine whether they are vulnerable to the same prompt-injection attack as ChatGPT Atlas.