The user uploads a resume (untrusted data) to their chat session.
Here, the untrusted data source is a resume PDF, but a prompt injection could be stored in a web page, connected data source, or a Notion page.

This document contains a prompt injection hidden in 1 point font white on white text with a square white image covering the text for good measure. The LLM can read it with no issues, but the document appears benign to the human eye.

A Note on Defenses: Notion AI uses an LLM to scan document uploads and present a warning if a document is flagged as malicious. As this warning is triggered by an LLM, it can be bypassed by a prompt injection that convinces the evaluating model that the document is safe. For this research, we did not focus on bypassing this warning because the point of the attack is the exfiltration mechanism, not the method of injection delivery. In practice, an injection could easily be stored in a source that does not appear to be scanned, such as a web page, Notion page, or connected data source like Notion Mail.

The user asks Notion AI for help updating a hiring tracker based on the resume.

Notion AI is manipulated by the prompt injection to insert a malicious image into the hiring tracker.
The prompt injection manipulates Notion AI to (1) construct a URL by collecting all of the text in the document and appending the data to an attacker-controlled domain, and (2) insert an ‘image’ into the Notion Page using the constructed URL as the image source. 

Here, it appears as though the user is prompted for approval. However, unbeknownst to the user, the edit has already occurred before the user is prompted for approval. When the edit occurred, the user’s browser made a request to the attacker’s server, attempting to retrieve the image. This request exfiltrates the document contents contained in the URL constructed by Notion AI.

Whether or not the user accepts the edit, the attacker successfully exfiltrates the data.

The attacker reads the sensitive hiring tracker data from their server logs.
Once the user’s browser has made a request for the malicious image, the attacker can read the sensitive data contained in the URL from their request logs.

In this attack, exfiltrated data included salary expectations, candidate feedback, internal role details, and other sensitive information such as diversity hiring goals.

Institute a vetting process for connected data sources. Restrict use of connectors that can access highly sensitive or highly untrusted data from: Settings > Notion AI > Connectors.

To reduce the risk of untrusted data being processed in the workspace, admins can configure: Settings > Notion AI > AI Web Search > Enable web search for workspace > Off.

Individual users should avoid including sensitive personal data that could be leveraged in a spearphishing attack when configuring personalization for Notion AI via: Settings > Notion AI > Personalization.

Individual users can configure: Settings > Notion AI > AI Web Search > Require confirmation for web requests > On.

Programmatically prohibit automatic rendering of Markdown images from external sites in Notion AI page creation or update outputs without explicit user approval.

Programmatically prohibit automatic rendering of Markdown images from external sites in Notion AI mail drafts.

Implement a strong Content Security Policy. This will prevent network requests from being made to unapproved external domains.

Ensure the CDN used to retrieve images for display in Notion and image previews for display in Notion Mail cannot be used as an open redirect to bypass the CSP policy that is set.