The unintended exposure of private or confidential data through the use of AI, for example by pasting it into an unapproved tool or through outputs, logs, or hidden instructions.
Definition
AI data leakage is the unintended exposure of private, confidential, or regulated data through the use of AI. The commonest route is an employee pasting data into a tool that is not approved for it. Others are data held in a vendor’s systems longer than intended or used to train its models, data repeated in outputs or copied into logs with wide access, an assistant tricked by hidden instructions into revealing files, and over-broad connections that let an AI reach more than its task needs.
History
Concern grew quickly after generative AI chat tools became widely available in 2022 and 2023, when several companies reported or restricted staff use after sensitive material was shared with public tools. The risk appears in lists such as the OWASP Top 10 for LLM Applications as sensitive information disclosure.
When to Use
Treat leakage as a design question for every AI use: which data goes in, where it goes, who can see the copies, and how long it stays.
Step-by-Step
- Classify the data and map classes to kinds of tool.
- Provide approved tools that are easy to reach.
- Minimize and mask what goes into prompts.
- Read the vendor terms: training, retention, location, subprocessors.
- Control logs and outputs.
- Limit connections and give least privilege.
- Report and learn from incidents.
Examples
- A customer drawing pasted into a public chatbot.
- An assistant with access to a whole drive that lists file names when a test email asks it to.
Common Pitfalls
- A ban with no alternative.
- Trusting a privacy setting unread.
- Poor masking.
- Forgetting logs and outputs.