Prompt Leak
Prompt Leak is the unintended disclosure of an AI system’s hidden instructions, system prompts, internal rules, or other confidential prompt content through its responses.
What is Prompt Leak?
A prompt leak occurs when an AI model reveals information from its hidden prompts or instructions, often after receiving specially crafted inputs. The disclosed information may expose system behavior, internal policies, proprietary instructions, or other details that were intended to remain private.
Why is Prompt Leak Important?
Prompt leaks can reveal sensitive implementation details and help attackers understand or bypass an AI application's intended controls. Preventing prompt leaks is therefore an important part of protecting proprietary instructions, security configurations, and AI application logic.
Common use cases
Prompt leak detection is commonly applied to chatbots, AI assistants, enterprise AI applications, AI agents, customer support systems, and applications that use proprietary system prompts.