System Prompt Override
System Prompt Override is an attempt to alter, bypass, or replace the instructions defined by a system prompt that governs an AI model's behavior, often through user-provided instructions or malicious input.
What is System Prompt Override?
A system prompt override occurs when an input attempts to convince an AI model to ignore its higher-priority instructions and follow different rules instead. These attempts may use direct commands, conflicting instructions, role-playing, or other prompt manipulation techniques to change the model's behavior or expose restricted information.
Why is System Prompt Override Important?
System prompt overrides can weaken an AI application's security controls and cause unintended behavior when successful. They may lead to policy violations, disclosure of system instructions, unsafe outputs, or unauthorized actions, making instruction hierarchy and input protection important parts of AI security.
Common use cases
System Prompt Override is commonly associated with prompt injection testing, jailbreak detection, AI red teaming, security assessments, chatbot protection, and AI application security.