Retrieval Augmentation Abuse
refers to attacks that manipulate or exploit the retrieval component of a RAG system to influence the information provided to an AI model.
What is Retrieval Augmentation Abuse?
Retrieval augmentation abuse can occur when attackers introduce malicious, misleading, or manipulated content into a knowledge base or other retrieval source. When that content is retrieved in response to a query, it can influence the model's context and potentially lead to unsafe, inaccurate, or unintended outputs.
Why is Retrieval Augmentation Abuse Important?
RAG systems depend on the quality and integrity of retrieved information. Malicious content in retrieval sources can bypass some traditional input controls and influence model behavior through trusted context. Protecting retrieval pipelines helps maintain the integrity, reliability, and security of AI applications.
Common use cases
Retrieval Augmentation Abuse is relevant to enterprise RAG systems, AI knowledge bases, document assistants, enterprise search, customer support systems, and applications using external or user-controlled data sources.