Prompt injection
Prompt injection: Malicious instructions embedded in inputs or retrieved content to divert an AI system’s behavior. It can arrive from users, web pages, documents or tool outputs. Mitigation separates data from instructions, limits permissions and filters actions rather than relying on a textual prohibition alone.
In italiano: Prompt InjectionLegge III · Come una rete impara dai datiIt can arrive from users, web pages, documents or tool outputs. Mitigation separates data from instructions, limits permissions and filters actions rather than relying on a textual prohibition alone.
Examples
- Hidden text in a web page
- Document instructing the model to ignore rules
- Tool returning unexpected commands