Adversarial prompt
Adversarial prompt: An input designed to cause errors, evasion or undesired behavior in a model. It can exploit ambiguity, hidden instructions or targeted perturbations and is used in red teaming. Robustness must be evaluated across the whole system, including retrieval and tools.
In italiano: Adversarial PromptLegge V · Overfitting, bias e collasso dei modelliIt can exploit ambiguity, hidden instructions or targeted perturbations and is used in red teaming. Robustness must be evaluated across the whole system, including retrieval and tools.
Examples
- Contradictory instructions
- Markup hiding commands
- Automated attack variants