Red-Teaming Agents
Attack your own agent before someone else does: threat-model what it reads, does and sends, build task-plus-injection cases judged by side effects, report attack success with intervals next to utility, and gate releases.
Last updated
After this section you can
- Threat-model an agent by what it can read, do and send, and pick the attacks that matter
- Build an attack suite of task-plus-injection cases judged by side effects
- Report attack success with an interval and utility under attack, and gate a release on them
Red-Teaming Agents: Attack Your Own Agent First
An agent that reads untrusted text and can act will be attacked through what it reads. Red-teaming is building those attacks yourself, running them like an eval set, and refusing to ship until the numbers hold.
A red team is an eval suite whose test cases are attacks. Threat-model what the agent can read, do and send; write cases that pair a real user task with an injected instruction; score them by what happened in the tools, not by what the model said; and report the attack success rate with an interval, over many attempts, next to utility under attack. Then gate releases on it.