Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security
LLM-based agents process external content, exposing them to prompt injection and multi-turn manipulation. Most safety benchmarks evaluate defenders against fixed attack pools collected before evaluation, single-turn or multi-turn. We present a 21-scenario benchmark for \emph{adaptive multi-round attacks against memoryless LLM defenders}: an autonomous LLM attacker observes prior defender responses and pivots across rounds, while each defender response is evaluated as a fresh interaction. Holding the 21 scenarios, attackers, defenders, and structured-output scoring fixed, restricting scoring to
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 59%AgentCore-8B →
“Fuzzy title match (0.73): “Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for ” ≈ “AgentCore-8B””
- PossiblePossibly related (embedding) · 56%New LLM Coordination Benchmark - Benchmarking Open-Ended Multi-Agent Coordination in Language Agents [R] →
- FuzzyOverlapping authors or contributors · 62%firecrawl/firecrawl →
“Shared author/contributor keys: jain”
- FuzzySimilar title/name (fuzzy) · 87%SWE-agent/SWE-agent →
“Fuzzy title match (0.94): “Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for ” ≈ “SWE-agent/SWE-agent””
- FuzzySimilar title/name (fuzzy) · 87%zhayujie/CowAgent →
“Fuzzy title match (0.94): “Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for ” ≈ “zhayujie/CowAgent””
- FuzzySimilar title/name (fuzzy) · 66%open-multi-agent/open-multi-agent →
“Fuzzy title match (0.78): “Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for ” ≈ “open-multi-agent/open-multi-agent””
- FuzzySimilar title/name (fuzzy) · 59%NousResearch/hermes-agent →
“Fuzzy title match (0.73): “Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for ” ≈ “NousResearch/hermes-agent””
- LinkedLinked via arxiv author · 85%Devina Jain →
“Adaptive Adversaries: A Multi-Turn, Multi-LLM Benchmark for LLM Agent Security”