Otap:Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories
Large language model agents solve tasks by generating trajectories that interleave planning, tool calls, and intermediate results. Current evaluation metrics reduce such a trajectory to a binary success flag or compare it against a reference by exact matching. A success flag cannot distinguish a sound solution from one that succeeds by luck, and says nothing about why a failed run went wrong. Exact matching penalizes plans that are valid but reordered or decomposed differently from the reference. We reframe trajectory evaluation as a distance between the agent's execution graph and a set of va
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 59%AgentCore-8B →
“Fuzzy title match (0.73): “Otap:Structure-Aware Optimal Transport for Evaluating Planni” ≈ “AgentCore-8B””
- FuzzySimilar title/name (fuzzy) · 87%SWE-agent/SWE-agent →
“Fuzzy title match (0.94): “Otap:Structure-Aware Optimal Transport for Evaluating Planni” ≈ “SWE-agent/SWE-agent””
- FuzzySimilar title/name (fuzzy) · 87%zhayujie/CowAgent →
“Fuzzy title match (0.94): “Otap:Structure-Aware Optimal Transport for Evaluating Planni” ≈ “zhayujie/CowAgent””
- FuzzySimilar title/name (fuzzy) · 59%NousResearch/hermes-agent →
“Fuzzy title match (0.73): “Otap:Structure-Aware Optimal Transport for Evaluating Planni” ≈ “NousResearch/hermes-agent””
- FuzzySimilar title/name (fuzzy) · 59%bojieli/ai-agent-book →
“Fuzzy title match (0.73): “Otap:Structure-Aware Optimal Transport for Evaluating Planni” ≈ “bojieli/ai-agent-book””
- FuzzySimilar title/name (fuzzy) · 59%2FastLabs/agent-squad →
“Fuzzy title match (0.73): “Otap:Structure-Aware Optimal Transport for Evaluating Planni” ≈ “2FastLabs/agent-squad””
- LinkedLinked via arxiv author · 85%Babak Barazandeh →
“Otap:Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories”
- LinkedLinked via arxiv author · 85%Subhabrata Majumdar →
“Otap:Structure-Aware Optimal Transport for Evaluating Planning and Execution in Agent Trajectories”