TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development
Large language models write correct code for isolated problems but remain far weaker at autonomous machine-learning development, where an agent must revise data pipelines, models, and validation over hours of feedback, and on most competitions still finishes below strong human competitors. Outcome-based benchmarks record this gap but not its cause, because they grade the final submission and discard the development process behind it. We introduce TraceML, which pairs human and agent work on the same competitions under one version-level schema: 4,465 human Kaggle trajectories across 134 competi
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- FuzzySimilar title/name (fuzzy) · 59%AgentCore-8B →
“Fuzzy title match (0.73): “TraceML: An Empirical Analysis of Human-Agent Planning in Ma” ≈ “AgentCore-8B””
- PossiblePossibly related (embedding) · 55%Beyond benchmarks: The 5 pillars of AI evaluation systems →
- FuzzySimilar title/name (fuzzy) · 87%SWE-agent/SWE-agent →
“Fuzzy title match (0.94): “TraceML: An Empirical Analysis of Human-Agent Planning in Ma” ≈ “SWE-agent/SWE-agent””
- FuzzySimilar title/name (fuzzy) · 87%zhayujie/CowAgent →
“Fuzzy title match (0.94): “TraceML: An Empirical Analysis of Human-Agent Planning in Ma” ≈ “zhayujie/CowAgent””
- FuzzySimilar title/name (fuzzy) · 66%stefan-jansen/machine-learning-for-trading →
“Fuzzy title match (0.78): “TraceML: An Empirical Analysis of Human-Agent Planning in Ma” ≈ “stefan-jansen/machine-learning-for-trading””
- FuzzySimilar title/name (fuzzy) · 66%DataTalksClub/machine-learning-zoomcamp →
“Fuzzy title match (0.78): “TraceML: An Empirical Analysis of Human-Agent Planning in Ma” ≈ “DataTalksClub/machine-learning-zoomcamp””
- FuzzyOverlapping authors or contributors · 62%google-research/google-research →
“Shared author/contributor keys: sun”
- LinkedLinked via arxiv author · 85%Jiarui Yan →
“TraceML: An Empirical Analysis of Human-Agent Planning in Machine Learning Development”