Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies
Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? We introduce Reconstruction, a blind idea-recovery benchmark that withholds the seed paper and all contemporaneous or future literature, and asks models to propose hypotheses that an independent large language model judge matches against the held-out ground-truth idea. A strict anti-leakage protocol-temporal citation cutoff, anonymous reference IDs, and frozen per-paper bibliographies, which prevents prompt-time leakage of the seed idea. Across six scientific doma
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 46%AI for Scientific Literature Mining: PubMed, Semantic Scholar, and What Actually Works - Technology Networks →
- PossiblePossibly related (embedding) · 45%When AI agrees with us: large language models as amplifiers of scientific bias - Springer Nature Link →
- FuzzyOverlapping authors or contributors · 62%modular/modular →
“Shared author/contributor keys: liu”
- FuzzySimilar title/name (fuzzy) · 59%jeinlee1991/chinese-llm-benchmark →
“Fuzzy title match (0.73): “Reconstruction: A Blind Benchmark for Recovering Research Id” ≈ “jeinlee1991/chinese-llm-benchmark””
- FuzzySimilar title/name (fuzzy) · 59%google-research/google-research →
“Fuzzy title match (0.73): “Reconstruction: A Blind Benchmark for Recovering Research Id” ≈ “google-research/google-research””
- LinkedLinked via arxiv author · 85%Shaolong Chen →
“Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies”
- LinkedLinked via arxiv author · 85%Yanlin Fei →
“Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies”
- LinkedLinked via arxiv author · 85%Nazhou Liu →
“Reconstruction: A Blind Benchmark for Recovering Research Ideas from Pre-Publication Bibliographies”