MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents
AI agents in partially observable environments need to coordinate active sensing with working memory to maintain an evolving perceptual state. However, existing benchmarks struggle to isolate this perceptual-state construction and interpretation capability because they introduce physical and control complexities. We address this with MNIST-PRO, a benchmark that isolates agentic perception by converting MNIST digit recognition into a sequential, glimpse-based search task with lookback constraints. We evaluate ten multimodal models across four memory representations, including raw visual history
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 53%Ex-Meta scientists want to bring visual AI to the factory floor →
- LinkedLinked via arxiv author · 85%Vernon Toh →
“MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents”
- LinkedLinked via arxiv author · 85%Navonil Majumder →
“MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents”
- FuzzySimilar title/name (fuzzy) · 87%NirDiamant/GenAI_Agents →
“Fuzzy title match (0.94): “MNIST-PRO: MNIST is Back as a Partially Observable World for” ≈ “NirDiamant/GenAI_Agents””
- FuzzySimilar title/name (fuzzy) · 84%Unity-Technologies/ml-agents →
“Fuzzy title match (0.92): “MNIST-PRO: MNIST is Back as a Partially Observable World for” ≈ “Unity-Technologies/ml-agents””
- FuzzyOverlapping authors or contributors · 62%modular/modular →
“Shared author/contributor keys: liu”
- FuzzySimilar title/name (fuzzy) · 59%datawhalechina/hello-agents →
“Fuzzy title match (0.73): “MNIST-PRO: MNIST is Back as a Partially Observable World for” ≈ “datawhalechina/hello-agents””
- FuzzySimilar title/name (fuzzy) · 59%jnMetaCode/agency-agents-zh →
“Fuzzy title match (0.73): “MNIST-PRO: MNIST is Back as a Partially Observable World for” ≈ “jnMetaCode/agency-agents-zh””