paperarXivTrust 82 · PrimaryPublished 1mo agoLive · 1mo ago

The Signal-Coverage Matrix: Stratifying Type and Semantic Errors in Statement Autoformalization

Headline type-correctness (TC\%) of LLM autoformalization has climbed from $\sim$53\% to $\sim$76\% in two years, yet this scalar conceals which errors each method resolves. We propose a signal-coverage matrix that crosses the Lean elaborator (pass/fail) with a semantic-equivalence judgment (equivalent/not), sorting every output into one of four cells: true success (TS), type-only (TO), semantic-only (SO), or both fail (BF). On ProofNet\# and MiniF2F-test with DeepSeek V4-Pro across Vanilla, Lean-Retry, Sample-Filter, and Stratified Autoformalization (SAF): (1) the +34 to +36 TS gain across th

Lineage graph

Paper → model → repo connections mined from source citations (Tier-1 exact match).

Why these links exist

Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.

LinkedLinked via unknownBenchmarking Self-Hosted Gemma 2 9B vs. Frontier APIs: The FP8 Quantization Prefill Tax and VRAM Realities on an NVIDIA L4 [P] →
PossiblePossibly related (embedding) · 45%At matched bits, the downstream winner between two codes flips when you change the consumer — while reconstruction error says they're identical. 12-domain study, pre-registered, negative results included` [R] →

Covers

newsBenchmarking Self-Hosted Gemma 2 9B vs. Frontier APIs: The FP8 Quantization Prefill Tax and VRAM Realities on an NVIDIA L4 [P]

Covers (incoming)

Related across the graph

newsAt matched bits, the downstream winner between two codes flips when you change the consumer — while reconstruction error says they're identical. 12-domain study, pre-registered, negative results included` [R]newsBenchmarking Self-Hosted Gemma 2 9B vs. Frontier APIs: The FP8 Quantization Prefill Tax and VRAM Realities on an NVIDIA L4 [P]

Topics

cs.CL