Challenges and Recommendations for LLMs-as-a-Judge in Multilingual Settings and Low-Resource Languages
LLM-as-a-Judge has become the dominant evaluation paradigm for many natural language generation tasks, due to shortcomings of conventional metrics and high correlations with human judgment, albeit mostly in English. There are now attempts to extend LLM-as-a-Judge to multilingual settings including low-resource languages. However, LLMs have limited proficiency in low-resource languages, and there is often no adequate human validation in these settings. To highlight the scope of the problem and current practices, we explore the use of LLM-as-a-Judge evaluators in ACL Anthology papers focusing on
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 52%chrisliu298/awesome-llm-unlearning →
- PossiblePossibly related (embedding) · 51%Book Review: Domain-Specific Small Language Models by Guglielmo Iozzia →
- PossiblePossibly related (embedding) · 46%IEEE Rolls Out Large Language Models Virtual Training Course →
- LinkedLinked via arxiv author · 85%A. Seza Doğruöz →
“Challenges and Recommendations for LLMs-as-a-Judge in Multilingual Settings and Low-Resource Languages”
- LinkedLinked via arxiv author · 85%Xixian Liao →
“Challenges and Recommendations for LLMs-as-a-Judge in Multilingual Settings and Low-Resource Languages”
- LinkedLinked via arxiv author · 85%Verena Blaschke →
“Challenges and Recommendations for LLMs-as-a-Judge in Multilingual Settings and Low-Resource Languages”
- LinkedLinked via arxiv author · 85%Jakob Prange →
“Challenges and Recommendations for LLMs-as-a-Judge in Multilingual Settings and Low-Resource Languages”
- LinkedLinked via arxiv author · 85%Senyu Li →
“Challenges and Recommendations for LLMs-as-a-Judge in Multilingual Settings and Low-Resource Languages”