paperarXivTrust 82 · PrimaryPublished 4d agoLive · 3d ago
Little Brains, Big Feats: Exploring Compact Language Models
While large language models have been dominating the research landscape recently, small language models remain highly relevant across various domains; yet, they receive far less attention. In this study, we investigate how smaller language models perform during the generation stage within a Retrieval-Augmented Generation (RAG) system. To benchmark these models effectively, we utilised both open-source and proprietary datasets covering diverse subject areas and question types. Our findings demonstrate that a RAG system with small language models can be executed directly on-device without requir
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Covers
newsRAGless: Q-Q retrieval with score aggregation for closed-domain FAQ [P]newsBook Review: Domain-Specific Small Language Models by Guglielmo IozzianewsDiffusionGemma: 4x faster text generationnewsKnowledge Distillation of Black-Box Large Language ModelsnewsKnowledge Distillation of Black-Box Large Language Models (2024)
Implements (incoming)
Covers (incoming)
Related across the graph
newsDiffusionGemma: 4x faster text generationnewsKnowledge Distillation of Black-Box Large Language ModelsnewsDoes intelligence ‘emerge’ in large language models? - Santa Fe Instituterepox-tabdeveloping/turftopicnewsRAGless: Q-Q retrieval with score aggregation for closed-domain FAQ [P]reposgl-project/sglangnewsBook Review: Domain-Specific Small Language Models by Guglielmo IozzianewsKnowledge Distillation of Black-Box Large Language Models (2024)