Probing Chemical Language Models: Effects of Pre-training and Fine-tuning
Chemical language models (CLMs) are trained with linearized representations such as SMILES, yet it remains unclear which chemically meaningful substructures they encode. To foster a better understanding of CLMs, we conduct a systematic study and probe for 78 molecular substructures across eight pre-trained and six randomly initialized models. We furthermore study how fine-tuning on chemical downstream tasks affects the learned representations of molecular substructures. Our results show that pre-training generally improves molecular structure awareness of CLMs, particularly in the upper layers
Lineage graph
Paper → model → repo connections mined from source citations (Tier-1 exact match).
Why these links exist
Every edge carries a method, confidence, and the source snippet that justified it — so bad links are debuggable.
- PossiblePossibly related (embedding) · 60%Bridging three-dimensional molecular structures and artificial intelligence with a conformation description language →
- PossiblePossibly related (embedding) · 49%Reshaping biomolecular structure prediction through strategic conformational exploration with HelixFold-S1 →
- PossiblePossibly related (embedding) · 46%chrisliu298/awesome-llm-unlearning →
- LinkedLinked via arxiv author · 85%Anna Karnysheva →
“Probing Chemical Language Models: Effects of Pre-training and Fine-tuning”
- LinkedLinked via arxiv author · 85%Dietrich Klakow →
“Probing Chemical Language Models: Effects of Pre-training and Fine-tuning”
- LinkedLinked via arxiv author · 85%Ji-Ung Lee →
“Probing Chemical Language Models: Effects of Pre-training and Fine-tuning”
- PossiblePossibly related (embedding) · 50%Efficient and valid large molecule generation via self-supervised generative models - Nature →