Supervisor: Chao Zhang (Department of Chemistry, Uppsala University)
Group Members: André Ramos Ekengren, Ramkishor Prabhu Ramlal, Yue Zhou
Large language models (LLMs) have shown impressive capabilities in domains such as natural language processing, code generation, and reasoning. In parallel, chemistry and materials science are becoming increasingly data-driven, with growing databases of reactions, molecules, and material properties. Bridging these two developments creates an opportunity: if molecular information can be expressed in a form that is compatible with LLMs, then models originally designed for human language might also support molecular property prediction and materials design.
In this project we focus on molecular prediction problems in material design that are relevant for battery research. Electrolytes play a central role in battery performance, safety, and lifetime, yet their design space is extremely large. Traditional machine-learning models have been successfully used for some property-prediction tasks, but they often require carefully engineered descriptors and do not easily generalize across different tasks or datasets. LLMs offer an alternative in which the same transformer architecture can be applied to a wide range of problems once the molecules are encoded in a suitable way.
As a first step, we conducted a literature survey on LLM applications in chemistry and materials science (primarily from 2022 onwards). The survey covers:
- Property prediction from molecular or materials representations, often using SMILES or graph-based encodings.
- Generative design of molecules and materials with targeted properties.
- Studies that probe the limitations of LLMs in scientific and multimodal settings.
The relevant papers are collected and summarized in a the table below.
| Model name | Architecture | Data link | Description | Adaptation | Purpose | Year | DOI | Github link |
|---|---|---|---|---|---|---|---|---|
| LLM-prop | T5 (encoder) | Benchmark data | predicting the properties of crystalline materials using large language models | Fine-tuning | Regression | 2025 | DOI | Github |
| N/A | GPT-J-6B, Llama-3.1-8B, Mistral-7B | Github | Assessment of fine-tuned large language models for real-world chemistry and material science applications | Fine-tuning | Classification Regression Inverse design | 2025 | DOI | Github |
| MolRAG | Llama-3-8B-Instruct GPT-4o Qwen2.5-7B-Instruct | N/A | N/A | N/A | N/A | N/A | DOI | Github, Code not available yet. |
| ElaTBot ElaTBot-DFT | Llama2-7b, GPT-4o | Data | Large language models for material property predictions: elastic constant tensor prediction and materials design | Fine-tuning, (LORA+), Prompt engineering, RAG | Regression, Inverse design / Material generation | 2025 | DOI | Github |
| N/A | LLaMA 3 | N/A | Regression with Large Language Models for Materials and Molecular Property Prediction | Fine-tuning | Regression | 2024 | DOI | Code and data: part 1 Code and data: part 2 |
| GPT 4 for Chemistry | GPT 4 | N/A | ExploringGPT-4’s potentialin chemical tasks, such as foundational chemistry knowledge, cheminformatics, data analysis,problem prediction, and proposal abilities | Prompt engineering | N/A | 2023 | DOI | No github |
| N/A | GPT 3.5 | No clear data source | Polymer Solubility Prediction Using Large Language Models | Fine-tuning | Classification | 2025 | DOI | No github |
| N/A | Gemini 1.5 | Github | Can large language models predict the hydrophobicity of metal–organic frameworks? | Fine-tuning | Classification | 2025 | DOI | Github |
| LLM4Mat-bench | LLM-Prop, MatBERT, Llama 2-7b-chat, | Data and checkpoints | Benchmarking large language models for materials property prediction | Fine-tuning, In-context learning | Classification Regression | 2025 | DOI | Github |
| PolyLLMem | Llama-3 Uni-Mol | Github | Multimodal Machine Learning with Large Language Embedding Model for Polymer Property Prediction | Fine-tuning, LoRA | Regression | 2025 | DOI | Github |
| gptchem | GPT3 | The repository includes example scripts and demo data sources | Natural-language pretrained | Prompt engineering, not for LoRA | Classification Regression | 2022 | DOI | Github |
| ChemLLM | InternLM-2 | ChemData,ChemBench | Pretrained for chemistry | Fine-tuning,open weights(LoRA applicable) | Classification | 2024 | DOI | Github |
| chemlift | General LLMs “The framework can be used with any HuggingFace model that accepts text input.” | N/A | Natural-language pretrained | Prompt engineering(Prompt + RAG) | Classification Regression | 2023 | DOI | Github |
| De novo Design of Polymer Electrolytes with High Conductivity using GPT-based and Diffusion-based Generative Models | minGPT | HTP-MD polymer-electrolyte dataset (SMILES) | Chemistry-pretrained SMILES-based generative | Fine-tuning hyperparameter grid search provided. (LoRA not described.) | Inverse design | 2024 | DOI | Github |
| ChemLoRA | General LLMs”GPT-3 via API; any HuggingFace base such as GPT-2) with LoRA/PEFT adapters. | QM9-G4MP2 small-molecule dataset(SMILES) | Natural-language pretrained | Prompt engineering (GPTChem-style prompts) **,**Fine-tuning(LoRA) | Regression(molecular energy predictions) | 2023 | N/A | Github |
| Text2Concrete | GPT-3.5-turbo | Data Known,Relevant,Random ( PubChem molecules) | Natural-language pretrained(Gaussian Process Regression and Random Forest for benchmarks) | Prompt engineering | Regression | 2023 | DOI | Github |
| Molecule-Discovery-by-Context | ScholarBERT | literature corpus(for context) and Known,Relevant,Random ( PubChem molecules) | Natural-language pretrained | Prompt engineering(prompt, RAG) | Ranking for candidate discovery | 2023 | DOI | Github |
| Using GPT-4 in parameter selection of polymer informatics: improving predictive accuracy amidst data scarcity and ‘Ugly Duckling’ dilemma | GPT-4 | Data | GPT4 is used to predict the refractive index of a molecule and select necessary descriptors. Comparative study | N/A | Prediction Descriptor | 2023 | DOI | Github |
| SMI-TED-IC | SMI-TED | Github | Chemical foundation model-guided design of high ionic conductivity electrolyte formulations | Fine-tuning | Regression | 2025 | DOI | Github |
| Leveraging GPT-4 to transform chemistry from paper to practice | GPT-4 | Data | 1. Using GPT 4 to read scientific literature and generate actionable steps 2.Generate a script to communicate the steps to EasyMax Reactor | N/A | N/A | 2024 | N/A | Github |
| ChemLLMBench | GPT-4, GPT-3, Davinci-003, Llama, Galactica | Data | What can Large Language Models do in chemistry? A comprehensive benchmark on eight tasks | In-context learning | Generation, Classification, Ranking | 2023 | DOI | Github |
Within this context, our project addresses three main tasks:
-
Molecular property prediction
Use LLMs to predict molecular properties, for example the conductivity of candidate electrolyte molecules. -
Classification
Formulate classification problems such as distinguishing high-conductivity from low-conductivity molecules or grouping molecules by other experimentally relevant labels. -
Molecule generation
Explore generative modelling, where the goal is to generate new SMILES strings that are chemically valid and have desirable predicted properties. These tasks together span both predictive and generative aspects of molecular design.
Each group member re-implemented selected models from the literature, covering property prediction, classification, and molecule generation tasks. These implementations serve both as baselines and as starting points for small methodological variations and improvements.
We then experimented with commercial LLMs (e.g., GPT-4) and selected a common open-source backbone, Llama-3.2-3B-Instruct, as our main model. Across the three subtasks, each group member implements and evaluates the model against their respective literature baseline, and explores modifications to improve performance.
Code for the three subtasks is organised as:
./regression/– LLM and baseline models for regression-style property prediction (implemented by André Ramos Ekengren)./classification/– LLM and baseline models for molecular classification (implemented by Ramkishor Prabhu Ramlal)./inverse-design/– LLM-based SMILES generation and post-processing (implemented by Yue Zhou)
- Investigating improved fine-tuning and evaluation strategies, including robustness in low-data regimes and better uncertainty estimates.
- Developing more targeted generative workflows that couple LLM-based generation with domain-specific filters or chemistry-based checks.
Contributions and suggestions are welcome, especially regarding datasets, baselines, and evaluation protocols.
This project is released under the MIT License.