OLEDLM: A UNIFIED LANGUAGE MODEL FOR OLED MOLECULAR DESIGN

Research Question

De novo generative approaches targeting strict multi-objective optimization for OLEDs remain scarce. We trained a BERT-style encoder from scratch on 9 million SMILES strings using the Masked Language Modeling objective.

This confirms our hypothesis that SFT struggles to learn generalizable structure-property relationships, instead memorizing specific (property, SMILES) pairs from the limited labeled dataset.

Methodology

We model molecular generation as a causal language modeling task. The predictor is fine-tuned for S 1 and f regression tasks using mean pooling and an MLP head.

Study Design

We evaluate the main generation benchmark by sampling 10,000 SMILES sequences per method.

We subsequently completed the remaining calculations and now report 20 converged molecules per method.

Results & Findings

Discovering high-performance OLED materials requires navigating a vast chemical space to find structures meeting optoelectronic constraints like specific S 1 levels and high f. Large Language Models trained on SMILES strings demonstrate proficiency in learning chemical language.

• Discovering high-performance OLED materials requires navigating a vast chemical space to find structures meeting optoelectronic constraints like specific S 1 levels and high f.
• Large Language Models trained on SMILES strings demonstrate proficiency in learning chemical language.
• We use a LLaMA-style Transformer and Reinforcement Learning to improve the precision of property-targeted OLED molecule design.
• We adopt Group Relative Policy Optimization (GRPO) to align the generative model with target properties.
• We pre-train a LLaMA-style Transformer on a large OLED dataset to create a foundational model for generating valid, novel OLED structures.

Practical Applications

General-purpose models pre-trained on broad datasets may not capture the electronic nuances of OLED materials. No meaningful generation metrics could be computed for the LSTM.

A higher masking ratio may encourage learning of global molecular features for oscillator-strength prediction.

Interestingly, removing the validity constraint slightly increases uniqueness (96.9% vs 93.2%), suggesting that some diverse structural patterns may be discouraged by strict validity enforcement.

Encoder-only Models for Molecular Representation

This section describes the adaptation of the BERT architecture for chemical representation, emphasizing its role as a property predictor in the OLED domain. It contrasts general-purpose encoders with the customized BERT-style architecture used in the proposed framework.

Large Language Models for De Novo Design

The section outlines the capabilities of autoregressive Transformer models, particularly LLaMA, in modeling discrete sequential data like SMILES strings. It discusses the importance of training stability and long-range dependency modeling in generating novel molecular structures.

Reinforcement Learning for Alignment

The section addresses the limitations of SFT in property-conditioned generation due to sparse supervision. It introduces Reinforcement Learning (RL) as a solution, specifically using Group Relative Policy Optimization (GRPO) to provide stable feedback and improve the model’s generalization capabilities.

Figures Explained

The paper’s visual material highlights the workflow and the main system components.

• Figure 1 :: Figure 1: Overview of our unified bidirectional OLED molecular design framework. Starting from a 9M SMILES pre-training corpus, we train two complementary models via self-supervised learning: (1) OLED-BERT for property prediction, and (2) OLED-LLaMA for molecular generation. The 9k DFT-verified dataset enables fine-tuning OLED-BERT into specialized S 1 and f predictors, while guiding conditional SFT of OLED-LLaMA. The predictors then serve as reward models to further refine the generator via GRPO, achieving precise property-conditioned molecular design.
• Figure 2 :: Figure 2: Joint property distribution for DFT validation target selection. Left: Scatter plot with kernel density estimation contours showing the training set distribution (blue, n=9,000) and matched DFT validation samples (red, n=20). Right: 2D histogram with matched sample overlay. The 20 target pairs span diverse regions of the property space, covering S 1 \u2208 [2.25, 3.07] eV and f \u2208 [0.002, 0.53].
• Figure 3 :: Figure 3: Marginal distributions of target properties. The training set distribution (blue, n=9000) is compared with the matched DFT validation samples (red, n=20). The selected samples cover the high-density regions of the training distribution while ensuring diversity in both S 1 and oscillator strength f .
• Figure 4 :: Figure 4: Generated OLED molecules from Bucket 1 (1-3).
• Figure 5 :: Figure 5: Generated OLED molecules from Bucket 1 (4-6).