How to explore model uncertainty to enhance generative molecular design

In recent years, artificial intelligence (AI) has made big strides in helping scientists design new molecules—whether for life-saving drugs or advanced materials. Especially, generative AI models have received significant interest from the research community for designing new molecules, opening exciting possibilities in science and medicine.

However, one big challenge remains: these models are usually trained once and then used “as-is.” Adapting them to target-specific molecular characteristics – for example, to improve a molecule’s activity against specific targets or to make the designed molecule easily synthesizable – is challenging.

In a recent publication entitled “Enhancing generative molecular design via uncertainty-guided fine-tuning of variational autoencoders,” published in RSC Molecular Systems Design & Engineering, we have proposed a smarter way to fine-tune generative AI models by quantifying and leveraging model uncertainty.

A N M Nafiz Abeer, Sanket Jantre, Nathan M Urban, and Byung-Jun Yoon, “Enhancing Generative Molecular Design via Uncertainty-guided Fine-tuning of Variational Autoencoders,” Molecular Systems Design & Engineering, 2025, https://doi.org/10.1039/D5ME00081E.

In this paper, we proposed an uncertainty-guided fine-tuning strategy that can effectively enhance a pre-trained variational autoencoder (VAE) for generative molecular design (GMD) through performance feedback in an active learning setting. The strategy begins by quantifying the model uncertainty of the generative model using an efficient active subspace-based UQ (uncertainty quantification) scheme. Next, the decoder diversity within the characterized model uncertainty class is explored to expand the viable space of molecular generation. Empirical results across six target molecular properties demonstrate that the uncertainty-guided fine-tuning strategy consistently leads to improved models that outperform the original pre-trained generative models.

For further details, please read the full paper at the following link: https://doi.org/10.1039/D5ME00081E