Category Archives: Uncategorized

Automatic Feature Engineering (AFE) Using Deep Reinforcement Learning for Materials Science

The final version of our paper entitled “Physics-constrained Automatic Feature Engineering for Predictive Modeling in Materials Science” recently presented at the 35th AAAI Conference on Artificial Intelligence (AAAI-21) is now accessible at the link below:

Ziyu Xiang, Mingzhou Fan, Guillermo Vázquez Tovar, William Trehem, Byung-Jun Yoon, Xiaofeng Qian, Raymundo Arroyave, Xiaoning Qian, “Physics-constrained Automatic Feature Engineering for Predictive Modeling in Materials Science“, 35th AAAI Conference on artificial intelligence (AAAI-21), Feb. 2-9, 2021.

Automatic Feature Engineering (AFE) aims to extract useful knowledge for interpretable predictions given data for the machine learning tasks of interest. In this work, we presented a novel AFE scheme that effectively extracts relationships from data that can be interpreted based on functional formulas to discover their “physical meaning” or “new hypotheses”. Here we focused on materials science applications, where interpretable predictive modeling may enhance our understanding of materials systems and also guide the discovery of new materials.

Typically, it is computationally prohibitive to exhaustively explore all potential relationships to identify interpretable and predictive features. We overcome this challenge by designing an AFE strategy that efficiently explores a feature generation tree (FGT) using a deep Q-network (DQN) for scalable and efficient exploration of the feature space in an automated manner. We demonstrate that our proposed DQN-based AFE strategy yields promising results when benchmarked against existing AFE methods based on several materials science datasets.

Paper on optimal experimental design (OED) strategy for uncertain Kuramoto oscillator models has been published

We are happy to announce that our paper entitled “Optimal Experimental Design for Uncertain Systems Based on Coupled Differential Equations,” has now been published in IEEE Access.

Youngjoon Hong, Bongsuk Kwon, and Byung-Jun Yoon, “Optimal Experimental Design for Uncertain Systems Based on Coupled Differential Equations,” IEEE Access, doi: 10.1109/ACCESS.2021.3071038.

In this work, we present a general optimal experimental design (OED) strategy for an uncertain system that is described by coupled ordinary differential equations (ODEs), whose parameters are not completely known. As a vehicle to develop the OED strategy, we focus on non-homogeneous Kuramoto models in this study as the primary example. The proposed OED strategy quantifies the objective uncertainty of the Kuramoto model based on the mean objective cost of uncertainty (MOCU), where the optimal experiment can be identified by predicting the one in a given experimental design space that is expected to maximally reduce the MOCU.

Illustrative overview of the proposed optimal experimental design (OED) framework. Objective uncertainty quantification based on MOCU (mean objective cost of uncertainty) is used to iteratively predict and perform the optimal experiment that is expected to most effectively reduce the uncertainty that affects the control (synchronization) performance.

Our study highlights the importance of quantifying the operational impact of the potential experiments in selecting the optimal experiment and it demonstrates that the MOCU-based OED scheme enables us to minimize the objective cost (i.e., cost of robust control in the application considered in this paper) of the uncertain Kuramoto model with the fewest experiments compared to other alternatives.

This work was performed in collaboration with Prof. Youngjoon Hong (Department of Mathematics, Sungkyunkwan University) and Prof. Bongsuk Kwon (Department of Mathematical Sciences, Ulsan National Institute of Science and Technology).

Brookhaven National Laboratory officially joins the ATOM consortium for Accelerating Therapeutics for Opportunities in Medicine (ATOM)

The Brookhaven National Laboratory (BNL), where Dr. Byung-Jun Yoon is working as a Scientist (via joint appointment), has officially joined the ATOM consortium for Accelerating Therapeutics for Opportunities in Medicine (ATOM).

“At Brookhaven, we are excited to apply our team’s work developing and using optimization algorithms directly to ATOM’s diverse computational data-driven modeling efforts,” said Francis J. “Frank” Alexander, deputy director of the Computational Science Initiative. “Often, mathematical models and systems of interest to ATOM cancer therapy problems are uncertain and under-characterized due to their extremely complex nature. At Brookhaven, our artificial intelligence, machine learning and applied mathematics work aims to unravel complexities to design computational and laboratory experiments that achieve discovery goals in the most efficient manner. We believe these efforts will have significant applications in ATOM that can greatly benefit and enhance the program’s impact. We look forward to contributing as part of the collaboration.”

Further information is available on the official ATOM website at the link below:

Our AISTATS 2021 paper entitled “Bayesian Active Learning by Soft Mean Objective Cost of Uncertainty” can now be accessed

We are happy to announce that our AISTATS 2021 paper entitled “Bayesian Active Learning by Soft Mean Objective Cost of Uncertainty” is now available in the Proceedings of Machine Learning Research, which can be accessed at the following link:

Guang Zhao, Edward Dougherty, Byung-Jun Yoon, Francis Alexander, Xiaoning Qian, “Bayesian Active Learning by Soft Mean Objective Cost of Uncertainty,” 24th International Conference on Artificial Intelligence and Statistics (AISTATS), April 13 – 15, 2021.

In this paper, we suggest a strictly concave approximation of MOCU – referred to as “Soft MOCU” – that can be used to define an acquisition function for Bayesian active learning with a theoretical convergence guarantee. We show in this study that the Soft MOCU based Bayesian active learning outperforms other existing methods, with the important benefit of theoretical guarantee of convergence to the optimal classifier.

ICLR 2021 paper entitled “Uncertainty-aware Active Learning for Optimal Bayesian Classifier” now available online

We are happy to announce that our ICLR 2021 paper below can now be accessed online on

Guang Zhao, Edward Dougherty, Byung-Jun Yoon, Francis Alexander, Xiaoning Qian, “Uncertainty-aware Active Learning for Optimal Bayesian Classifier,” 9th International Conference on Learning Representations (ICLR), May 4-8, 2021.

In this paper, we propose an acquisition function for active learning of a Bayesian classifier based on a weighted form of MOCU (mean objective cost of uncertainty). By quantifying the uncertainty that directly affects the classification error, the proposed method avoids the shortcoming of the previous expected Loss Reduction (ELR) methods by avoiding their myopic behavior. Unlike existing ELR methods, which may get stuck before reaching the optimal classifier, the proposed weighted-MOCU based strategy provides the critical advantage that the resulting Bayesian active learning algorithm guarantees convergence to the optimal classifier of the true model. We demonstrate its performance with both synthetic and real-world datasets.

A new named postdoctoral fellowship position in Applied Mathematics and Scientific Computing is available at Brookhaven National Lab (BNL)

The Applied Mathematics Group of the Computational Science Initiative (CSI) at Brookhaven National Laboratory (BNL) invites exceptional candidates to apply for the Amalie Emmy Noether Fellowship in applied mathematics and scientific computing.

This fellowship offers a unique opportunity to conduct research in a broad set of fields, including reduced order modeling, uncertainty quantification and scalable computational statistics for Bayesian inference, optimization and control for decision making under uncertainty, scientific machine learning, high-dimensional inverse problems, multiscale modeling, integrated computational modeling frameworks, data science for streaming or “in-situ” (within simulation) analytics in high performance computing (HPC), and numerical methods. The methods and fundamental advances made in the course of this research will further the progress of applications of interest to BNL and the Department of Energy (DOE). Examples of such applications might include: data-driven uncertainty quantification and hybrid process-based/data-driven modeling for climate prediction and resilience planning, optimal experimental and simulation design for drug discovery and materials science, and large-scale data processing for particle accelerator experiments. The followship includes access to world-class HPC resources, such as the BNL Institutional Cluster and DOE leadership computing facilities. Access to these platforms will allow computing at scale and will ensure that the successful candidate will have the necessary resources to solve challenging DOE problems of interest.

This program provides full support for a period of two years at CSI. Candidates must have received a doctorate (Ph.D.) in applied mathematics or a related field (e.g., mathematics, physics, engineering, statistics, operations research, or computer science) within the past five years. This fellowship presents a unique chance to conduct interdisciplinary collaborative research in BNL programs with a strongly competitive salary. Recipients will be allowed to select a direct mentor from a list of CSI staff scientists. This mentor will help the recipient define and pursue their own research agenda during their appointment.

For more information, visit:

Apply for 2021 NSF Math Sciences Graduate Internship (MSGI)

We are happy to announce the opportunity to apply for 2021 NSF Math Sciences Graduate Internship (MSGI) to work on the research project entitled: Uncertainty-Aware Data-Driven Models for Optimal Learning and Robust Decision Making Under Uncertainty. (Mentors: Drs. Nathan Urban & Byung-Jun Yoon)

This project aims to develop Scientific ML techniques that enable objective-driven uncertainty quantification (UQ) for data-driven models. We will focus on developing theories and algorithms that can ultimately lead to an automated learning procedure of effective surrogates for complex systems that can be used for making optimal decisions robust to system uncertainties and surrogate approximation errors. These goals will be attained based on a Bayesian ML paradigm, in which we integrate scientific prior knowledge on the system and the available data to obtain a prior directly characterizing the scientific uncertainty in the physical system, quantify the uncertainty relative to the objective, develop optimal operators robust to the uncertainty, and design strategies that can optimally reduce the uncertainty and thereby directly contribute to the attainment of the objective. Potential applications of this methodology will be discussed with the student, but may focus on biological and biomedical discovery science. Detailed information of this project can be found in the project catalog at the following link (search for reference code: BNL-URBAN1):

The NSF Mathematical Sciences Graduate Internship (MSGI) program is aimed at students who are interested in understanding the application of advanced mathematical and statistical techniques to “real world” problems, regardless of whether you plan to pursue an academic or nonacademic career. Internship activities will vary based on the assigned research project and hosting facility. As part of your application, you will identify your top 3 research projects from the 2021 NSF MSGI Project Catalog:

Further information about the NSF Mathematical Sciences Graduate Internship program can be found at:

Application Deadline: January 13, 2021 4PM Eastern Time Zone

A new website launched for objective uncertainty quantification

Drs. Byung-Jun and Xiaoning Qian have jointly launched a new website, which aims to provide a convenient centralized access point to the latest research findings and developments relevant to objective-based uncertainty quantification (UQ) and optimal experimental design (OED).

This new website will provide a comprehensive list of research papers, software, resources, as well as the latest news and developments relevant to objective UQ. Main topics of interest that will be primarily featured on this new website include:

  • Mean objective cost of uncertainty (MOCU)
  • Optimal experimental design (OED)
  • Robust operator design for operations such as: classification, filtering, compression, control, and clustering
  • Optimal Bayesian Transfer Learning (OBTL)
  • Maximal knowledge-driven information priors (MKDIP)
  • Diverse scientific applications of MOCU and MOCU-based OED

Please visit for further information.


BioMLSP lab has openings for MS/PhD students

The BioMLSP lab is actively looking for 1~2 MS students and 1~2 PhD students who are strongly motivated to work on emerging research topics that involve machine learning & signal processing with applications in life science and materials science.

Potential applicants should contact Prof. Byung-Jun Yoon via email (bjyoon at ece dot tamu dot edu), which should include: (1) a brief description of the applicant’s general research interests and career goals, (2) CV, (3) undergraduate/graduate transcripts.




CNB-MAC 2019 will be held in Niagara Falls, NY, on September 7, 2019

The Sixth International Workshop on Computational Network Biology: Modeling, Analysis, and Control (CNB-MAC 2019) will be held in Niagara Falls, NY, on September 7, 2019. As in previous years, the upcoming CNB-MAC workshop will be organized in conjunction with the 10th ACM Conference on Bioinformatics, Computational Biology, and Health Informatics (ACM-BCB 2019), the flagship conference of the ACM SIGBio.

Continue reading