Early Ideas and Innovations in Bayesian and Model-Assisted Multiobjective Optimization

Early Ideas and Innovations in Bayesian and Model-Assisted Multiobjective Optimization

Abstract. The early two thousands saw rapid growth in multiobjective design optimization, fueled by advances in evolutionary algorithms, Bayesian optimization, and surrogate models such as neural networks, Kriging, and Gaussian processes. These methods enabled efficient exploration of complex design spaces while reducing the cost of expensive function evaluations. Model-Assisted Multiobjective Optimization transformed the field by integrating surrogate-assisted techniques to enhance both efficiency and solution quality. This chapter reviews its historical development, emphasizing key contributions from European research efforts-including the European Community on Computational Methods in Applied Sciences and the INGENET network. We highlight foundational methodologies such as Pareto Efficient Global Optimization, Expected Hypervolume Improvement, and confidence-bound and interval-based pre-selection in Bayesian optimization. The chapter also examines the impact of these approaches on engineering applications, particularly in computational fluid dynamics and finite element method simulations, offering a comparative analysis of core concepts and their roles in shaping the landscape of modern multiobjective optimization.

One Introduction

The early two thousands marked an explosion of ideas in multiobjective design optimization, fueled by the convergence of several key advancements: the emergence of multiobjective evolutionary algorithms, the increasing adoption of Bayesian optimization for black-box problems, and the computational maturity of surrogate models-such as response surface models based on neural networks and Kriging (or Gaussian Process Regression)-that could handle multidimensional and multimodal functions. Together, these innovations transformed the approach to complex optimization problems, shifting the paradigm from exhaustive brute-force methods to intelligent, model-assisted strategies.

In many engineering and scientific applications, optimization problems involve multiple conflicting objectives. Traditional solution methods relied on either exhaustive simulation-based approaches or heuristic search techniques, both of which are computationally intensive. As simulation models grew more complex, particularly in areas like computational fluid dynamics and finite element method simulations, conventional optimization techniques struggled to cope with the high cost of function evaluations. This challenge led to the development of Model-Assisted Multiobjective Optimization, a framework that integrates surrogate models to approximate expensive functions and efficiently explore the Pareto-optimal set.

By making use of data-driven approximations, MAMO reduces the number of costly function evaluations while maintaining high-quality approximations of Pareto-optimal solutions. Early research demonstrated that surrogate-assisted optimization could accelerate convergence and yield robust solutions across various engineering disciplines. The integration of Kriging-based Gaussian processes, neural network surrogates, and Bayesian sampling strategies allowed optimization algorithms to learn from previous evaluations, predict promising regions of the search space, and iteratively refine solutions.

The paper is organized as follows. In Section Two, we provide historical context by discussing early research initiatives that addressed computationally expensive design problems in computational fluid dynamics and finite element method simulations. Section Three introduces key methodological foundations in optimization, covering single-objective and multiobjective optimization as well as surrogate modeling techniques, including Radial Basis Function Networks and Gaussian Process Regression. Section Four surveys surrogate models in single-objective optimization.

The next three sections present foundational contributions to surrogate-assisted multiobjective optimization: Section Five reviews the Radial Basis Function Network-based approach introduced by Giotis and Giannakoglou, Section Six details the ParEGO method that employs a scalarization-based strategy with Gaussian Process models, and Section Seven discusses the Expected Hypervolume Improvement framework that targets Pareto front enhancement directly.

Sections Eight, Nine, and Ten discuss alternative infill criteria and early methods for using surrogate models: Section Eight examines non-dominated sorting of expected improvements, Section Nine describes the SMS-EGO framework that exploits lower confidence bounds, and Section Ten introduces interval filters that compare two-sided confidence intervals for candidate selection.

Section Eleven provides a comparative analysis of these early contributions, highlighting their respective advantages and computational trade-offs. Finally, Section Twelve concludes with a summary of findings and an outlook on future research directions. An appendix is provided with detailed examples, including a comprehensive Kriging prediction with confidence intervals.