Exploring Dimensionality Reduction Techniques for Efficient Surrogate-Assisted optimization

IEEE Symposium Series on Computational Intelligence (SSCI), pp. 2965–2974

Abstract

Constructing surrogate models of high dimensional optimization problems is challenging due to the computational complexity involved. This paper empirically investigates the practicality of major dimensionality reduction techniques for encapsulating the high dimensional design space into compact representations. Such low dimensional representations of the design space can be utilized for constructing the surrogate models efficiently. Based on historical mainstays and recent developments in deep learning, we study four dimensionality reduction techniques in this paper, namely Principal Component Analysis, Kernel Principal Component Analysis, Autoencoders and Variational Autoencoders. We evaluate and compare these techniques based on quality assessments of the corresponding low dimensional surrogate models on a diverse range of test cases. These test cases are defined on combinations of three dimensionsalities, ten well-known benchmark problems from the continuous optimization domain and two surrogate modeling techniques, namely Kriging and Polynomials. Our results clearly demonstrate the superiority of Autoencoders and Principal Component Analysis on the criteria of modeling accuracy and global optimality respectively.

Authors 6

1. Sibghat Ullah Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Netherlands

2. Duc Anh Nguyen Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Netherlands

3. Hao Wang

   Centre National de la Recherche Scientifique · Sorbonne Université · LIP6

   Affiliation as printed

   CNRS, LIP6, Sorbonne Université, Paris, France

4. Stefan Menzel

   Honda (Germany)

   Affiliation as printed

   Honda Research Institute Europe GmbH (HRI-EU), Offenbach/Main, Germany

5. Bernhard Sendhoff

   Honda (Germany)

   Affiliation as printed

   Honda Research Institute Europe GmbH (HRI-EU), Offenbach/Main, Germany

6. Thomas Bäck Aachen Leiden Institute of Advanced Computer Science

   Leiden University

   Affiliation as printed

   Leiden Institute of Advanced Computer Science (LIACS), Leiden University, Netherlands