EvoMO-SR: Multiobjective LLM-based Evolution of Symbolic Expressions with substructure guidance

arXiv (Cornell University)

Abstract

Symbolic Regression (SR) is a data-driven method for scientific discovery which searches for interpretable analytical relationships within data. Recently, Large Language Models (LLMs) have also had a significant impact on scientific discovery, enabling the automation of various stages of the process. For these reasons, the possibility of harnessing the embedded scientific knowledge and programming capabilities of LLMs to solve SR tasks has emerged, showing promising performance compared with traditional methods. We propose EvoMO-SR, a novel LLM-driven SR framework in which the LLM generates equation skeletons, with their coefficients fitted separately by an external optimizer. The framework includes a multi-objective survival selection which controls bloating by balancing accuracy and complexity, and a substructure guidance mechanism which mutates expressions with candidate reusable building blocks. EvoMO-SR achieves the best accuracy in seven of the eight in-domain and out-of-domain settings for LSR-Synth, using a small LLM model, i.e., Llama-3.1-8B-Instruct. We also evaluated structural recovery through two symbolic accuracy metrics based on canonicalized subtree overlap and term matching, showing that our method has a greater probability of recovering highly accurate symbolic structures.

Authors 5

1. Cristina Rossetti

   Politecnico di Torino

   Affiliation as printed

   Department of Automation and Computer Science Polytechnic Institute of Turin Corso Duca degli Abruzzi , 24 , Torino , 10129 , Italy

2. Anna V. Kononova Aachen

   Leiden University · University of Applied Sciences Leiden

   Affiliation as printed

   LIACS Leiden University Einsteinweg 55 , 2333 CC , Leiden , The Netherlands

3. Thomas Bäck Aachen

   Leiden University · University of Applied Sciences Leiden

   Affiliation as printed

   LIACS Leiden University Einsteinweg 55 , 2333 CC , Leiden , The Netherlands

4. Fei Liu

   University of Zurich · ETH Zurich

   Affiliation as printed

   ETH Zurich

   University of Zurich &

5. Niki van Stein Aachen

   Leiden University · University of Applied Sciences Leiden

   Affiliation as printed

   LIACS Leiden University Einsteinweg 55 , 2333 CC , Leiden , The Netherlands