Controlling the Mutation in Large Language Models for the Efficient Evolution of Algorithms

arXiv (Cornell University)

Abstract

The integration of Large Language Models (LLMs) with evolutionary computation (EC) has introduced a promising paradigm for automating the design of metaheuristic algorithms. However, existing frameworks, such as the Large Language Model Evolutionary Algorithm (LLaMEA), often lack precise control over mutation mechanisms, leading to inefficiencies in solution space exploration and potentially suboptimal convergence. This paper introduces a novel approach to mutation control within LLM-driven evolutionary frameworks, inspired by theory of genetic algorithms. Specifically, we propose dynamic mutation prompts that adaptively regulate mutation rates, leveraging a heavy-tailed power-law distribution to balance exploration and exploitation. Experiments using GPT-3.5-turbo and GPT-4o models demonstrate that GPT-3.5-turbo fails to adhere to the specific mutation instructions, while GPT-4o is able to adapt its mutation based on the prompt engineered dynamic prompts. Further experiments show that the introduction of these dynamic rates can improve the convergence speed and adaptability of LLaMEA, when using GPT-4o. This work sets the starting point for better controlled LLM-based mutations in code optimization tasks, paving the way for further advancements in automated metaheuristic design.

Authors 4

1. Haoran Yin Aachen

   Leiden University

   Affiliation as printed

   LIACS , Leiden University , The Netherlands

2. Anna V. Kononova Aachen

   Leiden University

   Affiliation as printed

   LIACS , Leiden University , The Netherlands

3. Thomas Bäck
4. Bas van Stein Aachen

   Leiden University

   Affiliation as printed

   LIACS , Leiden University , The Netherlands