Stitching for Neuroevolution: Recombining Deep Neural Networks without Breaking Them
Abstract
Traditional approaches to neuroevolution often start from scratch. This becomes prohibitively expensive in terms of computational and data requirements when targeting modern, deep neural networks. Using a warm start could be highly advantageous, e.g., using previously trained networks, potentially from different sources. This moreover enables leveraging the benefits of transfer learning (in particular vastly reduced training effort). However, recombining trained networks is non-trivial because architectures and feature representations typically differ. Consequently, a straightforward exchange of layers tends to lead to a performance breakdown. We overcome this by matching the layers of parent networks based on their connectivity, identifying potential crossover points. To correct for differing feature representations between these layers we employ stitching, which merges the networks by introducing new layers at crossover points. To train the merged network, only stitching layers need to be considered. New networks can then be created by selecting a subnetwork by choosing which stitching layers to (not) use. Assessing their performance is efficient as only their evaluation on data is required. We experimentally show that our approach enables finding networks that represent novel trade-offs between performance and computational cost, with some even dominating the original networks.
Authors 4
-
Centrum Wiskunde & Informatica
Affiliation as printed
Centrum Wiskunde & Informatica Amsterdam , The Netherlands
-
Affiliation as printed
Utrecht University Utrecht , The Netherlands
-
Tanja Alderliesten Aachen
Centrum Wiskunde & Informatica · Leiden University Medical Center
Affiliation as printed
Centrum Wiskunde & Informatica Amsterdam , The Netherlands
Leiden University Medical Center Leiden , The Netherlands
Cited by 0 stored of 0
No patents citing this paper on Lens.org (checked 2026-10-11).