Galaxy merger challenge: A comparison study between machine learning-based detection methods

Astronomy and Astrophysics, vol. 687, pp. A24

Abstract

Aims. Various galaxy merger detection methods have been applied to diverse datasets. However, it is difficult to understand how they compare. Our aim is to benchmark the relative performance of merger detection methods based on machine learning (ML). Methods. We explore six leading ML methods using three main datasets. The first dataset consists of mock observations from the IllustrisTNG simulations, which acts as the training data and allows us to quantify the performance metrics of the detection methods. The second dataset consists of mock observations from the Horizon-AGN simulations, introduced to evaluate the performance of classifiers trained on different, but comparable data to those employed for training. The third dataset is composed of real observations from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) survey. We also compare mergers and non-mergers detected by the different methods with a subset of HSC-SSP visually identified galaxies. Results. For the simplest binary classification task (i.e. mergers vs. non-mergers), all six methods perform reasonably well in the domain of the training data. At the lowest redshift explored 0.1 < ɀ<0.3, precision and recall generally range between ~70% and 80%, both of which decrease with increasing ɀ as expected (by ~5% for precision and ~10% for recall at the highest ɀ explored 0.76 < ɀ < 1.0). When transferred to a different domain, the precision of all classifiers is only slightly reduced, but the recall is significantly worse (by ~20–40% depending on the method). Zoobot offers the best overall performance in terms of precision and F1 score. When applied to real HSC observations, different methods agree well with visual labels of clear mergers, but can differ by more than an order of magnitude in predicting the overall fraction of major mergers. For the more challenging multi-class classification task to distinguish between pre-mergers, ongoing-mergers, and post-mergers, none of the methods in their current set-ups offer good performance, which could be partly due to the limitations in resolution and the depth of the data. In particular, ongoing-mergers and post-mergers are much more difficult to classify than pre-mergers. With the advent of better quality data (e.g. from JWST andEuclid), it is of great importance to improve our ability to detect mergers and distinguish between merger stages.

Authors 15

1. Berta Margalef-Bentabol Aachen SRON Netherlands Institute for Space Research

   Space Research Organisation Netherlands

   Affiliation as printed

   SRON Netherlands Institute for Space Research, Landleven 12, 9747 AD Groningen, The Netherlands

2. Lingyu Wang Aachen SRON Netherlands Institute for Space Research

   University of Groningen · Space Research Organisation Netherlands

   Affiliation as printed

   Kapteyn Astronomical Institute, University of Groningen, Postbus 800, 9700 AV Groningen, The Netherlands

   SRON Netherlands Institute for Space Research, Landleven 12, 9747 AD Groningen, The Netherlands

3. Antonio La Marca Aachen SRON Netherlands Institute for Space Research

   University of Groningen · Space Research Organisation Netherlands

   Affiliation as printed

   Kapteyn Astronomical Institute, University of Groningen, Postbus 800, 9700 AV Groningen, The Netherlands

   SRON Netherlands Institute for Space Research, Landleven 12, 9747 AD Groningen, The Netherlands

4. Carmen Blanco-Prieto

   Centro de Astrobiología

   Affiliation as printed

   Centro de Astrobiología (CAB), CSIC-INTA, Carretera de Ajalvir km4, 28850 Torrejón de Ardoz, Madrid, Spain

5. H. Domínguez Sánchez

   Centro de Estudios de Física del Cosmos de Aragón

   Affiliation as printed

   Centro de Estudios de Física del Cosmos de Aragón (CEFCA), Plaza San Juan 1, 44001 Teruel, Spain

6. Andy D. Goulding

   Princeton University

   Affiliation as printed

   Department of Astrophysical Sciences, Princeton University, 4 Ivy Lane, Princeton, NJ 08544, USA

7. Alejandro Guzmán-Ortega

   Universidad Nacional Autónoma de México

   Affiliation as printed

   Instituto de Radioastronomía y Astrofísica, Universidad Nacional Autónoma de México, Apdo. Postal 72-3, 58089 Morelia, Mexico

8. G. Martin

   University of Arizona · Korea Astronomy and Space Science Institute

   Affiliation as printed

   Korea Astronomy and Space Science Institute, 776 Daedeokdae-ro, Yuseong-gu, Daejeon 34055, Korea

   Steward Observatory, University of Arizona, 933 N. Cherry Ave, Tucson, AZ, USA

9. William Pearson

   National Centre for Nuclear Research

   Affiliation as printed

   National Centre for Nuclear Research, Pasteura 7, 02-093 Warszawa, Poland

10. Vicente Rodríguez-Gómez

    Universidad Nacional Autónoma de México

    Affiliation as printed

    Instituto de Radioastronomía y Astrofísica, Universidad Nacional Autónoma de México, Apdo. Postal 72-3, 58089 Morelia, Mexico

11. Mike Walmsley

    University of Manchester

    Affiliation as printed

    Jodrell Bank Centre for Astrophysics, Department of Physics & Astronomy, University of Manchester, Oxford Road, Manchester M13 9PL, UK

12. Robert W. Bickley

    University of Victoria

    Affiliation as printed

    Department of Physics and Astronomy, University of Victoria, Victoria, British Columbia V8P 1A1, Canada

13. Connor Bottrell

    The University of Western Australia · International Centre for Radio Astronomy Research

    Affiliation as printed

    International Centre for Radio Astronomy Research, University of Western Australia, 35 Stirling Hwy, Crawley, WA 6009, Australia

14. Christopher J. Conselice

    University of Manchester

    Affiliation as printed

    Jodrell Bank Centre for Astrophysics, Department of Physics & Astronomy, University of Manchester, Oxford Road, Manchester M13 9PL, UK

15. David O’Ryan

    Lancaster University

    Affiliation as printed

    Department of Physics, Lancaster University, Bailrigg, Lancaster LA1 4YB, UK