A benchmark dataset and workflow for landslide susceptibility zonation
Earth-Science Reviews, vol. 258, pp. 104927
Abstract
Landslide susceptibility shows the spatial likelihood of landslide occurrence in a specific geographical area and is a relevant tool for mitigating the impact of landslides worldwide. As such, it is the subject of countless scientific studies. Many methods exist for generating a susceptibility map, mostly falling under the definition of statistical or machine learning. These models try to solve a classification problem: given a collection of spatial variables, and their combination associated with landslide presence or absence, a model should be trained, tested to reproduce the target outcome, and eventually applied to unseen data. Contrary to many fields of science that use machine learning for specific tasks, no reference data exist to assess the performance of a given method for landslide susceptibility. Here, we propose a benchmark dataset consisting of 7360 slope units encompassing an area of about 4,100 km 2 in Central Italy. Using the dataset, we tried to answer two open questions in landslide research: (1) what effect does the human variability have in creating susceptibility models; (2) how can we develop a reproducible workflow for allowing meaningful model comparisons within the landslide susceptibility research community. With these questions in mind, we released a preliminary version of the dataset, along with a “call for collaboration,” aimed at collecting different calculations using the proposed data, and leaving the freedom of implementation to the respondents. Contributions were different in many respects, including classification methods, use of predictors, implementation of training/validation, and performance assessment. That feedback suggested refining the initial dataset, and constraining the implementation workflow. This resulted in a final benchmark dataset and landslide susceptibility maps obtained with many classification methods. Values of area under the receiver operating characteristic curve obtained with the final benchmark dataset were rather similar, as an effect of constraints on training, cross–validation, and use of data. Brier score results show larger variability, instead, ascribed to different model predictive abilities. Correlation plots show similarities between results of different methods applied by the same group, ascribed to a residual implementation dependence. We stress that the experiment did not intend to select the “best” method but only to establish a first benchmark dataset and workflow, that may be useful as a standard reference for calculations by other scholars. The experiment, to our knowledge, is the first of its kind for landslide susceptibility modeling. The data and workflow presented here comparatively assess the performance of independent methods for landslide susceptibility and we suggest the benchmark approach as a best practice for quantitative research in geosciences.
Authors 28
-
Massimiliano Alvioli corresponding
Research Institute for Geo-Hydrological Protection · National Research Council
Affiliation as printed
Consiglio Nazionale delle Ricerche, Istituto di Ricerca per la Protezione Idrogeologica, via Madonna Alta 126, I-06128 Perugia, Italy
-
Czech Academy of Sciences · Charles University · Czech Academy of Sciences, Institute of Rock Structure and Mechanics
Affiliation as printed
Institute of Hydrogeology, Engineering Geology and Applied Geophysics, Charles University, Albertov 6, 128 43 Prague, Czech Republic
Institute of Rock Structure and Mechanics, Czech Academy of Sciences, V. Holešovičkách 41, 182 09 Prague, Czech Republic
-
Affiliation as printed
Institute for Biodiversity and Ecosystem Dynamics, University of Amsterdam, Amsterdam, the Netherlands
-
Affiliation as printed
Institute of Astronomy, Geophysics and Atmospheric Sciences, Universidade de São Paulo, Rua do Matão 1226, 05508-090, São Paulo, SP, Brazil
-
Norwegian Geotechnical Institute · RWTH Aachen University
Affiliation as printed
Methods for Model–based Development in Computational Engineering, RWTH Aachen, 52062, Germany
Norwegian Geotechnical Institute, 0484 Oslo, Norway
-
Indian Institute of Technology Indore
Affiliation as printed
Department of Civil Engineering, Indian Institute of Technology, Indore, India
-
Indian Institute of Technology Indore
Affiliation as printed
Department of Civil Engineering, Indian Institute of Technology, Indore, India
-
Affiliation as printed
Institute of Hydrogeology, Engineering Geology and Applied Geophysics, Charles University, Albertov 6, 128 43 Prague, Czech Republic
-
University of the Basque Country · Research Institute for Geo-Hydrological Protection · National Research Council
Affiliation as printed
Consiglio Nazionale delle Ricerche, Istituto di Ricerca per la Protezione Idrogeologica, via Madonna Alta 126, I-06128 Perugia, Italy
Departamento de Geografía, Prehistoria y Arqueología, Universidad del País Vasco/Euskal Herriko Unibertsitatea (UPV/EHU), Vitoria-Gasteiz 01006, Spain
-
Research Institute for Geo-Hydrological Protection · National Research Council
Affiliation as printed
Consiglio Nazionale delle Ricerche, Istituto di Ricerca per la Protezione Idrogeologica, via Madonna Alta 126, I-06128 Perugia, Italy
-
Research Institute for Geo-Hydrological Protection · National Research Council
Affiliation as printed
Consiglio Nazionale delle Ricerche, Istituto di Ricerca per la Protezione Idrogeologica, via Madonna Alta 126, I-06128 Perugia, Italy
-
International Institute for Geo-Information Science and Earth Observation · University of Twente
Affiliation as printed
Faculty of Geo–Information Science and Earth Observation (ITC), University of Twente, PO Box 217, Enschede AE 7500, the Netherlands
-
Eurac Research · International Institute for Geo-Information Science and Earth Observation · University of Twente
Affiliation as printed
Center for Climate Change and Transformation, Eurac Research, Bolzano, Italy
Faculty of Geo–Information Science and Earth Observation (ITC), University of Twente, PO Box 217, Enschede AE 7500, the Netherlands
-
Affiliation as printed
GeoSphere Austria, Vienna, Austria
-
Affiliation as printed
Dipartimento di Scienze della Terra "A. Desio", Università degli Studi di Milano, Milan, Italy
-
Affiliation as printed
Dipartimento di Scienze della Terra "A. Desio", Università degli Studi di Milano, Milan, Italy
-
Affiliation as printed
Department of Environmental Geography, Faculty of Geography, Universitas Gadjah Mada, Indonesia
-
Affiliation as printed
Department of Computer Science and Electronics, Faculty of Mathematics and Natural Sciences, Universitas Gadjah Mada, Indonesia
-
Affiliation as printed
Department of Mathematics, Faculty of Mathematics and Natural Sciences, Universitas Gadjah Mada, Indonesia
-
Affiliation as printed
Department of Geology and Geological Engineering, Faculty of Mining, Geology and Petroleum Engineering, University of Zagreb, Pierottijeva 6, HR-10000 Zagreb, Croatia
Cited by 56 stored of 57
No patents citing this paper on Lens.org (checked 2026-10-06).