Importance sampling for rare event tracking within the ensemble Kalman filtering framework

Statistics and Computing, vol. 36

Abstract

In this work we employ importance sampling (IS) techniques to track a small over-threshold probability of a running maximum associated with the solution of a stochastic differential equation (SDE) within the framework of ensemble Kalman filtering (EnKF). The proposed method acts as a post-processing step applied to the EnKF output: it uses the ensemble at a given observation time to estimate the probability of a rare event occurring before the next observation, without altering the EnKF itself. Between two observation times of the EnKF, we propose to use IS with respect to the initial condition of the SDE, IS with respect to the Wiener process via a stochastic optimal control formulation, and combined IS with respect to both initial condition and Wiener process. Both IS strategies require the approximation of the solution of Kolmogorov Backward equation (KBE) with boundary conditions. In multidimensional settings, we employ a Markovian projection dimension reduction technique to obtain an approximation of the solution of the KBE by just solving a one dimensional PDE. The proposed ideas are tested on three illustrative examples: Double Well SDE, Langevin dynamics and noisy Charney-deVore model, and showcase a significant variance reduction compared to the standard Monte Carlo method and another sampling-based IS technique, namely, multilevel cross entropy.

Authors 4

1. Nadhir Ben Rached corresponding

   University of Leeds

   Affiliation as printed

   School of Mathematics, Faculty of Engineering and Physical Sciences, University of Leeds, Leeds, UK

2. Erik von Schwerin

   King Abdullah University of Science and Technology

   Affiliation as printed

   Applied Mathematics and Computational Sciences , KAUST , Thuwal , Saudi Arabia

3. Gaukhar Shaimerdenova corresponding

   Astana IT University · King Abdullah University of Science and Technology

   Affiliation as printed

   Applied Mathematics and Computational Sciences, KAUST, Thuwal, Saudi Arabia

   School of Artificial Intelligence and Data Science, Astana IT University, Astana, Kazakhstan

4. Raul F. Tempone corresponding Aachen Chair of Mathematics for Uncertainty Quantification

   King Abdullah University of Science and Technology · RWTH Aachen University

   Affiliation as printed

   Applied Mathematics and Computational Sciences, KAUST, Thuwal, Saudi Arabia

   Chair of Mathematics for Uncertainty Quantification, RWTH Aachen University, Aachen, Germany