Uncovering Selective State Space Model's Capabilities in Lifelong Sequential Recommendation

arXiv (Cornell University)

Abstract

Sequential Recommenders have been widely applied in various online services, aiming to model users' dynamic interests from their sequential interactions. With users increasingly engaging with online platforms, vast amounts of lifelong user behavioral sequences have been generated. However, existing sequential recommender models often struggle to handle such lifelong sequences. The primary challenges stem from computational complexity and the ability to capture long-range dependencies within the sequence. Recently, a state space model featuring a selective mechanism (i.e., Mamba) has emerged. In this work, we investigate the performance of Mamba for lifelong sequential recommendation (i.e., length>=2k). More specifically, we leverage the Mamba block to model lifelong user sequences selectively. We conduct extensive experiments to evaluate the performance of representative sequential recommendation models in the setting of lifelong sequences. Experiments on two real-world datasets demonstrate the superiority of Mamba. We found that RecMamba achieves performance comparable to the representative model while significantly reducing training duration by approximately 70% and memory costs by 80%. Codes and data are available at \url{https://github.com/nancheng58/RecMamba}.

Authors 11

1. Ji-yuan YANG

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

2. Yuanzi Li

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

3. Jingyu Zhao

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

4. Hanbing Wang

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

5. Muyang Ma

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

6. Jun Ma

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

7. Zhaochun Ren Aachen

   Leiden University · Michigan State University

   Affiliation as printed

   Leiden University Leiden , The Netherlands

   Michigan State University East Lansing , USA

8. Mengqi Zhang

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

9. Xin Xin

   Shandong University

   Affiliation as printed

   Shandong University Qingdao , China

10. Zhumin Chen

    Shandong University

    Affiliation as printed

    Shandong University Qingdao , China

11. Pengjie Ren

    Shandong University

    Affiliation as printed

    Shandong University Qingdao , China