Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

arXiv (Cornell University)

Abstract

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.

Authors 5

1. Jujia Zhao Aachen

   Leiden University

   Affiliation as printed

   Leiden University

2. Zihan Wang

   Helmholtz Center for Information Security

   Affiliation as printed

   CISPA Helmholtz Center for Information Security

3. Shuaiqun Pan Aachen

   Leiden University

   Affiliation as printed

   Leiden University

4. Suzan Verberne Aachen

   Leiden University

   Affiliation as printed

   Leiden University

5. Zhaochun Ren Aachen

   Leiden University

   Affiliation as printed

   Leiden University