With the thriving of the pre-trained language model (PLM) widely verified in various NLP tasks, pioneer efforts attempt to explore the possible cooperation of the general textual information in PLM with the personalized behavioral information in user historical behavior sequences to enhance sequential recommendation (SR). However, despite the commonalities of input format and task goal, there are huge gaps between the behavioral and textual information, which obstruct thoroughly modeling SR as language modeling via PLM. To bridge the gap, we propose a novel unified pre-trained language model enhanced sequential recommendation (UPSR) that thoroughly transfers the next item prediction task to a text generation task, aiming to build a unified pre-trained recommendation model for multi-domain recommendation tasks. We formally design five key indicators, namely naturalness, domain consistency, informativeness, noise &amp; ambiguity, and text length, to guide the text \(\rightarrow\) item adaptation (selecting appropriate text to form the item textual representation) and behavior sequence \(\rightarrow\) text sequence adaptation (transferring the sequence of item textual representations into a text sequence) differently for pre-training and fine-tuning stages, which are essential but under-explored by previous works. In experiments, we conduct extensive evaluations on seven datasets with both supervised and zero-shot settings and achieve the overall best performance. Comprehensive model analyses also provide valuable insights for behavior modeling via PLM, shedding light on large pre-trained recommendation models. The source codes will be released in the future.

Zekai Qu

Ruobing Xie

Chaojun Xiao

Yuan Yao

Zhiyuan Liu

Fengzong Lian

Zhanhui Kang

Jie Zhou

ACM transactions on office information systems

ACM Transactions on Office Information Systems

Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language

全面建模多领域预训练推荐系统为语言模型

Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info