AI Open

While GPTs with traditional fine-tuning fail to achieve strong results on natural language understanding (NLU), we show that GPTs can be better than or comparable to similar-sized BERTs on NLU tasks with a novel method P-tuning -- which employs trainable continuous prompt embeddings. On the knowledge probing (LAMA) benchmark, the best GPT recovers 64\% (P@1) of world knowledge without any additional text provided during test time, which substantially improves the previous best by 20+ percentage points. On the SuperGlue benchmark, GPTs achieve comparable and sometimes better performance to similar-sized BERTs in supervised learning. Importantly, we find that P-tuning also improves BERTs' performance in both few-shot and supervised settings while largely reducing the need for prompt engineering. Consequently, P-tuning outperforms the state-of-the-art approaches on the few-shot SuperGlue benchmark.

Xiao Liu

Yanan Zheng

Zhengxiao Du

Ming Ding

Yujie Qian

Zhilin Yang

Jie Tang

GPT understands, too

GPT 也懂

Lots of learning tasks require dealing with graph data which contains rich relation information among elements. Modeling physics system, learning molecular fingerprints, predicting protein interface, and classifying diseases require a model to learn from graph inputs. In other domains such as learning from non-structural data like texts and images, reasoning on extracted structures, like the dependency tree of sentences and the scene graph of images, is an important research topic which also needs graph reasoning models. Graph neural networks (GNNs) are connectionist models that capture the dependence of graphs via message passing between the nodes of graphs. Unlike standard neural networks, graph neural networks retain a state that can represent information from its neighborhood with arbitrary depth. Although the primitive GNNs have been found difficult to train for a fixed point, recent advances in network architectures, optimization techniques, and parallel computation have enabled successful learning with them. In recent years, systems based on variants of graph neural networks such as graph convolutional network (GCN), graph attention network (GAT), gated graph neural network (GGNN) have demonstrated ground-breaking performance on many tasks mentioned above. In this survey, we provide a detailed review over existing graph neural network models, systematically categorize the applications, and propose four open problems for future research.

Jie Zhou

Ganqu Cui

Shengding Hu

Zhengyan Zhang

Cheng Yang

Zhiyuan Liu

Lifeng Wang

Changcheng Li

Maosong Sun

Graph neural networks: A review of methods and applications

图神经网络：方法与应用综述

Conversational recommender system (CRS) interacts with users through multi-turn dialogues in natural language, which aims to provide high-quality recommendations for user's instant information need. Although great efforts have been made to develop effective CRS, most of them still focus on the contextual information from the current dialogue, usually suffering from the data scarcity issue. Therefore, we consider leveraging historical dialogue data to enrich the limited contexts of the current dialogue session. In this paper, we propose a novel multi-grained hypergraph interest modeling approach to capture user interest beneath intricate historical data from different perspectives. As the core idea, we employ hypergraph to represent complicated semantic relations underlying historical dialogues. In our approach, we first employ the hypergraph structure to model users' historical dialogue sessions and form a session-based hypergraph, which captures coarse-grained, session-level relations. Second, to alleviate the issue of data scarcity, we use an external knowledge graph and construct a knowledge-based hypergraph considering fine-grained, entity-level semantics. We further conduct multi-grained hypergraph convolution on the two kinds of hypergraphs, and utilize the enhanced representations to develop interest-aware CRS. Extensive experiments on two benchmarks ReDial and TG-ReDial validate the effectiveness of our approach on both recommendation and conversation tasks. Code is available at: https://github.com/RUCAIBox/MHIM.

Chenzhan Shang

Yupeng Hou

Wayne Xin Zhao

Yaliang Li

Jing Zhang

Multi-grained hypergraph interest modeling for conversational recommendation

多粒度超图兴趣建模用于对话推荐

The research field of Information Retrieval (IR) has evolved significantly, expanding beyond traditional search to meet diverse user information needs. Recently, Large Language Models (LLMs) have demonstrated exceptional capabilities in text understanding, generation, and knowledge inference, opening up exciting avenues for IR research. LLMs not only facilitate generative retrieval but also offer improved solutions for user understanding, model evaluation, and user-system interactions. More importantly, the synergistic relationship among IR models, LLMs, and humans forms a new technical paradigm that is more powerful for information seeking. IR models provide real-time and relevant information, LLMs contribute internal knowledge, and humans play a central role of demanders and evaluators to the reliability of information services. Nevertheless, significant challenges exist, including computational costs, credibility concerns, domain-specific limitations, and ethical considerations. To thoroughly discuss the transformative impact of LLMs on IR research, the Chinese IR community conducted a strategic workshop in April 2023, yielding valuable insights. This paper provides a summary of the workshop's outcomes, including the rethinking of IR's core values, the mutual enhancement of LLMs and IR, the proposal of a novel IR technical paradigm, and open challenges.

Qingyao Ai

Ting Bai

Zhao Cao

Yi Chang

Jiawei Chen

Zhumin Chen

Zhiyong Cheng

Shoubin Dong

Zhicheng Dou

Fuli Feng

Shen Gao

Jiafeng Guo

Xiangnan He

Yanyan Lan

Chenliang Li

Yiqun Liu

Ziyu Lyu

Weizhi Ma

Jun Ma

Zhaochun Ren

Pengjie Ren

Zhiqiang Wang

Mingwen Wang

Ji-Rong Wen

Le Wu

Xin Xin

Jun Xu

Dawei Yin

Peng Zhang

Fan Zhang

Weinan Zhang

Min Zhang

Xiaofei Zhu

Information Retrieval meets Large Language Models: A strategic report from Chinese IR community

信息检索与大型语言模型相遇：中国信息检索社区的战略报告

Answering complex factual questions has drawn a lot of attention. Researchers leverage various data sources to support complex QA, such as unstructured texts, structured knowledge graphs and relational databases, semi-structured web tables, or even hybrid data sources. However, although the ideas behind these approaches show similarity to some extent, there is not yet a consistent strategy to deal with various data sources. In this survey, we carefully examine how complex factual question answering has evolved across various data sources. We list the similarities among these approaches and group them into the analysis–extend–reason framework, despite the various question types and data sources that they focus on. We also address future directions for difficult factual question answering as well as the relevant benchmarks.

Lingxi Zhang

Xirui Ke

Haoyang Li

Xinmei Huang

Zhonghui Shao

Shulin Cao

Xin Lv

A survey on complex factual question answering

复杂事实问答调查

Adikarige Madanayake

Kyungmi Lee

Ickjai Lee

Mining contacts from spatio-temporal trajectories

从时空轨迹中挖掘联系

Sofiane Ennadir

Giannis Nikolentzos

Michalis Vazirgiannis

Henrik Boström

Generating graph perturbations to enhance the generalization of GNNs

生成图扰动以增强GNN的泛化能力

Jinqi Lai

Wensheng Gan

Jiayang Wu

Zhenlian Qi

Philip S. Yu

Large language models in law: A survey

法律领域的大语言模型综述

Kui Qian

Beth Friedman

Jun Takatoh

Alexander Groisman

Fan Wang

David Kleinfeld

Yoav Freund

CellBoost: A pipeline for machine assisted annotation in neuroanatomy

CellBoost：一种用于神经解剖学机器辅助注释的管道

This paper investigates a novel graph-based representation of sound waves inspired by the physical phenomenon of correlated vibrations. We propose a Wave2Graph framework for integrating multiple acoustic representations, including the spectrum of frequencies and correlations, into various neural computing architectures to achieve new state-of-the-art performances in sound classification. The capability and reliability of our end-to-end framework are evidently demonstrated in voice pathology for low-cost and non-invasive mass-screening of medical conditions, including respiratory illnesses and Alzheimer's Dementia. We conduct extensive experiments on multiple public benchmark datasets (ICBHI and ADReSSo) and our real-world dataset (IJSound: Respiratory disease detection using coughs and breaths). Wave2Graph framework consistently outperforms previous state-of-the-art methods with a large magnitude, up to 7.65% improvement, promising the usefulness of graph-based representation in signal processing and machine learning. The code and data will be made publicly available for reproducibility and future work.

Hoang Van Truong

Khanh-Tung Tran

Xuan-Son Vu

Duy-Khuong Nguyen

Monowar Bhuyan

Hoang D. Nguyen

Wave2Graph: Integrating spectral features and correlations for graph-based learning in sound waves

Wave2Graph：结合频谱特征和相关性进行基于图的声音波形学习

ISSN	2666-6510
h-index	16

Publication Information	Publisher: KeAi Communications Co.，Publishing cycle: ，Journal Type: journal，Open Access Journals: Yes
Basic data	Year of publication: 2020，Proportion of original research papers: ，Self Citation Rate:， Gold OA Rate:
Journal Fees	$700Details
Average review cycle	16 weeks

AI Open

Journal Citation Format

A journal article with 1 author

A journal article with 2 authors

A journal article with 3 authors

A journal article with 5 or more authors

Books Citation Format

Thesis Citation Format

Web sites Citation Format

Patent Citation Format

Staying up late manually editing references? ivySCI automatically matches journals and helps you generate references with a single click.

Share Submission Experience