Image-text retrieval (ITR) is a challenging task in the field of multimodal information processing due to the semantic gap between different modalities. In recent years, researchers have made great progress in exploring the accurate alignment between image and text. However, existing works mainly focus on the fine-grained alignment between image regions and sentence fragments, which ignores the guiding significance of context background information. Actually, integrating the local fine-grained information and global context background information can provide more semantic clues for retrieval. In this paper, we propose a novel Hierarchical Graph Alignment Network (HGAN) for image-text retrieval. First, to capture the comprehensive multimodal features, we construct the feature graphs for the image and text modality respectively. Then, a multi-granularity shared space is established with a designed Multi-granularity Feature Aggregation and Rearrangement (MFAR) module, which enhances the semantic corresponding relations between the local and global information, and obtains more accurate feature representations for the image and text modalities. Finally, the ultimate image and text features are further refined through three-level similarity functions to achieve the hierarchical alignment. To justify the proposed model, we perform extensive experiments on MS-COCO and Flickr30 K datasets. Experimental results show that the proposed HGAN outperforms the state-of-the-art methods on both datasets, which demonstrates the effectiveness and superiority of our model.

Jie Guo

Meiting Wang

Yan Zhou

Bin Song

Yuhao Chi

Wei Fan

Jianglong Chang

IEEE Transactions on Multimedia

期刊的范围是多媒体技术和多媒体应用研究的各个方面，包括但不限于电路、网络、信号处理、系统、软件和系统集成，以“感兴趣的领域”为代表。赞助商。

The IEEE Transactions on Multimedia journal covers various aspects of multimedia technology and application research, including but not limited to circuits, networks, signal processing, systems, software, and system integration. The journal is sponsored by stakeholders in the field.

《IEEE Transactions on Multimedia》期刊涵盖多媒体技术及其应用研究的各个方面，包括但不限于电路、网络、信号处理、系统、软件和系统集成等领域。该期刊由相关领域的赞助商支持。

HGAN: Hierarchical Graph Alignment Network for Image-Text Retrieval

Experimental results show that the proposed HGAN outperforms the state-of-the-art methods on both datasets, which demonstrates the effectiveness and superiority of the model.

Category	Quartile
计算机科学	1
计算机科学, 计算机信息系统	1
计算机科学, 计算机软件工程	1
计算机科学, 电信学	2

HGAN: Hierarchical Graph Alignment Network for Image-Text Retrieval

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info