ACM TRANSACTIONS ON MULTIMEDIA COMPUTING, COMMUNICATIONS, AND APPLICATIONS 是 ACM 多媒体特别兴趣小组 (SIGMM) 的旗舰出版物。它正在征集有关多媒体各个方面的论文提交。也欢迎有关单一媒体（例如，音频、视频、动画）及其处理的论文。TOMM 是一份经过同行评审的档案期刊，有印刷和数字两种形式。该杂志每季度出版一次；每期大约有 7 篇 23 页的文章。此外，所有特刊均仅在线发布，以确保及时发布。交易主要包括研究论文。这是一份档案期刊，旨在使论文随着时间的推移具有持久的重要性和价值。一般而言，不包括主要关注特定多媒体产品或行业现状的论文。

ACM Transactions on Multimedia Computing, Communications and Applications is the flagship publication of the ACM Special Interest Group on Multimedia (SIGMM). It invites submissions on all aspects of multimedia, as well as papers focused on individual media (e.g., audio, video, animation) and their processing. TOMM is a peer-reviewed archival journal available in both print and digital formats, published quarterly with approximately 7 articles per issue, totaling around 23 pages. Additionally, all special issues are published online only to ensure timely dissemination. The journal primarily includes research papers and aims to maintain the enduring significance and value of the papers over time. Generally, papers focusing primarily on specific multimedia products or the current state of the industry are not included.

《ACM Transactions on Multimedia Computing, Communications and Applications》是ACM多媒体特别兴趣小组（SIGMM）的旗舰期刊，欢迎提交关于多媒体各个方面的论文。我们也接受关于单一媒体（如音频、视频、动画）及其处理的研究。该期刊经过同行评审，提供印刷和数字两种形式，每季度出版一次，每期约有7篇、23页的文章。此外，所有特刊均仅在线发布，以确保及时性。期刊主要发表研究论文，旨在使论文在时间的推移中保持持久的重要性和价值。一般而言，不包括主要关注特定多媒体产品或行业现状的论文。

ACM Transactions on Multimedia Computing, Communications and Applications

Deep Convolutional Neural Network (CNN) has achieved remarkable results in computer vision tasks for end-to-end learning. We evaluate here the power of a deep CNN to learn robust features from raw Electroencephalogram (EEG) data to detect seizures. Seizures are hard to detect, as they vary both inter- and intra-patient. In this article, we use a deep CNN model for seizure detection task on an open-access EEG epilepsy dataset collected at the Boston Children's Hospital. Our deep learning model is able to extract spectral, temporal features from EEG epilepsy data and use them to learn the general structure of a seizure that is less sensitive to variations. For cross-patient EEG data, our method produced an overall sensitivity of 90.00%, specificity of 91.65%, and overall accuracy of 98.05% for the whole dataset of 23 patients. The system can detect seizures with an accuracy of 99.46%. Thus, it can be used as an excellent cross-patient seizure classifier. The results show that our model performs better than the previous state-of-the-art models for patient-specific and cross-patient seizure detection task. The method gave an overall accuracy of 99.65% for patient-specific data. The system can also visualize the special orientation of band power features. We use correlation maps to relate spectral amplitude features to the output in the form of images. By using the results from our deep learning model, this visualization method can be used as an effective multimedia tool for producing quick and relevant brain mapping images that can be used by medical experts for further investigation.

M. Shamim Hossain

Syed Umar Amin

Mansour Alsulaiman

Ghulam Muhammad

ACM Transactions on Multimedia Computing Communications and Applications

Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization

With the rapid development of smart device technology, the current version of the Internet of Things (IoT) is moving towards a multimedia IoT because of multimedia data. This innovative concept seamlessly integrates multimedia data with the IoT-Edge Continuum. Recently, a distributed learning framework shows promise in revolutionizing various industries, including smart cities, healthcare, etc. However, these applications may face challenges such as the presence of malicious devices that invade the privacy of other devices or corrupt uploaded model parameters. Additionally, the existing synchronous federated learning (FL) methods face challenges in effectively training models on local datasets due to the diversity of IoT devices. To tackle these concerns, we propose an efficient and privacy-enhanced asynchronous federated learning approach for multimedia data in edge-based IoT. In contrast to traditional FL methods, our approach combines revocable attribute-based encryption (RABE) and differential privacy (DP). This guarantees the privacy of the entire process while allowing seamless collaboration between multiple devices and the aggregation server during model training. Also, this combination brings a dynamic nature to the system. Furthermore, we utilize an asynchronous weight-based aggregation algorithm to improve the efficiency of training and the quality of the final returned model. Our proposed scheme is confirmed by theoretical safety proofs and experimental results with multimedia data. Performance evaluation shows that our framework reduces the cryptography runtime by 63.3% and the global model aggregation time by 61.9% compared to cutting-edge schemes. Moreover, our accuracy is comparable to the most primitive FL schemes, maintaining 86.7%, 70.8%, and 86.1% on MNIST, CIFAR-10, and Fashion-MNIST, respectively. The experimental results highlight the remarkable practicality, resilience and effectiveness of the proposed scheme.

Hu Xiong

Hang Yan

Mohammad S. Obaidat

Jingxue Chen

Mingsheng Cao

Sachin Kumar

Kadambri Agarwal

Saru Kumari

Efficient and Privacy-Enhanced Asynchronous Federated Learning for Multimedia Data in Edge-based IoT

基于边缘物联网的多媒体数据高效且隐私增强的异步联邦学习

Foggy days limit the functionality of outdoor surveillance systems. However, it is still a challenge for existing methods to maintain the uniformity of defogging between image regions with a similar depth of field and large differences in appearance. To address above problem, this article proposes a regional gradient constrained prior (RGCP) for defogging that uses the piecewise smoothing characteristic of the scene structure to achieve accurate estimation and reliable constraint of the transmission. RGCP first derives that when adjacent similar pixels in the fog image are aggregated and spatially divided into regions, clusters of region pixels in RGB space conform to a chi-square distribution. The offset of the confidence boundary of the clusters can be regarded as the initial transmission of each region. RGCP further uses a gradient distribution to distinguish different regional appearances and formulate an interregional constraint function to constrain the overestimation of the transmission in the flat region, thereby maintaining the consistency between the estimated transmission map and the depth map. The experimental results demonstrate that the proposed method can achieve natural defogging performance in terms of various foggy conditions.

Qiang Guo

Zhi Zhang

Mingliang Zhou

Hong Yue

Huayan Pu

Jun Luo

Image Defogging Based on Regional Gradient Constrained Prior

基于区域梯度约束先验的图像去雾

This article tackles the challenging yet important task of Visual Grounding (VG), which aims to localize a visual region in the given image referred by a natural language query. Existing efforts on the VG task are twofold: (1) two-stage methods first extract region proposals and then rank them according to their similarities with the referring expression, which usually leads to suboptimal results due to the quality of region proposals; (2) one-stage methods usually predict all the possible coordinates of the target region online by leveraging modern object detection architectures, which pay little attention to cross-modality correlations and have limited generalization ability. To better address the task, we present an effective transformer-based end-to-end visual grounding approach, which focuses on capturing the cross-modality correlations between the referring expression and visual regions for accurately reasoning the location of the target region. Specifically, our model consists of a feature encoder, a cross-modality interactor, and a modality-agnostic decoder. The feature encoder is employed to capture the intra-modality correlation, which models the linguistic context in query and the spatial dependency in image respectively. The cross-modality interactor endows the model with the capability of highlighting the localization-relevant visual and textual cues by mutual verification of vision and language, which plays a key role in our model. The decoder learns a consolidated token representation enriched by multi-modal contexts and further directly predicts the box coordinates. Extensive experiments on five public benchmark datasets with quantitative and qualitative analysis clearly demonstrate the effectiveness and rationale of our proposed method.

Kun Li

Jiaxiu Li

Dan Guo

Xun Yang

Meng Wang

Transformer-Based Visual Grounding with Cross-Modality Interaction

基于Transformer的跨模态交互视觉定位

Generative adversarial networks (GANs)-based person re-identification (re-id) schemes provide potential ways to augment data in practical applications. However, existing solutions perform poorly because of the separation of data generation and re-id training and a lack of diverse data in real-world scenarios. In this paper, a person re-id model (IDGAN) based on semantic map guided identity transfer GAN is proposed to improve the person re-id performance. With the aid of the semantic map, IDGAN generates pedestrian images with varying poses, perspectives, and backgrounds efficiently and accurately, improving the diversity of training data. To increase the visual realism, IDGAN utilizes a gradient augmentation method based on local quality attention to refine the generated image locally. Then, a two-stage joint training framework is employed to allow the GAN and the person re-id network to learn from each other to better use the generated data. Detailed experimental results demonstrate that, compared with the existing state-of-the-art methods, IDGAN is capable of producing high-quality images and significantly enhancing re-id performance, with the FID of generated images on the Market-1501 dataset being reduced by 1.15, and mAP on the Market-1501 and DukeMTMC-reID datasets being increased by 3.3% and 2.6%, respectively.

Tian Wu

Rongbo Zhu

Shaohua Wan

Semantic Map Guided Identity Transfer GAN for Person Re-identification

语义地图引导的身份迁移生成对抗网络用于行人重识别

Deep cross-modal hashing methods provide effective and efficient solutions for large-scale cross-modal retrieval. However, existing cross-modal hashing methods fail to capture the dynamic changes of real-world data, and suffer from serious performance degradation when retrieving streaming data. In this paper, we propose a novel hashing method to achieve accurate cross-modal retrieval under continuous and streaming scenarios. Specifically, regularization-based lifelong learning module is introduced to balance plasticity for learning new knowledge and stability for maintaining old knowledge, and update incremental hash codes without retraining cumulative data. Then, multi-scale consistency network which employs multi-scale feature fusion module to extract fine-grained features among multi-scale modalities is introduced to learn multi-level semantic representations with consistency. Additionally, modality alignment with variational information bottleneck is designed to remove irrelevant information and obtain unified representation, which can be proved to be effective to yield high-quality hash code with new and old knowledge. Extensive experiments show that ours gains the advanced performance and the better adaptability to continuous and streaming environments.

Liming Xu

Hanqi Li

Jie Shao

Xianhua Zeng

Weisheng Li

Multi-scale Consistency Deep Lifelong Cross-modal Hashing

多尺度一致性深度终身跨模态哈希

In this paper, the novel adjacent two-end link or network multi-end link concept of software-defined network is presented to support the data transmission of electroencephalogram, audio and video streaming in spread spectrum communications. Instead of solving the problem of software-defined prioritization scheme in networking calculation and communication, the southbound-northbound structure of existing neural or information network is found in control and perceptual interactions. The proposed framework of multi-lead and multi-media structures allow the human-centric Metaverse to execute different operations for local and global planning schemes with the high-speed lead or media data transmission of spread spectrum streaming, depending on the kind of brain-computer interaction, and similar to the system of multi-modal perception in the routing or switching. Firstly, this research designed the filed programmable gate array for conventional routing nodes in the computation of different neural networks. Secondly, this study demonstrates its enhanced applicability with software core for multiple routers in the computing power network of the Internet of brains in different brain-computer smart terminals. Experiment results also show the benefits of proposed model and structure, and exposing the different running indicators in the simulation.

Yuxuan Zhou

Li Mingyang

Tong Jingze

Li Linlin

Yang Zhiwei

SD-Meta: The Software-Defined Network of Human-Centric Metaverse for Multi-Lead or Multi-Media Data in Spread Spectrum Communications

SD-Meta：面向多导或多媒体数据的人本元宇宙软件定义网络在扩频通信中的应用

Learned video compression has developed rapidly and shown competitive rate-distortion performance compared with the latest traditional video coding standard H.266 (VVC). However, existing works were restricted to fixed prediction direction and GoP size. The inflexibility on prediction structure hinders learned video compression towards optimal compression efficiency in diverse motion scenarios. In this paper, we propose to advance learned video compression with adaptive prediction structure decision. Specifically, we propose a unified compression framework that supports both forward prediction and bi-directional prediction. The framework can flexibly switch to different prediction direction to achieve better prediction performance. Meanwhile, we propose a low-complexity prediction structure decision algorithm, where prediction direction and GoP size are adaptively determined based on motion complexity to achieve optimal compression efficiency. Experimental results demonstrate that the proposed unified framework with adaptive decision algorithm improves compression efficiency of pure forward prediction-based or bi-directional prediction-based framework with neglectable ( \(0.9\%\) ) encoding time increment. Meanwhile, it achieves comparable compression performance with VVC and recent learned video coding methods.

Jiayu Yang

Yongqi Zhai

Wei Jiang

Chunhui Yang

Feng Gao

Ronggang Wang

Adaptive Prediction Structure for Learned Video Compression

学习型视频压缩的自适应预测结构

Person in Uniforms Re-identification (PU-ReID) is an emerging computer vision task for various intelligent video surveillance applications. PU-ReID is much understudied due to the absence of large-scale annotated datasets, also this task is extremely challenging because many individuals captured in surveillance videos wear same clothing, introducing significant interference for retrieval tasks owing to the high visual similarity of outfits and subtle differences among individuals. This research initiates the exploration of person in uniform re-identification, a novel and challenging task tailored for real industrial scenarios. To address these issues, a novel framework is proposed for PU-ReID, which aims to reduce the visual impact of similar uniforms and learn the unique cues derived from human parts and detailed visual features. Specifically, several novel techniques are built in this study: first, a uniform feature separation method with orthogonal constraints is proposed to extract non-uniform features. Second, multi-view subspace feature alignment is introduced to integrate soft-biometrics including optics-related visual features, contextual information of human parts, and cloth-invariant biometric features. In addition, to close the gap between academic research and real world settings, a new person in uniforms ReID dataset named PU-151 is constructed, which consists of 151 gas station employees in uniforms from 1,488 videos. At last, extensive experiments conducted on five datasets demonstrate that the proposed approach significantly outperforms the state-of-the-art methods. This advancement can drive further developments in re-identification and person search technologies.

Chong-Yang Xiang

Xiao Wu

Jun-Yan He

Zhaoquan Yuan

Tingquan He

Person in Uniforms Re-Identification

制服人员再识别

This is the foreword of our special issue volume on Realistic Synthetic Data: Generation, Learning, Evaluation organized with the ACM Transactions on Multimedia Computing, Communications, and Applications. It presents the target of the special issue that relates to synthetic data for various modalities, e.g., signals, images, volumes, audio, etc., controllable generation for learning from synthetic data, transfer learning and generalization of models, causality in data generation, addressing bias, limitations and trustworthiness in data generation, evaluation measures/protocols and benchmarks to assess quality of synthetic content, open synthetic datasets and software tools, and ethical aspects of synthetic data. The call for papers received a record number of 40 submissions out of which 15 were finally accepted for publication. This introduction provides an overview of the topics of each of the articles.

Bogdan Ionescu

Ioannis Patras

Henning Müller

Alberto Del Bimbo

Introduction to the Special Issue on Realistic Synthetic Data: Generation, Learning, Evaluation

《真实合成数据特刊导言：生成、学习与评估》

acBWQ6
Journal Impact	5.35
mbqgSu	COMPUTER SCIENCE, THEORY & METHODS(Q1)
xGGr6y	SCI Q1中科院 3 区

ISSN	1551-6857, 1551-6865
h-index	63
hQoIdU	计算机科学(3区)计算机科学计算机信息系统(3区)计算机科学计算机软件工程(3区)计算机科学计算机理论方法(3区)

u9zg3I	计算机科学人工智能数学操作系统工程类计算机视觉心理学哲学物理机器学习程序设计语言生物认知心理学语言学模式识别（心理学）统计计算机网络万维网政治学法学电信图像（数学）量子力学地理经济
+5d/bs	e+/GRg: Association for Computing Machinery (ACM)，uzYam8: Quarterly，dxwaWB: journal，8x2mMQ: TY4X7V
ak9yM6	A7IHrH: 2005，sQJ/D8: 98.61%，sE9og+:13.50%， wzlKC/: 0.57%
JrpmBe	网友分享经验：>12周，或约稿
jdemjX	网友分享经验：容易

ACM Transactions on Multimedia Computing, Communications and Applications

uM3Gfz

sZFHBm

sZFHBm

sZFHBm

sZFHBm

Hi2Sc4

e7MLUV

XjnQ+b

Kq5Kd6

LB2xFn

XmiM++

Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization

Efficient and Privacy-Enhanced Asynchronous Federated Learning for Multimedia Data in Edge-based IoT

Image Defogging Based on Regional Gradient Constrained Prior

Transformer-Based Visual Grounding with Cross-Modality Interaction

Semantic Map Guided Identity Transfer GAN for Person Re-identification

Multi-scale Consistency Deep Lifelong Cross-modal Hashing

SD-Meta: The Software-Defined Network of Human-Centric Metaverse for Multi-Lead or Multi-Media Data in Spread Spectrum Communications

Adaptive Prediction Structure for Learned Video Compression

Person in Uniforms Re-Identification

Introduction to the Special Issue on Realistic Synthetic Data: Generation, Learning, Evaluation