Computational Visual Media

Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repositoryhttps://github.com/MenghaoGuo/Awesome-Vision-Attentionsis dedicated to collecting related work. We also suggest future directions for attention mechanism research.

Meng-Hao Guo

Tian-Xing Xu

Jiang-Jiang Liu

Zheng-Ning Liu

Peng-Tao Jiang

Tai-Jiang Mu

Song-Hai Zhang

Ralph R. Martin

Ming-Ming Cheng

Shi-Min Hu

Comp. Visual Media

Attention mechanisms in computer vision: A survey

计算机视觉中的注意力机制：综述

In this paper, we propose a simple-yet-effective method for isotropic meshing relying on Euclidean distance transformation based centroidal Voronoi tessellation (CVT). Our approach improves the performance and robustness of computing CVT on curved domains while simultaneously providing high-quality output meshes. While conventional extrinsic methods compute CVTs in the entire volume bounded by the input model, we restrict the computation to a 3D shell of user-controlled thickness. Taking voxels which contain surface samples as sites, we compute the exact Euclidean distance transform on the GPU. Our algorithm is parallel and memory-efficient, and can construct the shell space for resolutions up to 20483 at interactive speed. The 3D centroidal Voronoi tessellation and restricted Voronoi diagrams are also computed efficiently on the GPU. Since the shell space can bridge holes and gaps smaller than a certain tolerance, and tolerate non-manifold edges and degenerate triangles, our algorithm can handle models with such defects, which typically cause conventional remeshing methods to fail. Our method can process implicit surfaces, polyhedral surfaces, and point clouds in a unified framework. Computational results show that our GPU-based isotropic meshing algorithm produces results comparable to state-of- the-art techniques, but is significantly faster than conventional CPU-based implementations.

Yuen-Shan Leung

Xiaoning Wang

Ying He

Yong-Jin Liu

Charlie C. L. Wang

A unified framework for isotropic meshing based on narrow-band Euclidean distance transformation

基于窄带欧几里得距离变换的各向同性网格化统一框架

Abstract A thin shell model refers to a surface or structure, where the object’s thickness is considered negligible. In the context of 3D printing, thin shell models are characterized by having lightweight, hollow structures, and reduced material usage. Their versatility and visual appeal make them popular in various fields, such as cloth simulation, character skinning, and for thin-walled structures like leaves, paper, or metal sheets. Nevertheless, optimization of thin shell models without external support remains a challenge due to their minimal interior operational space. For the same reasons, hollowing methods are also unsuitable for this task. In fact, thin shell modulation methods are required to preserve the visual appearance of a two-sided surface which further constrain the problem space. In this paper, we introduce a new visual disparity metric tailored for shell models, integrating local details and global shape attributes in terms of visual perception. Our method modulates thin shell models using global deformations and local thickening while accounting for visual saliency, stability, and structural integrity. Thereby, thin shell models such as bas-reliefs, hollow shapes, and cloth can be stabilized to stand in arbitrary orientations, making them ideal for 3D printing.

Yu Xing

Xiaoxuan Wang

Lin Lu

Andrei Sharf

Daniel Cohen-Or

Changhe Tu

Shell stand: Stable thin shell models for 3D fabrication

壳体支架：用于3D打印的稳定薄壳模型

While originally designed for natural language processing tasks, the self-attention mechanism has recently taken various computer vision areas by storm. However, the 2D nature of images brings three challenges for applying self-attention in computer vision: (1) treating images as 1D sequences neglects their 2D structures; (2) the quadratic complexity is too expensive for high-resolution images; (3) it only captures spatial adaptability but ignores channel adaptability. In this paper, we propose a novel linear attention named large kernel attention (LKA) to enable self-adaptive and long-range correlations in self-attention while avoiding its shortcomings. Furthermore, we present a neural network based on LKA, namely Visual Attention Network (VAN). While extremely simple, VAN achieves comparable results with similar size convolutional neural networks (CNNs) and vision transformers (ViTs) in various tasks, including image classification, object detection, semantic segmentation, panoptic segmentation, pose estimation, etc. For example, VAN-B6 achieves 87.8% accuracy on ImageNet benchmark, and sets new state-of-the-art performance (58.2 PQ) for panoptic segmentation. Besides, VAN-B2 surpasses Swin-T 4 mIoU (50.1 vs. 46.1) for semantic segmentation on ADE20K benchmark, 2.6 AP (48.8 vs. 46.2) for object detection on COCO dataset. It provides a novel method and a simple yet strong baseline for the community. The code is available athttps://github.com/Visual-Attention-Network.

Cheng-Ze Lu

Visual attention network

视觉注意力网络

Abstract Gradient-domain rendering estimates finite difference gradients of image intensities and reconstructs the final result by solving a screened Poisson problem, which shows improvements over merely sampling pixel intensities. Adaptive sampling is another orthogonal research area that focuses on distributing samples adaptively in the primal domain. However, adaptive sampling in the gradient domain with low sampling budget has been less explored. Our idea is based on the observation that signals in the gradient domain are sparse, which provides more flexibility for adaptive sampling. We propose a deep-learning-based end-to-end sampling and reconstruction framework in gradient-domain rendering, enabling adaptive sampling gradient and the primal maps simultaneously. We conducted extensive experiments for evaluation and showed that our method produces better reconstruction quality than other methods in the test dataset.

Yuzhi Liang

Tao Liu

Yuchi Huo

Rui Wang

Hujun Bao

Adaptive sampling and reconstruction for gradient-domain rendering

梯度域渲染的自适应采样与重构

Abstract Deep generative models allow the synthesis of realistic human faces from freehand sketches or semantic maps. However, although they are flexible, sketches and semantic maps provide too much freedom for manipulation, and thus, are not easy for novice users to control. In this study, we present DeepFaceReshaping, a novel landmark-based deep generative framework for interactive face reshaping. To edit the shape of a face realistically by manipulating a small number of face landmarks, we employ neural shape deformation to reshape individual face components. Furthermore, we propose a novel Transformer-based partial refinement network to synthesize the reshaped face components conditioned on the edited landmarks, and fuse the components to generate the entire face using a local-to-global approach. In this manner, we limit possible reshaping effects within a feasible component-based face space. Thus, our interface is intuitive even for novice users, as confirmed by a user study. Our experiments demonstrate that our method outperforms traditional warping-based approaches and recent deep generative techniques.

Shu-Yu Chen

Yue-Ren Jiang

Hongbo Fu

Xinyang Han

Zitao Liu

Rong Li

Lin Gao

DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation

DeepFaceReshaping：通过地标操作实现交互式深度人脸重塑

Abstract Highly scattering media, such as milk, skin, and clouds, are common in the real world. Rendering participating media is challenging, especially for high-order scattering dominant media, because the light may undergo a large number of scattering events before leaving the surface. Monte Carlo-based methods typically require a long time to produce noise-free results. Based on the observation that low-albedo media contain less noise than high-albedo media, we propose reducing the variance of the rendered results using differentiable regularization. We first render an image with low-albedo participating media together with the gradient with respect to the albedo, and then predict the final rendered image with a low-albedo image and gradient image via a novel prediction function. To achieve high quality, we also consider the gradients of neighboring frames to provide a noise-free gradient image. Ultimately, our method can produce results with much less overall error than equal-time path tracing methods.

Wenshi Wu

Beibei Wang

Miloš Hašan

Lei Zhang

Zhong Jin

Ling-Qi Yan

Efficient participating media rendering with differentiable regularization

高效参与介质渲染与可微正则化

Abstract The cross-view matching of local image features is a fundamental task in visual localization and 3D reconstruction. This study proposes FilterGNN, a transformer-based graph neural network (GNN), aiming to improve the matching efficiency and accuracy of visual descriptors. Based on high matching sparseness and coarse-to-fine covisible area detection, FilterGNN utilizes cascaded optimal graph-matching filter modules to dynamically reject outlier matches. Moreover, we successfully adapted linear attention in FilterGNN with post-instance normalization support, which significantly reduces the complexity of complete graph learning from O ( N 2 ) to O ( N ). Experiments show that FilterGNN requires only 6% of the time cost and 33.3% of the memory cost compared with SuperGlue under a large-scale input size and achieves a competitive performance in various tasks, such as pose estimation, visual localization, and sparse 3D reconstruction.

Jun-Xiong Cai

Yu-Kun Lai

FilterGNN: Image feature matching with cascaded outlier filters and linear attention

FilterGNN：基于级联离群点过滤器和线性注意力的图像特征匹配

Abstract Global illumination (GI) plays a crucial role in rendering realistic results for virtual exhibitions, such as virtual car exhibitions. These scenarios usually include all-frequency bidirectional reflectance distribution functions (BRDFs), although their geometries and light configurations may be static. Rendering all-frequency BRDFs in real time remains challenging due to the complex light transport. Existing approaches, including precomputed radiance transfer, light probes, and the most recent path-tracing-based approaches (ReSTIR PT), cannot satisfy both quality and performance requirements simultaneously. Herein, we propose a practical hybrid global illumination approach that combines ray tracing and cached GI by caching the incoming radiance with wavelets. Our approach can produce results close to those of offline renderers at the cost of only approximately 17 ms at runtime and is robust over all-frequency BRDFs. Our approach is designed for applications involving static lighting and geometries, such as virtual exhibitions.

Abbreviation	Comp. Visual Media
Journal Impact	17.21
Quartiles(Global)	COMPUTER SCIENCE, SOFTWARE ENGINEERING(Q1)

ISSN	2096-0433, 2096-0662
h-index	27

Publication Information	Publisher: Tsinghua University Press，Publishing cycle: 4 issues per year，Journal Type: journal，Open Access Journals: Yes
Basic data	Year of publication: 2015，Proportion of original research papers: 81.58%，Self Citation Rate:3.50%， Gold OA Rate: 100.00%
Average review cycle	网友分享经验：6 Weeks

Computational Visual Media

Journal Citation Format

A journal article with 1 author

A journal article with 2 authors

A journal article with 3 authors

A journal article with 5 or more authors

Books Citation Format

Thesis Citation Format

Web sites Citation Format

Patent Citation Format

Staying up late manually editing references? ivySCI automatically matches journals and helps you generate references with a single click.

Share Submission Experience

Attention mechanisms in computer vision: A survey

A unified framework for isotropic meshing based on narrow-band Euclidean distance transformation

Shell stand: Stable thin shell models for 3D fabrication

Visual attention network

Adaptive sampling and reconstruction for gradient-domain rendering

Adaptive sampling and reconstruction for gradient-domain rendering

DeepFaceReshaping: Interactive deep face reshaping via landmark manipulation

Efficient participating media rendering with differentiable regularization

FilterGNN: Image feature matching with cascaded outlier filters and linear attention

Real-time all-frequency global illumination with radiance caching