Image translation has various applications in computer graphics and computer vision, aiming to transfer images from one domain to another. Thanks to the excellent generation capability of diffusion models, recent single-view image translation methods achieve realistic results. However, directly applying diffusion models for multi-view image translation remains challenging for two major obstacles: the need for paired training data and the limited view consistency. To overcome the obstacles, we present a first unified multi-view image to multi-view image translation framework based on diffusion models, called MV2MV. Firstly, we propose a novel self-supervised training strategy that exploits the success of off-the-shelf single-view image translators and the 3D Gaussian Splatting (3DGS) technique to generate pseudo ground truths as supervisory signals, leading to enhanced consistency and fine details. Additionally, we propose a latent multi-view consistency block, which utilizes the latent-3DGS as the underlying 3D representation to facilitate information exchange across multi-view images and inject 3D prior into the diffusion model to enforce consistency. Finally, our approach simultaneously optimizes the diffusion model and 3DGS to achieve a better trade-off between consistency and realism. Extensive experiments across various translation tasks demonstrate that MV2MV outperforms task-specific specialists in both quantitative and qualitative.

Youcheng Cai

Runshi Li

Ligang Liu

ACM Transactions on Graphics

ACM TRANSACTIONS ON GRAPHICS (TOG) 是一份经过同行评审的科学期刊，旨在传播计算机图形学领域的最新发现。它自 1982 年以来由计算机协会出版。从 2003 年开始，所有在 SIGGRAPH 年度会议上被接受发表的论文都印在该期刊的夏季特刊上。

ACM Transactions on Graphics (TOG) is a peer-reviewed scientific journal aimed at disseminating the latest research findings in the field of computer graphics. It has been published by the Association for Computing Machinery since 1982. Since 2003, all papers accepted for presentation at the SIGGRAPH annual conference are included in the summer issue of this journal.

《ACM Transactions on Graphics》（TOG）是一本经过同行评审的科学期刊，旨在传播计算机图形学领域的最新研究成果。该期刊自1982年由计算机协会出版。从2003年起，所有在SIGGRAPH年度会议上接受发表的论文均会收录在该期刊的夏季特刊中。

MV2MV: Multi-View Image Translation via View-Consistent Diffusion Models

This work proposes a novel self-supervised training strategy that exploits the success of off-the-shelf single-view image translators and the 3D Gaussian Splatting technique to generate pseudo ground truths as supervisory signals, leading to enhanced consistency and fine details.

Category	Quartile
计算机科学	1
计算机科学, 计算机软件工程	1

MV2MV: Multi-View Image Translation via View-Consistent Diffusion Models

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
COMPUTER SCIENCE, SOFTWARE ENGINEERING	1