检索结果-南通市图书馆

Single-view facial reflectance inference with a differentiable renderer

Science China(Information Sciences) 2021年第11期64卷 5-21页

作者： Jiahao GENG Yanlin WENG Lvdi WANG Kun ZHOU State Key Lab of CAD&CG Zhejiang University ZJU-Face Unity Joint Lab of Intelligent Graphics

We introduce a deep-learning based algorithm to infer high-fidelity facial reflectance from a single image. The algorithm uses convolutional neural networks to encode the input image into a latent representation, from which a decoder and a detail enhancing network reconstruct decoupled facial reflectance(albedo, specular, and normal) as well as the environmental lighting. These decoupled components, together with a 3 D facial mesh estimated from the image, are then fed into a differentiable renderer to produce a rendered facial image. This allows us to iteratively optimize the latent representation of the facial image by minimizing the image-space reconstruction loss. Experimental results show that optimizing the latent representation through the differentiable renderer can effectively reduce the discrepancy between the original image and the rendered one, leading to a more accurate reconstruction of characteristic facial features such as skin tone, lip color, and facial hair.

关键词： facial modeling reflectance inference differentiable renderer

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

Weakly-Supervised Single-view Dense 3D Point Cloud Reconstruction via differentiable renderer

引用

Chinese Journal of Mechanical Engineering 2021年第5期34卷 195-205页

作者： Peng Jin Shaoli Liu Jianhua Liu Hao Huang Linlin Yang Michael Weinmann Reinhard Klein School of Mechanical Engineering Beijing Institute of TechnologyBeijing 100081China Institute of Computer Science II University of Bonn53115 BonnGermany

In recent years,addressing ill-posed problems by leveraging prior knowledge contained in databases on learning techniques has gained much *** this paper,we focus on complete three-dimensional(3D)point cloud reconstruction based on a single red-green-blue(RGB)image,a task that cannot be approached using classical reconstruction *** this purpose,we used an encoder-decoder framework to encode the RGB information in latent space,and to predict the 3D structure of the considered object from different *** individual predictions are combined to yield a common representation that is used in a module combining camera pose estimation and rendering,thereby achieving differentiability with respect to imaging process and the camera pose,and optimization of the two-dimensional prediction error of novel ***,our method allows end-to-end training and does not require supervision based on additional ground-truth(GT)mask annotations or ground-truth camera pose *** evaluation of synthetic and real-world data demonstrates the robustness of our approach to appearance changes and self-occlusions,through outperformance of current state-of-the-art methods in terms of accuracy,density,and model completeness.

关键词： Point clouds reconstruction differentiable renderer Neural networks Single-view configuration

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Neural compositing for real-time augmented reality rendering in low-frequency lighting environments

引用

Science China(Information Sciences) 2021年第2期64卷 139-153页

作者： Shengjie MA Qian SHEN Qiming HOU Zhong REN Kun ZHOU State Key Lab of CAD&CG Zhejiang University ZJU-FaceUnity Joint Lab of Intelligent Graphics

We present neural compositing, a deep-learning based method for augmented reality rendering,which uses convolutional neural networks to composite rendered layers of a virtual object with a real photograph to emulate shadow and reflection effects. The method starts from estimating the lighting and roughness information from the photograph using neural networks, renders the virtual object with a virtual floor into color, shadow and reflection layers by applying the estimated lighting, and finally refines the reflection and shadow layers using neural networks and blends them with the color layer and input image to yield the output image. We assume low-frequency lighting environments and adopt PRT(precomputed radiance transfer) for layer rendering, which makes the whole pipeline differentiable and enables fast end-to-end network training with synthetic scenes. Working on a single photograph, our method can produce realistic reflections in a real scene with spatially-varying material and cast shadows on background objects with unknown geometry and material at real-time frame rates.

关键词： augmented reality neural networks differentiable renderer reflection shadow

来源：

同方期刊数据库评论

在线全文

同方期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：