WCSE

View Synthesis from Silhouette Using Deep Convolutional Neural Network

Abstract— Multiview video could be the basis to support various applications, such as Three-Dimensional video (3DV), Virtual Reality (VR), and Free Viewpoint Video (FVV). Mult iview data is intrinsically redundant, in fact, the semantic contents of different views are almost similar, and this is especially true for small baseline views. Obviously, wide baseline views might significantly differ in their contents, where some objects might be completely absent in some views. However, the current approaches of representing multiview data, even if they exploit inter -view correlation, require large bandwidth for transmission. This bandwidth is almost linear with the number of transmitted views. Thus, in this paper we propose to address this problem by representing lateral views solely using their edges, while dropping their texture content. The texture content is synthesized, at the receiver side, by a convolutional neural network (CNN) exploiting the edges and the information in the central view. The edges of the lateral views represent the location of the “objects” in their corresponding views, whereas, we assume that their te xture in other vie ws does not change significantly, consequently there is no need to represent them in the lateral views. In this work, in addit ion to the proposed paradigm of representing mult iview data, we also propose a training framework for the CNN network. Experimental results demonstrate the effectiveness of the proposed framework and demonstrate that the network is able to synthesis accurate and reliable lateral views starting from their edges .

Index Terms— View synthesis, silhouette, edge map, stereo video, convolutional neural network.

Samer JAMMAL
Liverpool University, UK
Tammam TILLO
University of Bozen-Bolzano, ITALY
Jimin XIAO
Xi'an Jiaotong Liverpool University, CHINA

[Download]

Cite: Samer JAMMAL, Tammam TILLO, Jimin XIAO, "View Synthesis from Silhouette Using Deep Convolutional Neural Network," Proceedings of 2019 the 9th International Workshop on Computer Science and Engineering WCSE_2019_SPRING, pp. 17-21, Yangon, Myanmar, February 27-March 1, 2019.