Multi-view video reconstruction method based on GAN latent codes

A multi-viewpoint video and viewpoint technology, applied in image communication, electrical components, stereoscopic systems, etc., can solve the problems of reconstructed images, difficult to assist reconstructed image quality, limited reconstruction quality of intermediate viewpoints, etc., and achieve good quality results

Active Publication Date: 2021-05-28
FUZHOU UNIVERSITY
View PDF3 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The 3D scene free-viewpoint VR system is suitable for using camera arrays to obtain multi-viewpoint image sets, and perform scene reconstruction through multi-view rendering. Acquisition, storage and transmission present great challenges
The depth value of the 3D scene is often used as a kind of auxiliary information, such as the video coding method based on multi-view plus depth (MVD), the corresponding depth is coded and transmitted, and the virtual viewpoint image is synthesized by depth image-based rendering (DIBR), but this The depth information in the method is difficult to obtain, and a lot of information in the d

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Examples

Experimental program
Comparison scheme
Effect test

Example Embodiment

[0071]The invention will be further described below with reference to the accompanying drawings and examples.

[0072]It should be noted that the following detailed description is illustrative and is intended to provide a further description of the present application. Unless otherwise indicated, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art of this application.

[0073]It should be noted that the terms used herein are intended to describe specific embodiments, and not intended to limit the exemplary embodiments of the present application. As used herein, unless the context further clearly indicates that the singular form is intended to include a plural form, but it should be understood that when the term "including" and / or "including" is used in this specification, it indicates There is a combination of features, steps, operations, devices, components, and / or their combinations.

[0074]Such asfigure 1 As shown,

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention relates to a multi-view video reconstruction method based on GAN latent codes, which can be used in the fields of virtual view synthesis, multi-view video compression post-processing, virtual reality and the like, and has wide application value. According to the method, the lost mapping relation between adjacent viewpoints is compensated through the generation capability of the GAN, and the balance between the reconstruction quality and the bit rate of the latent code is considered in the training process of the extraction network. At an encoder end, a convolutional network is used to extract latent codes of the GAN as auxiliary information, and a code rate and reconstruction quality joint cost function of the model is constructed. At the decoder end, a generator of the GAN combines adjacent viewpoints with latent codes to reconstruct an intermediate view. The GAN latent code can be used as auxiliary information of a multi-view video so as to realize better rate distortion performance.

Description

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Owner FUZHOU UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap