Sketch-based modeling with a differentiable renderer
Authors: Xiang, N., Wang, R., Jiang, T., Wang, L., Li, Y., Yang, X. and Zhang, J.
Journal: Computer Animation and Virtual Worlds
Volume: 31
Issue: 4-5
eISSN: 1546-427X
ISSN: 1546-4261
DOI: 10.1002/cav.1939
Abstract:Sketch-based modeling aims to recover three-dimensional (3D) shape from two-dimensional line drawings. However, due to the sparsity and ambiguity of the sketch, it is extremely challenging for computers to interpret line drawings of physical objects. Most conventional systems are restricted to specific scenarios such as recovering for specific shapes, which are not conducive to generalize. Recent progress of deep learning methods have sparked new ideas for solving computer vision and pattern recognition issues. In this work, we present an end-to-end learning framework to predict 3D shape from line drawings. Our approach is based on a two-steps strategy, it converts the sketch image to its normal image, then recover the 3D shape subsequently. A differentiable renderer is proposed and incorporated into this framework, it allows the integration of the rendering pipeline with neural networks. Experimental results show our method outperforms the state-of-art, which demonstrates that our framework is able to cope with the challenges in single sketch-based 3D shape modeling.
https://eprints.bournemouth.ac.uk/34480/
Source: Scopus
Sketch-based modeling with a differentiable renderer
Authors: Xiang, N., Wang, R., Jiang, T., Wang, L., Li, Y., Yang, X. and Zhang, J.
Journal: Computer Animation and Virtual Worlds
Volume: 31
Issue: 4-5
ISSN: 1546-4261
Abstract:© 2020 The Authors. Computer Animation and Virtual Worlds published by John Wiley & Sons, Ltd. Sketch-based modeling aims to recover three-dimensional (3D) shape from two-dimensional line drawings. However, due to the sparsity and ambiguity of the sketch, it is extremely challenging for computers to interpret line drawings of physical objects. Most conventional systems are restricted to specific scenarios such as recovering for specific shapes, which are not conducive to generalize. Recent progress of deep learning methods have sparked new ideas for solving computer vision and pattern recognition issues. In this work, we present an end-to-end learning framework to predict 3D shape from line drawings. Our approach is based on a two-steps strategy, it converts the sketch image to its normal image, then recover the 3D shape subsequently. A differentiable renderer is proposed and incorporated into this framework, it allows the integration of the rendering pipeline with neural networks. Experimental results show our method outperforms the state-of-art, which demonstrates that our framework is able to cope with the challenges in single sketch-based 3D shape modeling.
https://eprints.bournemouth.ac.uk/34480/
Source: BURO EPrints