Semantic Augmented Reality Environment with Material-Aware Physical Interactions
Authors: Chen, L., Francis, K. and Tang, W.
Journal: Adjunct Proceedings of the 2017 IEEE International Symposium on Mixed and Augmented Reality, ISMAR-Adjunct 2017
Pages: 135-136
ISBN: 9780769563275
DOI: 10.1109/ISMAR-Adjunct.2017.49
Abstract:In Augmented Reality (AR) environment, realistic interactions between the virtual and real objects play a crucial role in user experience. Much of recent advances in AR has been largely focused on developing geometry-aware environment, but little has been done in dealing with interactions at the semantic level. High-level scene understanding and semantic descriptions in AR would allow effective design of complex applications and enhanced user experience. In this paper, we present a novel approach and a prototype system that enables the deeper understanding of semantic properties of the real world environment, so that realistic physical interactions between the real and the virtual objects can be generated. A material-aware AR environment has been created based on the deep material learning using a fully convolutional network (FCN). The state-of-the-art dense Simultaneous Localisation and Mapping (SLAM) has been used for the semantic mapping. Together with efficient accelerated 3D ray casting, natural and realistic physical interactions are generated for interactive AR games. Our approach has significant impact on the future development of advanced AR systems and applications.
https://eprints.bournemouth.ac.uk/30318/
Source: Scopus
Semantic Augmented Reality Environment with Material-Aware Physical Interactions
Authors: Chen, L., Francis, K. and Tang, W.
Conference: ISMAR 2017: IEEE International Symposium on Mixed and Augmented Reality
Pages: 135-136
ISBN: 9780769563275
Abstract:© 2017 IEEE. In Augmented Reality (AR) environment, realistic interactions between the virtual and real objects play a crucial role in user experience. Much of recent advances in AR has been largely focused on developing geometry-aware environment, but little has been done in dealing with interactions at the semantic level. High-level scene understanding and semantic descriptions in AR would allow effective design of complex applications and enhanced user experience. In this paper, we present a novel approach and a prototype system that enables the deeper understanding of semantic properties of the real world environment, so that realistic physical interactions between the real and the virtual objects can be generated. A material-aware AR environment has been created based on the deep material learning using a fully convolutional network (FCN). The state-of-the-art dense Simultaneous Localisation and Mapping (SLAM) has been used for the semantic mapping. Together with efficient accelerated 3D ray casting, natural and realistic physical interactions are generated for interactive AR games. Our approach has significant impact on the future development of advanced AR systems and applications.
https://eprints.bournemouth.ac.uk/30318/
https://ismar2017.sciencesconf.org/
Source: BURO EPrints