[ACM Press the 24th ACM Symposium - Tokyo, Japan (2018.11.28-2018.12.01)] Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology - VRST '18 - Real-time human motion forecasting using a RGB camera

摘要： We propose a real-time human motion forecasting system which visualize the future pose in virtual reality using a RGB camera. Our system consists of three parts: 2D pose estimation from RGB frames using a residual neural network, 2D pose forecasting using a recurrent neural network, and 3D recovery from the predicted 2D pose using a residual linear network. To improve the prediction learning quantity of temporal feature, we propose a special method using lattice optical flow for the joints movement estimation. After fitting the skeleton, a predicted 3d model of target human will be built 0.5s in advance in a 30-fps video.