Human Reconstruction Using mmWave Technology by Kong, Hao; Yu, Jiadi; Shen, Xuemin;

Long description:

"

This book introduces human reconstruction technologies in smart homes leveraging mmWave signals. It begins by presenting the overview of human reconstruction and the development of mmWave sensing technology. The authors introduce a mmWave sensing-based human posture reconstruction approach, which exploits mmWave signals to sense and track multiple users’ postures as they move, walk or sit. It also presents a facial expression reconstruction system that reconstructs 3D human faces and continuously exhibits facial expressions using mmWave. The authors describe a hand gesture reconstruction system that utilizes mmWave signals to generate 3D hand skeletons and reconstruct hand meshes. A thorough investigation of state-of-art research work is further presented covering human reconstruction, mmWave sensing and mmWave-based human reconstruction. Finally, conclusions and the direction of future research are highlighted for this book.

Human and the cyberworld are moving towards high synchronization as more IoT devices are integrated in smart homes. By reconstructing human postures, facial expressions, and hand gestures in the cyberworld, a virtual human avatar can map the state of human into the physical world and deeply integrate reality and virtuality. The process strongly energizes emerging applications such as virtual reality, augmented reality, meta-universe, immersive games, etc.

This book targets graduate-level students majoring in the computer science and engineering, and electrical engineering. Professionals working in wireless signal-based human sensing will also find this book a valuable resource.

Table of Contents:

Chapter 1 Introduction.- 1.1 An Overview of Human Reconstruction.- 1.2 Development of mmWave Sensing.- 1.3 Organization of Monograph.- Chapter 2 mmWave-based Human Posture Reconstruction.- 2.1 Background and Motivation.- 2.2 System Overview.- 2.3 Multi-User Detection and Separation.- 2.3.1 User Detection on mmWave Signals.- 2.3.2 User Separation on mmWave Signals.- 2.4 Single-User Posture Reconstruction.- 2.4.1 Posture Feature Representation.- 2.4.2 3D Posture Reconstruction.- 2.5 Multi-User 3D Posture Tracking.- 2.5.1 Posture Mapping with Point Cloud.- 2.5.2 3D Posture Tracking.- 2.6 Evaluation.- 2.6.1 Evaluation Setup.- 2.6.2 Overall Performance.- 2.6.3 Quantitative Result.- 2.6.4 Localization Performance.- 2.6.5 Performance in Different Scenarios.- 2.6.6 Comparison with SOTA Systems.- 2.6.7 Impact of Factors.- 2.7 Summary and Further Reading.- Reference.- Chapter 3 mmWave-based Facial Expression Reconstruction.- 3.1 Background and Motivation.- 3.2 System Overview.- 3.3 Facial Feature Extraction.- 3.3.1 Feature Representation Extraction.- 3.3.2 Geometric Feature Extraction.- 3.4 3D Facial Expression Reconstruction.- 3.4.1 Facial Shape Reconstruction.- 3.4.2 Facial Expression Reconstruction.- 3.4.3 3D Avatar Generation.- 3.5 Evaluation.- 3.5.1 Evaluation Setup.- 3.5.2 Overall Performance.- 3.5.3 Comparison with SOTA Systems.- 3.5.4 Facial Expression Recognition.- 3.5.5 Impact of Mask.- 3.5.6 Impact of Distance and Orientation.- 3.5.7 Impact of Background Environment.- 3.5.8 In-the-Wild Evaluation.- 3.6 Summary and Further Reading.- Reference.- Chapter 4 mmWave-based Hand Gesture Reconstruction.- 4.1 Background and Motivation.- 4.2 System Overview.- 4.3 Hand Joint Regression.- 4.3.1 Hand Detection on mmWave Signals.- 4.3.2 Hand Feature Extraction.- 4.3.3 3D Hand Joint Regression.- 4.4 Mesh Reconstruction.- 4.4.1 3D Hand Model.- 4.4.2 Hand Mesh Reconstruction.- 4.5 Evaluation.- 4.5.1 Evaluation Setup.- 4.5.2 Overall Performance.- 4.4.3 Comparison with SOTA Systems.- 4.4.4 Impact of Distance and Angle.- 4.4.5 Impact of Human Body.- 4.4.6 Impact of Gloves and Handheld Object.- 4.4.7 Impact of Background Environment.- 4.6 Summary and Further Reading.- Reference.- Chapter 5 State-of-Art Research.- 5.1 Human .-Pose Reconstruction.- 5.2 mmWave Sensing in Smart Homes.- 5.3 mmWave-based Human Sensing Applications.- 5.4 Summary of Existing Research.- Reference.- Chapter 6 Conclusion and Future Work.- 6.1 Conclusion of the Book.- 6.2 Future Work.- Reference.