Topic 1 : Shared Sensory Perception Between the Human Worker and the Robot
I am testing various haptic interfaces for promoting the shared sensory perception between the human agents and the robot. The findings are expected to help construction workers control the robot in a more natural manner. Meanwhile, the perception of how construction workers manipulating and interacting with the environment can be transferred to robots and thus improves their automation algorithms.
Payload Sway Simulator for Tower Crane Teleoperation
Topic 2: VR-Based Robotic Simulation
My research involves reconstructing the contextual environment that both humans and robots are involved in. To realize the digital twins of construction robots in the VR environment, I bridged the Robot Operating System (ROS) and the game engine (Unity). The inverse kinematic (IK) algorithms include interactive manipulation, animation control, and collision avoidance methods are applied via the Unity and the ROS, and are transferred to the real robots (e.g., robotic arms and quadrupedal robots) for further validation if required.
Collaborative Robot Arm Simulation : Franka Emika Panda
Zhu, Q., Zhou, T., Xia, P., & Du, J. (2022). Robot Planning for Active Collision Avoidance in Modular Construction: Pipe Skids Example. Journal of Construction Engineering and Management, 148(10), 04022114.
Two-Armed Robot Simulation : Baxter
Zhou, T., Zhu, Q., Du, J., (2020). “Intuitive Robot Teleoperation for Civil Engineering Operations with Virtual Reality and Deep Learning Scene Reconstruction”. ELSEVIER Advanced Engineering Informatics. (46) 101170.
Upasani, S., Zhu, Q., Herron, C., Du, J., Leonessa, A., & Srinivasan, D., “Sensitivity of Eye-Tracking Measures to Variations in Mental Workload while Learning to Operate a Physically-Coupled Robot. “ Proceedings of the Human Factors and Ergonomics Society Annual Meeting., Baltimore, MD.2021. (Accepted)
Biomorphic HyperRedundant Robot Simulation : Snake Robot
Zhu, Q., Xiang Xia, P., Zhou, T., & Du, J. Snake Robot In-Pipe Teleoperation System with Upper-Body Haptic Simulator. In Computing in Civil Engineering 2021 (pp. 1110-1118).
Topic 3: Human Factors Engineering for Augmented Cognitive and Physical Functions
I developed a real-time human behavior data analytics system that integrates neuroimaging, eye tracking, and motion analysis to achieve this goal. This system is embedded in a VR simulator and supports online visualization and computing (www.cogdna.com).
Cognition‑Driven Display for Navigation Activities (Cog‑DNA): Personalized AR Spatial Information System
Du, J.,Zhu,Q., Shi, Y., Wang, Q., Lin, Y., and Zhao D. (2020). “Cognition‑Digital Twins (Cog‑DT) for Personalized InformationSystems of Smart Cities – A Proof of Concept”.ASCE Journal of Management in Engineering. 36 (2), 04019052.
Zhu,Q., Shi, Y., Du, J. (2021). “ Wayfinding Information Cognitive Load Classification based on Functional Near‑InfraredSpectroscopy (fNIRS).”ASCE Journal of Computing in Civil Engineering.35(5), 04021016.
Sensory Manipulation as a Countermeasure to Robot Teleoperation Delays
This research proposes an innovative sensory manipulation approach to help reduce risks related to teleoperation delays. The neural, perception, and performance evidence contributes to the formulation of effective space teleoperation designs.