Embedded Edge Intelligence and Flexible Dual-Modal Sensor Systems for Robotic Electronic Skin

Robot hand requires more sensing data to perform grasping tasks safely. It is difficult to ensure the stability and adaptability of the robot hand during the grasping process with single sensing data, which hinders the advanced capabilities and real time performance of robot hand. The real time control system based on flexible dual-modal sensors (FDMSs) and embedded multilayer perceptron (MLP) model is built to enhance performance of robot hand in precision operations and complex tasks. FDMSs are applied to simultaneously detect the proximity distance and contact pressure of the target objects for providing richer tactile information during the grasping process of the robot hand. Lightweight neural network models are deployed on STM32F4 microprocessor for processing the modal sensing data from FDMSs. Robot hand with FDMSs could grasp different objects successfully with proximity distance and contact pressure signal feedback control strategies, which improves the operational precision and response time of robot hand during the grasping process. Comparative experiments indicate that the embedded AI system with sensory feedback could adjust grasping force to avoid damaging objects. Robot hand with FDMSs and embedded AI demonstrates good adaptability of grasping various objects by adjusting the grasping strategy based on multi-sensing data, ensuring safe and effective task completion.