Somatosensory networks that provide sophisticated sensory feedback and enable the dexterous manipulation of the human grasp remain difficult to replicate in robots, which is attributed to the grand challenge of densely covering the hand with tactile arrays. Here, a multisensory tactile glove is reported that is capable of object recognition with dense coverage of pressure and temperature sensing arrays. The synergistic effect of the multimodal configuration allows the tactile arrays to perceive contact pressure and thermal conductivity of an object involved in grasping motion, thus enhancing the accuracy via the combination of the mechanical features with thermal properties. By leveraging the multiple scanning technology and wireless transmission system, the tactile glove achieves a recognition accuracy of 94.2% in differentiating 20 types of objects with a modified deep learning algorithm. The large-area sensing arrays with high spatiotemporal resolution and multimodal sensing capabilities, which paves the way for the development of robot grasping tools, human–machine interfacing, and advanced prosthetics.
Robot hand requires more sensing data to perform grasping tasks safely. It is difficult to ensure the stability and adaptability of the robot hand during the grasping process with single sensing data, which hinders the advanced capabilities and real time performance of robot hand. The real time control system based on flexible dual-modal sensors (FDMSs) and embedded multilayer perceptron (MLP) model is built to enhance performance of robot hand in precision operations and complex tasks. FDMSs are applied to simultaneously detect the proximity distance and contact pressure of the target objects for providing richer tactile information during the grasping process of the robot hand. Lightweight neural network models are deployed on STM32F4 microprocessor for processing the modal sensing data from FDMSs. Robot hand with FDMSs could grasp different objects successfully with proximity distance and contact pressure signal feedback control strategies, which improves the operational precision and response time of robot hand during the grasping process. Comparative experiments indicate that the embedded AI system with sensory feedback could adjust grasping force to avoid damaging objects. Robot hand with FDMSs and embedded AI demonstrates good adaptability of grasping various objects by adjusting the grasping strategy based on multi-sensing data, ensuring safe and effective task completion.
危险品化学品经营许可证（带存储）