Multimode-fused reservoir computing based on CH3NH3PbI3 nanowire optoelectronic memristor for spatiotemporal information processing

Multiple sensing and processing capabilities enable an organism to obtain comprehensive information and make real-time decisions in unstructured environments. Inspired by the biological counterpart, developing neuromorphic hardware with multimodal fusion is supposed to realize high-precision processing for complex information. In this work, we developed a visual-auditory-fused reservoir computing system by combining a CH 3 NH 3 PbI 3 nanowire/pentacene optoelectronic memristor and a sound sensor. Under the stimulation of optical/electrical signals, the CH 3 NH 3 PbI 3 nanowire-based device exhibited volatility under optical or electrical stimulation, providing an essential foundation for visual-auditory-fused reservoir computing. The motion detection function was demonstrated using synchronous visual and auditory sensory information. Compared with single sensory input, the detection accuracy increased from 51.2% to 95.5% by combining the visual and auditory modalities. The results presented in this work provide a promising strategy to develop efficient multimodal neuromorphic hardware for human-robot interactions.