Broad spectrum detection and information processing have important application requirements in many fields such as earth remote sensing, environmental monitoring, and unmanned driving. The current wide-spectrum information detection and information processing are completed by different types of image sensors and digital image processors, respectively, which makes traditional wide-spectrum machine vision systems face the challenges of large power consumption and high time delay. The sensor-computing integration technology is considered to be an effective way to solve the above challenges. However, how to design quantum materials that meet the above requirements and realize a new sensor-computing integrated device with wide-spectrum and multi-information synchronous detection and processing functions is a widely concerned topic. issue.
Two-dimensional layered materials are one of the fundamental electronic materials that have high hopes in the post-Moore era. The team of Professor Feng Miao from the School of Physics, Nanjing University has been focusing on exploring the unique physical properties and regulation mechanisms of two-dimensional materials, as well as the design and implementation of new-principle devices, pioneering and developing the direction of “Atomic Lego” electronics and optoelectronics. Recently, the team of Professor Feng Miao of Nanjing University and the team of Professor Tianyou Zhai of Huazhong University of Science and Technology have realized the wide spectral convolution processing and recognition in the sensor by using the characteristics of the reversible control of the electric field of the band-matching type of bipolar two-dimensional van der Waals heterojunction. , which provides a new idea for the development of sensor-computing integrated image sensors for various complex pattern recognition tasks.
In this work, the collaborative team systematically studied the photoresponsivity of the PdSe2/MoTe2 heterojunction under different optical powers and gate voltages, and found that the device has electric field-tunable positive and negative photoresponse in the wavelength range from ultraviolet to near-infrared. Moreover, the optical responsivity, optical power, and gate voltage maintain an excellent linear dependence within a certain range, which enables the linear dot product operation between vectors and matrices to be accurately mapped to the physical process (as shown in Figure 1). shown). Therefore, the use of the excellent linear response characteristics of the heterojunction can realize the accurate conversion and reconfigurable processing of image information, and the accurate weight update in the image recognition training process.
Figure 1: Schematic diagram of the integrated information processing mechanism of wide-spectrum sensing and computing and the PdSe2/MoTe2 heterojunction device. (a-b) Schematic diagram of wide spectral convolution based on sensor-computing integration; (c) PdSe2/MoTe2 bipolar van der Waals heterojunction device; (d) Under the control of gate voltage, the device has three wavelengths of ultraviolet-visible-near-infrared. exhibit tunable positive and negative photovoltaic responses.
By configuring heterojunction devices with different convolution kernels, the collaborative team demonstrated the processing results of performing different types of convolution filtering on remote sensing images containing broad spectral information (e.g., remote sensing image sharpening, edge enhancement) (Fig. 2a-d). Further, the cooperative team constructed a wide-spectrum convolutional neural network using the sensor-computing integrated heterojunction sensor, which realized the accurate identification of wide-spectrum images. For the displayed wide-spectrum letter image dataset (as shown in Figure 2e), each pixel corresponds to information of different wavelength bands and different light intensities. The integrated convolutional network can detect and calculate the image of this mixed light intensity and band information synchronously, and associate the image feature information of different bands. Combined with the back-propagation training method, dynamic training is continuously performed to update the grid voltage. After only 3 training cycles, the recognition accuracy of the network can be close to 100%. In contrast, single-spectral convolution devices can only detect and process information in specific bands in the image. In order to achieve broad spectrum recognition, it is necessary to use multiple specific convolution kernels to detect information in different bands, and then integrate them. This method of splitting the image separation band detection processing not only weakens the relationship between the intrinsic features in the image, but also increases the amount of processed data, but also makes the training speed and recognition effect significantly weaker than the wide-spectrum convolution processing (as shown in Figure 2f). shown). This work provides a new idea for solving the problem of sensor-computing separation of wide-spectrum information in traditional machine vision systems, and for developing a sensor-computing integrated full-hardware image recognition system suitable for complex pattern recognition tasks.
Figure 2: Broad spectral information processing and identification and classification of mixed spectral images based on PdSe2/MoTe2 heterojunction devices. (a-d) Reconfigurable processing of wide-spectral remote sensing images; (e) Alphabet image dataset, each pixel contains different band and intensity information; (f) The recognition results of wide-spectral convolution are compared with those of single-spectral convolution.
The related research results of this work are published on April 25, 2022 in Electronics International under the title “Broadband convolution processing using band-alignment-tunable heterostructures” Authoritative journal “Nature Electronics” (“Nature Electronics”).