INTRODUCTION
As the name suggests, humanoid robots are expected to emulate both the structures and behaviors of humans1. They can act as powerful assistants for humans in a variety of situations and have broad application prospects in the fields of healthcare, education, service, etc.2. Compared with conventional industrial robots working in a fixed environment and performing repeated tasks, humanoid robots aim to face changeable environments and fulfil various tasks3,4. In this process, capturing multimodal characteristic information of the surrounding environment and converting the collected data into interpreted information are key perceptual functions5. As the amount of data grows increasingly under such requirements, great challenges arise and remain to be solved for data processing performance of electronic systems.
At present, sensing data processing is mainly based on the conventional von Neumann architecture in which the storage unit and the processing unit are separated, leading to frequent data transmission between the two units6,7. Therefore, the computation performance is severely limited by the transmission and storage speed since the computing speed has become far faster than the storage speed with the upgrade of technology8-10. Additionally, the continual data transmission also leads to a huge demand for energy consumption. Furthermore, the von Neumann architecture is competent in executing structured and prespecified tasks, however, it could hardly handle indeterministic, everchanging, and real-time problems8,11. For these reasons, it is of great difficulty for the conventional computation architecture to realize perception functions on the background of the explosive growth of sensing data and complex working scenarios.
To overcome these dilemmas, a new architecture for data processing is highly demanded. The biological nervous system suggests a unique computing paradigm to process data through an interconnected network consisting of billions of neurons and synapses. Neurons are special cells in charge of information processing12. Synapses allow communications between neurons by electrical pulses, and perform storage function in the form of connection weight13,14. This structure allows simultaneous implementation of storage and computation by the nervous system, which effectively avoids the efficiency limit caused by data transmission in the von Neumann architecture9,13,15. Meanwhile, a synapse only consumes very little energy (10 fJ per synaptic event), and the whole human brain consumes only about 20 W of energy, which is much lower than that consumed by a digital computer16,17. Besides, the biological nervous system can naturally encode temporal information, which is of vital importance to the implementation of self-adaption and learning. This parallel distributed information processing paradigm enables the brain to work in a robust, energy-efficient, and self-adaptable way, underpinning the implementation of thinking, learning, emotion, and other complex behaviors of creatures14,18.
In recent years, more and more attention has been paid to the hardware implementation of neuromorphic systems. To achieve hardware-based neuromorphic systems, the implementation of synapses is considered a prerequisite. Historically, complementary metal-oxide-semiconductor (CMOS) technology was used to mimic synaptic behaviors, but the complex circuit design hindered the reduction of size and led to high energy consumption19-22. To overcome this technical issue, two-terminal memristors and three-terminal synaptic transistors emerged. In general, the conductance of these synaptic devices can be tuned by external stimuli, analog to the changing connection weights of synapses in the biological nervous system23,24. These synaptic devices have exhibited low energy consumption and robust performance, and been widely investigated in applications such as data storage25,26, brain-like computing27-29, and etc. Recently, biomimetic sensory systems based on the incorporation of advanced sensing technologies and synaptic devices have also become hot issues and attracted more and more attention form scholars. By mimicking the neurobiological architecture of biological sensory organs, they provide a neuromorphic way to perceive, transmit, and process sensory information, which helps promoting a notable evolution in next-generation humanoid robotics, prosthetics, wearable electronics, and etc. Fig. 1 shows the frame work of the advantages of synaptic devices and their applications in biomimetic sensory neural systems.
Herein, this review presented a summary of the development of synaptic devices and the recent advances in biomimetic sensory neural systems. It started with a brief introduction of the biological synapse and synaptic plasticity. After that, the development of synaptic devices including memristors and synaptic transistors was reviewed. Afterwards, advanced applications of biomimetic sensory neural systems based on synaptic devices were demonstrated, which mainly focused on tactile and visual sensory systems. Finally, several remaining challenges and future trends of the biomimetic sensory neural system were also discussed.
BIOLOGICAL SYNAPSES AND SYNAPTIC PLASTICITY
The human brain operates in a highly parallel and efficient manner through a densely interconnected network of neurons and synapses18. There are approximately 1012 neurons and 1015 synapses in human brain, and each neuron connects with 103-104 other neurons through synapses30. A neuron consists of a cell body named soma, dendrites, and axons (Fig. 2). Dendrites are short and branching protrusions extending from the cell body of a neuron (soma), responsible for receiving signals from other neurons. The axon is a long and slender fiber, transmitting signals generated by the cell body to other neurons or effectors (such as glands or muscle cells). Between two neurons, a specialized area named synapse allows one neuron (called the presynaptic neuron) to pass an electrical or chemical signal to the other (called the postsynaptic neuron)31,32.
Fig. 2. Biological neuron (left) and synapse (right). Reprinted with permission from ref.33. © 2014 Nature Publishing Group. |
In contrast to invertebrates, most vertebrates are more adaptable to the environment. One major difference lies in the structure of synapses. Electrical synapses are commonly found in invertebrates, where electrical signals are rapidly transmitted through ion migration in gap junctions and low-impedance bridges33. In vertebrates, however, synapses deliver nerve impulses in a different way by releasing and receiving chemical substances called the neurotransmitters (Fig. 2). Action potential in presynaptic membrane causes the Ca2+ channel to open, then the ionic influx stimulates the release of neurotransmitters from synaptic vesicles. Neurotransmitters diffuse across the 20-40 nm synapse cleft, and bind to specific receptors on the postsynaptic membrane. These receptors control the ion channels on the postsynaptic membrane to open (or close), thus changing the membrane potential of the postsynaptic neuron33-35.
Although chemical synapses are of slower signal transmission rate, the additional modulation through neurotransmitters and their receptors significantly enrich the complexity and adaptability of the nervous system. In this process, the efficiency of information transmission between presynaptic and postsynaptic neurons is quantified by synaptic weight. The synaptic weight change can be either positive or negative (termed potentiation and depression36), depending on the types of the receptors and the recent activity history of both sides of the synapse37,38. The synaptic weight variation could last from seconds to months and is thus described as synaptic plasticity and considered as a key mechanism for information processing, learning, and memory39.
Generally, Researchers characterize synaptic activities by their retention time and synaptic weight modulation approaches. Classified by the retention time, synaptic plasticity can be divided into short-term plasticity (STP) and long-term plasticity (LTP). STP, including short-term potentiation and short-term depression. Generally, STP lasts over milliseconds to minutes before decaying to its initial state40.STP is widely believed to play an important role in information processing, and contributes to abundant essential computational functions in neural circuits37,41. Paired-pulse facilitation (PPF) and paired-pulse depression (PPD) are two representative effects of STP. PPF describes the phenomenon that when two action potentials are generated by the presynaptic neuron in rapid succession, the amplitude of the second excitatory postsynaptic current (EPSC) tends to be larger than of the first one. On the contrary, PPD refers to the phenomenon that the amplitude of inhibitory postsynaptic current (IPSC) shows a decreasing trend under paired pulses.
Applying repeated stimulation leads to changes in the structure of synaptic connection, and hereby STP can be transformed into LTP42. LTP generally lasts for hours or longer, and it is in charge of learning and memory of human brain43-45. It could also be divided into long-term potentiation and depression. In 1949, Donald Hebb proposed a learning rule that the synaptic weight increases if a synapse continues to cause its postsynaptic neuron to produce action potentials46. As an extension of it, spike-timing-dependent plasticity (STDP) changes synaptic weight according to the timing between the presynaptic and the postsynaptic action potential47,48. In a typical STDP model, the presynaptic spike occurs a few milliseconds before the postsynaptic spike, which increases the synaptic weight. In contrast, the occurrence of a postsynaptic spike proceeds a few milliseconds before a presynaptic spike, this weakens the synaptic weight. The shorter the time elapsed between the two spikes, the larger the change in synaptic weight is. Spike-rate-dependent plasticity (SRDP), another basic learning rule for LTP, modifies synaptic weight through the presynaptic firing rate. A high frequency (20-100 Hz) train of presynaptic spikes results in potentiation, whereas a low frequency (1-5 Hz) one leads to depression49.
SYNAPTIC DEVICES
As the first step to build an artificial neuromorphic system, synaptic devices mimic biological synapses in terms of multi-state memory, in-memory computing, and non-volatility properties50. In 2008, Hewlett-Packard Company presented titanium dioxide thin film with adjustable resistance and time-dependent memory characteristics. It is believed to be the first synaptic device realized through a single electronic component51. Since then, growing efforts have been made to fabricate synaptic devices with diverse structures and working mechanisms. Nowadays, artificial synapses are mainly classified into two types according to their device structures: two-terminal memristors and three-terminal synaptic transistors.
Two-terminal memristors
Two-terminal devices resemble biological synapses in their structures and thus have been widely investigated since early researches52. Leon Chua first proposed the concept of a memristor, a two-terminal non-volatile memory electrical component based on resistance switching, regardless of the device material and physical operation mechanisms53,54. In most cases, a memristor consists of a top electrode, a bottom electrode, and an insulating layer sandwiched between them55. The top electrode takes the role of a presynaptic neuron, and the applied voltage pulse is regarded as an action potential. Meanwhile, the bottom electrode is regarded as a postsynaptic neuron. Under applied voltage pulse, the insulating layer switches between a low-resistance state (LRS) and a high-resistance state (HRS), mimicking the update of synaptic weight56,57. The process of resistance switching from HRS to LRS is named SET, while the reverse process RESET. Mechanisms of the resistance switching behavior mainly include electrochemical metallization mechanism (ECM), valence change mechanism (VCM), proton-migration mechanism, phase-change mechanism (PCM), and ferroelectric tunnel junction (FTJ). Table 1 summarizes the recent advances and benchmarks of two-terminal memristors.
Table 1. Recent advances of memristor-based synaptic devices. |
| Mechanism | Structure | SET/RESET voltage | ON/OFF ratio | Retention time | Synaptic functions | Application | Ref |
|---|---|---|---|---|---|---|---|
| ECM | Ag/ZrO2/WS2/Pt | 0.16 V/−0.06 V | ∼106 | 4 × 104 s | PPF, STDP | Handwriting recognition | 72 |
| ECM | Cu/MoS2/Au | 0.25 V/−0.15 V | ∼10 | ∼1.8 × 103 s | STDP | - | 63 |
| ECM | Ag/WSe2/Ag | 4 V/−3.5 V | 1.6 × 103 | 3.6 × 104 s | STP to LTP transition | - | 75 |
| VCM | W/MgO/ZnO/Mo | 1.32 V/−1.32 V | ∼7.6 | 104 s | LTP, STDP | Security data storage | 45 |
| VCM | Pt/Ta2O5/HfO2/TiN | ∼−1.0 V/∼1.1 V | ∼10.7 | 8000 s | PPD, STDP | Pattern recognition | 158 |
| VCM | ITO/BN/TaN | 0.81 V/−0.79 V | ∼98.5 | > 105 s | PPF, LTP, STDP | Non-volatile logic circuits | 235 |
| Proton-migration mechanism | Au/C3N/PVPy/ITO | 5 V/−5 V | - | - | PPF, PPD, SRDP | - | 91 |
| PCM | Al/TiN/OTSTa/W | 1.15-3.25 V/ - | ∼100 | - | Linear resistance change | - | 236 |
| PCM | ITO/Lignin/Au | 0.7 V/−0.7 V | - | 50 s | SRDP, STP to LTP transition | - | 237 |
| FTJ | Ag/BTO/NSTOb | 3 V/−3.4 V | 200 | 104 s | STDP | Supervised learning | 104 |
| FTJ | Au/HZO/p+-Si | ∼2 V/∼ −2 V | 15 | 7200 s | PPF, LTP, STDP | - | 105 |
aO-Ti0.4Sb2Te3. bNb:SrTiO3. |
Electrochemical metallization mechanism
A typical ECM-based memristor consists of an anode made from electrochemically active metals (e.g., Ag58-60 and Cu61-63), an insulating layer, and a cathode made from electrochemically inert materials (e.g., Au64-66 and ITO67-69). It results in resistance switching through the formation of metallic filaments in the switching layer. Under a positive bias voltage applied to the anode, part of the anode is ionized to cations, while the cations are driven into the insulating layer. Then they are reduced to atoms by electrons, and the metallic conductive filaments are formed, realizing switching from HRS to LRS70. While under negative bias, the metallic filaments rupture, and the device switches back to HRS.
Inorganic materials (oxides71-73, two-dimensional materials63,66,74,75, perovskites76-78, etc.) and organic materials64,68,69 have been widely involved in ECM-based devices as the switching layer. Currently, the key challenge in developing ECM-based memristors is to precisely control the filament growth so as to enable stable and continuous switching21,79. Yan et al. proposed a memristor with Ag/ZrO2/WS2/Pt structure (Fig. 3a)72. Owing to different ion-mobility characteristics, an hourglass-shaped Ag filament was formed when a positive voltage was firstly applied, and the weakest position was the bilayer interface. Afterwards the application of RESET/SET voltages would cause the rupture/formation of the filament at the bilayer interface (Fig. 3b). Therefore, the Ag filament rupture/formation was shortened and limited, greatly reducing the randomness. The device exhibited high speed switching (∼10 ns) and endurance (> 109 cycles), which was superior to those with single-layer ZrO2 or WS2. Jang et al. found that by simply reducing the lateral size of the Cu filament, the RESET behavior of their poly(1,3,5-trivinyl-1,3,5-trimethyl cyclotrisiloxane) (pV3D3)-based memristor transmitted from abrupt to gradual mode (Fig. 3c, d)80. Potentiation and depression characteristics, PPF, and STDP were successfully implemented with the pV3D3-based memristor with an atomically thin filament (Fig, 3e). Xu et al. demonstrated a flexible multilevel memristor using the biocompatible pectin extracted from the natural orange peel as the switching layer67. With pectin enabling fast migration of Ag ions, the Ag/pectin/ITO device exhibited excellent characteristics such as low operating voltage, fast switching speed, and long retention time (> 104 s). Besides, the device can be dissolved in deionized water rapidly, demonstrating the transient characteristics. Such transient electronic devices are potentially suitable for several applications including biocompatible electronics, green electronics, and secure data storage81.
Fig. 3. Memristors based on ECM, VCM, and proton-migration mechanism. a, Schematic of the Ag/ZrO2/WS2/Pt memristor. b, Schematic of SET and RESET process of the Ag/ZrO2/WS2/Pt memristor.72 Reprinted with permission from ref.72. © 2019 American Chemical Society. c, Schematic of a flexible pV3D3 memristor-based electronic synapse array. d, Analog RESET behavior of the flexible pV3D3 memristor with a thin filament. e, Potentiation-depression characteristics of the flexible pV3D3 memristor with a thin filament.80 Reprinted with permission from ref.80. © 2019 American Chemical Society. f, DC characteristic of the Pd/WOx/W device. Inset: Scanning electron microscope image of a typical device. Scale bar: 20 µm. g, Schematic of the internal VO dynamics showing: (1) electric field-driven VO drift and (2) spontaneous diffusion. h, STDP implemented by the device. Reprinted with permission from ref.86. © 2019 Wiley-VCH. i, Schematic of the C3N/PVPy-based memristor. j, Molecular structure of C3N sheet (top) and cross-sectional SEM image of the memristor (bottom). k, PPF index as a function of pulse interval. Reprinted with permission from ref.91. © 2019 Elsevier B.V. |
Valence change mechanism
Resistance switching of VCM also relies on electrochemical reactions and migration of ions. Different from ECM, it is mainly driven by the redistribution of vacancies in the switching layer under the applied voltage. Therefore, electrochemically active electrodes are not required in VCM-based memristors.
Oxides are often used as the switching layer of VCM-memristors82-85. Understanding the different time scales of ionic dynamics helps implementing synaptic functions in oxides-based memristors. Du et al. demonstrated that the migration of oxygen vacancies was driven by two dynamics: the electric-field-driven drift process and the diffusion process (Fig. 3f, g)86. Taking advantage of the different time scales of dynamics, diverse synaptic functions at different time scales such as PPF and STDP were implemented by their Pd/WOx/W memristor (Fig. 3h). Wang et al. designed a memristor with a van der Waals heterostructure composed of graphene/MoS2-xOx/graphene87. The device was able to work under a high operating temperature of up to 340 °C, and exhibited good endurance against over 1000 bending cycles on a polyimide substrate. Besides oxides, sulfides are also adopted as the switching layer of VCM-based memristors. Yan et al. demonstrated a high-performance and low-power consumption memristor based on two-dimensional (2D) WS2 with 2H phase88. The SET/RESET energy consumption of the device reached the level of femtojoules. Synaptic behaviors including PPF and STDP were simulated by the device.
Proton-migration mechanism
In proton memristors, proton-conducting materials are used as the switching layer. With the effect of applied voltage, protons will move in the direction of the electric field, which resembles the proton conduction phenomenon in biological systems89,90. By tuning the proton flux, continuous resistance states can be obtained.
Nafion is a proton conductor which is adopted widely as proton exchange membranes for fuel cell applications. Zhang et al. presented a memristor with a Au/Nafion/ITO structure21. The memristor exhibited several synaptic behaviors including PPF, PPD, and STDP. Zhou et al. demonstrated a memristor with tunable synaptic behaviors based on 2D C3N/polyvinylpyrrolidone (PVPy) (Fig. 3i, j)91. The hydrogen bonding network between C3N and PVPy as well as the large amount of ordered nitrogen atoms in C3N resulted in the high proton conductivity. Essential synaptic behaviors including EPSC, PPF, PPD, and STDP were demonstrated (Fig. 3k).
Phase-change mechanism
PCM-based devices utilize the reversible phase change between the amorphous (HRS) and crystalline (LRS) states to achieve resistance switching. The applied voltage pulse generates heat via Joule heating and induces phase transformation of the phase-change layer92,93. In particular, the resistance can be modulated by applying voltage pulses with designed amplitude and duration, resulting in multiple states between HRS and LRS94. This wide programmable resistance state range allows PCM-based devices to act as electronic synapses. Besides, PCM-based devices also show the advantages of fast operating speed and good scalability95.
Kuzum et al. pioneered a common approach to fabricate PCM-based synapse by applying multiple pulse schemes in 201196. In their strategy, the presynaptic is composed of depression pulses with increasing amplitudes and potentiation pulses with decreasing amplitude. The postsynaptic pulse was a single pulse with a negative magnitude. They successfully implemented the STDP behavior by modulating time spacing between the well-designed presynaptic and postsynaptic spikes.
In the traditional lance-type PCM-based devices, the amorphization process is usually abrupt, which resulted in asymmetric weight-update in programming97. In order to slow down the amorphization process, Barbera et al. optimized the device structure (Fig. 4a)98. The narrow heater bottom electrode exhibited greater resistance, contributing to a better control of the current density. The gradual depression was achieved through the following two steps: initializing the device to a partial amorphization state and gradually increasing the resistance by a train of identical fast pulses. With their scheme, both long term potentiation and depression were successfully mimicked with identical pulses (Fig. 4b).
Fig. 4. Memristors based on PCM and FTJ. a, Schematic of the PCM-based device with narrow heater bottom electrode. b, Long term potentiation and depression implemented with identical pulses.98 Reprinted with permission from ref.98. © 2018 Wiley-VCH. c, Long-term potentiation and depression behavior of the HZO device. d, STDP implemented by the HZO device. e, PPF ratios of positive and negative pulse.106 Reprinted with permission from ref.106. © 2018 Wiley-VCH. f, STDP implemented by the organic FTJ-based memristors. g, LTP and STP behavior with 20 ns voltage pulses.108 Reprinted with permission from ref.108. © 2019 Wiley-VCH. |
Ferroelectric tunnel junction
Ferroelectric materials are insolating materials with a spontaneous electric polarization which can be reversed under an external electric field99. For FTJ-based memristors, in which an ultrathin ferroelectric film is sandwiched between two electrodes, the resistance sensitively depends on the polarization orientation of the ferroelectric layer100,101. Applying voltage pulses changes the ferroelectric polarization, thereby inducing variations of the resistance state102. FTJ-based memristors are endowed with the advantages of clearly physical mechanisms, fast operation speed, and low energy consumption, making them promising candidates as artificial synapses103.
Ferroelectric oxides are widely applied in FTJ-based memristors104,105. Yoong et al. reported the epitaxial growth of single orthorhombic phase Hf0.5Zr0.5O2 (HZO) thin film using the pulsed laser deposition technique106. The HZO-based memristor showed synaptic behaviors including long-term potentiation/depression, PPF, and STDP (Fig. 4c-e). Sun et al. successfully constructed epitaxial BiFeO3 (BFO) ferroelectric film on flexible mica substrates, and the polarization remained stable in 104 bending cycles under 5mm radius107. STDP was implemented by the flexible BFO-based memristor, demonstrating its potential as artificial synapses.
Compared with ferroelectric oxides whose growth was highly restricted, ferroelectric copolymers such as poly(vinylidene fluoride-trifluoroethylene) (PVDF-TrFE) can be easily obtained from solutions. Majumdar et al. reported a FTJ-based memristor consisting of Nb-doped SrTiO3 bottom electrode, a PVDF-TrFE switching layer, and a patterned Au top electrode108. The PVDF-TrFE film was prepared by a simple spin-coating process. The memristor exhibited essential synaptic behaviors including STP, LTP, and STDP (Fig. 4f). Notably, it could operate at an ultrafast speed, and proper emulation of synaptic behaviors were achieved on nanosecond timescales (Fig. 4g).
Three-terminal transistors
A transistor is a three-terminal device composed of a semiconducting channel layer, a gate dielectric layer, and a source/drain/gate electrode. To mimic a synapse, the gate electrode and the channel layer are treated as presynaptic terminal and postsynaptic terminal, while the channel conductance modified by the gate voltage acts as synaptic weight109. Compared with memristors, the additional gate terminal allows synaptic transistors to perform concurrent learning. In other words, the signal transmission and synaptic weight updating can be implemented simultaneously90,110,111. Therefore, synaptic transistors allow more flexible simulation of synaptic behaviors, and exhibit high promise for next-generation advanced neuromorphic electronics112. Typical synaptic transistors mainly include floating-gate transistors (FGT), electric-double-layer transistors (EDLT), electrochemical transistors (ECT), and ferroelectric field-effect transistors (FeFET). Working mechanism, material selection, device performance, synaptic behavior, and application of synaptic transistors are given in Table 2.
Table 2. Recent advances of transistor-based synaptic devices. |
| Mechanism | Material | Channel size | VDS | ON/OFF ratio | Synaptic functions | Application | Ref |
|---|---|---|---|---|---|---|---|
| FGT | Channel: IDTBT Floating gate: PVPa/QDb | - | −20 V | > 104 | PPF, PPD, STDP | - | 111 |
| FGT | Channel: MoS2 Floating gate: graphene | - | 2 V | > 103 | Linear synaptic weight updates, STDP | - | 116 |
| FGT | Channel: IGZO Floating gate: ITO | W = 1000 µm L = 80 µm | 50 mV | - | EPSC | ANN | 162 |
| EDLT | Channel: ITO Dielectric: PSGc | W = 1000 µm L = 80 µm | 1.5 V | 1.2 × 107 | EPSC,PPF,high-pass filtering | - | 163 |
| EDLT | Channel: IGZO Dielectric: chitiosan | W = 1000 µm L = 80 µm | 1.0 V | - | ESPC, PPF, LTP | - | 193 |
| EDLT | Channel: MoS2 Dielectric: sodium alginate | - | 0.1 V | ∼ 105 | EPSC, STDP | Photoelectric hybrid integrated neural networks | 112 |
| EDLT | Channel: IGZO Dielectric: chitosan | - | 1 V | 3.7 × 105 | PPF, STDP | - | 127 |
| ECT | Channel: PEDOT: PSS/PEI Dielectric: Nafion | W = 2275 µm L = 65 µm | 0.3 V | - | PPF, STDP | Neuromorphic functionality in stretchable systems | 143 |
| ECT | Channel: ETE-Sd Dielectric: NaCl | W = 1000 µm L = 30 µm | −0.2 V | - | STP, LTP | - | 138 |
| ECT | Channel: PEDOT:Tos/PTHF Dielectric: NaCl | W = 500 µm L = 10 µm | −0.2 V | - | PPF, STP to LTP transition | Associative learning | 144 |
| FeFET | Channel: MoS2 Dielectric: P(VDF-T rFE) | W = 5 µm L = 5 µm | −1 V | 104 | LTP, STDP | - | 238 |
| FeFET | Channel: WS2 Dielectric: HZO | W = 500 nm L = 8 µm | 0.1 V | 105 | Potentiation and depression, SRDP | - | 99 |
| FeFET | Channel: GOe/PVA Dielectric: P(VDF-T rFE)/PMMA | W = 100 nm L = 100 nm | 0.2 V | 104 | PPF, PPD, potentiation and depression, SRDP | Vector-matrix multiplication | 239 |
apolyvinylpyrrolidone. ball-inorganic CsPbBr3 perovskite quantum dot. cphosphorosilicate glass. d4-(2-(2,5-bis(2,3-dihydrothieno[4-b][1,4]dioxin-5-yl)thiophen-3-yl)ethoxy)butane-1-sulfonate. egraphene oxide. |
Floating-gate transistors
A FGT has a similar device structure as a conventional field-effect transistor (FET), except for a floating gate embedded in the dielectric layer113. In FGTs, charges accumulated by the gate bias can be injected into the floating gate through thermal emission or quantum tunneling. The charges remain trapped even after the gate bias is removed, while in conventional FETs, charges dissipate quickly after the removal of bias114. The trapped charges modulate the effective threshold voltage, thus tuning the channel conductance115,116. The tunable nonvolatile channel conductance makes FGTs a promising device to mimic synaptic functions, especially LTP117.
Kim et al. demonstrated a synaptic FGT based on highly purified semiconducting carbon nanotubes (CNT)118. A thin Au layer was introduced as a floating gate inside the gate dielectric. By designing the amplitude and duration time of the gate voltage pulse, the amount of charges in the Au layer could be adjusted accurately, enabling the linearity and variation margin of the weight update. In spite of the fact that great successes have been achieved in utilizing conducting thin films as floating gates by continuous floating gate devices 119, limitations also exist in terms of retention ability owing to lateral leakage and increased cell-to-cell interferences when scaled down113. As an optimization strategy, nano-floating gate transistors use metallic or semiconducting nanoparticles as the floating gates, which can effectively improve the retention ability and stabilize the synaptic weight changing process120,121. Ren et al. proposed a flexible organic synaptic transistor containing a combined tunneling and floating-gate hybrid layer, and the C60 floating-gate domains were well dispersed in the tunneling poly(methyl methacrylate) (PMMA) dielectric layers (Fig. 5a)117. This transistor exhibited a memory window of 2.95 V, a high current on/off ratio above 103, and over 500 times of program/erase endurance. In addition, several important synaptic behaviors including EPSC, PPF/PPD, and LTP were emulated (Fig. 5b, c). Park et al. developed an UV-responsive photonic FGT which used 2D nitric acid-treated carbon nitride (NT-CN) nanodot materials as an UV-responsive floating-gate layer122. Under UV light, the photo-induced electrons were trapped in the NT-CN layer. After the UV light was removed, long recombination time of electrons led to the realization of synaptic plasticity. The device worked at a low energy consumption of 18.06 fJ, and synaptic behaviors including PPF and SRDP were achieved.
Fig. 5. FGTs and EDLTs. a, Schematic of the transistor with C60/PMMA hybrid film as the floating gate. b, PPF index as a function of pulse interval. c, Channel conductance modulation by applying repeated positive and negative gate spikes. Reprinted with permission from ref.117. © 2018 Wiley-VCH. d, Schematic of the PVA coupled 2D MoS2 transistor. e, EPSC triggered by a presynaptic spike; spike duration-dependent EPSC where the spike amplitude is 1.5 V. f, Schematic of the 2D MoS2 neuromorphic transistor with a grid of 3 × 3 coplanar-gate arrays for the visual system. Reprinted with permission from ref.134. © 2018 American Chemical Society. g, Cross-section view of the EDLT. h, EPSCs in response to stimuli separately or corporately. i, Emulation of eyeblink reflex in response to external stimuli in five different situations. Reprinted with permission from ref.136. © 2021 IEEE. |
Electric-double-layer transistors
An EDLT predominantly employs electrolyte as the gate dielectric. Electrolyte-gated transistors can operate in electrostatic mode (called EDLTs) or electrochemical doping modes (called ECTs)123. The main difference of the two modes lies in the ionic permeability of the channel material, which is a property related to both the electrolyte and the semiconductor124,125. For the electrostatic mode, the semiconductor is ion-impermeable. Under an electric field, the drift cations and anions create strong accumulation of space charges at the electrolyte/semiconductor interface, which results in an electric double layer (EDL)31,126,127. The EDL acts as a nanogap capacitor, leading to a high specific capacitance above 1.0 µF/cm2 and much-improved carrier densities of 1014 cm−2128,129. Due to the strong capacitive coupling, EDLTs can operate at low voltage (< 10 V) and consume a low amount of energy. Furthermore, contrary to FGTs, the volatile changes in the conductance of EDLTs are favorable for STP emulation, which is an important feature in artificial sensory systems130. In addition, compared with electrons, the mobility of ions is relatively slow. The ion accumulation/relaxation time in the gate electrolyte is in the order of milliseconds, and is comparable with the ionic influx in biological synapses131. These unique characteristics have made EDLTs an attractive candidate in the field of biomimetic synaptic devices in the past few years.
When evaluating artificial neural system performance, energy consumption is always an important issue. The low-voltage operation of EDLTs could potentially provide a solution. Zhou et al. fabricated a flexible low-voltage indium-gallium-zinc-oxide (IGZO) EDLT for energy-efficient artificial synapse application132. The energy consumption of the transistor was estimated to be as low as ∼0.23 pJ per synaptic event. Rohit et al. presented a dual-gated EDLT configuration with semiconducting indium-tungsten-oxide (IWO) channel and ion gel-based dielectric133. The transistor could operate at low voltage (Vgs ∼ 1.5 V and Vds ∼ 1-100 mV) owing to the large EDL capacitance. With Vds being set to 1 mV, the energy consumption was as low as ∼ 9.3 fJ per synaptic event, which is comparable to biological synapses.
Another huge advantage of EDLTs lies in the fact that they are compatible with multiple gates to synthetically process input signals, which is similar to the function of neural dendrites. Xie et al. demonstrated a coplanar multigate 2D molybdenum disulfide (MoS2) EDLT based on poly(vinyl alcohol) (PVA) proton-conducting electrolyte (Fig. 5d)134. Apart from successful mimicking synaptic behaviors such as EPSC and PPF (Fig. 5e), the proposed transistor also experimentally demonstrated features of the visual cortex cells including spatiotemporal coordinate and visual orientation recognition with multiple in-plane gates as the modulatory input terminals (Fig. 5f). He et al. proposed multiterminal oxide-based EDLTs for mimicking the dynamic discrimination of different spatiotemporal inputs sequences135. As an example of it, sound location of human brain was emulated by an artificial network based on these multiterminal EDLTs. Li et al. introduced a multiterminal ion gel-gated synaptic transistor which could be used to emulate the human blink reflex (Fig. 5g)136. Two stimuli with the same amplitude and different frequencies were applied to the multiterminal EDLT as an emulation of biological stimuli sensed by various receptors, which were tactile stimuli on the corneal and visual information of approaching object in this case (Fig. 5h). The transistor acted as a combinatory calculation unit to distinguish multi-channel signal patterns by comparing with a predetermined threshold (Fig. 5i).
Electrochemical transistors
In ECTs, ions penetrate the semiconductor and cause electrochemical reactions to change the electrical properties of the channel137,138. Under applied gate voltage, ions are injected into the semiconductor from the electrolyte and modify the doping level, hence modifying the conductance of the semiconductor139. Unlike EDLTs whose semiconductor/dielectric interface functions as the major channel, the effective channel of ECTs employs the whole volume of the semiconductor layer125,140. Therefore, large modulation in the drain current could be achieved by ECTs under low gate voltages and impressive signal amplification properties are realized141,142.
Similar to EDLTs, ECTs can also work under low voltage. Burgt et al. fabricated a nonvolatile, flexible ECT-based artificial synapse143. A poly(ethylenimine) (PEI)/ poly(3,4-ethylene dioxythiophene): polystyrene sulfonate (PEDOT: PSS) film was used as the channel material, along with PEDOT: PSS presynaptic electrodes and Nafion electrolyte. This device exhibited a large number of nonvolatile and reproducible states (> 500), and operated at low voltage (10 mV) due to the electrochemical overpotential of the organic channel. Ji et al. presented a nonvolatile organic electrochemical transistor (OECT) by adopting a vapor phase polymerized poly(3,4-ethylenedioxythiophene): tosylate (PEDOT: Tos)/ polytetrahydrofuran (PTHF) composite as the channel layer (Fig. 6a)144. The PEDOT: Tos/PTHF-based OECT showed a fast response time (∼1 ms), low working voltage (< 0.8 V), and long retention time (> 200 min). Different synaptic behaviors from short-term to long-term plasticity were emulated as well (Fig. 6b).
Fig. 6. ECTs and FeFETs. a, Schematic of synapse and the PEDOT: Tos-based OECT (left) and the chemical structure of PEDOT+, Tos−, and PTHF (right). b, Short-term memory (STM) to long-term memory (LTM) transition in the P-80% PTHF-based OECT. Reprinted with permission from ref.144. © 2021 Nature Publishing Group. c, Micrograph of the OECT with patterned solid electrolyte. d, Composition of the solid electrolyte (precursor). e, Transfer characteristics of an OECT with patterned vs. non-patterned solid electrolyte. Reprinted with permission from ref.146. © 2022 Royal Society of Chemistry. f, Schematic of the flexible FeFET consisting of mica/SRO/PZT/IGZO heterostructure. g, Photo of a flexible FeFET on the bended finger. h, Long term potentiation and depression. Reprinted with permission from ref.152. © 2020 American Institute of Physics. i, The double sweep transfer curves of NOFST. j, The comparison of conductance update between NOFST and conventional organic ferroelectric synaptic transistor (COFST). k, The Gmax/Gmin of NOFST and COFST. Reprinted with permission from ref.153. © 2021 Elsevier B.V. |
Liquid electrolytes lead to incompatibility with traditional photolithography technology, making it challenging to achieve high-density, large-area and uniform fabrication56. Consequently, recent researches are focusing on developing solid electrolytes with high ion mobility. Yang et al. proposed an all-solid-state electrochemical transistor with Li ion-based solid dielectric and 2D α-phase molybdenum oxide (α-MoO3) nanosheets as the channel145. Their device achieved nonvolatile conductance modulation through gate voltage-induced reversible intercalation of Li ions into the α-MoO3 lattice. Synaptic behaviors including PPF and LTP were simulated. Weissbach et al. presented a photopatternable solid electrolyte based on ionic liquid 1-ethyl-3-methylimidazolium ethyl sulfate ([EMIM][EtSO4]) in a polymer matrix (Fig. 6c,d)146. The electrolyte could be patterned with standard photolithographic techniques down to a resolution of 10 µm. The proposed OECT exhibited excellent performance with ON/OFF ratio of 105, a threshold voltage of 200 mV, and a sub-threshold swing of 61 mV dec−1 (Fig. 6e). Similarly, Tuchman et al. put forward a hybrid electrolyte design by permeating an ionic liquid in a porous inorganic matrix147. The hybrid electrolyte enabled electrochemical doping of the semiconductor, tuning the device conductance in a linear and analog way. They fabricated single-microscale stacked hybrid organic/inorganic electrochemical random-access memories adopting traditional lithography and encapsulation methods.
Ferroelectric field-effect transistors
In FeFETs, ferroelectric materials are generally utilized as the gate dielectric. By applying a gate voltage, the carrier concentration of channel layer can be tuned by the changes in the ferroelectric polarization, and hence providing multiple conductance states, which can be utilized to record synaptic weight148-150.
Kim and Lee demonstrated the analog modulation behavior in FeFET with nanoscale zirconium-doped hafnium oxide (HfZrOx) gate dielectric and IGZO as the channel layer151. The gradual change in the polarization state of HfZrOx was implemented by controlling the applied gate voltage, hence making it possible that the conductance of the channel can be controlled gradually. Their FeFET exhibited good potentiation and depression properties including 64 level conductance state, good linearity, and large conductance range (Gmax/Gmin > 10). Zhong et al. developed an all-inorganic FeFET consisting of a SrRuO3 (SRO) gate electrode on the flexible mica substrate, a PbZr0.2Ti0.8O3 (PZT) ferroelectric layer, an IGZO channel, and Au source and drain electrodes (Fig. 6f)152. The device showed high flexibility and robust performance (Fig. 6g). Under a series of positive/negative presynaptic spikes, the device conductance showed continuous increase/reduction, which was regarded as long-term potentiation/depression (Fig. 6h). Li et al. presented a nanoscale organic ferroelectric synaptic transistor (NOFST) based on ferroelectric material PVDF-TrFE and semiconductor copolymer poly[4-c]pyrrole-1,4(2H,5H)-dione-alt-5,50-di(thiophen-2-yl)−2,20-(E)−2-(2-(thiophen-2-yl)vinyl)thiophene] (PDVT-10)153. The NOFST applied a vertical structure, in which the semiconducting layer was sandwiched between mesh source and drain electrodes (Fig. 6i). Therefore, the charge was transmitted through the entire semiconducting layer, and the channel length was determined by the thickness of the semiconducting layer. Due to its unique operation mechanism and nanoscale channel length, the device exhibited linear and symmetric weight update (Fig. 6j) and improved conductance variation compared with transistors with planar structure (Fig. 6k).
In summary, memristors and synaptic transistors are two main synaptic devices. Among them, memristors stand out due to the advantageous properties of fast accessing speed, low power consumption, high density, and configuration convenience154-157. They are regarded as the most promising candidate for highly scalable and low-power synaptic devices156,158, and have been widely used for neuromorphic computing159-161. However, the linearity of the synaptic response of memristors remains to be further improved, and they have limitations in emulating sophisticated biological synapses because of the simple device structure. These drawbacks make them relatively less used in developing biomimetic sensory neural systems130.
Meanwhile, controllable and continuous conductance states can be easily achieved by synaptic transistors under the control of the gate voltage. FGTs and FeFETs usually exhibit a controllable and stable channel conductance, making them ideal candidates for realizing LTP162. Electrolyte-gated transistors including EDLTs and ECTs show low working voltage and energy consumption. More interestingly, the ion-related processes are quite similar to ion migration-related information processing in biological synapses163. These features make them ideal artificial synapses for developing artificial sensory systems in which STP is critical130. However, compared with memristors, synaptic transistors show relatively high energy consumption and low conductance states. They also face severe challenges in device fabrication. Difficulties in device integration and scaling could be also caused by poor device uniformity and the relatively complicated three-terminal structure110. In addition, the incompatibility existing between solution processing technology widely used in EDLTs and ECTs, and standard microelectronic processing technology is still a tough problem remains to be solved.
BIOMIMETIC SENSORY NEURAL SYSTEMS
In every biological sensory system, the process of sensory perception begins with a receptor sensitive to external stimuli. For humans, there are four main types of receptors: mechanoreceptors, thermoreceptors, photoreceptors, and chemoreceptors, forming five essential senses: sight, hearing, smell, taste, and touch164. These receptors form synapse-like connections to afferent neurons so as to detect diverse stimuli and convert them into electrical impulses165. These electrical impulses are subsequently transferred through afferent nerves to the central nervous system for further processing and interpreting. Owing to these sophisticated and operative sensory systems, organisms are empowered to interact with the colorful world.
A biomimetic sensory neural system simulates how the biological sensory systems process external sensory information. It allows in situ sensing, processing, and memory of external sensory signals without external computing units, thus making it possible that a low-latency and energy-saving perception system could be implemented. To develop a biomimetic sensory system, there are two main approaches to sense external stimuli and convert them into postsynaptic outputs: developing synaptic devices with sensing capabilities and integrating synaptic devices with separated sensing elements. The processed postsynaptic output signals are further used for advanced functions such as recognition, learning, and motor control. Through proper material selection, structure design, and device integration strategies, different sensory systems have been successfully developed, as showed in Table 3. Also, discussions about recent advances in the biomimetic tactile sensory system, visual sensory system, and multisensory integration neural system are presented in the upcoming section.
Table 3. Biomimetic sensory neural systems mimicking senses of human. |
| Sense | Stimuli | Key devices | Applications |
|---|---|---|---|
| Touch | Pressure | Pressure sensors and synaptic devices176,183,240 | Logic gates, pattern recognition, artificial somatic reflex arcs. |
| Vision | Light | Optoelectronic synapses208,209 | Human visual memory, pattern recognition. |
| Auditory | Voice | Piezoelectric/ triboelectric vibration sensors and synaptic devices233 | Instruction recognition, sound azimuth detection. |
| Olfactory | Gas | Synaptic sensory transistors sensing chemicals234,241 | Artificial organ-damage memory system, associative learning. |
Biomimetic tactile sensory system
The tactile perception system is the earliest developed, most widely distributed, and most complicated sensory system of human166,167. It is responsible for interpreting and representing touch sensing information to explore the shape, weight, surface structure, and other physical properties of objects, largely shaping our interactions with the external environment168,169. As the sensory organs of human skin, mechanoreceptors convert external touch stimuli on the skin into electrical signals, and are mainly divided into slowly adapting (SA) and fast adapting (FA) mechanoreceptors to detect static and dynamic forces, respectively170. Subsequently, the electrical signals flow through nerve fibers, and then arrive in the synapses as the presynaptic action potentials for further signal processing171.
Generally, biomimetic tactile perception systems are achieved by combining pressure sensors and bioinspired synaptic devices. According to the form of the output pressure signal, pressure sensors can be mainly divided into the following two types. Piezoresistive and capacitive pressure sensors produce continuous changes of electrical properties (e.g., resistance, capacitance) in the active layer under stimuli. These changes are later read out by circuits to reflect the magnitude of the applied pressure. Another type of pressure sensor is self-powered, including piezoelectric and triboelectric pressure sensors which could convert external tactile signals directly into voltage pulses.
Among them, piezoresistive pressure sensors show high sensitivity to static forces, which is similar to the function of skin's SA mechanoreceptors172-174. Zang et al. presented a dual-organic-transistor-based tactile-perception element (DOT-TPE) (Fig. 7a)175. The DOT-TPE consisted of a suspended-gate transistor as the pressure sensor and a synaptic transistor. The sensor converted dynamic tactile information into electrical signals, and then the signals were processed by the synaptic transistor. Different from the output of a single sensor, Ipost of DOT-TPE contained combined information on the intensity, frequency, and duration time of the pressure (Fig. 7b, c).
Fig. 7. Biomimetic tactile sensory systems detecting static forces. a, Schematic (left) and equivalent electrical circuit (right) for the DOT-TPS. b, The relative changes in current in the sensing device and Ipost responses of the synaptic transistor under different pressures. c, The relative changes in current in the sensing device and the Ipost responses of the synaptic transistor under a pressure of 50 Pa175. Reprinted with permission from ref.175. © 2017 Wiley-VCH. d, Schematic of the NeuTap. e, Tactile pattern recognition and perceptual learning by the NeuTap. Left: Digital image of the NeuTap on a finger and schematic diagrams illustrating the pattern pairs and their corresponding two-bit binary code labels. Right: The typical responses to three types of pattern pairs176. Reprinted with permission from ref.176. © 2018 Wiley-VCH. f, Schematic of the artificial haptic perception system consisting of a pressure sensor and a Nafion-based memristor. g, Handwriting recognition by the haptic sensory system. Top: Illustration of the writing of English characters by the sensory system assembled on a “pen”. Bottom: Current response of different characters. Reprinted with permission from ref.177. © 2019 Wiley-VCH. |
The features extracted from a biomimetic tactile sensory system could be used to help achieving pattern recognition. Wan et al. designed a neuromorphic tactile processing system (NeuTap) which integrated and differentiated the spatiotemporal features of touch patterns176. The NeuTap consisted of a resistive pressure sensor, soft ionic cable, and a synaptic transistor (Fig. 7d). Electrical signals generated by the pressure sensor were transmitted to the synaptic transistor via the ionic cable. The ionic cable separated sensing and processing elements, which reduced the interferences and improved the flexibility of the system as well. They adopted the NeuTap to implement tactile pattern recognition, where the conductance of the transistor was measured as output (Fig. 7e). Through repeated training, the recognition accuracy could be improved, illustrating high similarity to perceptual learning. Zhang et al. designed an artificial sensory neuron system by integrating a piezoresistive sensor and a Nafion-based organic memristor (Fig. 7f)177. The artificial sensory system was capable of processing and learning the pressure information encoded with temporal information, including frequency, duration and speed. The 91.7% accuracy of English characters recognition could be achieved when adopting a supervised learning method (Fig. 7g).
In biological systems, spike sequences are used to transmit and process information. Compared with continuous analog and digital electrical signals, spiking patterns are less affected by noise during transmission, and consumes less power178. The active potential of receptor under external stimulation is transformed into a series of neural pulse codes, in which the frequency reflects the strength of stimulation179. Therefore, to be better compatible with the spike-coding mode of the biological nervous system, it's necessary to transform the voltage signals produced by sensors into pulse mode, especially for potential applications like human-machine interfaces and prosthetics. Kim et al. introduced ring oscillators into artificial sensory systems, and first invented an artificial afferent nerve to emulate the functions of the SA-Ⅰ afferent neuron (Fig. 8a)180. This biomimetic system consisted of a cluster of pressure sensors, an organic ring oscillator, and a synaptic transistor. Among them, the ring oscillator converted constant voltage signals from pressure sensors into voltage pulses with various frequencies that matched the action potentials of biological sensory neurons. Afterwards, pulse sequences were integrated and converted into postsynaptic currents by the synaptic transistor. This artificial afferent nerve was subsequently interfaced with biological efferent nerves (a cockroach leg) to form a complete monosynaptic reflex arc (Fig. 8b). The oscillating signals from the artificial afferent nerve controlled the responses of the muscle. As the intensity and duration of the stimulus applied increased, the maximum isometric contraction force of the cockroach leg increased accordingly, demonstrating the hybrid bioelectronic reflex arc was feasible.
Fig. 8. A bioinspired flexible organic artificial afferent nerve. a, Schematic of the afferent nerve made of pressure sensors, an organic ring oscillator, and a synaptic transistor. b, Hybrid reflex arc made of the artificial afferent nerve and a biological efferent nerve. Reprinted with permission from ref.180. © 2018 AAAS. |
Although piezoresistive pressure sensors are capable of detecting static forces, they show limitations when facing dynamic forces with high frequency. By comparison, piezoelectric and triboelectric pressure sensors are more sensitive to dynamic forces and endowed with the advantage of fast response, making them suitable for simulating FA mechanoreceptors181,182. Additionally, these self-powered sensors also decrease the energy consumption of the artificial sensory system183. Chen et al. developed a piezoelectric graphene artificial sensory synapse which integrated a piezoelectric nanogenerator (PENG) with an ionic gel-gated transistor (Fig. 9a)184. The piezopotential, which was triggered by external strain spikes, was applied to the gate of the synaptic transistor for mimicking the presynaptic input. The device exhibited a variety of synaptic behaviors under stimuli, EPSC, IPSC, and PPF included (Fig. 9b, c). Liu et al. proposed a rapid-response and high-sensitivity artificial sensory system integrated with a triboelectric nanogenerator (TENG) and a synaptic transistor (Fig. 9d)185. Hierarchical memorial processes from sensory memory to short-term memory and long-term memory were successfully demonstrated (Fig. 9e). A 28 × 28 matrix was fabricated to connect the real-time handwritten image with large-scale data processing (Fig. 9f). Zhang et al. presented a tactile element composed of a TENG and an electrolyte-gated synaptic transistor (Fig. 9g)186. Several important behaviors of sensory neurons including EPSC, PPF, and high-pass filtering were simulated. In addition, spiders’ ability to identify prey by sensing the vibrations of cobwebs was also emulated. A threshold was set to distinguish an insert trapped on the web or environmental disturbance, according to the amplitude of EPSCs in response to different vibration frequency applied on the TENG (Fig. 9h).
Fig. 9. Biomimetic tactile sensory systems detecting dynamic forces. a, Schematic of the piezotronic grapheme artificial sensory synapse. b, EPSC triggered by different strain inputs. c, EPSC versus strain pulses of different duration time.Reprinted with permission from ref.184. © 2019 Wiley-VCH. d, Artificial sensory system integrated with a triboelectric nanogenerator (TENG) and a synaptic transistor. e, Sensory memory to STM and to LTM transfer process. f, Diagrams of the real-time handwritten digit number and the final tactile mapping. Reprinted with permission from ref.185. © 2020 Elsevier B.V. g, Schematic of biological sensory neuron (top) and the proposed tactile-sensing element (bottom). h, Simulation of the functions of spider's mechanical sensory neuron. Reprinted with permission from ref.186. © 2020 IEEE. |
The above works were mainly based on the integration of separated sensors and synaptic devices. These separated systems require complicated fabrication processes for discrete components, and the complexity of wiring and array implementation remains to be optimized. A potentially effective strategy to solve the above problems could be integrating the tactile-sensing functions and synaptic behaviors into a single device. Yang et al. proposed a versatile mechanoplastic artificial synapse based on tribotronic floating-gate MoS2 synaptic transistor, in which mechanical displacement was used to modulate the synaptic weights by inducing triboelectric potential coupling to the transistor (Fig. 10a, b)187. Typical synaptic behaviors including STP and LTP were successfully imitated by mechanical displacements. Kim et al. presented an intelligent haptic perception device (IHPD) that combined pressure sensing function with an OECT-based synaptic transistor into a simple device architecture (Fig. 10c)188. The pyramid-patterned ion gel could simultaneously function as the dielectric layer and the pressure sensing element. The IHPD was capable of rapid and reversible switching between STP and LTP (Fig. 10d, e). Lee et al. reported an artificial intrinsic-synaptic tactile sensory organ (AiS-TSO) based on a ferroelectric organic field-effect transistor(Fig. 10f)165. When applying a touch stimulation on the substrate, alignment of dipoles in the ferroelectric gate dielectric would lead to the modulation of postsynaptic current signal due to tribo-capacitive coupling effect. Synaptic characteristics of the AiS-TSO were demonstrated with different forces and frequencies (Fig. 10g), in which the deviceexhibited sensory intelligence including adaptation, filtering, and memory functions.
Fig. 10. Synaptic devices with pressure sensing capabilities. a, Schematic of the mechanoplastic MoS2 synaptic transistor (left) and working mechanism of Au NPs floating-gate layer (right). b, Pulse number dependent facilitation gain with applied displacement (D) pulses. Reprinted with permission from ref.187. © 2020 Wiley-VCH. c, Schematic of the IHPD. d, PPF behaviors of the IHPD. e, Gradual potentiation and depression in postsynaptic current under successive pulses under LTP operation. Reprinted with permission from ref.188. © 2020 American Chemical Society. f, Schematic of the organic synaptic transistor with ferroelectric nanocomposite gate dielectric. g, Dependence of the EPSC of the AiS-TSO on force (left) and touch rate (right). Reprinted with permission from ref.165. © 2020 Nature Publishing Group. |
Biomimetic visual sensory systems
Vision plays a critical role in the task of information perceiving for human being, and nearly 80% of information from the external environment is derived visually189. Realizing artificial vision systems is an important step for the development of humanoid robotics, computer vision, and etc. An optoelectronic synapse responding to light stimulation is the basic unit for mimicking visual sensory functions. In recent years, artificial optoelectronic synapses based on diverse materials (metallic oxides190-193, 2D materials,194-196. perovskites197-200, organic materials201,202, etc.) and device structures (memristors203-206, transistors122,198,207, integrated structures208-210, etc.) have been used to build biomimetic visual neural systems.
As a necessary part of the human visual system, Retina receives various light information (wavelength, frequency, intensity, etc.), and serves as the first-stage processing unit before cortex via the optical nerves (Fig. 11)211. Various human retinal functions have been successfully realized using optoelectronic synapses. Zhou et al. designed an optoelectronic resistive random access memory (ORRAM) synaptic device with a two-terminal structure of Pd/MoOx/ITO212. The device showed UV-light-tunable synaptic plasticity. An ORRAM array was fabricated to perform a first-stage image processing including image contrast enhancement and noise reduction, emulating the sensing and processing functions of the human retina. The preprocessed images were sent to an artificial neural network (ANN) for further recognition functions, mimicking the visual cortex. Seo et al. presented an optic-neural synaptic device (ONS) by integrating a synaptic device with an optical-sensing device on the same h-BN/WSe2 heterostructure (Fig. 12a)213. The synaptic dynamic properties of the ONS were modified according to the light wavelength conditions. An optic neural network where optical-sensing function was added to the synaptic connection was fabricated to demonstrate the capability of colored and color-mixed pattern recognition (Fig. 12b). Kwon et al. designed a light-adjustable optoelectronic neuromorphic circuit consisting of a metal chalcogenide photosensor, a load transistor, and a metal oxide synaptic transistor (Fig. 12c)214. Synaptic behaviors such as STP, LTP, and neural facilitation were achieved under light stimulations of a variety of intensities and wavelengths (Fig. 12d, e). More importantly, the environment-adaptable perception behaviors at various levels of light illumination were well reproduced by adjusting the load circuit, exhibiting the acts of the variable dynamic range of the biological system (Fig. 12f).
Fig. 11. Schematic of a human eye and the multilayer structure of a retina. Reprinted with permission from ref.122. © 2020 Wiley-VCH. |
Fig. 12. Biomimetic visual sensory systems mimicking the human visual system. a, Schematic of the human visual system (top) and the h-BN/WSe2 synaptic device integrated with a photodetector (bottom). b, Colored and color-mixed pattern recognition based on an artificial optic-neural network. Reprinted with permission from ref.213. © 2018 Nature Publishing Group. c, Optical microscopy image of the adjustable neuromorphic circuit. d, Variation of EPSC triggered by visible-light spikes with different wavelengths and intensities. e, PPF curve triggered by two paired visible-light spikes with various pulse interval. f, The 3 × 3 light-adaptable optoelectronic neuromorphic circuit array under high-intensity (left) and low-intensity (right). Reprinted with permission from ref.214. © 2019 Nature Publishing Group. |
In addition to simulating retinal functions for visual recognition, optogenetics referring to the control of biological tissues by light stimulation have also been studied in detail215. These works suggested the potential applications of optoelectronic synapse in the era of next-generation prosthetics and neurorobotics. Lee et al. reported an organic optoelectronic sensorimotor synapse (Fig. 13a)216. The voltage pulses generated by a self-powered photodetector drove a designed stretchable organic nanowire synaptic transistor (s-ONWST), forming an optoelectronic synapse. Then, by connecting the s-ONWST to a polymer actuator through a transimpedance circuit, a neuromuscular electronic system was assembled. When applying short light pulses, the EPSCs were converted to voltages by the transimpedance circuit to drive the actuator (Fig. 13b, c). Inspired by cephalopods’ ability to change their skin color without the involvement of brains, Li et al. designed a tunable, environment-adaptive camouflagic artificial reflex arc that could independently activate electrochromism in response to optical stimuli (Fig. 13d)217. An IGZO-based light-responsive transistor with ion gel dielectric acted as both the receptor and artificial synapse. Subsequently, EPSC generated from the synaptic transistor was amplified, and then triggered the color transform of an electrochromic device, mimicking the phenomenon of an octopus changing color according to the background light (Fig. 13e).
Fig. 13. Artificial reflex arcs in response to optical stimuli. a, Schematic of organic optoelectronic synapse and neuromuscular electronic system. b, Visible light-triggered EPSC amplitudes of s-ONWST from 0 to 100% strains with 1 to 30 spikes. c, digital images of the polymer actuator under 0 to 100 spikes with 0 or 100% strain. Reprinted with permission from ref.216. © 2018 AAAS. d, Schematic of the neuromorphic camouflage device. e, Emulation of octopus’ peripheral nervous stimulated by the background light and color changing of the chromatophores accordingly. Reprinted with permission from ref.217. © 2017 IEEE. |
Multisensory integration neural systems
The biological sensory system has a unique feature that it can integrate multiple sensory inputs into one synthetical perception for better monitoring of environmental information218. For example, the vertebrate detects tactile and visual information in parallel through peripheral nervous system219. The incorporation of multiple sensory into artificial sensory systems will allow robots to perform complex recognition and decision tasks in unknown environments. To this end, several biomimetic sensory systems which sense both tactile and visual information have been reported.
Chen et al. developed a bimodal artificial sensory neuron (BASE) to implement the visual-haptic fusion (Fig. 14a)220. The artificial neuron collected optic and pressure information based on a photodetector and pressure sensors. Bimodal signals were transmitted through an ionic cable and were integrated into postsynaptic currents by a synaptic transistor. Motor control and pattern recognition were demonstrated through this bimodal artificial sensory neuron (Fig. 14b, c). Wu et al. developed a multisensory integration neural system with haptic and optic perception behaviors by integrating TENG as a skin receptor with an organic optoelectronic synaptic transistor as a retina receptor and synaptic device218. Principles of multisensory integration including inverse effectiveness and temporal congruency were successfully mimicked. Their pattern recognition results showed that this artificial nervous system based on multisensory integration exhibited higher accuracy than the one with single sensory component.
Fig. 14. A bimodal artificial sensory neuron with visual-haptic fusion. a, The BASE patch for visual-haptic fusion. b, Motion control based on visual-haptic fusion. c, Multi-transparency pattern recognition based on visual-haptic fusion. Reprinted with permission from ref.220. © 2020 Nature Publishing Group. |
CONCLUSIONS AND OUTLOOK
This review summarized state-of-the-art synaptic devices and the recent trends of their applications in biomimetic sensory neural systems. In recent years, considerable achievements have been made in the development of synaptic devices. Emerging synaptic devices with various materials, device structures, and working mechanisms have successfully emulated various synaptic behaviors, and been regarded as a promising alternative to the conventional von Neumann architecture. As two main types of synaptic devices, two-terminal memristors and three-terminal transistors possess their advantages and application prospects, respectively. Two-terminal memristors are endowed with low energy consumption, fast response time, and optimum scale integration, thus exhibiting great potential in storage and neuromorphic computing. In comparison, three-terminal transistors have the merits of controllable parameters and multiple inputs, allowing them to achieve simultaneous signal transmission and learning functions. Therefore, synaptic transistors are more advantageous in perceiving various stimuli (light, chemical composition, pressure, etc.), and even rebuilding artificial sensory system.
Although great progresses have been made in single synaptic devices, lack of sophistication and practicality for the proposed systemsis is still a tough problem remains to be solved. In the future, we look upon the combination of living organisms and biomimetic neural systems to be a mainstream trend. As mentioned above, by connecting an artificial afferent nerve and biological motor nerves, a hybrid bioelectronic reflex arc has been successfully constructed180. The same research group also reported an artificial efferent nerve used to control the leg muscles of an anaesthetized mouse221. Notably, the device could be operated with recorded public neural data as input, further demonstrating its applicability in neurorehabilitation. These simple and low-power systems, which could replace the functions of damaged nerves and generate voluntary motions, provide a new paradigm for future neuromorphic prosthetic devices. In addition, biomimetic sensory neural systems transfer sensors’ outputs into synaptic signals, thus could act as human-machine interfaces which bridge the gap between prostheses and the nervous system222. For example, prosthetic limbs with biomimetic tactile system could provide sensory feedback to restore tactile sensation, which largely improve the experience of amputees223. Furthermore, it is even possible to design more sophisticated bionic prostheses which perform beyond the human sensory organs, thus improving the level of human perception.
Humans perform not only sensing and neural signal processing but also motor responses with the participation of the efferent nervous system. The integration of the sensory and motor systems allows humans to execute and adjust motions under continuous sensory guidance224-226. This flexible and dynamic neural system provides new inspiration for the future development of robotics. Using output signals of artificial synapses to drive soft actuators, biomimetic motor systems that mimic the muscle movement have been fabricated227-229. The application of the biomimetic motor system enables robots to perform life-like motions without complex circuit setup. Besides, biomimetic neural systems have also been utilized in robotic systems for learning. Complex learning tasks including pathfinding224 and human body-like pain reflex230 have been achieved. These works have proved the potential of using such neuromorphic systems, instead of conventional microprocessors and circuits, to implement complicated behaviors of robots. It is expected that biomimetic neural systems will become vital components of future robotic systems21,231.
In order to realize these fascinating conceptions, the current biomimetic neural systems should be optimized and improved in the following aspects. (1) A deeper understanding of the biological nervous system is essential to enable accurate matching of signals between the biomimetic neural system and biological nervous system. (2) Applications and synthetical strategies are still limited in developing multimodal artificial sensory systems. For example, few works could integrate three or more types of sensory information to perform tasks like grabbing objects by cooperating touch, proprioception, and vision232. (3) Implementing advanced neuromorphic algorithms has great potential on all the above-mentioned hardware platforms. More efforts should be made in incorporating ANN or spiking neural network (SNN) to realize intelligent functions including pattern recognition and perceptual learning. (4) If the biological nervous system in various biomimetic electronic devices including prostheses and robots is to be comprehensively mimiced, designing actuators driven by neural spikes is required.
Biomimetic sensory neural systems based on synaptic devices reported today are still in the early stage and there is still a long way to go before they are put into commercial application. We look forward that biomimetic sensory neural systems will greatly improve existing electronic systems with joint efforts of fields such as biology, chemistry, material science, and computer science.
MISCELLANEA
Acknowledgements This work was supported by the National Key Research and Development Program of China (2021YFA1401103), the National Natural Science Foundation of China (61825403, 61921005, and 61674078), the Priority Academic Program Development of Jiangsu Higher Education Institutions. The Postgraduate Research & Innovation Program of Jiangsu Province (KYCX21_0049 to J.-H.Z.).
Declaration of Competing Interest The authors declare no competing interests.
