Temporal processing with volatile memristors

Temporal Processing with Volatile Memristors R. Berdan, T. Prodromakis, A. Khiat, I. Salaoru, C. Toumazou F. Perez-Diaz, E. Vasilaki Department of Computer Science University of Sheffield Sheffield, S1 4DP, UK Centre for Bio-Inspired Technology Imperial College London London, SW7 2AZ, UK Abstract— Short-term synaptic plasticity (STP) is a mechanism identified in brain systems according to which the effective connection strength (synaptic strength) between two neurons varies dynamically with recent communication history. As a consequence, the amplitude of the post-synaptic potential in response to a single pre-synaptic event, so-called “spike”, may increase (short-term facilitation) or decrease (short-term depression) with consecutive presynaptic stimulation. However, in contrast to Long-term Synaptic plasticity, these changes are temporary and are typically restored in the absence of input. Interestingly, however, a single neuron which receives input via both facilitating and depressing synapses has improved discrimination capability, distinguishing, for instance, between a sequence of events and a sequence of the same events presented in the reversed order. We, therefore, studied the memory mechanisms in emerging non-CMOS devices with a view to application in temporal pattern recognition and detection, inspired by the STP mechanisms. In particular, we demonstrate that memristors can exhibit a resembling behavior to STP due to an inherent volatility and hysteresis. When stimulated by closely spaced pulse waves, the conductance of the device decreases similar to what a depressing synapse would do if presented with consecutive pre-synaptic spikes. This work paves the way for employing memristors in solving spatio-temporal sequence learning problems. I. INTRODUCTION Memristors have recently fueled research interest owing to their massive potential for non-volatile memory applications [1], but also neuromorphic engineering [2], reconfigurable architectures [3], and artificial synapses [4][5]. The functional properties of these devices, mainly their simple two terminal topology and infinitesimal device structure (reported state-ofart in size 10x10nm2 [6]), low power consumption (<50fJ/bit) and high-speed operation (ns switching) have been the driving force in this research. The theoretical basis of memristors has first been conceived by L. Chua in 1971 [7], with this field attracting significant interest only after a Hewlett Packard team observed memristive signatures in nanoscale TiO2-based resistive switch [8]; despite the fact that similar signatures have been observed over the past two centuries [9]. So far memristive attributes have been traditionally researched towards establishing next-generation memory cells 978-1-4673-5762-3/13/$31.00 ©2013 IEEE and as such emphasis has been given in attaining devices with very long retention characteristics (non-volatile). Nonetheless, the functionality of memristors extends beyond that of memory elements, including the possibility of employing their unconventional dynamics to perform temporal processing. In fact, Chua, in his paper [7] posted the possibility of employing memristors as neuronal synapse emulators, while more recently it was shown how memristors can even support biological learning rules, such as Spike-Timing Dependent Plasticity (STDP) [4]. Yet these exploit memristors as static dynamic loads that are incapable of recounting the short-term dynamic behavior of biological synapses [10][11]. In this work we present solid-state memristors that demonstrate volatility, which can effectively be employed to emulate short-term synaptic dynamics. Section II describes the fabrication of the devices and the instrumentation set up we employed to characterize these. Finally, section III delineates our experimental results that are fitted through an existing biological model to verify our hypothesis. II. METHODOLOGY This work exploits solid-state TiO2-based memristors fabricated in-house and specific instrumentation was designed for effectively evaluating the devices dynamics when subjected to distinct pulsing schemes. A. Fabrication All employed memristors throughout this work consist of a Figure 1. Solid-state memristors based on TiO2 thin films fabricated at Imperial College London. a) Microphotograph of the prototyped cross-bar arrays, with inset of (a) showing a cross-section schematic of the device’s architecture. b) Employed devices in a conventional 24-pin package and corresponding inset showing a close-up of the bonded devices. 425 25nm thin film of TiO2 sandwiched within two 90nm thick Pt electrodes in a cross-bar architecture of 5x5 µm2 active area, as illustrated in Fig. 1a. The TiO2 thin film was conformally deposited via RF sputtering from a stoichiometric target. Conventional optical lithography and lift-off techniques were employed for patterning the bottom and top electrodes (BE and TE) of the devices, while a wet-etching step was used for removing the TiO2 on top of the interfacing electrodes. For ease-of-use purposes, these prototypes were bonded on standard packages, as shown on Fig. 1b and its inset figure. B. Instrumentation In order to measure the temporal variability of resistive state of a memristor, we have designed (circuit schematic is depicted on Fig. 2a) and constructed (Fig. 2b illustrates our test board) a versatile and flexible programming system that is capable of applying sharp voltage pulses on a series resistormemristor (RP – M) combination. Our instrumentation is also able to measure the value of memristance over increased periods of time with good accuracy. This system utilizes an NXP mBed micro-controller and feeds back data to a local PC through a serial communication link. Measuring the resistive state of the memristor at any time instance is achieved by applying a 0.9V pulse on the memristor branch and reading the voltage drop across the device through a 12 bit ADC located at the Mread input of the mBed. The memristance is then computed through a simple voltage divider equation Figure 2. a) NXP mBed testbed circuit schematic. b) Breadboard assembly with associated control circuitry and memristor package devices under test. between RP (the series resistor of 3.9 kΩ) and M (Fig. 2a). Pulsing the memristive device through a series resistor decreases the chances of accidentally damaging the device when it’s in a low resistive state, as the voltage drop across the memristor becomes smaller than the threshold voltage for which resistive switching occurs. III. EXPERIMENTAL RESULTS A. Memristor Characterisation Our experiment involves a pulsing sequence of 30 read pulses (the memristance is evaluated 30 times at the start of the experiment) followed by 10 programming pulses of 4.8V and 100µs wide applied on the resistor-memristor branch. This sequence is repeated 5 times over the course of 1 minute. Fig. 3a shows volatile behavior of our TiO2-based memristor when excited by the pulsing sequence depicted in Fig. 3b. Initially, the memristance is read 30 times showing that the resistive state is stable. When programming pulses are applied, the memristor switches to a low resistive state after which the memristance drifts toward its original value indicating that the attained state is volatile. This effect occurs for subsequent pulsing of the memristor branch. Considering that the initial resistive state of the memristor is Mi = 5.5 kΩ, this implies that Read pulses contribute to a voltage drop of VMR=0.9V*Mi/(Mi+RP)=0.51V across the memristor. At this level, there is no resistive switching while reading the device. Programming pulses contribute to a voltage drop of VMP=2.75V across the device when the first resistive switching occurs, therefore the threshold voltage for which an observable change in the resistive state takes place is definitely larger than VMR=0.51V. For subsequent read pulses, the voltage dropped across the memristor is lower than VMR simply because the memristance is smaller than the initial value of 5.5 kΩ. Therefore, the instrumentation has no influence on the observed drift of resistive state after it has been SET by the above pulsing sequence. Figure 3. a) Memristance dynamics under pulse sequence shown in (b). b) Pulse sequence comprised of 30 Read pulses of 0.9V and 10 SET pulses of 4.8V, 100 µs wide. 426 Figure 4. STP mechanism and behaviour as described by Markram and Tsodyks. a) Neuronal connection or synapse. The pre-synaptic neuron (PRE) is the input to the post-synaptic neuron (POST). b) Activity of the presynaptic neuron, where each spike is represented as a Dirac delta function. Post-synaptic potential in response to the presynaptic input for c) a depressing synapse and d) a facilitating synapse. e) STP mechanism represented as a continuous Markov chain, where the synaptic resources can be in three different states (active, inactive and recovered). The change from one state to another occurs with certain probability (USE) or according to specified decay constants (τin and τrec). B. Memristor Dynamics Short-term synaptic plasticity (STP) is an experimentally observed type of neural plasticity [10] where the effective strength of a connection (synapse) between two neurons (Fig 4a) varies dramatically over the course of hundreds of milliseconds as a result of recent activity. However, these changes are temporary in nature and will be restored in the absence of synaptic input for a sufficiently long time (Fig. 4bd). In general, artificial neural networks and hardware implementations have mainly focused on long-term plasticity, permanent changes that have its effects on longer time scales, ignoring STP. Nevertheless, this is a feature worthwhile of consideration as the importance of STP has been demonstrated, among others, in enhancing spatiotemporal stimuli discrimination [12] and in the emergence of connectivity motifs [13]. The basic mechanism of STP can be understood as a continuous Markov chain (Fig. 4e), where the synaptic resources (e.g.: Calcium ions, Ca2+) can be in three possible states: active, inactive or recovered. The amplitude of the postsynaptic potential, i.e. the response of the post-synaptic neuron to a spike emitted by the pre-synaptic neuron depends on the amount of available (active) resources at the synapse. When a signal (spike) is emitted by the pre-synaptic neuron, these resources are used with a certain time constant τin and then become inactive. Subsequently, the resources will recover with a time constant τrec. Once they are recovered they can be released again upon the arrival of a new incoming spike (input to the post-synaptic neuron) with certain release probability USE. This is equivalent to a set of three differential equations describing this effect [14]. Depending of the values of the parameters various behaviors can be observed. For instance, a Figure 5. Fitting of the memristor behaviour to STP. a) Appearance of a biologically plausible train of presynaptic spikes. b) Input to the memristor, where each spike in (a) is represented by 10 short voltage pulses. c) Solid line: memristor response to the input shown in (b). Crosses: STP fitting given spikes timed as in (a), with ÛSE=0.8, τrec=3900, τin=1, τfac=30. Lower line: weight given to each point in the fitting function, one for the spike times, and zero elsewhere. release probability close to the unit and a large recovery time constant, short-term depression is obtained, where the postsynaptic potential amplitude decreases upon consecutive presynaptic stimulation (Fig. 4c). This is due to a great initial usage of the resources, which do not recover sufficiently fast to be available on the following input spike. On the other hand, a small release probability together with a low recovery time lead to short-term facilitation, where the postsynaptic potential amplitude increases with consecutive stimulation (Fig. 4d). In this case, the resources built up as a result of a fast recovery rate. The equations that model STP and were introduced by Markram and Tsodyks in [14] are: dx z   U SE  x   (t  t sp ) dt τ rec dy y   U SE  x   (t  t sp ) τ in dt (1) (2) dz y z   dt τ in τ rec (3) dU SE U   SE  Û SE  1  U SE   (t  t sp ) dt τ fac (4) where x, y, z are the fractions of resources in the recovered, active and inactive states, respectively, USE is the probability of release, ÛSE determines the increase in the value of USE at each spike, τin, τrec and τfac are the inactivation, recovery and facilitation time constants respectively, tsp is the arrival time of 427 a spike emitted by the pre-synaptic neuron and δ(t) is the Dirac delta function. In our preliminary experiments we represented each presynaptic spike as a series of 10 short voltage pulses applied to the memristor (Fig. 5a). When stimulated by a train of closely spaced spikes, the conductance of the material (solid line in Fig. 5b reproduces what a depressing synapse would do if presented with consecutive presynaptic spikes. We use the STP model described above, to show how the inherent behavior and volatility of the memristor could reproduce the response of a dynamic synapse. We have fitted the STP model to the experimental by minimizing the squared difference to the characteristic points of the curve (conductance at the time of a spike). The corresponding short-term facilitation behavior is represented in Fig. 5c, showing a great degree of resemblance between the conductance of the memristor (solid line) and the synaptic efficacy of a depressing neural connection (crosses). The error, ε, that we minimized is shown in (5), with α(t) being the weight given to each point (lower line, Fig. 5c, d(t) being the memristor measurement and f(t) the STP model. (t )(d (t )  f (t ))2  (5) [3] [4] [5] [6] [7] [8] [10] CONCLUSION In this paper we have presented solid-state TiO2 based memristors that exhibit volatility. Contrary to the needs of memory elements, we deliberately employ this volatility behavior to emulate the short-term dynamics of biological synapses. Our experimental results were fitted with an existing synaptic model demonstrating the potential of nanoscale inorganic cells in resembling “dynamic synapses” and particularly short-term depression. This work opens the road for harnessing such elements in devising large-scale systems with notable benefits in enhancing the current state-of-art in temporal processing. V. [2] [9] t IV. VI. [1] [11] [12] [13] [14] ACKNOWLEDGMENTS The authors wish to acknowledge the financial support of the CHIST-ERA ERA-Net, EPSRC EP/J00801X/1 and eFutures XD EFXD12003-4. 428 REFERENCES M. Rozenberg, I. Inoue, and M. Sánchez, “Nonvolatile Memory with Multilevel Switching: A Basic Model,” Phys. Rev. Lett., vol. 92, no. 17, p. 178302, Apr. 2004. A. Gelencsér, T. Prodromakis, C. Toumazou, and T. Roska, “Biomimetic model of the outer plexiform layer by incorporating memristive devices,” Phys. Rev. E, vol. 85, no. 4, Apr. 2012. Q. Xia, W. Robinett, M. Cumbie, and N. Banerjee, “Memristor–CMOS hybrid integrated circuits for reconfigurable logic,” Nano Lett, 2009. C. Zamarreño-Ramos, L. A. Camuñas-Mesa, J. A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona, and B. Linares-Barranco, “On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex,” Frontiers in neuroscience, vol. 5, 2011. S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviya, P. Mazumder, and W. Lu, “Nanoscale Memristor Device as Synapse in Neuromorphic Systems,” Nano Lett, vol. 10, no. 4, pp. 1297–1301, Apr. 2010. B. Govoreanu et al., “10×10nm2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation,” International Technical Digest on Electron Devices Meeting, pp. 31– 34, Dec. 2011. L. Chua, “Memristor - Missing Circuit Element,” IEEE Trans. Circuit Theory, vol. 18, no. 5, pp. 507–&, 1971. D. B. Strukov, D. B. Strukov, G. S. Snider, G. S. Snider, D. R. Stewart, D. R. Stewart, R. S. Williams, and R. S. Williams, “The missing memristor found,” Nature, vol. 453, no. 7191, pp. 80–83, May 2008. T. Prodromakis, C. Toumazou, and L. Chua, “Two centuries of memristors,” Nature Materials, vol. 11, no. 6, pp. 478–481, 2012. H. Markram and M. Tsodyks, “Redistribution of synaptic efficacy between neocortical pyramidal neurons,” Nature, vol. 382, no. 6594, pp. 807–810, 1996. W. Maass and A. Zador, “Dynamic stochastic synapses as computational units,” Neural computation, vol. 11, no. 4, pp. 903–917, 1999. T. Carvalho, D. Buonomano, “A novel learning rule for long-term plasticity of short-term synaptic plasticity enhances temporal processing”, Frontiers in Integrative Neuroscience, 2011. E. Vasilaki, M. Giugliano, “Emergence of Connectivity Patterns from Long-term and Short-term Plasticities”, ICANN 2012, Part I, LNCS Vol 7552. M. Tsodyks, H. Markram, “The neural code between neocortical pyramidal neurons depends on neurotransmitter release probability”, Proc. Natl. Acad. Sci, 1997.