Statistical mechanics of neocortical interactions: Constraints on 40-Hz models of short-term memory Lester Ingber Lester Ingber Research, P.O.

Box 857, McLean, Virginia 22101 [email protected] Calculations presented in L. Ingber and P.L. Nunez, Phys. Rev. E 51, 5074 (1995) detailed the evolution of short-term memory in the neocortex, supporting the empirical 72 rule of constraints on the capacity of neocortical processing. These results are given further support when other recent models of 40-Hz subcycles of low-frequency oscillations are considered.

PACS Nos.: 87.10.+e, 05.40.+j, 02.50.-r, 02.70.-c

Statistical Mechanics of Neocortical ...

-2I. BACKGROUND

Lester Ingber

In a recent paper [1], high-resolution path-integral calculations of a previously developed model of short-term memory (STM) in the neocortex were presented. These calculations, made possible with supercomputer resources, supplanted similar calculations [2] and supported coarser estimates made a decade earlier [3,4]. A more detailed description of the theoretical basis for these calculations and description of other phenomena, e.g., electroencephalographic (EEG) data, can be found in previous papers in this series of statistical mechanics of neocortical interactions (SMNI) [2,5-16]. The SMNI approach is to develop mesoscopic scales of neuronal interactions at columnar levels of hundreds of neurons from the statistical mechanics of relatively microscopic interactions at neuronal and synaptic scales, poised to study relatively macroscopic dynamics at regional scales as measured by scalp EEG. In SMNI, the derived rings of columnar activity, considered as order parameters of the mesoscopic system, develop multiple attractors, which illuminate attractors that may be present in the macroscopic regional dynamics of the neocortex. The SMNI calculations are of minicolumnar interactions among hundreds of neurons, within a macrocolumnar extent of hundreds of thousands of neurons. Such interactions take place on time scales of several , where is on the order of 10 msec (of the order of time constants of cortical pyramidal cells). This also is the observed time scale of the dynamics of STM. We hypothesized that columnar interactions within and/or between regions containing many millions of neurons are responsible for phenomena at time scales of several seconds. The nonlinear evolution was calculated at relatively ner temporal scales, giving a base of support for the phenomena observed at the coarser temporal scales, e.g., by establishing mesoscopic attractors at many macrocolumnar spatial locations to process patterns at larger region domains. As pointed out in a recent paper, long-ranged minicolumnar circuitry across regions and across macrocolumns within regions is quite important in the neocortex and those SMNI calculations only represented a model of minicolumnar interactions within a macrocolumn [1]. Therefore, only the rst few foldings should be considered as having much physical signicance. It must be assumed that other effects, e.g., those contained in long-ranged neocortical circuitry, should sufce to replenish these

Statistical Mechanics of Neocortical ...

-3-

Lester Ingber

memories, to account for the observed durations of many tenths of a second to many seconds of STM. A recent proposal has been advanced that STM is processed by information coded in approximately 40-Hz (approximately 2.5 foldings of ) bursts per stored memory, permitting up to seven such memories to be processed serially within single waves of lower frequencies on the order of 5 to 12 Hz [17]. To account for the observed duration of STM, those authors propose that observed after-depolarization (ADP) at synaptic sites, affected by the action of relatively long-time acting neuromodulators, e.g., acetylcholine and serotonin, acts to regularly refresh the stored memories in subsequent oscillatory cycles. A recent study of the action of neuromodulators in the neocortex supports the premise of their effects on broad spatial and temporal scales [18], but the ADP model is much more specic in its proposed spatial and temporal inuences. Section II gives a set of comparisons between the SMNI and ADP approaches, organized according to several experimental issues. Section III presents the short but important conclusion that a conuence of models gives us a deeper understanding of STM, SMNIs presenting statistical constraints on large-scale neuronal interactions, while other ner-resolution approaches present plausible models for specic neuronal interactions.

II. SMNI AND ADP APPROACHES

A. 72 rule STM is observed to transpire on the order of tenths of a second to seconds, generally limited to the retention of 7 2 items [19]. This is true even for apparently exceptional memory performers who, while they may be capable of more efcient encoding and retrieval of STM, and while they may be more efcient in chunking larger patterns of information into single items, nevertheless are limited to a STM capacity of 7 2 items [20]. Mechanisms for various STM phenomena have been proposed across many spatial scales [21].

Statistical Mechanics of Neocortical ...

-4-

Lester Ingber

SMNI addresses the 72 rule as the number of attractors possible in minicolumnar interactions, using synaptic and neuronal parameters taken from experimental data. These attractors are most-likely states of an evolving conditional probability distribution of columnar rings. The ADP model is based on neural network simulations. The model is based on the memory storage possible within single neurons, using changes in phases of cell rings to store information across the network. Both assumptions must of course be considered as tentative, awaiting more conclusive experimental conrmation than that offered in the present formulation of the ADP model. Furthermore, it is well known from many neural network studies that ensembles of neurons can simultaneously support multiple memories [22], so that it is unlikely that just serially connecting 40-Hz subcycles within larger oscillatory cycles can provide the 72 constraint. The SMNI approach gives a consistent interpretation of STM across the many observed levels of information processes that presumably take place across most of the neocortex. For example, this is the reasonable interpretation of how the above phenomenon of chunking is made possible at various levels of abstraction. The ADP approach must require similar neuromodulator controls over all regions of the neocortex, which does not seem likely [18]. The SMNI approach by itself fails to account for the duration of STM. Early papers [3,4], suggested the possibility of sustenance of STM over epochs of tens of seconds just due to localized columnar interactions, but it was already clear that longer-ranged inuences were important to the development of the SMNI approach [5-7]. More recent calculations show that this duration of STM may not be possible if only localized columnar interactions are considered [1,2]. After approximately 5 , the clear separation between peaks of most-likely states in the evolving conditional probability distribution soon overlap. After approximately 10 , the separation hardly exists. All four models considered, representing dominant inhibition, dominant excitation, a balanced case in between these two (model BC illustrated here), and the latter for the visual neocortex, exhibit similar decays of their peaks over these time scales. Future calculations, including all nonlinear SMNI effects might change this numerical result, but still the action of long-

Statistical Mechanics of Neocortical ...

-5-

Lester Ingber

ranged neuron-neuron and diffuse neuromodulator interactions are known to be important to neocortical function, and they must be addressed. The ADP model has an explicit mechanism for sustenance of rings into durations of tens of seconds. Unfortunately, since some neuromodulators can act for much longer times, the ADP model can only put an upper bound on STM duration by limiting the model to specic neuromodulators, which may not affect all neocortex involved with STM. The ADP approach states that observed STM durations are the result of how 40-Hz signals can be stacked within waves of to oscillations (5 to 12 Hz). This appears to be a quite naive view of generation in neocortex, as it is extremely likely that there are many generators of across many scales of interaction [23]; e.g., ranging from hemispheric global resonance effects [24] to local columnar generators [6,7], and to more specic neuronal interactions [25].

B. 42 rule The authors of the ADP model, in their Ref. 25, acknowledge another neuroscientist for making them aware of the pioneering work on memory limits [19]. However, their paper does not mention other pioneering work of the 42 rule [26]. The 72 rule is veried for acoustical STM, but for visual or semantic STM, which typically require longer times for rehearsal in an hypothesized articulatory loop of individual items, STM capacity appears to be limited to 42. The 42 rule is not mentioned in the ADP model. In SMNI, the 42 rule is established by virtue of detailed calculations using the experimental observation that the number of neurons per minicolumn in the visual cortex is about twice that in other regions of the neocortex [1-4]. While each attractor of the conditional probability distribution becomes sharper and more stable, the number of effective attractors is reduced to within the 42 constraint.

Statistical Mechanics of Neocortical ...

-6-

Lester Ingber

C. Primacy versus recency rule Another interesting phenomenon of STM capacity explained by the SMNI is the primacy versus recency effect in STM serial processing, wherein rst-learned items are recalled most error-free, with lastlearned items still more error-free than those in the middle [27]. In the SMNI approach, the basic assumption is made that a pattern of neuronal ring that persists for many cycles is a candidate to store the memory of activity that gave rise to this pattern. If several ring patterns can simultaneously exist, then there is the capability of storing several memories. The short-time conditional probability distribution derived for the neocortex is the primary tool to seek such ring patterns. The deepest minima of the Lagrangian are more likely accessed than the others of this probability distribution, and these valleys are sharper than the others. I.e., they are more readily accessed and sustain their patterns against uctuations more accurately than the relatively more shallow minima. The more recent memories or newer patterns may be presumed to be those having synaptic parameters more recently tuned and/or more actively rehearsed. It is difcult to see how invoking the ADP mechanism can explain the primacy versus frequency rule, since the ADP mechanism assumes that newly acquired memories simply replace old ones, and it also assumes that memories are scanned serially [17]. The primacy versus recency rule is veried for acoustical STM, but visual or semantic STM typically requires longer times for rehearsal in an hypothesized articulatory loop of individual items [26]. Experimental paradigms that do not account for the 42 rule of visual STM of course will tend to lose the recency part of this rule.

D. Large scale inuences of STM The SMNI model develops STM as attractors of the conditional probability of rings, and these attractors are candidates for inuencing large scale circuitry across regions of the neocortex [1]. This concept was tested by demonstrating that these attractors could t EEG data taken under conditions of stressing STM in subjects [13]. The statistical results of that study negated suggestions in the previous

Statistical Mechanics of Neocortical ...

-7-

Lester Ingber

literature that the EEG by itself might determine a genetic predisposition to alcoholism. More recent studies on this clinical problem look for multiple factors beyond just EEG, although EEG may still be an extremely important diagnostic component of such studies. The ADP model has not yet been tested with any clinical source of data.

III. CONFLUENCE OF MODELS SMNI details the capability of minicolumnar activity to develop multiple memories of capacity 72 with duration on the order of tenths of a second. These memories are described as most-likely states of an evolving probability distribution of minicolumnar rings. It also details an explanation of related phenomena, e.g., the observed random access phenomena of these memories, the 42 rule, the primacy versus recency rule, and the inuence of STM in observed EEG patterns. The SMNI does not detail any specic synaptic or neuronal mechanisms that might refresh these most-likely states to reinforce multiple short-term memories. However, the calculated evolution of states is consistent with the observation that an oscillatory subcycle of 40 Hz may be the bare minimal threshold of self-sustaining minicolumnar rings before they begin to degrade. The mechanism of ADP details a specic synaptic mechanism that, when coupled with additional proposals of neuronal oscillatory cycles of 512 Hz and oscillatory subcycles of 40 Hz, can sustain these memories for longer durations on the order of seconds. By itself, ADP does not provide a constraint such as the 72 rule. The ADP approach does not address the observed random access phenomena of these memories, the 42 rule, the primacy versus recency rule, or the inuence of STM in observed EEG patterns. The SMNI and ADP models are complementary to the understanding of STM. Research is in progress to implement various synaptic mechanics into the SMNI framework. For example, the SMNI approach has been proposed to study minicolumnar interactions across macrocolumns and across regions. This could be approached with a mesoscopic neural network using a conuence of techniques drawn from the SMNI, modern methods of functional stochastic calculus dening nonlinear Lagrangians, adaptive

Statistical Mechanics of Neocortical ...

-8-

Lester Ingber

simulated annealing (ASA) [28], and parallel-processing computation, to develop a generic nonlinear stochastic mesoscopic neural network (MNN) [14]. The MNN can be used to overlap the spatial scales studied by the SMNI with the ner spatial scales typically studied by other relatively more microscopic neural networks. At this scale, such models as ADP are candidates for providing an extended duration of ring patterns within the microscopic networks.

Statistical Mechanics of Neocortical ...

-9REFERENCES

Lester Ingber

[1]

L. Ingber and P.L. Nunez, Statistical mechanics of neocortical interactions: High resolution pathintegral calculation of short-term memory, Phys. Rev. E 51, 5074-5083 (1995).

[2]

L. Ingber, Statistical mechanics of neocortical interactions: Path-integral evolution of short-term memory, Phys. Rev. E 49, 4652-4664 (1994).

[3]

L. Ingber, Statistical mechanics of neocortical interactions. Derivation of short-term-memory capacity, Phys. Rev. A 29, 3346-3358 (1984).

[4]

L. Ingber, Statistical mechanics of neocortical interactions: Stability and duration of the 72 rule of short-term-memory capacity, Phys. Rev. A 31, 1183-1186 (1985).

[5] [6]

L. Ingber, Towards a unied brain theory, J. Social Biol. Struct. 4, 211-224 (1981). L. Ingber, Statistical mechanics of neocortical interactions. I. Basic formulation, Physica D 5, 83-107 (1982).

[7]

L. Ingber, Statistical mechanics of neocortical interactions. Dynamics of synaptic modication, Phys. Rev. A 28, 395-416 (1983).

[8]

L. Ingber, Statistical mechanics of neocortical interactions. EEG dispersion relations, IEEE Trans. Biomed. Eng. 32, 91-94 (1985).

[9]

L. Ingber, Towards clinical applications of statistical mechanics of neocortical interactions, Innov. Tech. Biol. Med. 6, 753-758 (1985).

[10] L. Ingber, Applications of biological intelligence to Command, Control and Communications, in Computer Simulation in Brain Science: Proceedings, University of Copenhagen, 20-22 August 1986, ed. by R. Cotterill (Cambridge University Press, London, 1988), p. 513-533. [11] L. Ingber, Mesoscales in neocortex and in command, control and communications (C3) systems, in Systems with Learning and Memory Abilities: Proceedings, University of Paris 15-19 June 1987, ed. by J. Delacour and J.C.S. Levy (Elsevier, Amsterdam, 1988), p. 387-409.

Statistical Mechanics of Neocortical ...

-10-

Lester Ingber

[12] L. Ingber and P.L. Nunez, Multiple scales of statistical physics of neocortex: Application to electroencephalography, Mathl. Comput. Modelling 13, 83-95 (1990). [13] L. Ingber, Statistical mechanics of neocortical interactions: A scaling paradigm applied to electroencephalography, Phys. Rev. A 44, 4017-4060 (1991). [14] L. Ingber, Generic mesoscopic neural networks based on statistical mechanics of neocortical interactions, Phys. Rev. A 45, R2183-R2186 (1992). [15] L. Ingber, Statistical mechanics of multiple scales of neocortical interactions, in Neocortical Dynamics and Human EEG Rhythms, ed. by P.L. Nunez (Oxford University Press, New York, NY, 1995), p. 628-681. [16] L. Ingber, Statistical mechanics of neocortical interactions: Multiple scales of EEG, Electroencephal. clin. Neurophysiol. (to be published) (1996). [17] J.E. Lisman and M.A.P. Idiart, Storage of 7 2 short-term memories in oscillatory subcycles, Science 267, 1512-1515 (1995). [18] R.N. Silberstein, Neuromodulation of neocortical dynamics, in Neocortical Dynamics and Human EEG Rhythms, ed. by P.L. Nunez (Oxford University Press, New York, NY, 1995), p. 628-681. [19] G.A. Miller, The magical number seven, plus or minus two, Psychol. Rev. 63, 81-97 (1956). [20] K.A. Ericsson and W.G. Chase, Exceptional memory, Am. Sci. 70, 607-615 (1982). [21] H. Eichenbaum, Thinking about brain cell assemblies, Science 261, 993-994 (1993). [22] J. Hertz, A. Krogh, and R.G. Palmer, Introduction to the Theory of Neural Computation (AddisonWesley, Redwood City, CA, 1991). [23] P.L. Nunez, Neocortical Dynamics and Human EEG Rhythms (Oxford University Press, New York, NY, 1995).

Statistical Mechanics of Neocortical ...

-11-

Lester Ingber

[24] P.L. Nunez, Electric Fields of the Brain: The Neurophysics of EEG (Oxford University Press, London, 1981). [25] A. Van Rotterdam, F.H. Lopes da Silva, J. van der Ende, M.A. Viergever, and A.J. Hermans, A model of the spatial-temporal characteristics of the alpha rhythm, Bull. Math. Biol. 44, 283-305 (1982). [26] G. Zhang and H.A. Simon, STM capacity for Chinese words and idioms: Chunking and acoustical loop hypotheses, Memory & Cognition 13, 193-201 (1985). [27] B.B. Murdock, Jr., A distributed memory model for serial-order information, Psychol. Rev. 90, 316-338 (1983). [28] L. Ingber, Adaptive Simulated Annealing (ASA), [ftp.alumni.caltech.edu: /pub/ingber/ASA-shar, ASA-shar.Z, ASA.tar.Z, ASA.tar.gz, ASA.zip], Lester Ingber Research, McLean, VA, 1993.