Emotional simulations and depression diagnostics

Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx Contents lists available at ScienceDirect Biologically Inspired Cognitive Architectures journal homepage: www.elsevier.com/locate/bica Research article Emotional simulations and depression diagnostics Max Talanov b,⇑, Jordi Vallverdú a, Bin Hu c, Philip Moore c, Alexander Toschev b, Diana Shatunova b, Anzhela Maganova b, Denis Sedlenko b, Alexey Leukhin b a Universitat Autònoma de Barcelona, Catalonia, Spain Kazan Federal University, Russia c Lanzhou University, China b a r t i c l e i n f o Article history: Received 10 June 2016 Revised 22 September 2016 Accepted 25 September 2016 Available online xxxx Keywords: Dopamine Serotonin Fear Artificial intelligence Simulation Rat brain Affective computing Emotion modelling Neuromodulation a b s t r a c t In this work we propose the following hypothesis: the neuromodulatory mechanisms that control the emotional states of mammals can be translated and re-implemented in a computer by controlling the computational performance of a hosted computational system. In our specific implementation, we represent the simulation of the ‘fear-like’ state based on the three dimensional neuromodulatory model of affects, in this paper ‘affects’ refer to the basic emotional inborn states, inherited from works of Hugo Lövheim. Whilst dopamine controls attention, serotonin is the key for inhibition, and fear is a elicitator for inhibitory and protective processes. This inhibition can promote [in a cognitive system] to blocking behaviour which can be labelled as ’depression’. Therefore, our interest is how to reimplement biomimetically both action-regulators without the computational system to resulting in a ‘failed’ scenario. We have simulated 1000 ms of the dopamine system using NEST Neural Simulation Tool with the rat brain as the model. The results of the simulation experiments are reported with an evaluation to demonstrate the correctness of our hypothesis. Ó 2016 Elsevier B.V. All rights reserved. 1. Introduction The current rapid developments in neurocognitive sciences and new discoveries related to the core mechanisms of natural intelligence have triggered new insights and opportunities in the field of biologically inspired cognitive systems. There are new and reliable data related to a key aspect [previously undervalued or hidden] which builds the entire cognitive processes architecture: emotions (Minsky, 2007). Research has shown that emotions play a significant role in natural intelligence and adaptive behaviour (Damasio, 1999; Picard, Vyzas, & Healey, 2001). Additionally, the intrinsic value of emotions in cognitive processes remains undervalued by researchers who take a behaviourist approach to artificial emotions based on basic observable actions; we term this: the ‘skinnerian’ approach to emotions. This approach basically considers the emotional performance as epiphenomenalist without considering the deep mech- ⇑ Corresponding author. E-mail addresses: [email protected] (M. Talanov), [email protected] (J. Vallverdú), [email protected] (B. Hu), [email protected] (P. Moore), [email protected] (A. Toschev), [email protected] (D. Shatunova), [email protected] (A. Maganova), [email protected] (D. Sedlenko), alexey. [email protected] (A. Leukhin). anisms that are hidden under this black box. This view can be somehow useful for ‘real-time’ emotional detection during Human-Robot Interactions or Human-Computer Interactions. Attempts to design computer emotional architectures have been proposed, consider for example CogAff (Sloman, 1994) or LIDA (Franklin, Madl, D’Mello, & Snaider, 2014) which are architectonically modulatory and whilst they simulate the homeostatic role of emotional mechanisms, they fail to provide an integrative way to implement emotional design into all areas of computational activity. As a departure point of our model, we consider a simple ‘‘fear”, which is necessary to evaluate ‘‘fly-or-fight” actions (Stevenson & Rillich, 2012). Our study focuses on two opposing and complimentary neuromodulators: dopamine and serotonin (Daw, Kakade, & Dayan, 2002). Dopamine is related to brain reward processes, whilst serotonin is implied into aversive or inhibitory processes; used in combination we may design a system that manages ‘flyor-fight’ actions in which several learning procedures could be easily implemented. We argue that our proposal represents a milestone in the creation of a new generation AI intelligence incorporating the capability to create neuromodulatory architectures which can run over several conceptual models, languages and systems. The posited http://dx.doi.org/10.1016/j.bica.2016.09.002 2212-683X/Ó 2016 Elsevier B.V. All rights reserved. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 2 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx approach is designed to model human cognitive processes such as: creating unique and plastic cognitive identities. This is a basic, but solid, first step towards grounded cognitive systems that share across the whole cognitive multi-heuristic architecture the same controlling mechanisms: synthetic neuromodulators. 2. The problem Dating from the seminal work of Rosalind Picard in the mid 1990s (Picard, 1997), interest in emotion(s) has gained significant traction; this is particulary evident where the understanding of the connection between cognition and thinking is concerned (Ahn & Picard, 2006; Damasio, 1999; Picard, 2003). We have indicated the cognitive perspectives which seem to offer the most promise in the context of the reimplementation of emotions in a computised machine (Bridges et al., 2015). Based on these cognitive perspectives, we propose an approach for a neurobiologically plausible model to represent the influence of emotions over neurobiological processes of thinking mapped to the emotional influence over computational processes using neuromodulation. Research, for exmple see: Arbib and Fellous (2004), Picard (1997), Kort, Reilly, and Picard (2001), Ahn and Picard (2006), and Damasio (1994, 1998), has identified that emotions form a fundamental part of human cognition and are important in cognitive processes which include: attention, motivation, strategy selection, mood disposal, and reaction, invention. In this paper we have paid special attention to mental disorders (see Section 3.1 for a detailed discussion). It is clear that emotions and neuromodulators play a crucial role in the diagnosis and management of depression, schizophrenia and addictive personalities which includes drug addition. Once explained, the crucial role of grounded and emotional aspects in cognitive architectures, we have adopted the Lövheim model as a reference for our own architecture. Lövheim (2012) has developed a three-dimensional model of emotions based on monoamine neurotransmitters: serotonin, dopamine, and noradrenaline. The vertexes of the model are affects as defined in the Tomkins ‘‘Theory of affects” (Kelly, 2009). Tomkins (1962, 1963, 1991) referres to basic emotions as ‘‘innate affects”, where affect in his theory, stands for the ‘‘strictly biological portion of emotion”. Basic affects are: enjoyment/joy, interest/excitement, surprise, anger/rage, disgust, distress/anguish, fear/terror, and shame/humiliation. There are basic affects which are managed soley by the internal cognitive processes of an individual, other basic affects are intrinsically related to social interactions; however in all cases ‘‘fly-or-fight” or even neutral states are hormonally controlled. In our previous publications we have demonstrated the attention regulation of dopamine (Vallverdú et al., 2015); in the studu presented in this paper we have extend our implementation with a second monoamine neuromodulator: serotonin (Meltzer, 1998). The serotonin (5-HT) neurotransmitter is main actor in how a body prepares to deal with some perceived danger. Additionally the serotonin neurotransmitter acts as a bridge channel for an action regulation: it controls sexual satiation (Arnone et al., 1995). There is a final aspect related to the serotonin: loss is also related to ageing and several neuropsychiatric diseases (Pfaff, 2002); for this reason we explore the possible outcomes of an implemented bioinspired architecture which uses serotonin. There is extensive evidence which implicates a deficit in serotonergic neurotransmission in the development of stress or major depression; this is related to the combination of disturbances in cholinergic and serotonergic functions. We use a naturalistically inspired model of cognition where the management of a selec tion/suppression/reinforcement actions are neuromodulatory managed following a complex flow of hormonal interactions. 3. Illustrative scenario The aim of intelligent systems is to realise set goals in dynamic environments such as the healthcare domain. The concept of self has been identified along with the internalised and externalised concepts in respect of self forms an important component in cognitive modelling and emotive response; this has synergies with the two types of stimuli which are: (1) internal stimuli (events triggered by an individual with a response from their external environment) and (2) external stimuli (events triggered in their external environment which prompt an emotional response. In this paper we are addressing emotive response to events (stimuli) with a particular focus on the healthcare domain and mental disorders. In the following illustrative scenarios we discuss two specific areas in relation to mental disorders: (1) depressive states and (2) Schizophrenia; the concept of self forms an important element in such mental disorders in terms of the diagnosis and management in terms of short, medium, and long term time scales. We argue that emotive responses are central to this process. 3.1. Depression and depressive states Depression is a prototypical mental disorder and we argue that cognitive factors, taken with physical symptoms, forms a pivotal feature is the diagnosis and treatment of the condition. Depression is a feature of a number of mental disorders (American Psychiatric Association, 2013) we may however view depression on a spectrum as, whilst it may be clinical depression, it may also be a normal reaction to life events or side effects as a result of drugs and medical treatments. Recent evidence has suggested that recurrent episodes of severe depression are associated with changes in brain function that further heighten vulnerability and functional impairment. It is argued in Thase (1999) that an: ‘‘integrated approach” to diagnosis and management imbues profound beneficial effects for all stakeholders in the treatment of depression. There is a very large (and growing) body of of documented research in the literature addressing depression and depressive states, for example see: Dyrbye, Thomas, and Shanafelt (2006), Lawlor and Hopker (2001), Blatt (2004), Joiner, Coyne, and Blalock (1999), Blanchard, Waterreus, and Mann (1994), Thase (1999), Greenberg and Watson (2006), Spitzer, Md, and Williams (1980), and Sandra (1997). Depression is a highly variable condition that impacts individuals across demographics and gender (Blanchard et al., 1994; Thase, 1999) and Greenberg in Greenberg and Watson (2006) introduces the emotional component into the treatment of depression. Depression is a highly variable condition that impacts individuals across demographics and gender (Blanchard et al., 1994; Thase, 1999); Greenberg in Greenberg and Watson (2006) introduces the emotional component into the treatment of depression. We have considered the concept of self self in both internalised and externalised forms and this forms an important element in such mental disorders in terms of the diagnosis and management (both short and long term) of the conditions and we argue that emotive responses are central to this process. Depression and depressive states reflect an individuals low mood and aversion to activity such that there can be significant affects in terms of an individuals thoughts, behaviour, feelings and sense of well-being (American Psychiatric Association, 2013; Sandra, 1997; Spitzer et al., 1980). Depression may trigger mood swings including feeling: sad, anxious, empty, hopeless, helpless, worthless, guilty, irritable, ashamed or restless. Additionally, interest in activities (that were previously enjoyed) along with eating disorders, problems in concentration, remembering details or making decisions. In extreme cases, individuals suffering from depression Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx [along a spectrum of depressive states] may contemplate, attempt or commit suicide. Insomnia, excessive sleeping, fatigue, aches, pains, digestive problems or reduced energy may also be present (NIMH, 2015a). There is a clear synergy between these observations and the emotive responses identified by behavioural biologists such as Plutchik in Plutchik (1980) (see Section 4) where similar negative and positive reactions are discussed. In this paper we are interested in the cognitive aspects of depression and it’s treatment. Whilst an ‘‘integrated approach” Thase (1999) may include: (1) cognitive behavioural therapies (CBT) Beck (2011) using both face-to-face and remote (possibly on-line) consultations and (2) drug treatments, our aim is to provide an effective basis upon which the cognitive and physiological factors that relate to depressive states are captured and processed to improve the diagnosis and treatment of depression. Moreover, our aim is to include emotional response (based on objective data (much of the cognitive response data is subjective) using cognitive modelling) to stimuli in the diagnosis and treatment of patients in the depressive spectrum as this aspect of the condition is, we argue, central to understanding depression. In considering depression we may also identify Alzhieimers’ disease and dementia as a primary cause [of depression] (Walker et al., 2012). In this research the Cornell Scale for Depression in Dementia (CSDD) which is a scale specifically developed to assess signs and symptoms of major depression in patients with dementia (Alexopoulos, Abrams, Young, & Shamoian, 1988). The mood related factors identified in the Cornell Scale do, in fact, bear a similarity to the Behavioural and Psychological Symptoms of Dementia (BPSD) as discussed in Qassem, Tadros, Moore, and Xhafa (2014) and Moore, Thomas, Tadros, Xhafa, and Barolli (2013). Clearly, the impact of dementia may be reflected in depression but also it may be interesting to note that the research we present in this paper may be useful in the management of dementia. Recent evidence has suggested that recurrent episodes of severe depression are associated with changes in brain function that further heighten vulnerability and functional impairment. Thase in Thase (1999) argues that: ‘‘integrated approach to preventative therapy saves lives and has profoundly beneficial effects for our patients, their loved ones, and society”. 3.2. Schizophrenia As with depression there is a very large (and growing) body of of documented research in the literature addressing Schizophrenia, for example see: Saha, Chant, Welham, and McGrath (2005), McGorry (2000), Wolf and Berle (1976), Bleuler (1968), Wynne (1978), and NIMH (2015b). Schizophrenia is a mental disorder often characterised by abnormal social behaviour and failure to recognise ‘what is real’. Common symptoms include: false beliefs, unclear or confused thinking, auditory hallucinations, reduced social engagement and emotional expression, and inactivity. Diagnosis is based on observed behaviour and the person’s reported experiences. Precursors to Schizophrenia include: genetic, environmental, and factors psychological and social processes; such precursors appear to be important contributory factors (Singh, Burns, Amin, Jones, & Harrison (2004) and Large, Ryan, Singh, Paton, & Nielssen (2011)) and importantly, there are recognised issues with recreational and prescription drugs appear to cause or exacerbate symptoms. The derivation of the term Schizophrenia is Greek (skhizein - ‘‘to split”, and phren - ‘‘mind”). Notwithstanding the genesis of the term, Schizophrenia does not imply a ‘‘split personality”, or a ‘‘multiple personality disorder” (a condition with which it is often confused in public perception of the condition). Rather, the term means a ”splitting of mental functions”, reflecting the presentation of the illness (Baucum, 2006). 3 The mainstay of treatment is antipsychotic medication, which primarily (and importantly for the discussion presented in this paper) suppresses dopamine receptor activity. Counseling, job training and social rehabilitation are also important in treatment. In more serious cases (where there are risks asociated with self harm or harm to others) patients may be subject to involuntary hospitalisation (Becker & Kilian, 2006). It has been argued that symptoms may begin in young adulthood with around 0.3–0.7% of people being affected during their lifetime (van Os & Kapur, 2009). The disorder is thought to mainly affect the ability to think; however it also usually contributes to chronic problems with behaviour and emotion. People with schizophrenia are likely to have additional conditions, including major depression and anxiety disorders; the lifetime occurrence of substance use disorder is almost 50% (Buckley, Miller, Lehrer, & Castle, 2009). The average life expectancy of people with the disorder is ten to twenty-five years less than the average life expectancy (Laursen, Munk-Olsen, & Vestergaard, 2012) and this may be the result of increased physical health problems and a higher suicide rate (about 5%) (Hor & Taylor, 2010; van Os & Kapur, 2009). It has been estimated that in 2013 16,000 people died from behaviour related-to or caused by schizophrenia (Naghavi et al., 2015). In considering treatment options (see Section 3.3) Swaran Singh has argued in Singh (2010) that: ‘‘Early intervention in psychosis services produce better clinical outcomes than generic teams and are also cost-effective. Clinical gains made within such services are robust as long as the interventions are actively provided. Longer-term data show that some of these gains are lost when care is transferred back to generic teams. This paper argues that sustaining these early gains requires both a reappraisal of generic services and an understanding of the active ingredients of early intervention, which can be tailored for longer input in cases with poorer outcome trajectories.” As evidence accumulates, implementation of evidence-based practice in real work settings is a major challenge as it is throughout the mental health service system. The momentum of preventively orientated treatment must be maintained through the 2nd National Mental Health Strategy and in the face of recent misleading polemic regarding the treatability of psychotic disorders, especially schizophrenia. The evidence demonstrates that schizophrenia and related disorders have never been more treatable (McGorry, 2000). Furthermore, aspects of the minimal self that involve senses of ownership and agency in the context of both motor action and cognition can be clarified by neurocognitive models of schizophrenia that suggest the involvement of specific brain systems (including prefrontal cortex, supplementary motor area, and cerebellum) in the manifestation of neurological symptoms in this disorder (Gallagher, 2000). Primary prevention requires a sophisticated knowledge of key causal risk factors relevant to the expression of a disorder. The causal risk factors most useful from an intervention standpoint may turn out to be somewhat removed from the neurobiology of the disorder and may even be relatively non-specific, so that tackling them could reduce the risks for a range of mental disorders. The frontier for more specific prevention in schizophrenia and related psychosis is currently represented by indicated preventive interventions for subthreshold symptoms. Again, these may be relatively broad spectrum early in the prepsychotic phase but more proximal to onset, greater treatment specificity can be explored. However, this can be viewed more as preventively orientated treatment rather than primary prevention per se. Early detection of first episode psychosis and optimal intensive treatment of first episodes and the critical early years after diagnosis also represent increasingly attractive preventive foci in psychotic disorders. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 4 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx In relation to a neurocognitive model of schizophrenia that requires us to make a distinction between two aspects of the minimal sense of self: the sense of self-ownership and the sense of selfagency (Gallagher, 2000). Schizophrenic patients who suffer from thought insertion and delusions of control also have problems with this forward, pre-action monitoring of movement, but not with motor control based on a comparison of intended movement and sensory feedback. The control based on sensory feedback is thought to involve the cerebellum. By contrast, problems with forward monitoring are consistent with studies of schizophrenia that show abnormal pre-movement brain potentials associated with elements of a neural network involving supplementary motor, premotor and prefrontal cortex. Problems with these mechanisms might therefore result in the lack of a sense of agency that is characteristic of these kinds of schizophrenic experience. 3.3. Non-drug related treatment strategies The treatment options for the management of depression and schzophrenia will include both drug and non-drug related therapies (Roshanaei-Moghaddam et al., 2011) and must include cognitive and emotional factors (Schupp, Junghöfer, Weike, & Hamm, 2003) including affective factors (Lang, Greenwald, Bradley, & Hamm, 1993). In a meta-analysis of 56 studies related to the relationship between psychotherapy and drug therapy in the treatment of unipolar depression in adults (Roshanaei-Moghaddam et al., 2011), the research findings suggest that psychotherapy was superior to drug therapies (Roshanaei-Moghaddam et al., 2011). An interesting novel technology is Transcranial Magnetic Stimulation (TMS) therapy which has been widely used and investigated in depression (Lopez-Ibor, López-Ibor, & Pastrana, 2008). In 2007 TMS was approved for treatment-resistant depression in Canada, Australia, New Zealand, the European Union and Israel; in other countries it is undergoing trials and remains largely experimental. A multi-modal [integrated] approach to the management of depression will generally include both psychotherapy and drug therapy with consultations conducted on both a F2F and remote basis which will include the monitoring of patients emotional and physical current prevailing state which will include the monitoring of medication and its effect. 4. Our idea To reimplemet emotional psychoneurobiological phenomena in the machine we used neurobiologically plausible simulation approach, this was needed to keep the connection with biological nature of emotions. We have created the simulation of the brain pathways or subsystems that plays important role in the neurobiological emotional processes reimplementing emotion-like processes on a cellular level. Based on roles of the neuromodulators we have created a mapping from one neuropsychological model of affects in order to implement it into computing system parameters. Fig. 1 shows the dimensional model of mapping in which levels of neuromodulators are mapped to simulate ‘‘innate emotional states” or affects. For a detailed exposition see our previous articles: Bridges et al. (2015), Talanov, Vallverdú, Distefano, Mazzara, and Delhibabu (2015), and Vallverdú et al. (2015). In summary and more specifically, the parameters of a computational system taken into account are as follows: Computing utilisation is a metric able to quantify how busy the processing resources of the system are. It can be expressed by the average value of all the single processing resources’ utilisation. Fig. 1. The correlation of emotional states with monoamines levels and computational system parameters, based on Lövheim Cube of emotion (Lövheim, 2012). Computing distribution aims at quantifying the load balancing amongst processing resources. It can be expressed as the variance of single resources’ utilisation. Memory distribution is associated with the amount of memory allocated to the processing resources. It can be quantified by the variance of the amount of memory per single resource. Storage volume is an index related to the the amount of data and information used by the system. Storage bandwidth quantifies the number of connections between resources, i.e. processing and data nodes. Human depressive states are associated with low serotonin levels which promote inhibitory states which are understood as depressive states. In computational systems, the depressive state of all computing parameters along to serotonin axis is close to the lowest operational state. A depressive state can be triggered by a long running computational process which is not completed and should be [normally] terminated, the reason being that it will run infinitely without any realistic probability of a successful conclusion. Alternatively, any successful process can trigger off depression state. We recognise that the nature of the depression is more complex than the present serotonin modelling may realise. However, it is also true that in considering the fundamentamechanistic processes, direct correlations to levels of neuromodulators can be indicated and employed to control basic excitatory/inhibitory processes management. Thus, we are not claiming to emulate computationally depressive states; we suggest a biomimetic computational mechanism to implement neuromodulatory-like strategies which include inhibitory processes. 5. Details on modelling dopamine and serotonin pathways Four practical purposes, our model is predicated on the dopamine and serotonin systems of a rat brain based on works of: Sprague-Dawley and Long-Evans (Nair-Roberts et al., 2008; Sadowski, Wise, Park, Schantz, & Juraska, 2014; Wang & Morales, 2009). We have modelled two main sub-systems involved in the ‘‘cube of emotions” described in Section 4. The sub-systems are:
Dopamine pathways: – Nigrostriatal pathway including thalamus, striatum and substantia nigra. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx – Mesolimbic and mesocortical pathways from VTA to prefrontal cortex.
Serotonin pathways: – Middle pathway from raphe neurons to cerebral cortex, cerebellum, basal ganglia. – Rostral pathway from raphe nuclei to 5 areas: uses 5HT1 ‘‘slow inhibition” and 5HT2 ‘‘slow excitation”. Pathways or sub-systems were selected based on their role in the emotional processing of a mammalian brain. As indicated above (see Section 4), dopamine and serotonin are key players in an emotional picture of mammalian neurobiology and selected pathways play important role in a mammalian thinking that we reference as being analogous to computational processes of modern computers. 5.1. The dopamine pathway In previous research we have have based our studies on the dopamine sub-system because it: (1) allows the simplest implementation in a realistic neural network and (2) controls basic features fundamental for the regulation of learning processes. The approximate principal schema of the dopaminergic nigrostriatal, mesolimbic and mesocortical pathways is presented on Fig. 2. The majority of the neurons in a striatum are GABA inhibitory cells and have dopamine receptors. The dopamine has a dual influence on striatum neurons as it: (1) inhibits the (D2-type) in the indirect pathway and (2) excites (D1-type) in the direct pathway. When dopamine is low [in the striatum] the indirect pathway becomes overactive and the direct pathway becomes underactive, thus balancing the distribution of neuronal activities between two parts of the nigrostriatal pathway. The implementation of the dopamine pathways is depicted on Fig. 3 where the pathways are indicated by text on arrows: 1. ‘‘Direct”: Cerebral cortex (stimulate) ? Striatum (inhibit) ? complex SNr-GPi (Thalamus is less inhibited) ? Thalamus (stimulate) ? Cerebral cortex (stimulate) ? muscles and etc. 2. ‘‘Indirect”: Cerebral cortex (stimulate) ? Striatum (inhibit) ? GPe (STN is less inhibited) ? STN (stimulate) ? complex SNrGPi (inhibit) ? Thalamus (is less stimulated) ? Cerebral cortex (is less stimulated) ? muscles and etc. 5 5.2. The serotonin pathways The diagram presented in Fig. 4 depicts the overall modelled structure of the serotonin pathways. Efferent projections from the rostral group to raphe nucleus, cerebral cortex, thalamus, basal forebrain (includes septum), hypothalamus, locus coeruleus, ventral tegmental area, limbic system (includes hippocampus, amygdala), basal ganglia, such as neostriatum, substanta nigra and nucleus accumbens, also to other nucleus, e.g. trigeminal nucleus, facial nucleus, laterodoral tegmental nucleus (Müller & Jacobs, 2010). Afferent projections to the raphe nucleus are from ventral tegmental area, limbic system, basal ganglia, cerebral cortex, basal forebrain, hypothalamus, other nucleus, periaqueductal grey and reticular formation (Müller & Jacobs, 2010). Neurons of the serotonin pathways project to different parts of a brain and interact with other neurotransmitter pathways. The serotonergic projection from the dorsal raphe nucleus (DR) to the locus coeruleus (LC) which plays an inhibitory role. Serotonin transmission affects the dopamine pathways. The Activation of serotonin receptors in the prefrontal cortex (PFC) and in the nucleus accumbens stimulates the dopamine release. The activity of dopaminergic neurons of the ventral tegmental area (VTA) is under the excitatory control of 5-HT receptors located in the PFC. The effect on the dopamine system in the VTA is dependent on the type of activated serotonin receptor. The 5  HT1A, 5  HT3 receptor agonists increasing dopamine release in the VTA and PFC and the 5  HT2C receptor agonists decreasing dopamine release [in the VTA and PFC. In the substantia nigra, nucleus accumbens and in the striatum serotonin antagonizes the dopamine function (Daw et al., 2002; Müller & Jacobs, 2010). 5.3. The noradrenaline pathways The diagram 5 depicts a simplified structure of the schema of the simulation of noradrenaline pathways as discussed in following sections where we consider: (1) the motor cortex, (2) the bed nucleus of the stria terminal, (3) the raphe nuclei, (4) the perirhinal cortex and Nucleus paragigantocellularis lateral, (5) the lateral dorsal tegmental nucleus, (6) the nucleus tractus solitarii, and (7) the locus coeruleus. Fig. 2. The dopamine system of a rat brain (Kourrich et al., 2015). SNc - substantia nigra pars compacta, SNr - substantia nigra pars reticulata, STN - subthalamic nucleus, GPe - globus pallidus external, GPi - globus pallidus internal, Acb - nucleus accumbens, VTA - ventral tegmental area, PPTg - pedunculopontine tegmental nucleus, Amy - amygdala. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 6 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx Fig. 3. The simplified schema of the simulated dopamine sub-system including: nigrostriatal and mesolimbic and mesocortical pathways. 5.3.1. Motor cortex The motor cortex projects to noradrenaline receptors of locus coeruleus (LC in the diagram) (Ferrucci, Giorgi, Bartalucci, Busceti, & Fornai, 2013), via glutamate. 5.3.2. Bed nucleus of the stria terminal Cholinergic innervation to the bed nucleus of the stria terminal (BNST) is provided by the latero-dorsal tegmental nucleus, and the BNST in its turn projects to the amygdala. Glutamatergic inputs to the BNST originate from neurons located in the prefrontal cortex (Crestani et al., 2013) and regulate the activity of GABA inhibition. The activity of BNST neurons is regulated by GABAergic inputs from both intrinsic sources. 5.3.3. Raphe nuclei Both a1 - and a2 -adrenergic receptors play a role in mediating noradrenergic regulation of serotonin release in the raphe as well as in raphe projection areas (Ordway, Schwartz, & Frazer, 2007). 5.3.4. Perirhinal cortex and Nucleus paragigantocellularis lateral Prominent afferents to the LC include the nucleus paragigantocellularis (PGi) and the ventromedial aspect of the prepositus hypoglossi (PrH) in the rostroventrolateral and dorsomedial medulla, respectively. These nuclei provide strong excitatory and inhibitory influences on LC neurons (Ordway et al., 2007), respectively, and are also sources of several neurotransmitter inputs to the LC nucleus. The stimulation of the PGi strongly excites the LC neurons. In contrast, strong inhibition is produced by PrH stimulation (Berridge & Waterhouse, 2003). That inhibitory input also arises from the PGi is revealed when the strong glutamate input is antagonized pharmacologically. 5.3.5. Lateral dorsal tegmental nucleus The inbound axons project to the lateral dorsal tegmental nucleus (LDT) on a1 and a2 adrenergic receptors from the LC. In its turn the LDT cholinergic neurons project to the LC where noradrenaline neurons are activated. 5.3.6. Nucleus tractus solitarii There are strong ascending projections of these nucleus tractus solitarii (NTS) NE cells to forebrain areas such as the BNST (into GLUT receptor of BNST), nucleus accumbens (into GABA receptor) (Aston-Jones, 2002). Amygdala also plays important role in affective and cognitive processes. 5.3.7. Locus coeruleus [LC] The striatum receives a small amount of NE projections only via scattered fibres from the LC, nonetheless these striatal afferents seem to possess a high turnover rate. Stimulation DA neurons of VTA increases the activity of the LC (Ferrucci et al., 2013) receiving signals from the D1/D2 receptors and generating further exposure on them. The main afferents to the LC include projections from the prefrontal cortex (activates noradrenergic LC neurons via a glutamate), lateral hypothalamus, raphe nuclei, and amygdala (Aston-Jones, 2002). Furthermore, the LC receives NE afferents from lower medullary A1 and A2 regions. Additionally, LC receives DA afferents from VTA. The a1 - adrenergic receptor, receives signals from the LC and has an intensive influence over most DA neurons activity of the VTA. The PFC activates noradrenergic LC neurons via a glutamate. Inside the PFS there is an underlying inhibitory effect of prefrontal activation when the more potent glutamate-mediated excitation is antagonized (Ordway et al., Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx 7 N Fig. 4. The simplified schema of the serotonin sub-system including: middle and rostral pathways. Fig. 5. The simplified schema of the noradrenaline sub-system. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 8 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx Fig. 6. Experimental results: the computational power used per each 5 ms of the simulation. 2007). Gamma-aminobutyric acid (GABA) has a tonic inhibitory effect on the LC and regulates levels of norepinephrine in the cortex. 6. Results To evaluate and validate the posited approach the simulation is implmemted using the NEST the Neural Simulation tool (Gewaltig & Diesmann, 2007) (NEST) with the current validation being restricted to the dopamine pathways. We trigger the dopamine production in SNc, PPTG, amygdala programmatically on 400 ms of the simulation a shown in Fig. 3. The ‘spike generator’ of the motor cortex is triggered every 20 ms to initiate the neuronal activity. The overall duration of the simulation is 1000 ms during which we determine the reaction of simulated thalamus and motor cortex on the neuromodulation related to dopamine levels as a rise in spiking activity and the following slow decay. Spike detectors are installed in: the motor cortex ‘‘Glu[1]”, the prefrontal cortex, VTA, SNc, the thalamus. Spike generators of PPtg and the amygdala remain active until 600th ms of our simulation. We have registered every 0.1 ms the simulated neuronal activity and computational power consumed to compute every 10 ms of the simulation. The orange or topmost graph on Fig. 6 is the computational power used for the simulation of the rat brain with dopamine neuromodulation. The histogram indicates the frequency of spikes for: green – Motor cortex [Glu1], blue – Thalamus. The computational power shown in Fig. 6 is represented in number of seconds used by the CPU on 100% of utilisation of the machine with running simulation. According to the graph shown in Fig. 6 we indicate the rise of the computational power consumed at 410 ms of the simulation that we correlate with the dopamine neuromodulation that matches our hypothesis of the correlation of fear of a mammalian brain with the computational power consumed of a computational system. 7. Conclusions and future work In this, and previous works Talanov and Toschev (2014) and Vallverdú et al. (2015), we propose that mechanisms similar to bio- logical affects and emotions can be translated and implemented in computerised system parameters in artificial systems. Following the model of H. Lövheim we have considered the creation of dopaminergic, serotonergic and norepinephrinergic neuromodulation systems of mammalian brains in order to design our computational model. Thus, we introduce an ‘emotional-like’ mechanistic approach to artificial reasoning. Our intention is not only emulate the human cognitive architecture for human understanding to: (1) test if such evolutionary approach can be instantiated into computational frameworks and (2) evaluate the potential of this implementation to improve the quality and complexity of their computational/cognitive processes. We have implemented a model of dopamine system of a rat brain using NEST Neural Simulation Tool to evaluate and validate our posited approach in a simulation of 1000 ms of dopamine neuromodulation inside the model, the results identify the increase of computational power exactly during neuromodulation. This can be understood as partial proof of our hypothesis regarding dopamine axis. We have also created a model for the inhibitory role of neurotransmitters, as exemplified by serotonin. As a consequence of both models, dopamine and serotonin, we have demonstrated that both neuromodulators can be successfully implemented into computational environments along with the power of the combination of both synthetic hormones. This approach allows the management of increasingly complex cognitive behaviours into computational decision-taking AI. Whilst our study has identified and addressed a number of challenges we have identified a number of open remaining open research questions. In considering these questions we plan further experimental validation and enhancement of proposed model, modelling and validation of serotonin and norepinephrine systems, their integration, validation of integrated system, development of appraisal models, with implementation in a working cognitive architecture. Acknowledgments Part of the work was performed according to the Russian Government Program of Competitive Growth of Kazan Federal University. Prof. Vallverdu’s research is supported by DGICYT: Innovacion en la prctica cientfica: enfoques cognitivos y sus consecuencias filosficas, (FFI 2011-23238). Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx References Ahn, H., & Picard, R. W. (2006). Affective cognitive learning and decision making: The role of emotions. In The 18th European meeting on cybernetics and systems research (EMCSR 2006). . Alexopoulos, G. S., Abrams, R. C., Young, R. C., & Shamoian, C. A. (1988). Cornell scale for depression in dementia. Biological Psychiatry, 23(3), 271–284. American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorders. Arlington: American Psychiatric Publishing. Arbib, M., & Fellous, J.-M. (2004). Emotions: from brain to robot. Trends in Cognitive Sciences, 8(12), 554–559. Arnone, M., Baroni, M., Gai, J., Guzzi, U., Desclaux, M. F., Keane, P. E., ... Soubrié, P. (1995). Effect of sr 59026a, a new 5-ht(1a) receptor agonist, on sexual activity in male rats. Behavioural Pharmacology, 6, 276–282. Aston-Jones, G. (2002). Norepinephrine. Pennsylvania: University of Pennsylvania School of Medicine. Baucum, D. (2006). Psychiatric services for people with severe mental illness across western Europe: What can be generalized from current knowledge about differences in provision, costs and outcomes of mental health care? Acta Psychiatrica Scandinavica, 113(Supplement 429), 9–16. Beck, J. S. (2011). Cognitive behavior therapy: Basics and beyond. Guilford Press. Becker, T., & Kilian, R. (2006). Psychiatric services for people with severe mental illness across western Europe: What can be generalized from current knowledge about differences in provision, costs and outcomes of mental health care? Acta Psychiatrica Scandinavica, 113(s429), 9–16. Berridge, Craig W., & Waterhouse, B. D. (2003). The locus coeruleusnoradrenergic system: Modulation of behavioral state and state-dependent cognitive processes. Brain Research Reviews (42), 33–84. Blanchard, M., Waterreus, A., & Mann, A. (1994). The nature of depression among older people in inner london, and the contact with primary care. The British Journal of Psychiatry, 164(3), 396–402. Blatt, S. J. (2004). Experiences of depression: Theoretical, clinical, and research perspectives. American Psychological Association. Bleuler, M. (1968). A 23-year longitudinal study of 208 schizophrenics and impressions in regard to the nature of schizophrenia. Journal of Psychiatric Research, 6, 3–12. Bridges, M. W., Distefano, S., Mazzara, M., Minlebaev, M., Talanov, M., & Vallverdú, J. (2015). Towards anthropo-inspired computational systems: The p^3 model. In G. Jezic, R. J. Howlett, & L. C. Jain (Eds.). Smart Innovation, Systems and Technologies (vol. 38, pp. 311–321). Cham: Springer International Publishing. Buckley, P. F., Miller, B. J., Lehrer, D. S., & Castle, D. J. (2009). Psychiatric comorbidities and schizophrenia. Schizophrenia Bulletin, 35(2), 383–402. Crestani, C. C., Alves, F. H., Gomes, F. V., Resstel, L. B., Correa, F. M., & Herman, J. P. (2013). Mechanisms in the bed nucleus of the stria terminalis involved in control of autonomic and neuroendocrine functions: A review. Current Neuropharmacol, 2(11), 141–159. Damasio, A. (1994). Descartes’ error: Emotion, reason, and the human brain. Penguin Books. Damasio, A. R. (1998). Emotion in the perspective of an integrated nervous system. Brain Research Reviews, 26(2–3), 83–86. Damasio, A. (1999). The feeling of what happens: Body and emotion in the making of consciousness. New York. Daw, N. D., Kakade, S., & Dayan, P. (2002). Opponent interactions between serotonin and dopamine. Neural Networks, 15(4), 603–616. Dyrbye, L. N., Thomas, M. R., & Shanafelt, T. D. (2006). Systematic review of depression, anxiety, and other indicators of psychological distress among us and canadian medical students. Academic Medicine, 81(4), 354–373. Ferrucci, M., Giorgi, F. S., Bartalucci, A., Busceti, C. L., & Fornai, F. (2013). The effects of locus coeruleus and norepinephrine in methamphetamine toxicity. Current Neuropharmacol, 1(11), 80–94. Franklin, S., Madl, T., D’Mello, S. K., & Snaider, J. (2014). LIDA: A systems-level architecture for cognition, emotion, and learning. IEEE Transactions on Autonomous Mental Development, 6(1), 19–41. Gallagher, S. (2000). Philosophical conceptions of the self: Implications for cognitive science. Trends in cognitive sciences, 4(1), 14–21. Gewaltig, M.-O., & Diesmann, M. (2007). Nest (neural simulation tool). Scholarpedia, 2(4), 1430. Greenberg, L. S., & Watson, J. C. (2006). Emotion-focused therapy for depression. American Psychological Association. Hor, K., & Taylor, M. (2010). Review: Suicide and schizophrenia: A systematic review of rates and risk factors. Journal of Psychopharmacology, 24(4 suppl), 81–90. Joiner, T., Coyne, J. C., & Blalock, J. (1999). On the interpersonal nature of depression: Overview and synthesis. In The interactional nature of depression: Advances in interpersonal approaches (pp. 3–19). American Psychological Association. Kelly, V. C. (2009). A primer of affect psychology (on-line). Kort, B., Reilly, R., & Picard, R. W. (2001). An affective model of interplay between emotions and learning: Reengineering educational pedagogy-building a learning companion. In Proceedings of international conference on advanced learning technologies (ICALT 2001). . Kourrich, S., Calu, D. J., & Bonci, A. (2015). Intrinsic plasticity: An emerging player in addiction. Nature Reviews Neuroscience, 16(3), 173–184. Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology, 30(3), 261–273. 9 Large, M. M., Ryan, C. J., Singh, S. P., Paton, M. B., & Nielssen, O. B. (2011). The predictive value of risk categorization in schizophrenia. Harvard Review of Psychiatry, 19(1), 25–33. Laursen, T. M., Munk-Olsen, T., & Vestergaard, M. (2012). Life expectancy and cardiovascular mortality in persons with schizophrenia. Current Opinion in Psychiatry, 25(2), 83–88. Lawlor, D. A., & Hopker, S. W. (2001). The effectiveness of exercise as an intervention in the management of depression: Systematic review and metaregression analysis of randomised controlled trials. Bmj, 322(7289), 763. Lopez-Ibor, J. J., López-Ibor, M.-I., & Pastrana, J. I. (2008). Transcranial magnetic stimulation. Current Opinion in Psychiatry, 21(6), 640–644. Lövheim, H. (2012). A new three-dimensional model for emotions and monoamine neurotransmitters. Medical hypotheses, 78(2), 341–348. McGorry, P. D. (2000). The nature of schizophrenia: Signposts to prevention. Australasian Psychiatry, 34(S2), 14–S21. Meltzer, C. e. a. (1998). Serotonin in aging, late-life depression, and alzheimer’s disease: The emerging role of functional imaging. Neuropsychopharmacology, 18 (6), 407–430. Minsky, M. (2007). The emotion machine: Commonsense thinking, artificial intelligence, and the future of the human mind. Simon & Schuster. Moore, P., Thomas, A., Tadros, G., Xhafa, F., & Barolli, L. (2013). Detection of the onset of agitation in patients with dementia: Real-time monitoring and the application of big-data solutions. International Journal of Space-Based and Situated Computing, 3(3), 136–154. Müller, C. P., & Jacobs, B. L. (Eds.). (2010). Handbook of the behavioral neurobiology of serotonin (1st ed.. Number 21 in Handbook of Behavioral Neuroscience (1st ed.). Amsterdam: Elsevier/Academic Press. Naghavi, M., Wang, H., Lozano, R., Davis, A., Liang, X., Zhou, M., ... Aziz, M. I. A. (2015). Global, regional, and national age–sex specifi c all-cause and causespecifi c mortality for 240 causes of death, 1990–2013: A systematic analysis for the global burden of disease study 2013. The Lancet, 385(9963), 117–171. Nair-Roberts, R. G., Chatelain-Badie, S. D., Benson, E., White-Cooper, H., Bolam, J. P., & Ungless, M. A. (2008). Stereological estimates of dopaminergic, GABAergic and glutamatergic neurons in the ventral tegmental area, substantia nigra and retrorubral field in the rat. Neuroscience, 152(4), 1024–1031. NIMH (2015a). What is depression? NIMH (2015b). What is schizophrenia? Ordway, Gregory A., Schwartz, Michael A., & Frazer, A. (2007). Brain norepinephrine: Neurobiology and therapeutics. New York: United States of America by Cambridge University Press. Pfaff, D. e. a. (2002). Hormones, brain and behavior. Academic Press. Picard, R. W. (1997). Affective computing. Massachusets Institute of Technology. Picard, R. W. (2003). Affective computing: Challenges. International Journal of Human-Computer Studies, 59, 55–64. Picard, R. W., Vyzas, E., & Healey, J. (2001). Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10), 1175–1191. Plutchik, R. (1980). Emotion: A psychoevolutionary analysis. Nueva York: Harper and Row. Qassem, T., Tadros, G., Moore, P., & Xhafa, F. (2014). Emerging technologies for monitoring behavioural and psychological symptoms of dementia. In Ninth international conference on P2P, parallel, grid, cloud and internet computing (3PGCIC), 2014 (pp. 308–315). IEEE. Roshanaei-Moghaddam, B., Pauly, M. C., Atkins, D. C., Baldwin, S. A., Stein, M. B., & Roy-Byrne, P. (2011). Relative effects of cbt and pharmacotherapy in depression versus anxiety: Is medication somewhat better for depression, and cbt somewhat better for anxiety? Depression and Anxiety, 28(7), 560–567. Sadowski, R. N., Wise, L. M., Park, P. Y., Schantz, S. L., & Juraska, J. M. (2014). Early exposure to bisphenol A alters neuron and glia number in the rat prefrontal cortex of adult males, but not females. Neuroscience, 279, 122–131. Saha, S., Chant, D., Welham, J., & McGrath, J. (2005). A systematic review of the prevalence of schizophrenia. PLoS Medicine, 2(5), e141. Sandra, S. (1997). Depression: Questions you have-answers you need. People’s Medical Society. Schupp, H. T., Junghöfer, M., Weike, A. I., & Hamm, A. O. (2003). Attention and emotion: An erp analysis of facilitated emotional stimulus processing. Neuroreport, 14(8), 1107–1110. Singh, S. P. (2010). Early intervention in psychosis. The British Journal of Psychiatry, 196(5), 343–345. Singh, S. P., Burns, T., Amin, S., Jones, P. B., & Harrison, G. (2004). Acute and transient psychotic disorders: Precursors, epidemiology, course and outcome. The British Journal of Psychiatry, 185(6), 452–459. Sloman, A. (1994). Computational modelling of motive-management processes. In N. Frijda (Ed.), Proceedings of the conference of the international society for research in emotions (pp. 344–348). Cambridge: ISRE Publications. Spitzer, R. L., Md, K. K., & Williams, J. B. (1980). Diagnostic and statistical manual of mental disorders. Citeseer: American Psychiatric Association. Stevenson, P. A., & Rillich, J. (2012). The decision to fight or flee - Insights into underlying mechanism in crickets. Frontiers in Neuroscience, 6(118). Talanov, M., & Toschev, A. (2014). Computational emotional thinking and virtual neurotransmitters. International Journal of Synthetic Emotions (IJSE), 5(1). Talanov, M., Vallverdú, J., Distefano, S., Mazzara, M., & Delhibabu, R. (2015). Neuromodulating cognitive architecture: Towards biomimetic emotional AI. In 2015 IEEE 29th international conference on advanced information networking and applications, April. vol. 2015 (pp. 587–592). IEEE. Thase, M. E. (1999). Long-term nature of depression. Journal of Clinical Psychiatry. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002 10 M. Talanov et al. / Biologically Inspired Cognitive Architectures xxx (2016) xxx–xxx Tomkins, S. (1962). Affect imagery consciousness volume I the positive affects. New York: Springer Publishing Company. Tomkins, S. (1963). Affect imagery consciousness volume II the negative affects. New York: Springer Publishing Company. Tomkins, S. (1991). Affect imagery consciousness volume III the negative affects anger and fear. New York: Springer Publishing Company. Vallverdú, J., Talanov, M., Distefano, S., Mazzara, M., Tchitchigin, A., & Nurgaliev, I. (2015). A cognitive architecture for the implementation of emotions in computing systems. Biologically Inspired Cognitive Architectures. van Os, J., & Kapur, S. (2009). Schizophrenia. Lancet, 374(9690), 635–645. Walker, Z., McKeith, I., Rodda, J., Qassem, T., Tatsch, K., Booij, J., ... O’Brien, J. (2012). Comparison of cognitive decline between dementia with lewy bodies and alzheimer’s disease: A cohort study. BMJ Open, 2(1), e000380. Wang, H.-L., & Morales, M. (2009). Pedunculopontine and laterodorsal tegmental nuclei contain distinct populations of cholinergic, glutamatergic and GABAergic neurons in the rat. The European Journal of Neuroscience, 29(2), 340–358. Wolf, S., & Berle, B. B. (1976). The nature of schizophrenia. In The biology of the schizophrenic process (pp. 16–39). Springer. Wynne, L. C. (1978). The nature of schizophrenia Ph.D. thesis. McLean Hospital Belmont. Please cite this article in press as: Talanov, M., et al. Emotional simulations and depression diagnostics. Biologically Inspired Cognitive Architectures (2016), http://dx.doi.org/10.1016/j.bica.2016.09.002