Structure of semantic memory was investigated in the way of neural network simulations in detail.... more Structure of semantic memory was investigated in the way of neural network simulations in detail. In the literature, it is well-known that brain damaged patients often showed category specific disorder in various cognitive neuropsychological tasks like picture naming, categorisation, identification tasks and so on. In order to describe semantic memory disorder of brain damaged patients, the attractor neural network model originally proposed Hinton and Shallice (1991) was employed and was tried to re-evaluate the model performance. Especially, in order to answer the question about organization of semantic memory, how our semantic memories are organized, computer simulations were conducted. After the model learned data set (Tyler, Moss, Durrant-Peatfield, & Levy, 2000), units in hidden and cleanup layers were removed and observed its performances. The results showed category specificity. This model could also explain the double dissociation phenomena. In spite of the simplicity of its architecture, the attractor neural network might be considered to mimic human behavior in the meaning of semantic memory organization and its disorder. Although this model could explain various phenomenon in cognitive neuropsychology, it might become obvious that this model had one limitation to explain human behavior. As far as investigation in this study, asymmetry in category specificity between animate and inanimate objects might not be explained on this model without any additional assumptions. Therefore, further studies must be required to improve our understanding for semantic memory organisation.
Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02., 2002
ABSTRACT We propose SOMDS that is a combination of MDS (multidimensional scaling) and SOM. SOMDS ... more ABSTRACT We propose SOMDS that is a combination of MDS (multidimensional scaling) and SOM. SOMDS is a special type of MDS that can learn locally and adaptively the structure of similarity data. SOMDS is a special type of SOM without neighborhood functions and whose inputs are similarities between objects. Convergence properties of the algorithm and some applications are presented.
ABSTRACT In this article, I discuss the implementation of neural network models for reading Engli... more ABSTRACT In this article, I discuss the implementation of neural network models for reading English words aloud. Since 1989, there has been existing a debate about the models of reading English words aloud. One is the Dual Route Cascaded (DRC) model. The other is the Triangle model, whose original version was developed in 1989. Because there are arbitrary variables of both models, we did not decide which model gives better accounts for the numerous data given by psychological experiments and neuropsychological evidence. Therefore, in order to give a solution of this debate, an attempt to integrate both models was made. Introducing the Mixture of Experts Network, an elegant solution to overcome the arbitrariness of both models could be given. The Mixture of Expert Network can include both the models as a special case of this Network. From the Mixture of Expert Network's point of view, the difference between the Dual Route Cascaded model and the Triangle model would be able to describe as the quantitative difference of the dispersion parameters.
International Joint Conference on Computational Intelligence, 2012
It was investigated that the ability of an attractor neural network. The attractor neural network... more It was investigated that the ability of an attractor neural network. The attractor neural network can be applicable to various symptoms of brain damaged patients. It can account for delays in reaction times in word reading and word identification tasks. Because the iteration numbers of mutual connections between an output and a cleanup layers might increase, when they are partially damaged. This prolongation looks or behaves the delays of reaction times of brain damaged patients. When we applied the attractor neural network to the data of Tyler et al. (2000) for categorization task, it showed a kind of category specific phenomenon. In this sense, the attractor neural network could explain an aspect of the category specific disorders. In this sense the attractor network might simulate the human semantic memory organization. In spite of variations in data, and in spite of the simplicity of the architecture, the attractor network showed good performances. We could say that the attractor network succeeded in mimicking human normal subjects and brain damaged patients. The possibility of explaining the triangle model (
Abstract. An implementation of neural network models for reading English words aloud is proposed.... more Abstract. An implementation of neural network models for reading English words aloud is proposed. Since 1989, there has been existing a debate in neuropsycholgy and cognitive science about the models of reading. One is the Dual Route Cascaded model, another is the Triangle model. Since there exist arbitrary variables in both models, it was dif-ficult to decide which model would be appropriate to explain the data from psychological experiments and neuropsychological evidence. There-fore, in order to provide a solution of this debate, an attempt to integrate both models was attempted. By introducing the Mixtures of Experts Network model, a solution to overcome the arbitrariness of both models could be given. The Mixtures of Experts Network model could include both models as a special case. From the Mixtures of Experts network’s point of view, the difference between the Dual Route Cascaded model and the Triangle model would be considered as a quantitative difference of the dispersion p...
In this report, the author reviewed a brief history of neuropsychology, and its contribution of n... more In this report, the author reviewed a brief history of neuropsychology, and its contribution of neuroimaging study. In the contents of this review, the author focused on the role of the left angular gyrus on reading letters aloud. In general, the alexia is emerged by the damage of the left angular gyrus. On the other hand, no activation was observed at the angular gyrus in the neuroimaging study in the meaning of the statistically significant level. In order to solve this dissociation between neuropsychological evidence and neuroimaging data, the author proposed the 3 layered neural network model in which the hidden layer was the kWTA circuit [6]. This model could explain both neuropsychological evidence and neuroimaging data without contradiction. Neurocomputing would play an important role as a tool for an abstract and an interdisciplinary modeling of the brain.
Word embedding models such as word2vec (Mikolov,2013) and GloVe (Pennington et al.2014) became to... more Word embedding models such as word2vec (Mikolov,2013) and GloVe (Pennington et al.2014) became to be widely recognized, while similarity judgment of analogy and comprehension could be described on a Euclidian space. We tried to compare between word embedding models and human performances. Despite that several models were proposed to deal with adjust each domain in order to improve their performance, it has still remained unsolved the relation between human judgment and understanding of analogy and their performance of word embedding models. We investivated the relationship between them. Our results suggest that human understanding of analogy might be understood in terms of word embedding spaces. We disscussed the further possibilities to understannd the semantic space in human in these models.
The NTT Kanji database (天野・近藤, 1999) is one of the most popular kanji databases. However, it has ... more The NTT Kanji database (天野・近藤, 1999) is one of the most popular kanji databases. However, it has been long ever since it published. The word2vec (Mikolov, Yih, & Zweig, 2013; Mikolov, Sutskever, Chen, Corrado, & Dean, 2013; Mikolov, Chen, Corrado, & Dean, 2013; Mikolov, Le, & Sutskever, 2013) was proposed based upon large vocabulary. The word2vec was insisted that it could be dealt with semantics as well as syntax, got popularity among machine learning scientists. In spite of these popularities, no trials have been attempted to compare them so far. Here, we tried to compare them in term of a modern visualization, t-SNE Maaten & Hinton (2008).
An Extension of Elman Network and A Generalized Simple Recurrent Network Shin-ichi Asakawa Tokyo ... more An Extension of Elman Network and A Generalized Simple Recurrent Network Shin-ichi Asakawa Tokyo Woman’s Christian University Abstract: Although the Elman network is so powerful that it can deal with a variety of language processings, there exist some short comings about its ability. For example, the original Elman net cannot always deal with a long distance dependency appropriately, which is a number agreement between nouns and verbs with many relative pronouns in a sentence. This limitation might cause from the constraints of its structure of the context and the hidden layer, which can preserve only one time previous state of the network. Here, we propose an extension of the Elman network and a generalized simple recurrent neurual network. The proposed networks could preserve the n-th generations of inner states. When the model processed the corpus consisted of many relative pronouns with multi- center embeddings structure, it could deal with the long distance number agreement ade...
A Model of Cognitive Rehabilitation: Recovering with Constraints Shin-ichi Asakawa Tokyo Woman’s ... more A Model of Cognitive Rehabilitation: Recovering with Constraints Shin-ichi Asakawa Tokyo Woman’s Christian University Yoshihiro Itaguchi Waseda University Abstract: Neural network modeling offers a useful computational framework for exploring the nature of normal and impaired cognitive processes. Among such modelings, Hinton and Shalllice (1991) and Plaut (1996) lesioned recurrent neural networks and investigated the degree of recovery through retraining. However, many ways of brain damages has remained to be unresolved. The current works propose the method of constraints in which brain damages might occur. In order to understand the nature and variability of recovery in patients, we examined both simple three layerd perceptrons and attractor networks with lesions in various parts of networks, and observed recovery processes in retraining. The findings in this study revealed conditions to recover from brain damages and suggests good and proper ways of therapy in which therapists hav...
Structure of semantic memory was investigated in the way of neural network simulations in detail.... more Structure of semantic memory was investigated in the way of neural network simulations in detail. In the literature, it is well-known that brain damaged patients often showed category specific disorder in various cognitive neuropsychological tasks like picture naming, categorisation, identification tasks and so on. In order to describe semantic memory disorder of brain damaged patients, the attractor neural network model originally proposed Hinton and Shallice (1991) was employed and was tried to re-evaluate the model performance. Especially, in order to answer the question about organization of semantic memory, how our semantic memories are organized, computer simulations were conducted. After the model learned data set (Tyler, Moss, Durrant-Peatfield, & Levy, 2000), units in hidden and cleanup layers were removed and observed its performances. The results showed category specificity. This model could also explain the double dissociation phenomena. In spite of the simplicity of its architecture, the attractor neural network might be considered to mimic human behavior in the meaning of semantic memory organization and its disorder. Although this model could explain various phenomenon in cognitive neuropsychology, it might become obvious that this model had one limitation to explain human behavior. As far as investigation in this study, asymmetry in category specificity between animate and inanimate objects might not be explained on this model without any additional assumptions. Therefore, further studies must be required to improve our understanding for semantic memory organisation.
Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02., 2002
ABSTRACT We propose SOMDS that is a combination of MDS (multidimensional scaling) and SOM. SOMDS ... more ABSTRACT We propose SOMDS that is a combination of MDS (multidimensional scaling) and SOM. SOMDS is a special type of MDS that can learn locally and adaptively the structure of similarity data. SOMDS is a special type of SOM without neighborhood functions and whose inputs are similarities between objects. Convergence properties of the algorithm and some applications are presented.
ABSTRACT In this article, I discuss the implementation of neural network models for reading Engli... more ABSTRACT In this article, I discuss the implementation of neural network models for reading English words aloud. Since 1989, there has been existing a debate about the models of reading English words aloud. One is the Dual Route Cascaded (DRC) model. The other is the Triangle model, whose original version was developed in 1989. Because there are arbitrary variables of both models, we did not decide which model gives better accounts for the numerous data given by psychological experiments and neuropsychological evidence. Therefore, in order to give a solution of this debate, an attempt to integrate both models was made. Introducing the Mixture of Experts Network, an elegant solution to overcome the arbitrariness of both models could be given. The Mixture of Expert Network can include both the models as a special case of this Network. From the Mixture of Expert Network's point of view, the difference between the Dual Route Cascaded model and the Triangle model would be able to describe as the quantitative difference of the dispersion parameters.
International Joint Conference on Computational Intelligence, 2012
It was investigated that the ability of an attractor neural network. The attractor neural network... more It was investigated that the ability of an attractor neural network. The attractor neural network can be applicable to various symptoms of brain damaged patients. It can account for delays in reaction times in word reading and word identification tasks. Because the iteration numbers of mutual connections between an output and a cleanup layers might increase, when they are partially damaged. This prolongation looks or behaves the delays of reaction times of brain damaged patients. When we applied the attractor neural network to the data of Tyler et al. (2000) for categorization task, it showed a kind of category specific phenomenon. In this sense, the attractor neural network could explain an aspect of the category specific disorders. In this sense the attractor network might simulate the human semantic memory organization. In spite of variations in data, and in spite of the simplicity of the architecture, the attractor network showed good performances. We could say that the attractor network succeeded in mimicking human normal subjects and brain damaged patients. The possibility of explaining the triangle model (
Abstract. An implementation of neural network models for reading English words aloud is proposed.... more Abstract. An implementation of neural network models for reading English words aloud is proposed. Since 1989, there has been existing a debate in neuropsycholgy and cognitive science about the models of reading. One is the Dual Route Cascaded model, another is the Triangle model. Since there exist arbitrary variables in both models, it was dif-ficult to decide which model would be appropriate to explain the data from psychological experiments and neuropsychological evidence. There-fore, in order to provide a solution of this debate, an attempt to integrate both models was attempted. By introducing the Mixtures of Experts Network model, a solution to overcome the arbitrariness of both models could be given. The Mixtures of Experts Network model could include both models as a special case. From the Mixtures of Experts network’s point of view, the difference between the Dual Route Cascaded model and the Triangle model would be considered as a quantitative difference of the dispersion p...
In this report, the author reviewed a brief history of neuropsychology, and its contribution of n... more In this report, the author reviewed a brief history of neuropsychology, and its contribution of neuroimaging study. In the contents of this review, the author focused on the role of the left angular gyrus on reading letters aloud. In general, the alexia is emerged by the damage of the left angular gyrus. On the other hand, no activation was observed at the angular gyrus in the neuroimaging study in the meaning of the statistically significant level. In order to solve this dissociation between neuropsychological evidence and neuroimaging data, the author proposed the 3 layered neural network model in which the hidden layer was the kWTA circuit [6]. This model could explain both neuropsychological evidence and neuroimaging data without contradiction. Neurocomputing would play an important role as a tool for an abstract and an interdisciplinary modeling of the brain.
Word embedding models such as word2vec (Mikolov,2013) and GloVe (Pennington et al.2014) became to... more Word embedding models such as word2vec (Mikolov,2013) and GloVe (Pennington et al.2014) became to be widely recognized, while similarity judgment of analogy and comprehension could be described on a Euclidian space. We tried to compare between word embedding models and human performances. Despite that several models were proposed to deal with adjust each domain in order to improve their performance, it has still remained unsolved the relation between human judgment and understanding of analogy and their performance of word embedding models. We investivated the relationship between them. Our results suggest that human understanding of analogy might be understood in terms of word embedding spaces. We disscussed the further possibilities to understannd the semantic space in human in these models.
The NTT Kanji database (天野・近藤, 1999) is one of the most popular kanji databases. However, it has ... more The NTT Kanji database (天野・近藤, 1999) is one of the most popular kanji databases. However, it has been long ever since it published. The word2vec (Mikolov, Yih, & Zweig, 2013; Mikolov, Sutskever, Chen, Corrado, & Dean, 2013; Mikolov, Chen, Corrado, & Dean, 2013; Mikolov, Le, & Sutskever, 2013) was proposed based upon large vocabulary. The word2vec was insisted that it could be dealt with semantics as well as syntax, got popularity among machine learning scientists. In spite of these popularities, no trials have been attempted to compare them so far. Here, we tried to compare them in term of a modern visualization, t-SNE Maaten & Hinton (2008).
An Extension of Elman Network and A Generalized Simple Recurrent Network Shin-ichi Asakawa Tokyo ... more An Extension of Elman Network and A Generalized Simple Recurrent Network Shin-ichi Asakawa Tokyo Woman’s Christian University Abstract: Although the Elman network is so powerful that it can deal with a variety of language processings, there exist some short comings about its ability. For example, the original Elman net cannot always deal with a long distance dependency appropriately, which is a number agreement between nouns and verbs with many relative pronouns in a sentence. This limitation might cause from the constraints of its structure of the context and the hidden layer, which can preserve only one time previous state of the network. Here, we propose an extension of the Elman network and a generalized simple recurrent neurual network. The proposed networks could preserve the n-th generations of inner states. When the model processed the corpus consisted of many relative pronouns with multi- center embeddings structure, it could deal with the long distance number agreement ade...
A Model of Cognitive Rehabilitation: Recovering with Constraints Shin-ichi Asakawa Tokyo Woman’s ... more A Model of Cognitive Rehabilitation: Recovering with Constraints Shin-ichi Asakawa Tokyo Woman’s Christian University Yoshihiro Itaguchi Waseda University Abstract: Neural network modeling offers a useful computational framework for exploring the nature of normal and impaired cognitive processes. Among such modelings, Hinton and Shalllice (1991) and Plaut (1996) lesioned recurrent neural networks and investigated the degree of recovery through retraining. However, many ways of brain damages has remained to be unresolved. The current works propose the method of constraints in which brain damages might occur. In order to understand the nature and variability of recovery in patients, we examined both simple three layerd perceptrons and attractor networks with lesions in various parts of networks, and observed recovery processes in retraining. The findings in this study revealed conditions to recover from brain damages and suggests good and proper ways of therapy in which therapists hav...
Uploads
Papers by Shin Asakawa