Control of Attention in Neural Networks Eric Mjolsness Computer Science Department Yale Universit... more Control of Attention in Neural Networks Eric Mjolsness Computer Science Department Yale University New Haven, CT 06520 Abstract "Attention" - the sequential selection of portions of a large computation to be performed now, later, or not at all - is important in the study of neural networks and ...
Systems biology has experienced dramatic growth in the number, size, and complexity of computatio... more Systems biology has experienced dramatic growth in the number, size, and complexity of computational models. To reproduce simulation results and reuse models, researchers must exchange unambiguous model descriptions. We review the latest edition of the Systems Biology Markup Language (SBML), a format designed for this purpose. A community of modelers and software authors developed SBML Level 3 over the past decade. Its modular form consists of a core suited to representing reaction-based models and packages that extend the core with features suited to other model types including constraintbased models, reaction-diffusion models, logical network models, and rule-based models. The format leverages two decades of SBML and a rich software ecosystem that transformed how systems biologists build and interact with models. More recently, the rise of multiscale models of whole cells and organs, and new data sources such as single-cell measurements and live imaging, has precipitated new ways of integrating data with models. We provide our perspectives on the challenges presented by these developments and how SBML Level provides the foundation needed to support this evolution. 3
1990 IJCNN International Joint Conference on Neural Networks, 1990
Abstract One way to speed up convergence in a large optimization problem is to introduce a smalle... more Abstract One way to speed up convergence in a large optimization problem is to introduce a smaller, approximate version of the problem at a coarser scale and to alternate between relaxation steps for the fine-scale and the coarse-scale problems. Done recursively, this is the idea behind the Multigrid methods which are widely used in the solution of partial differential equations, usually by optimizing quadratic objective functions defined on geometric domains. We exhibit a similar optimization method for neural networks ...
We present a phenomenological modeling framework for development. Our purpose is to provide a sys... more We present a phenomenological modeling framework for development. Our purpose is to provide a systematic method for discovering and expressing correlations in experimental data on gene expression and other developmental processes. The modeling framework is based on a connectionist or "neural net" dynamics for biochemical regulators, coupled to "grammatical rules" which describe certain features of the birth, growth, and death of cells, synapses and other biological entities. We outline how spatial geometry can be included, although this part of the model is not complete. As an example of the application of our results to a specific biological system, we show in detail how to derive a rigorously testable model of the network of segmentation genes operating in the blastoderm of Drosophila. To further illustrate our methods, we sketch how they could be applied to two other important developmental processes: cell cycle control and cell-cell induction. We also present a simple biochemical model leading to our assumed connectionist dynamics which shows that the dynamics used is at least compatible with known chemical mechanisms.
One way to speed up convergence in a large optimization problem is to introduce a smaller, approx... more One way to speed up convergence in a large optimization problem is to introduce a smaller, approximate version of the problem at a coarser scale and to alternate between relaxation steps for the finescale and coarse-scale problems. We exhibit such an optimization method for neural networks governed by quite general objective functions. At the coarse scale there is a smaller approximating neural net which, like the original net, is nonlinear and has a nonquadratic objective function. The transitions and information flow from fine to coarse scale and back do not disrupt the optimization, and the user need only specify a partition of the original fine-scale variables. Thus the method can be applied easily to many problems and networks. We show positive experimental results including cost comparisons.
AGNS (Arabidopsis GeneNet supplementary) database is an Internet-available resource that provides... more AGNS (Arabidopsis GeneNet supplementary) database is an Internet-available resource that provides access to description of the functions of the known Arabidopsis genes at various levels-the levels of mRNA, protein, cell, tissue, and ultimately at the levels of organs and the organism in both wild type and mutant backgrounds. AGNS annotates published papers on gene expression and function and by this way integrates, systematizes, and classifies this heterogeneous, disparate, and scattered information. AGNS consists of ...
THE SIXTH INTERNATIONAL CONFERENCE ON BIOINFORMATICS OF GENOME REGULATION AND STRUCTURE, Jun 22, 2008
Results: The model (2r) did not give rise to any local maximum of auxin in the ring. We changed t... more Results: The model (2r) did not give rise to any local maximum of auxin in the ring. We changed the equation according to [3], where regulation of auxin transport from the cell depends on auxin concentration in the neighboring cell. This model (2s) demonstrated local auxin maxima similar to [3].
We introduce new diagrammatic notations for probabilistic independence networks (including Bayes ... more We introduce new diagrammatic notations for probabilistic independence networks (including Bayes nets and graphical models). These notations include new node and link types that allow for natural representation of a wide range of probabilistic data models including complex hierarchical models. The diagrammatic notations also support models defined on variable numbers of complex objects and relationships. Node types include random variable nodes, index nodes, constraint nodes, and an object supernode. Link ...
Accurate, automated crater counts will be essential in extrapolating from existing Mars crater ca... more Accurate, automated crater counts will be essential in extrapolating from existing Mars crater catalogs to much larger catalogs of impact craters in high-resolution orbital imagery for use in relative dating of surfaces in such imagery. Once validated, automatic methods for performing crater counts could be integrated into tools such as the Planetary Image Atlas, which is designed to be a convenient interface through which a user can search for, display, and download images and other ancillary data for planetary Missions, ...
IJCNN-91-Seattle International Joint Conference on Neural Networks
We derive a neural net for reconstructing a set of curves from ungrouped dot locations. The netwo... more We derive a neural net for reconstructing a set of curves from ungrouped dot locations. The network performs Bayesian inference on a visual grammer, which serves as probabilistic model of the image formation process, by means of quadratic matching objective function.
Proceedings of the First NASA/DoD Workshop on Evolvable Hardware
A key determinant of overall morphogenesis in flowering plants such as Arabidopsis thaliana is th... more A key determinant of overall morphogenesis in flowering plants such as Arabidopsis thaliana is the shoot apical meristem (growing tip of a shoot). Gene regulan'on networks can be used to model this system. We exhibit a very preliminary two.dimensional model including gene regulation and intercellular signaling, but omitting cell division and dynamical geometry. The model can be trained to have three stable regions of gene expression corresponding to the central zone, peripheral zone, and rib meristem. We also discuss a space-engineering motivation for studying and controlling the morphogenesis of plants using such computational models.
Bioinformatics of Genome Regulation and Structure II, 2006
AGNS (Arabidopsis GeneNet supplementary database) is an Internet-available resource that provides... more AGNS (Arabidopsis GeneNet supplementary database) is an Internet-available resource that provides access to description of the functions of the known Arabidopsis genes at various levels—the levels of mRNA, protein, cell, tissue, and ultimately at the levels of organs and the organism in both wild type and mutant backgrounds. AGNS annotates published papers on gene expression and function and by this way integrates, systematizes, and classifies this heterogeneous, disparate, and scattered information. AGNS consists of ...
2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops
In this paper we present the techniques for tracking cell signal in GFP (Green Fluorescent Protei... more In this paper we present the techniques for tracking cell signal in GFP (Green Fluorescent Protein) images of growing cell colonies. We use such tracking for both data extraction and dynamic modeling of intracellular processes. The techniques are based on optimization of energy functions, which simultaneously determines cell correspondences, while estimating the mapping functions. In addition to spatial mappings such as affine and Thin-Plate Spline mapping, the cell growth and cell division histories must be estimated as well. Different levels of joint optimization are discussed. The most unusual tracking feature addressed in this paper is the possibility of one-to-two correspondences caused by cell division. A novel extended softassign algorithm for solutions of one-to-many correspondences is detailed in this paper. The techniques are demonstrated on three sets of data: growing bacillus Subtillus and e-coli colonies and a developing plant shoot apical meristem. The techniques are currently used by biologists for data extraction and hypothesis formation.
The blastoderm of the fruit fly Drosophila melanogaster is unusually well suited for analysis of ... more The blastoderm of the fruit fly Drosophila melanogaster is unusually well suited for analysis of fundamental questions in animal development. One such question is how genes specify the positional information which determines the developmental pathways (fate) of cells a t appropriate spatial locations. In this paper we propose a dynamical model of gene regulation which explicitly describes how positional information is used in the blastoderm. The model is applied to analyze important experimental findings on the dependence of cell fate on the concentration of the Bicoid morphogen. The model shows that positional information in the presumptive middle body is cooperatively determined by maternal products of the bicoid and hunchback genes.
The softassign quadratic assignment algorithm is a discrete-time, continuous-state, synchronous u... more The softassign quadratic assignment algorithm is a discrete-time, continuous-state, synchronous updating optimizing neural network. While its effectiveness has been shown in the traveling salesman problem, graph matching, and graph partitioning in thousands of simulations, its convergence properties have not been studied. Here, we construct discrete-time Lyapunov functions for the cases of exact and approximate doubly stochastic constraint satisfaction, which show convergence to a fixed point. The combination of good convergence properties and experimental success makes the softassign algorithm an excellent choice for neural quadratic assignment optimization.
Control of Attention in Neural Networks Eric Mjolsness Computer Science Department Yale Universit... more Control of Attention in Neural Networks Eric Mjolsness Computer Science Department Yale University New Haven, CT 06520 Abstract "Attention" - the sequential selection of portions of a large computation to be performed now, later, or not at all - is important in the study of neural networks and ...
Systems biology has experienced dramatic growth in the number, size, and complexity of computatio... more Systems biology has experienced dramatic growth in the number, size, and complexity of computational models. To reproduce simulation results and reuse models, researchers must exchange unambiguous model descriptions. We review the latest edition of the Systems Biology Markup Language (SBML), a format designed for this purpose. A community of modelers and software authors developed SBML Level 3 over the past decade. Its modular form consists of a core suited to representing reaction-based models and packages that extend the core with features suited to other model types including constraintbased models, reaction-diffusion models, logical network models, and rule-based models. The format leverages two decades of SBML and a rich software ecosystem that transformed how systems biologists build and interact with models. More recently, the rise of multiscale models of whole cells and organs, and new data sources such as single-cell measurements and live imaging, has precipitated new ways of integrating data with models. We provide our perspectives on the challenges presented by these developments and how SBML Level provides the foundation needed to support this evolution. 3
1990 IJCNN International Joint Conference on Neural Networks, 1990
Abstract One way to speed up convergence in a large optimization problem is to introduce a smalle... more Abstract One way to speed up convergence in a large optimization problem is to introduce a smaller, approximate version of the problem at a coarser scale and to alternate between relaxation steps for the fine-scale and the coarse-scale problems. Done recursively, this is the idea behind the Multigrid methods which are widely used in the solution of partial differential equations, usually by optimizing quadratic objective functions defined on geometric domains. We exhibit a similar optimization method for neural networks ...
We present a phenomenological modeling framework for development. Our purpose is to provide a sys... more We present a phenomenological modeling framework for development. Our purpose is to provide a systematic method for discovering and expressing correlations in experimental data on gene expression and other developmental processes. The modeling framework is based on a connectionist or "neural net" dynamics for biochemical regulators, coupled to "grammatical rules" which describe certain features of the birth, growth, and death of cells, synapses and other biological entities. We outline how spatial geometry can be included, although this part of the model is not complete. As an example of the application of our results to a specific biological system, we show in detail how to derive a rigorously testable model of the network of segmentation genes operating in the blastoderm of Drosophila. To further illustrate our methods, we sketch how they could be applied to two other important developmental processes: cell cycle control and cell-cell induction. We also present a simple biochemical model leading to our assumed connectionist dynamics which shows that the dynamics used is at least compatible with known chemical mechanisms.
One way to speed up convergence in a large optimization problem is to introduce a smaller, approx... more One way to speed up convergence in a large optimization problem is to introduce a smaller, approximate version of the problem at a coarser scale and to alternate between relaxation steps for the finescale and coarse-scale problems. We exhibit such an optimization method for neural networks governed by quite general objective functions. At the coarse scale there is a smaller approximating neural net which, like the original net, is nonlinear and has a nonquadratic objective function. The transitions and information flow from fine to coarse scale and back do not disrupt the optimization, and the user need only specify a partition of the original fine-scale variables. Thus the method can be applied easily to many problems and networks. We show positive experimental results including cost comparisons.
AGNS (Arabidopsis GeneNet supplementary) database is an Internet-available resource that provides... more AGNS (Arabidopsis GeneNet supplementary) database is an Internet-available resource that provides access to description of the functions of the known Arabidopsis genes at various levels-the levels of mRNA, protein, cell, tissue, and ultimately at the levels of organs and the organism in both wild type and mutant backgrounds. AGNS annotates published papers on gene expression and function and by this way integrates, systematizes, and classifies this heterogeneous, disparate, and scattered information. AGNS consists of ...
THE SIXTH INTERNATIONAL CONFERENCE ON BIOINFORMATICS OF GENOME REGULATION AND STRUCTURE, Jun 22, 2008
Results: The model (2r) did not give rise to any local maximum of auxin in the ring. We changed t... more Results: The model (2r) did not give rise to any local maximum of auxin in the ring. We changed the equation according to [3], where regulation of auxin transport from the cell depends on auxin concentration in the neighboring cell. This model (2s) demonstrated local auxin maxima similar to [3].
We introduce new diagrammatic notations for probabilistic independence networks (including Bayes ... more We introduce new diagrammatic notations for probabilistic independence networks (including Bayes nets and graphical models). These notations include new node and link types that allow for natural representation of a wide range of probabilistic data models including complex hierarchical models. The diagrammatic notations also support models defined on variable numbers of complex objects and relationships. Node types include random variable nodes, index nodes, constraint nodes, and an object supernode. Link ...
Accurate, automated crater counts will be essential in extrapolating from existing Mars crater ca... more Accurate, automated crater counts will be essential in extrapolating from existing Mars crater catalogs to much larger catalogs of impact craters in high-resolution orbital imagery for use in relative dating of surfaces in such imagery. Once validated, automatic methods for performing crater counts could be integrated into tools such as the Planetary Image Atlas, which is designed to be a convenient interface through which a user can search for, display, and download images and other ancillary data for planetary Missions, ...
IJCNN-91-Seattle International Joint Conference on Neural Networks
We derive a neural net for reconstructing a set of curves from ungrouped dot locations. The netwo... more We derive a neural net for reconstructing a set of curves from ungrouped dot locations. The network performs Bayesian inference on a visual grammer, which serves as probabilistic model of the image formation process, by means of quadratic matching objective function.
Proceedings of the First NASA/DoD Workshop on Evolvable Hardware
A key determinant of overall morphogenesis in flowering plants such as Arabidopsis thaliana is th... more A key determinant of overall morphogenesis in flowering plants such as Arabidopsis thaliana is the shoot apical meristem (growing tip of a shoot). Gene regulan'on networks can be used to model this system. We exhibit a very preliminary two.dimensional model including gene regulation and intercellular signaling, but omitting cell division and dynamical geometry. The model can be trained to have three stable regions of gene expression corresponding to the central zone, peripheral zone, and rib meristem. We also discuss a space-engineering motivation for studying and controlling the morphogenesis of plants using such computational models.
Bioinformatics of Genome Regulation and Structure II, 2006
AGNS (Arabidopsis GeneNet supplementary database) is an Internet-available resource that provides... more AGNS (Arabidopsis GeneNet supplementary database) is an Internet-available resource that provides access to description of the functions of the known Arabidopsis genes at various levels—the levels of mRNA, protein, cell, tissue, and ultimately at the levels of organs and the organism in both wild type and mutant backgrounds. AGNS annotates published papers on gene expression and function and by this way integrates, systematizes, and classifies this heterogeneous, disparate, and scattered information. AGNS consists of ...
2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Workshops
In this paper we present the techniques for tracking cell signal in GFP (Green Fluorescent Protei... more In this paper we present the techniques for tracking cell signal in GFP (Green Fluorescent Protein) images of growing cell colonies. We use such tracking for both data extraction and dynamic modeling of intracellular processes. The techniques are based on optimization of energy functions, which simultaneously determines cell correspondences, while estimating the mapping functions. In addition to spatial mappings such as affine and Thin-Plate Spline mapping, the cell growth and cell division histories must be estimated as well. Different levels of joint optimization are discussed. The most unusual tracking feature addressed in this paper is the possibility of one-to-two correspondences caused by cell division. A novel extended softassign algorithm for solutions of one-to-many correspondences is detailed in this paper. The techniques are demonstrated on three sets of data: growing bacillus Subtillus and e-coli colonies and a developing plant shoot apical meristem. The techniques are currently used by biologists for data extraction and hypothesis formation.
The blastoderm of the fruit fly Drosophila melanogaster is unusually well suited for analysis of ... more The blastoderm of the fruit fly Drosophila melanogaster is unusually well suited for analysis of fundamental questions in animal development. One such question is how genes specify the positional information which determines the developmental pathways (fate) of cells a t appropriate spatial locations. In this paper we propose a dynamical model of gene regulation which explicitly describes how positional information is used in the blastoderm. The model is applied to analyze important experimental findings on the dependence of cell fate on the concentration of the Bicoid morphogen. The model shows that positional information in the presumptive middle body is cooperatively determined by maternal products of the bicoid and hunchback genes.
The softassign quadratic assignment algorithm is a discrete-time, continuous-state, synchronous u... more The softassign quadratic assignment algorithm is a discrete-time, continuous-state, synchronous updating optimizing neural network. While its effectiveness has been shown in the traveling salesman problem, graph matching, and graph partitioning in thousands of simulations, its convergence properties have not been studied. Here, we construct discrete-time Lyapunov functions for the cases of exact and approximate doubly stochastic constraint satisfaction, which show convergence to a fixed point. The combination of good convergence properties and experimental success makes the softassign algorithm an excellent choice for neural quadratic assignment optimization.
Uploads
Papers by E. Mjolsness