Recent Advances in Learning Automata

Studies in Computational Intelligence Volume 754 Series editor Janusz Kacprzyk, Polish Academy of Sciences, Warsaw, Poland e-mail: [email protected] The series “Studies in Computational Intelligence” (SCI) publishes new developments and advances in the various areas of computational intelligence—quickly and with a high quality. The intent is to cover the theory, applications, and design methods of computational intelligence, as embedded in the fields of engineering, computer science, physics and life sciences, as well as the methodologies behind them. The series contains monographs, lecture notes and edited volumes in computational intelligence spanning the areas of neural networks, connectionist systems, genetic algorithms, evolutionary computation, artificial intelligence, cellular automata, self-organizing systems, soft computing, fuzzy systems, and hybrid intelligent systems. Of particular value to both the contributors and the readership are the short publication timeframe and the world-wide distribution, which enable both wide and rapid dissemination of research output. More information about this series at http://www.springer.com/series/7092 Alireza Rezvanian Ali Mohammad Saghiri S. Mehdi Vahidipour Mehdi Esnaashari Mohammad Reza Meybodi • • Recent Advances in Learning Automata 123 Alireza Rezvanian School of Computer Science Institute for Research in Fundamental Sciences (IPM) Tehran Iran S. Mehdi Vahidipour Faculty of Electrical and Computer Engineering, Computer Engineering Department University of Kashan Kashan Iran and Computer Engineering and Information Technology Department Amirkabir University of Technology (Tehran Polytechnic) Tehran Iran Ali Mohammad Saghiri Computer Engineering and Information Technology Department Amirkabir University of Technology (Tehran Polytechnic) Tehran Iran Mehdi Esnaashari Faculty of Computer Engineering K.N.Toosi University of Technology Tehran Iran Mohammad Reza Meybodi Soft Computing Laboratory Amirkabir University of Technology (Tehran Polytechnic) Tehran Iran ISSN 1860-949X ISSN 1860-9503 (electronic) Studies in Computational Intelligence ISBN 978-3-319-72427-0 ISBN 978-3-319-72428-7 (eBook) https://doi.org/10.1007/978-3-319-72428-7 Library of Congress Control Number: 2017960204 © Springer International Publishing AG 2018 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Printed on acid-free paper This Springer imprint is published by Springer Nature The registered company is Springer International Publishing AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland To my lovely wife, my beloved dad, my merciful mom, and my dear sisters for their love and supports Alireza To my late mother, Dr. H. Afsar Lajevardi. I will never forget her kindness and support. To my brother, Dr. Mohammad Ali Saghiri and my father for their support during difficult days of my life Ali Mohammad To my family S. Mehdi To my love, my life, my soulmate, my ONE and only, Najmeh Mehdi Preface This book is written for computer engineers, scientists, and students studying/working in reinforcement learning and artificial intelligence domains. The book collects recent advances in learning automaton theory as well as its applications in different computer science problems and domains. The book, in detail, describes the distributed learning automata and the cellular learning automata models for solving a variety of complex problems in wireless sensor networks, complex social networks, cognitive peer-to-peer networks, and adaptive Petri nets. Validation of the given learning automata-based methodologies is provided through extensive computer simulations. In addition, the book presents detailed mathematical and theoretical aspects of recent developments of cellular learning automata in real-world problems. The mathematical level in all chapters is well-suited within the grasp of the scientists as well as the graduate students from the engineering and computer science streams. The reader is encouraged to have basic understanding of probability, stochastic processes, and related mathematical analyses. This book consists of six chapters dedicated toward using recent models of learning automata for computer science applications. Chapter 1 provides the necessary background about learning automata theory and distributed learning automata. Chapter 2 gives a brief introduction about recent cellular learning automata models including irregular cellular learning automata and dynamic cellular learning automata models. Chapter 3 introduces recent applications of learning automata for wireless sensor networks. Chapter 4 is devoted to applications of learning automata in cognitive peer-to-peer networks. Chapter 5 discusses about the applications of learning automata for social network analyses when the underlying graph model is assumed to be stochastic. Finally, Chap. 6 provides new models of adaptive Petri nets based on learning automata. The authors would like to thank Dr. Thomas Ditzinger, Springer, Executive Editor, Interdisciplinary and Applied Sciences and Engineering and Mr. Ramamoorthy Rajangam, Project Coordinator, Books Production, Springer Verlag, Heidelberg, for the editorial assistance and excellent cooperative collaboration to produce this vii viii Preface important scientific work. We hope that readers will share our pleasure to present this book on recent advances in learning automata and will find it useful in their careers. Tehran, Iran Tehran, Iran Kashan, Iran Tehran, Iran Tehran, Iran Alireza Rezvanian Ali Mohammad Saghiri S. Mehdi Vahidipour Mehdi Esnaashari Mohammad Reza Meybodi Acknowledgements We are grateful to many people who have helped us during the past few years, who have contributed to work presented here, and who have offered critical reviews of prior publications. We thank Springer for their assistance in publishing the book. We are also grateful to our academic supervisor, our family, our parents, and all our friends for their love and support. ix Contents Part I Models 1 Learning Automata Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 Fixed-Structure Learning Automata . . . . . . . . . . . . . . 1.1.2 Variable-Structure Learning Automata (VSLA) . . . . . 1.1.3 Learning Automata with Variable Number of Actions 1.1.4 Estimator Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 1.1.5 Pursuit Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.6 Continuous Action-Set Learning Automata . . . . . . . . 1.2 Interconnected Learning Automata . . . . . . . . . . . . . . . . . . . . 1.2.1 Hierarchical Structure Learning Automata . . . . . . . . . 1.2.2 Multi-Level Game of Learning Automata . . . . . . . . . 1.2.3 Network of Learning Automata . . . . . . . . . . . . . . . . 1.2.4 Distributed Learning Automata (DLA) . . . . . . . . . . . 1.2.5 Extended Distributed Learning Automata (eDLA) . . . 1.3 Cellular Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . 1.3.1 CLA-EC: CLA-Based Evolutionary Computing . . . . . 1.4 Applications of Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3 6 6 7 7 8 9 9 10 12 12 13 15 16 18 19 2 Cellular Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Irregular Cellular Learning Automata . . . . . . . . . . . . . . . . . 2.2.1 Definitions and Notations . . . . . . . . . . . . . . . . . . . . 2.2.2 Behavior of ICLA . . . . . . . . . . . . . . . . . . . . . . . . . 2.2.3 Expediency of ICLA . . . . . . . . . . . . . . . . . . . . . . . 2.3 Dynamic Models of Cellular Learning Automata . . . . . . . . 2.3.1 Dynamic Irregular Cellular Learning Automata . . . . 2.3.2 Heterogeneous Dynamic Irregular Cellular Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 21 22 24 26 37 39 40 ..... 52 . . . . . . . . xi xii Contents 2.3.3 Closed Asynchronous Dynamic Cellular Learning Automata . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.4 Closed Asynchronous Dynamic Cellular Learning Automata with Varying Number of LAs in Each Cell 2.3.5 Norms of Behavior . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Part II .... 68 .... .... .... 84 85 87 Recent Applications 3 Learning Automata for Wireless Sensor Networks . . . . . . . . . . . . 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Data Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.2 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.2 Analytical Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Area Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 CLA-DS: A Cellular Learning Automata-Based Deployment Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 CLA-EDS: An Extension to CLA-DS for Providing K-Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 Point Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5.1 SALA: A Learning Automata-Based Scheduling Solution to the Dynamic Point Coverage Problem . . . . . . . . . . . . 3.5.2 SACLA: A Cellular Learning Automata-Based Algorithm for Dynamic Point Coverage Problem . . . . . . . . . . . . . . 3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Learning Automata for Cognitive Peer-to-Peer Networks . . . . . . . 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 A Cognitive Engine for Solving Topology Mismatch Problem Based on Schelling Segregation Model . . . . . . . . . . . . . . . . . . 4.2.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2 An Approach Based on CADCLA for Designing Cognitive Engines and Its Application to Solve Topology Mismatch Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 A Cognitive Engine for Solving Super-Peer Selection Problem Based on Fungal Growth Model . . . . . . . . . . . . . . . . . . . . . . . 4.3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 91 92 93 95 101 105 107 111 119 122 . . 123 . . 147 . . 170 . . 172 . . 197 . . 218 . . 221 . . 221 . . 225 . . 225 . . 228 . . 237 . . 244 . . 245 Contents xiii 4.3.2 Proposed Algorithm: X-NET . . . . . . . . . . . . . . . . . . . 4.3.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 A Cognitive Engine for Solving Topology Mismatch Problem Based on Voronoi Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Proposed Algorithm: xOverlay . . . . . . . . . . . . . . . . . . 4.4.3 Experimental Results . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 . . . 252 . . . . . . . . . . . . . . . 261 262 265 273 278 5 Learning Automata for Complex Social Networks . . . . . . . . . . . . 5.1 Complex Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1.1 Stochastic Graphs as a Graph Model for Complex Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Stochastic Graph Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2.1 Maximum Clique Problem in Stochastic Graphs . . . . . . 5.2.2 Minimum Vertex Covering in Stochastic Graphs . . . . . . 5.3 Social Network Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.1 Network Centralities and Measures for Social Network Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.2 LA-Based Algorithm for Computing Network Centralities and Measures in Stochastic Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3.3 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Network Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.1 LA-Based Algorithms for Network Sampling in Deterministic Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4.2 LA-Based Algorithms for Network Sampling Algorithms in Stochastic Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Community Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5.1 LA-Based Algorithm for Community Detection . . . . . . . 5.5.2 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319 326 327 330 333 6 Adaptive Petri Net Based on Learning Automata . 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Petri Nets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2.1 Definitions and Notations . . . . . . . . . . . 6.2.2 Controlling Mechanisms in Petri Nets . . 6.2.3 Time in Petri Nets . . . . . . . . . . . . . . . . 6.2.4 Adaptive Petri Nets . . . . . . . . . . . . . . . 6.2.5 Applications of Petri Nets . . . . . . . . . . 6.3 Hybrid Machines Based on LAs and PNs . . . . 6.3.1 APN-LA: Adaptive PN Based on LA . . 6.3.2 ASPN-LA: Generalization of APN-LA . . . . . . . . . . . . . . . . . . . . . . . 335 335 337 339 346 348 352 353 354 354 361 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279 . . 279 . . . . . . . . . . 280 282 283 290 298 . . 300 . . 303 . . 306 . . 311 . . 311 xiv Contents 6.3.3 APN-ICLA: Adaptive PN Based on Irregular Cellular LA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3.4 CAPN-LA: Cellular Adaptive PN Based on LA . 6.3.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . 6.4 Priority Assignment Application . . . . . . . . . . . . . . . . . . 6.4.1 Priority Mechanisms . . . . . . . . . . . . . . . . . . . . . 6.4.2 ASPN-LA and Priority Assignment . . . . . . . . . . 6.4.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . 6.5 Graph Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5.1 The Shortest Path Problem in Stochastic Graphs: Application of ASPN-LA . . . . . . . . . . . . . . . . . . 6.5.2 Vertex Coloring Problem: Application of CAPN-LA . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.6 Conclusion and Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 366 376 377 378 380 394 404 . . . . . . . 405 . . . . . . . 421 . . . . . . . 436 7 Summary and Future Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439 About the Authors Alireza Rezvanian was born in Hamedan, Iran, in 1984. He received his B.Sc. from Bu-Ali Sina University of Hamedan, Iran, in 2007, M.Sc. in Computer Engineering with honors from Islamic Azad University of Qazvin, Iran, in 2010, and Ph.D. in Computer Engineering at the Computer Engineering and Information Technology Department from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran, in 2016. Currently, he works as a researcher in School of Computer Science, Institute for Research in Fundamental Sciences (IPM), Tehran, Iran. Prior to the current position, he joined the Department of Computer Engineering and Information Technology at Hamedan University of Technology, Hamedan, Iran as a lecturer. He has authored or coauthored more than 70 research publications in reputable peer-reviewed journals and conferences including IEEE, Elsevier, Springer, Wiley and Taylor & Francis. He has been Guest Editor of special Issue on new applications of learning automata-based techniques in real-world environments for Journal of Computational Science (Elsevier). He is an associate editor of the Human-centric Computing and Information Sciences (Springer). His research activities include soft computing, evolutionary algorithms, complex social networks, and learning automata. xv xvi About the Authors Ali Mohammad Saghiri received his B. Sc. and M. Sc. in computer engineering in Iran, in 2008 and 2010, respectively. He received Ph.D. in computer engineering at the Computer Engineering and Information Technology Department from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran, in 2017. His research interests include peer-to-peer networks, distributed systems, artificial intelligence, learning automata, reinforcement learning, parallel algorithms, and soft computing. S. Mehdi Vahidipour received his B.Sc. and M.Sc. in computer engineering in Iran, in 2000 and 2003, respectively. He received Ph.D. in computer engineering at the Computer Engineering and Information Technology Department from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran, in 2016. Currently, he is an Assistant Professor in Electrical and Computer Engineering department, University of Kashan. His research interests include distributed artificial intelligence, learning automata, and Adaptive Petri nets. Mehdi Esnaashari received his B.S., M.S., and the Ph.D. in computer engineering, all from the Amirkabir University of Technology, Tehran, Iran, in 2002, 2005, and 2011, respectively. Prior to his current position, he was an Assistant Professor with Iran Telecommunications Research Center, Tehran. He is currently an Assistant Professor with faculty of computer engineering, K. N. Toosi University of Technology, Tehran, Iran. His current research interests include computer networks, learning systems, learning automata, soft computing, and information retrieval. About the Authors xvii Mohammad Reza Meybodi received B.S. and M.S. in Economics from the Shahid Beheshti University in Iran, in 1973 and 1977, respectively. He also received M.S. and Ph.D. from the Oklahoma University, USA, in 1980 and 1983, respectively, in Computer Science. Currently, he is a Full Professor in Computer Engineering Department, Amirkabir University of Technology, Tehran, Iran. Prior to current position, he worked from 1983 to 1985 as an Assistant Professor at the Western Michigan University, and from 1985 to 1991 as an Associate Professor at the Ohio University, USA. His current research interests include, learning systems, cloud computing, soft computing, and social networks. Abstract Learning automaton (LA) as a promising field of artificial intelligence is a self-adaptive decision-making device that interacts with an unknown stochastic environment and progressively is able to find the optimal action even provided with probabilistic wrong hints. LA has made a significant impact in many areas of computer science and engineering problems. In the past decade, a wide range of learning automata theories, models, and paradigms have been published by researchers in vast areas of computer science domain such as resource allocation, pattern recognition, image processing, task scheduling, data mining, computer networks, communication networks, distributed adaptive systems, cognitive networks, vehicular sensor networks, grid computing, cloud computing, adaptive Perti-nets, complex social networks, optimization, and so on. Learning automata are extremely suitable for modeling, learning, controlling, and solving real-world problems, especially when the information is incomplete; that is, when the environment is noisy or has a high degree of uncertainty. This book is intended to collect recent advances in learning automata including research results that address key issues and topics related to learning automata theories, architecture, models, algorithms, and their applications. xix