Papers by Cleotilde Gonzalez
Cognitive Systems Research, 2011
Complex cognition addresses research on (a) high-level cognitive processes-mainly problem solving... more Complex cognition addresses research on (a) high-level cognitive processes-mainly problem solving, reasoning, and decision making-and their interaction with more basic processes such as perception, learning, motivation and emotion and (b) cognitive processes which take place in a complex, typically dynamic, environment. Our focus is on AI systems and cognitive models dealing with complexity and on psychological findings which can inspire or challenge cognitive systems research. In this overview we first motivate why we have to go beyond models for rather simple cognitive processes and reductionist experiments. Afterwards, we give a characterization of complexity from our perspective. We introduce the triad of cognitive science methods-analytical, empirical, and engineering methods-which in our opinion have all to be utilized to tackle complex cognition. Afterwards we highlight three aspects of complex cognition-complex problem solving, dynamic decision making, and learning of concepts, skills and strategies. We conclude with some reflections about and challenges for future research.
Psychology Press eBooks, Dec 22, 2020
Learning Under High Cognitive Workload F. Javier Lerch (flOc@ andrew. cmu. edu') Cle... more Learning Under High Cognitive Workload F. Javier Lerch (flOc@ andrew. cmu. edu') Cleotilde Gonzalez (conzalez@ andrew. cmu. edu) Christian Lebiere (cl@ andrew. cmu. edu) Center for Interactive Simulations Carnegie Mellon University 5000 Forbes Ave. Pittsburgh PA ...
Journal of Cognitive Engineering and Decision Making, Jun 1, 2009
We contrasted and compared independently developed computational models of human performance in a... more We contrasted and compared independently developed computational models of human performance in a common dynamic decision-making task. The task, called Dynamic Stocks and Flows, is simple and tractable enough for laboratory experiments yet exhibits many characteristics of macrocognition. A macrocognitive model was developed using a computational instantiation of Recognition-Primed Decision-Making. A microcognitive model was developed using the ACT-R cognitive architecture. Both models followed an instance-based learning paradigm and displayed striking similarities, including their constraints, limitations, and the key breakthrough that enabled satisfactory (though still short of human-like) performance, suggesting the emergence of a general design pattern. On the basis of this comparison we argue that while some substantive differences remain, microcognitive and macrocognitive approaches provide complementary rather than contradictory accounts of human behavior.
I. INTRODUCTION 'Cognitive architectures' are computer algorithms designed to model human behavio... more I. INTRODUCTION 'Cognitive architectures' are computer algorithms designed to model human behavior and to function in a way similar to the workings of the human mind. The breadth of cognitive architectures is one of their primary strengths. Rather than serving as special-purpose models engineered specifically for individual tasks, cognitive architectures provide general computational mechanisms and constraints that are applicable to the development of models for all kinds of tasks. ACT-R is a widely researched cognitive architecture that accounts for hundreds of empirical results obtained in the field of experimental psychology (Anderson and Lebiere, 1998). ACT-R is a hybrid architecture of cognition that combines a production system (to capture the sequential, symbolic structure of cognition) with a subsymbolic, statistical layer (to capture the adaptive nature of cognition). A goal of ACT-R researchers is to investigate the overall integration of cognition by building models designed to explain how all the components of the mind work together (Anderson, 2002). Although cognitive architectures like ACT-R can offer flexibility and precision in human-like behavior representation, they have rarely been used to study economic decision making. A reason for this state of affairs is that ACT-R has mistakenly been Instance-based decision making 2 conceptualized as a rule-based static theory that does not provide the flexibility necessary for uncertain decision situations, like economic settings. This chapter will demonstrate the potential of ACT-R to model economic decision making. Economic decision making should be modeled as a learning process, involving more than calculation of expected values, accounting for human cognitive limitations and abilities, and allowing for flexibility in transfer of knowledge. This chapter summarizes evidence of successful ACT-R modeling of decision making processes of this kind. Several examples of ACT-R decision making models show that the same architecture can be used in a variety of tasks including dynamic control tasks, backgammon players and simple 2 x 2 gamers like in the Prisoner's Dilemma. We argue that for economic decision making settings as well as for many other tasks in which learning and decision making occur in unison, instance-based decision making is the most plausible learning mechanism (Gonzalez, Lerch and Lebiere, 2003). Other researchers have also theorized that instance-based decision making is a general mechanism used for all types of decision making under uncertainty (Gilboa and Schmeidler, 1995). All the models reported in this chapter have successfully used this instance-based approach in ACT-R, concluding that ACT-R can provide an integrated account of the psychology of decision making. The rest of this chapter is organized as follows: Section 2 presents an introduction to the ACT-R cognitive architecture, its knowledge representation structures and memory and learning mechanisms. Section 3 summarizes the instance-based decision making approach from the psychology and economics perspectives. Section 4 presents a compilation of results from instance-based ACT-R models in individual decision making tasks and in 2 x 2 economic games. Section 5 further demonstrates the use of ACT-R instance-based decision making models in complex and more real-world tasks. Section 6 concludes.
Journal of Dynamic Decision Making, Nov 1, 2016
Proceedings of the Human Factors and Ergonomics Society ... Annual Meeting, Sep 1, 2016
An essential skill in security involves categorizing events based on observed event attributes. T... more An essential skill in security involves categorizing events based on observed event attributes. That is, determining threat level and priority of the event when choosing an appropriate response action. To explore the basic mechanisms of learning and decision making, we conducted two experiments wherein participants were asked to categorize security events into four categories on the basis of the cues that define each event. Participants had no prior knowledge about the relationship between events and categories and through 128 categorization trials they had to learn the relationship between them using feedback received per trial in terms of rewards (higher reward for appropriate categorization). Results from the experiments demonstrate the significant role of task abstraction and experiment context in the categorization success. The effect of heuristics and knowledge on categorization performance was measured and compared. We conclude with recommendation for future experiments on learning and decision making in security event categorization.
Cognitive Science, 2016
Given the global challenges of security, both in physical and cyber worlds, security agencies mus... more Given the global challenges of security, both in physical and cyber worlds, security agencies must optimize the use of their limited resources. To that end, many security agencies have begun to use "security game" algorithms, which optimally plan defender allocations, using models of adversary behavior that have originated in behavioral game theory. To advance our understanding of adversary behavior, this paper presents results from a study involving an opportunistic crime security game (OSG), where human participants play as opportunistic adversaries against an algorithm that optimizes defender allocations. In contrast with previous work which often assumes homogeneous adversarial behavior, our work demonstrates that participants are naturally grouped into multiple distinct categories that share similar behaviors. We capture the observed adversarial behaviors in a set of diverse models from different research traditions, behavioral game theory, and Cognitive Science, illustrating the need for heterogeneity in adversarial models.
Lecture Notes in Computer Science, 2020
The evaluation of an AGI system can take many forms. There is a long tradition in Artificial Inte... more The evaluation of an AGI system can take many forms. There is a long tradition in Artificial Intelligence (AI) of competitions focused on key challenges. A similar, but less celebrated trend has emerged in computational cognitive modeling, that of model comparison. As with AI competitions, model comparisons invite the development of different computational cognitive models on a well-defined task. However, unlike AI where the goal is to provide the maximum level of functionality up to and exceeding human capabilities, the goal of model comparisons is to simulate human performance. Usually, goodness-of-fit measures are calculated for the various models. Also unlike AI competitions where the best performer is declared the winner, model comparisons center on understanding in some detail how the different modeling "architectures" have been applied to the common task. In this paper we announce a new model comparison effort that will illuminate the general features of cognitive architectures as they are applied to control problems in dynamic environments. We begin by briefly describing the task to be modeled, our motivation for selecting that task and what we expect the comparison to reveal. Next, we describe the programmatic details of the comparison, including a quick survey of the requirements for accessing, downloading and connecting different models to the simulated task environment. We conclude with remarks on the general value in this and other model comparisons for advancing the science of AGI development.
Advances in information security, Oct 7, 2022
Advances in Intelligent Systems and Computing, 2019
Deception, an art of making someone believe in something that is not true, may provide a promisin... more Deception, an art of making someone believe in something that is not true, may provide a promising real-time solution against cyber-attacks. In this paper, we propose a human-in-the-loop real-world simulation tool called HackIT, which could be configured to create different cyber-security scenarios involving deception. We discuss how researchers can use HackIT to create networks of different sizes; use deception and configure different webservers as honeypots; and, create any number of fictitious ports, services, fake operating systems, and fake files on honeypots. Next, we report a case-study involving HackIT where adversaries were tasked with stealing information from a simulated network over multiple rounds. In one condition in HackIT, deception occurred early; and, in the other condition, it occurred late. Results revealed that participants used different attack strategies across the two conditions. We discuss the potential of using HackIT in helping cyber-security teams understand adversarial cognition in the laboratory.
Handbook of Computer Networks and Cyber Security, 2020
Computer-based simulation tools have an important role to play in helping us understand the behav... more Computer-based simulation tools have an important role to play in helping us understand the behavior of people performing as attackers (people who launch cyberattacks) and defenders (people who protects computer networks against cyberattacks) in complex cyber situations. In this paper, we introduce a simulation tool called HackIt that could be used to build dynamic cyberattack scenarios. We used the HackIt tool to investigate the influence of timing of deception strategies involving honeypots (computers that pretend to be real, but those that are actually fake) on the decisions of participants performing as attackers. In a lab-based experiment, participants performing as attackers were randomly assigned to two between-subjects conditions, each involving six repeated games: early (N = 20) and late (N = 20). In early condition, deception was present via honeypots on the second and third games, whereas in late condition, deception was present via honeypots on the fourth and fifth games. Presence of deception meant that the honeypots were easy to exploit in deception rounds. In both conditions, the goal of attacker was to steal credit-card information for computers on the network. Results revealed that the proportion of honeypot attacks were higher in late condition compared to early condition. Similarly, we found that the proportion of regular attacks were lower in late condition compared to early condition. We highlight the potential of using the HackIt tool for creating realistic cyberscenarios and evaluating the effectiveness of different deception strategies in reducing cyberattacks.
2017 International Conference On Cyber Situational Awareness, Data Analytics And Assessment (Cyber SA), 2017
With the growth of digital infrastructure, cyber-attacks are increasing in the real-world. Cyber-... more With the growth of digital infrastructure, cyber-attacks are increasing in the real-world. Cyber-attacks are deliberate exploitation of computer systems, technology-dependent enterprises, and networks. Deception, i.e., the act of making someone believe in something that is not true, could be a way of countering cyber-attacks. In this paper, we propose a real-time simulation environment (“Deception Game”), which we used to evaluate and model the decision making of hackers in the presence of deception. In an experiment, using a repeated Deception Game (N = 100 participants), we analyzed the effect of two factors on participants' decisions to attack a computer network: amount of deception used and the timing of deception. Across 10-attack trials, the amount of deception used was manipulated at 2-levels: low and high. The timing of deception was manipulated at 2-levels: early and late. Results revealed that using late and high deception caused a reduction in attacks on regular webserver compared to early and low deception. Furthermore, we developed a cognitive model of hacker's decision-making using Instance-Based Learning (IBL) Theory, a theory of decisions from experience. The parameters obtained from the model helped explain the reasons for our experimental results.
Journal of Dynamic Decision Making, 2015
Computational models of learning and the theories they represent are often validated by calibrati... more Computational models of learning and the theories they represent are often validated by calibrating them to human data on decision outcomes. However, only a few models explain the process by which these decision outcomes are reached. We argue that models of learning should be able to reflect the process through which the decision outcomes are reached, and validating a model on the process is likely to help simultaneously explain both the process as well as the decision outcome. To demonstrate the proposed validation, we use a large dataset from the Technion Prediction Tournament and an existing Instance-based Learning model. We present two ways of calibrating the model’s parameters to human data: on an outcome measure and on a process measure. In agreement with our expectations, we find that calibrating the model on the process measure helps to explain both the process and outcome measures compared to calibrating the model on the outcome measure. These results hold when the model is...
Journal of Dynamic Decision Making, 2016
Before making a choice we often search and explore the options available. For example, we try clo... more Before making a choice we often search and explore the options available. For example, we try clothes on before selecting the one to buy and we search for career options before deciding a career to pursue. Although the exploration process, where one is free to sample available options is pervasive, we know little about how and why humans explore an environment before making choices. This research contributes to the clarification of some of the phenomena that describe how people perform search during free sampling: we find a gradual decrease of exploration and, in parallel, a tendency to explore and choose options of high value. These patterns provide support to the existence of learning and an exploration-exploitation tradeoffs that may occur during free sampling. Thus, exploration in free sampling is not led by the purely epistemic value of the available options. Rather, exploration during free sampling is a learning process that is influenced by memory effects and by the value of the o...
Cognitive Science, 2021
This work is an initial step toward developing a cognitive theory of cyber deception. While widel... more This work is an initial step toward developing a cognitive theory of cyber deception. While widely studied, the psychology of deception has largely focused on physical cues of deception. Given that present‐day communication among humans is largely electronic, we focus on the cyber domain where physical cues are unavailable and for which there is less psychological research. To improve cyber defense, researchers have used signaling theory to extended algorithms developed for the optimal allocation of limited defense resources by using deceptive signals to trick the human mind. However, the algorithms are designed to protect against adversaries that make perfectly rational decisions. In behavioral experiments using an abstract cybersecurity game (i.e., Insider Attack Game), we examined human decision‐making when paired against the defense algorithm. We developed an instance‐based learning (IBL) model of an attacker using the Adaptive Control of Thought‐Rational (ACT‐R) cognitive archi...
Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2020
Many cybersecurity algorithms assume adversaries make perfectly rational decisions. However, huma... more Many cybersecurity algorithms assume adversaries make perfectly rational decisions. However, human decisions are only boundedly rational and, according to Instance-Based Learning Theory, are based on the similarity of the present contextual features to past experiences. More must be understood about what available features are represented in the decision and how outcomes are evaluated. To these ends, we examined human behavior in a cybersecurity game designed to simulate an insider attack scenario. In a human-subjects experiment, we manipulated the information made available to participants (concealed or revealed decision probabilities) and the framing of the outcome (as losses or not). An endowment was given to frame negative outcomes as losses, but these were not framed as losses when no endowment was given. The results reveal differences in behavior when some information is concealed, but the framing of outcomes only affects behavior when all information is available. A cognitive...
Proceedings of the Annual Hawaii International Conference on System Sciences, 2020
This paper improves upon recent game-theoretic deceptive signaling schemes for cyber defense usin... more This paper improves upon recent game-theoretic deceptive signaling schemes for cyber defense using the insights emerging from a cognitive model of human cognition. One particular defense allocation algorithm that uses a deceptive signaling scheme is the peSSE (Xu et al., 2015). However, this static signaling scheme optimizes the rate of deception for perfectly rational adversaries and is not personalized to individuals. Here we advance this research by developing a dynamic and personalized signaling scheme using cognitive modeling. A cognitive model based on a theory of experiential-choice (Instance-Based Learning Theory; IBLT), implemented in a cognitive architecture (Adaptive Control of Thought-Rational; ACT-R), and validated using human experimentation with deceptive signals informs the development of a cognitive signaling scheme. The predictions of the cognitive model show that the proposed solution increases the compliance to deceptive signals beyond the peSSE. These predictions were verified in human experiments, and the results shed additional light on human reactions towards adaptive deceptive signals.
Proceedings of the Annual Hawaii International Conference on System Sciences, 2020
Uploads
Papers by Cleotilde Gonzalez