Rationality and Ethics in Artificial Intelligence

Rationality and Ethics
in Artificial Intelligence
Rationality and Ethics
in Artificial Intelligence
Edited by
Boris D. Grozdanoff, Zdravko Popov

and Silviya Serafimova
Rationality and Ethics in Artificial Intelligence
Edited by Boris D. Grozdanoff, Zdravko Popov and Silviya Serafimova
This book first published 2023
Cambridge Scholars Publishing
Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK
British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library
Copyright © 2023 by Boris D. Grozdanoff, Zdravko Popov,

Silviya Serafimova and contributors
All rights for this book reserved. No part of this book may be reproduced,
stored in a retrieval system, or transmitted, in any form or by any means,
electronic, mechanical, photocopying, recording or otherwise, without
the prior permission of the copyright owner.
ISBN (10): 1-5275-9441-6

ISBN (13): 978-1-5275-9441-8
TABLE OF CONTENTS
Acknowledgements .................................................................................. vii
Chapter I ..................................................................................................... 1
A Brief History of Computer Ethics and how it is Connected to AI Ethics
Mariana Todorova
Chapter II .................................................................................................. 10
Ethical Clashes in the Prospects for Autonomous Vehicles
Silviya Serafimova
Chapter III ................................................................................................ 35

Ethical Challenges to Artificial Intelligence in the Context of Pandemic
and Afterwards
Iva Georgieva
Chapter IV ................................................................................................ 52
The AI-Run Economy: Some Ethical Issues
Anton Gerunov
Chapter V ................................................................................................. 71
Scratch My Back & I Will Scratch Yours: Putting Game Theory and
Artificial Intelligence on a Converging Path beyond Computational
Capabilities
Boris Gurov
Chapter VI ................................................................................................ 93
Artificial Intelligence in Defence
Todor Dimitrov
Chapter VII ............................................................................................. 116

Process Mining with Machine Learning
Nikola Sotirov
vi Table of Contents
Chapter VIII ........................................................................................... 131

Approaching the Advanced Artificial Intelligence
Alexander Lazarov
Chapter IX .............................................................................................. 152

The Structure of Artificial Rationality
Boris Grozdanoff
Chapter X ............................................................................................... 199

Discriminator of Well-Formed Formulae as a Component of Artificial
Human Rationality
Dimitar Popov
ACKNOWLEDGEMENTS
This volume includes a selection of papers, written on the basis of talks

delivered at three high-level conferences on Artificial Intelligence (AI), held
in Sofia in 2018 and 2019. The topics cover a broad spectrum of AI related
themes, among which approaches to artificial general intelligence (AGI)
and human-like reasoning models, the problem of ethical AI, modern
implementations of AI in the economy and defense sectors. The editors want
to thank cordially to Prof. Anastas Gerdjikov, rector of the University of
Sofia, Academician Julian Revalski, chairman of the Bulgarian Academy of
Sciences, Gen Maj. Grudi Angelov, rector of the National Defense Academy
“Georgi Sava Rakovski”, the Japanese Ambassador to the Republic of
Bulgaria H.E. Masato Watanabe, Dr. Ivo Trayanov, chairman of the
Defense and International Security Institute (DISI), Prof. Zdravko Popov,
in his quality of the chairman of the Public Policy Institute (PPI), Dr. Karina
Angelieva, deputy minister of science and education, Dr. Hiroshi Yamakawa,
the Director of the Dwango AI Laboratory and Chairperson of the Whole
Brain Architecture Initiative (WBAI), and former Chief Editor of the
Japanese Society for Artificial Intelligence (JSAI) and Dr. Momtchil
Karpouzanov from the American University in Bulgaria (AUBG). The
editors also wish to cordially thank Dr. Dimitar Popov for his invaluable
help in proofreading the manuscript and editing the formalism.
CHAPTER I
A BRIEF HISTORY OF COMPUTER ETHICS

AND HOW IT IS CONNECTED TO AI ETHICS
MARIANA TODOROVA
In his article “A very short history of computer ethics”,1 the author Terrell
Bynum defines computer ethics as a scientific field that emerged in the first
years of the outbreak of World War II, beginning with Massachusetts
Institute of Technology professor Norbert Wiener. He defined the new field
while participating in the development of an anti-aircraft weapon, the
purpose of which was to intercept and track an enemy aircraft, then calculate
its probable trajectory and inform the other parts of the weapon to activate
the projectile. The emerging challenges for engineers, according to Bynum,
led to the creation of a new branch of science by Wiener and his colleagues,
which they called cybernetics, the science of information feedback systems.
It was cybernetics combined with the digital computers created at the time
that motivated the inventor to draw several important ethical conclusions.
In 1950, he (Wiener) published the first book of its kind on computer ethics
(though he nowhere explicitly calls his reasoning that way): The Human Use
of Human Beings,2 where he spoke of the benefits and risks of automation
and the use of computers. The text sounds like a come true (self-fulfilling
prediction), as Wiener predicts that computers will enhance human
capabilities, free people from repetitive manual labor, but also allow for
processes of dehumanization and subordination of the human species. The
1 Bynum,”A Very Short History of Computer Ethics”, accessed July 8, 2022,

https://web.archive.org/web/20080418122849/http:/www.southernct.edu/organizati
ons/rccs/resources/research/introduction/bynum_shrt_hist.html.
2 Norbert Wiener, The Human Use of Human Beings. Cybernetics and Society (New
York: Doubleday Anchor Book, 1950).

2 Chapter I
author warns us not to accept computers as available entities, but to always

keep in mind that they are trained (something that actually happens with
machine learning) and can go beyond our control. Such a development is a
prerequisite for complete dependence and even for control over humanity.
The danger, he said, comes from the fact that computers cannot think
abstractly and therefore cannot comprehend and evaluate human values.
Wiener adds that the invention of digital computers will lead to a second
industrial revolution, which will have multi-layered dimensions, will unfold
for decades and will lead to radical changes. For these reasons, he explicitly
warns in the chapter, “Someone Communication Machines and Their
Future,” as well as throughout the book, that workers must adapt to changes
in their jobs. Governments need to draft new laws and regulations.
Industries and businesses need to create new policies and practices.
Professional organizations need to prepare new codes for their members.
Sociologists and psychologists need to study new phenomena, and
philosophers need to rethink and redefine outdated social and ethical
concepts.
Norbert Wiener made valuable recommendations nearly 70 years ago,

which unfortunately do not find systematic application to this day. The
changes he describes are under way, the exponential development of
technology is accelerating, and although there are many projects of
universities and research centers on the machine ethics of artificial
intelligence, there is still a lack of serious discussion and consensus on key
issues that touch.
Walter Manner was the researcher who formalized the term “computer
ethics”, defining it as part of the applied (ethics) in his work: Starter Kit on
Teaching Computer Ethics3 in 1976. He devoted his subsequent work to
efforts to emancipate this title as a separate scientific field. The intention to
strictly distinguish computer ethics from fundamental ethical issues is
implicit.
3Walter Maner, Starter Kit on Teaching Computer Ethics (Self-published in 1978.

Republished in 1980 by Helvetia Press in cooperation with the National Information
and Resource Center for Teaching Philosophy).
A Brief History of Computer Ethics and how it is Connected to AI Ethics 3
James Moore, also known by his law of the same name, also dedicated an
article4 on this issue. He believes that the ambiguity surrounding computer
ethics arises because there is a political vacuum over how to use computer
technology. Through computers, we acquire new abilities that provide us
with new opportunities and choices for action. Very often, according to him,
there are no political measures for such situations, and if there are any, they
are inadequate. For Moore, a central task in computer ethics is to determine
what we need to do in specific computer-related situations, such as
formulating policies and action guides. Computer ethics, according to him,
must take into account both individual and social rights and policies.
Therefore, it identifies four areas of computer ethics: 1) identifying the
computer-generated policy vacuum; 2) clarification of conceptual
ambiguities; 3) formulation of policies for the use of computer technologies
4) ethical justifications.
Moore correctly outlines the steps that should be taken to fill the ethical and
then legal and regulatory gaps, but fails to point out that this task is too
ambitious to be executed. Philosophers, anthropologists, psychologists and
neuroscientists must take the lead in such a task, but they must work
alongside representatives of labor and social systems, education, medicine,
security and military technology. That is, with experts from all fields who
will be influenced by computer technology and artificial intelligence.
Deborah Johnson also contributed to this issue. In her article, “Computer

Ethics”,5 she defines it as a field that explores how computers provoke new
varieties of classical moral problems and how they worsen and deepen when
we apply common moral norms to emerging spheres. She does not share the
opinion that this should be a new part of ethics, but simply that a new
perspective is set for problems concerning property, power, personal space
and responsibility, etc.
4 Moor, James H. “What Is Computer Ethics?” In Computers and Ethics, ed. Terrell
Ward Bynum (Basil Blackwell, 1985), 266 – 275.
5 Deborah Johnson, “Computer Ethics”, Prentice-Hall, reprinted by Metaphilosophy,
Vol.16, No. 4 (October 1985): 319-322.

4 Chapter I
Back in 1995, Krystyna Górniak-Kocikowska predicted in her article6 that

computer ethics, then considered still part of applied ethics, would evolve
into a system of global ethics applicable to every culture in the world.
She associates it with the times of the print media, arguing that Bentham
and Kant developed their ethical theories in response to this discovery, and
believes that the same will be repeated with computer ethics, which must
respond to the computer-induced revolution. According to her, the nature of
the expected phenomenon (computer revolution) is such that the ethics of
the future will have a global character, in the sense that it will also address
the integrity of human actions and relationships. Because computers know
no boundaries and operate globally, they will set the stage for a new global
universal ethic that must be universal to all human beings. The author adds
that all local ethics (considering both individual areas and cultural and
regional features (of Asia, Africa, Europe, America, etc.) may grow into a
global ethic, inheriting computer ethics in information era.
Johnson (1985) also talks about global ethics, but uses a different meaning.
For her, it will belong to new kinds of extensive moral problems. Inherited
and contemporary ethical theories will continue to be fundamental, and this
will not lead to a revolution in ethics. With the help of Terrell Bynum, two
opposite concepts of computer ethics are revealed to us. On the one hand,
there is the thesis of Wiener, Maner and Gorniyak-Kosikovska about a
revolution in ethics and an obligation of humanity to rethink its very
foundations, as well as of human life. On the other hand, Johnson's more
conservative view is presented, which defends the position that ethics will
remain “untouched” and that these are the same old ethical issues in a new
reading, which in turn will make computer ethics meaningless as a separate
part.
In the debate thus outlined, we can agree in part with statements from both
theses. Ethics that address information and computer technology, as well as
artificial intelligence, will be global and universal, as responses to the
6Górniak-Kocikowska, Krystyna. “The Computer Revolution and the Problem of

Global Ethics.” In Global Information Ethics, eds. Terrell Ward Bynum and Simon
Rogerson (Opragen Publications, 1996), 177–190.
consequences of interacting with artificial intelligence will not be a regional

problem or only within nation states. We can assume nuances in the attitude
and legal regulations towards the narrow, specialized artificial intelligence,
which is not yet competitive, in terms of human brain capacity and
awareness. For some cultures, nationalities, or global companies, it will
probably be permissible for the personal assistant, artificial intelligence, to
have a level of trust with which to perform delegated functions and
decisions. For others, this will not be equally valid.
However, when addressing general or superartificial intelligence, it will be

necessary for ethical aspects to be universally valid on a planetary level,
addressed to all mankind.
Will the code of ethics for artificial intelligence be objective if the

discussion is dominated by catastrophic or overly optimistic scenarios?
Mara Hvistendahl, a scientific correspondent for the Guardian, in her article

“Can we stop AI outsmarting humanity”7 considers (only) the emergence of
artificial intelligence after about 30 years as the end of human evolution,
began 50,000 years ago with the erection of the species Homo Sapiens and
5,000 years ago with the emergence of the phenomenon of civilization,
citing the scientist Jean Tallinn. She is a co-founder of Skype and is strongly
influenced by the scientist Elizer Yudkowski, according to whom artificial
intelligence can hypothetically destroy humanity. On this occasion, Tallinn
became a regular donor to the organization of Yudkowski, which dedicates
its work to a project for the so-called “friendly artificial intelligence”. The
category of “friendly,” according to Hvischendal, does not mean a reduction
to skills such as dialogue with people or that it will be guided only by love
and altruism. According to her, “friendly” can be defined as having human
motivation, impulses and values. That is, it should not be useful for the
machines of the future to arrive at the conclusion that they must erase us in
order to achieve their goals. For these reasons, Tallinn founded the Center
for the Study of Existential Risk in Cambridge.
7 Mara Hvistendahl, “Can We Stop Outsmarting Humanity?”, The Guardian, Match
28, 2019.
6 Chapter I
The concept behind Tallinn is that software does not need to be programmed
to destroy humanity, but can “decide” so along the course of its existence.
As we have already noted, this might be the product of a small error,
software bug, etc. The scientist also refers to the example of Bostrom, who
reveals that artificial intelligence could decide that the atoms in the human
body are a good raw material and can be used in another way as a resource.
Objections to such arguments come from the Technology Guild, which says
it is too early to seek a solution against hostility. They recommend shifting
the focus to current problems, such as the fact that most of the algorithms
were created by white men and that this fact has given rise to the biases that
accompany them.
Stuart Armstrong of the Future of humanity institute at the Oxford Institute

also deals with these issues. Armstrong even goes further and suggests that
purely physical intelligence be confined to a container and limited to
answering questions. Its strategy is to protect people from possible
manipulation. It also proposes to have a mechanism for disconnection from
people or self-exclusion from the software itself under certain conditions.
However, Armstrong fears that these conditions may be sufficient as a
measure because artificial intelligence can learn to protect itself or at least
develop “curiosity”, as there is such an option. In this context, Hvistendahl
reports on a programmer, Tom Murphy VII, who invented a program that
could teach itself to play Nintendo computer games. In practice, this
software is invincible and the only way to stop it is not to play (ibid.).
Tallinn, on the other hand, believes that even if the power button is masked
and not of interest to artificial intelligence, there is still no solution to the
problem of potential threat, as artificial intelligence may have secretly
replicated itself hundreds of thousands of times on the web. For these
reasons, researchers and practitioners are united around the idea of artificial
intelligence being taught to recognize and study human values. That is,
according to Tallinn, he must learn to evaluate people outside the canons of
strict logic. For example, that we often say one thing but think another, that
we enter conflicts or think differently when we are drunk, and so on. a state
of irrationality (ibid.).
Tallinn takes as its best formula the statement of the Cambridge philosopher
Hugh Price, who defines that artificial intelligence in ethical and cognitive
aspects should be like a “superman”. Other questions arise - if we do not
want artificial intelligence to dominate us, then should we surpass it. And
these questions again inevitably lead us to the presence of consciousness
and free will in artificial intelligence.
Boris Grozdanoff in his article “Prospects for a Computational Approach to

Savulescu’s Artificial Moral Advisor”8 proposes the creation of a universally
valid homogeneous human ethic that can be codified so that artificial
intelligence can study it and to create “Artificial moral agent” (AMA).
According to him, ethics is also a product of human evolution, which was
developed by science and, in particular by philosophy, to create a set of rules
to bring order to society through which it can survive and prosper.
Grozdanoff launched the thesis of circumstantial normativity in the form of
an ethical system. It has to be formalized and axiomatized, which seems like
a rather complicated and almost impossible task. In addition to being
unified, encircled, and translatable into languages that AI can understand, it
must also survive its incarnation in a general/general AI program. For
Grozdanoff, the solution lies in the construction of a semantic engine
(AMA) that can deliver and handle epistemological processes.
The formula that Hugh Price, Boris Grozdanoff and other scientists offer is
correct, but much work is needed to precede it. Today, we are witnessing a
resurgent wave of neoconservatism, which sharply criticizes liberal theories
such as multiculturalism, globalism, etc. In parallel with these processes, we
can predict a resurgence of nationalism, hardening of the concepts of
“nation states” and “identities”. This context would certainly prevent
attempts to seek a universally valid formula for human ethics that could
eventually be applied as a matrix for the training of general artificial
intelligence. Cultural diversity and different human civilizational norms do
not share the same views on the categories of “good” and “bad”, human
8Boris D. Grozdanoff, “Prospects for a Computational Approach to Savulescu’s

Artificial Moral Advisor,” ȿɬɢɱɟɫɤɢ ɢɡɫɥɟɞɜɚɧɢɹ, ɛɪ.5, No. 3 (December 2020):
107-120.
8 Chapter I
rights, etc. their definition, which mostly fit world-class institutions such as
the UN.
There is a call from the European Commission to companies, when creating

new software involving artificial intelligence, to use the integration of
ethical rules as a competitive advantage. In a document published on April
7, 2019, Europe stated that it would invent and propose global standards to
guide all other players in this field. Such requests provoked sharp
comments, including from Daniel Castro, vice president of the Information
Technology and Innovation Foundation (ITIF).
Undoubtedly, it is crucial that there is a global strategic factor in the face of

the European Commission in particular and the European Union in general,
which is concerned and will try to impose and take precedence in ethical
standards and frameworks for artificial intelligence. The problem is that this
position will most likely be peripheral and non-binding.
UNESCO is also developing a solid and intergovernmental document - an

ethical framework for artificial intelligence, which will also provide
important and meaningful recommendations. From now on, it is important
that all scientific papers and articles “meet” with political documents'
positions to find the best solution for the ethical development of artificial
intelligence.
References
Bynum, Terrell Ward. “A Very Short History of Computer Ethics”.
Accessed July 8, 2022.
https://web.archive.org/web/20080418122849/http:/www.southernct.edu/o
rganizations/rccs/resources/research/introduction/bynum_shrt_hist.html.
Górniak-Kocikowska, Krystyna. “The Computer Revolution and the
Problem of Global Ethics.” In Global Information Ethics, edited by
Terrell Ward Bynum and Simon Rogerson, 177–190. (Guildford:
Opragen Publications, 1996).
Grozdanoff, Boris D. “Prospects for a Computational Approach to
Savulescu’s Artificial Moral Advisor.” ȿɬɢɱɟɫɤɢ ɢɡɫɥɟɞɜɚɧɢɹ. ɛɪ. 5.,
No. 3 (2020): 107-120. https://jesbg.com/eticheski-izsledvania-br-5-

kn3-2020/
Hvistendahl, Mara. “Can We Stop Outsmarting Humanity?” The Guardian,
March 28, 2019.
https://www.theguardian.com/technology/2019/mar/28/can-we-stop-
robots-outsmarting-humanity-artificial-intelligence-singularity.
Johnson, Deborah. “Computer Ethics”, Prentice-Hall, reprinted by
Metaphilosophy, Vol.16, No. 4 (1985): 319-322.
Maner, Walter. 1978. Starter Kit on Teaching Computer Ethics (self-
published in 1978. Republished in 1980 by Helvetia Press in cooperation
with the National Information and Resource Center on Teaching
Philosophy).
Moor, James H. “What Is Computer Ethics?” In Computers and Ethics,
edited by Terrell Ward Bynum, 266–275. (Basil Blackwell, 1985).
Wiener, Norbert. 1950. The Human Use of Human Beings. Cybernetics and
Society. New York: Doubleday Anchor Book.
CHAPTER II
ETHICAL CLASHES IN THE PROSPECTS

FOR AUTONOMOUS VEHICLES
SILVIYA SERAFIMOVA
Introduction
Clarifications
The moral issues regarding the use of autonomous vehicles (AVs) are not a
new phenomenon in the ethical discourse.1 Some of them date back to the
concerns about the so-called trolley dilemma, introduced by Philippa Foot
in 1967. The dilemma is a thought experiment according to which a
fictitious onlooker can choose to save five people in danger of being hit by
a trolley, by diverting the trolley to kill just one person.2
At first sight, the trolley dilemma looks like a utilitarian moral dilemma,
which is based upon calculating the maximization of well-being for more
representatives at the expense of the suffering of the few. If that were the
case, there would be no dilemma whatsoever. The solution would be one to
switch the trolley so that the five people can be saved. However, such a
decision is an act utilitarian decision par excellence.
In turn, the trolley dilemma is modified within so-called moral design

problem, which addresses the moral challenges in building AVs.3 In this
1 For the challenges in relating the trolley cases to the ethics of AVs, see Geoff
Keeling,”The Ethics of Automated Vehicles,” (PhD thesis, University of Bristol,
2020), 45-68.
2 Philippa Foot, “The Problem of Abortion and the Doctrine of the Double Effect,”
Oxford Review, No. 5 (1967): 5-15.

3 Geoff Keeling, “Commentary: Using Virtual Reality to Assess Ethical Decisions
in Road Traffic Scenarios: Applicability of Value-of-Life-Based Models and

Ethical Clashes in the Prospects for Autonomous Vehicles 11
context, moral programming can be examined as displaying adaptations of

the trolley dilemma, with the difference that the AVs are preprogrammed to
make such decisions.4
Automated vehicle technologies are “the computer systems that assist

human drivers by automating aspects of vehicle control” including a wide
range of capabilities such as antilock brakes and forward collision warning,
adaptive cruise control and lane keeping, as well as fully automated
driving.5 The moral concerns derive from the findings of theoretical
research robotics, which show that crash-free environment is unrealistic.6
This means that if a crash is unavoidable, “a computer can quickly calculate
the best way to crash on the basis of a combination of safety, the likelihood
of the outcome, and certainty in measurements much faster and with greater
precision than a human can”.7
Current projects for AVs aim at building partially autonomous vehicles,

assuming that drivers can take back control of the vehicle under given
circumstances. Such AVs are considered as artificial moral agents (AMAs)
belonging to Levels 3 and 4 of NHTSA’s classification.8 According to the
more “optimistic” projects of fully autonomous AVs, one should build AVs
as artificial autonomous moral agents (AAMAs) belonging to Level 5 of the
same classification. The moral challenges in building AVs concern the
strive of the engineers for developing a “universally accepted moral code
Influences of Time Pressure,” Front. Behav. Neurosci, No. 11 (December 2017):

247; Keeling, Geoff. “Against Leben’s Rawlsian Collision Algorithm for Autonomous
Vehicles.” In Philosophy and Theory of Artificial Intelligence 2017 (Studies in
Applied Philosophy, Epistemology and Rational Ethics. 44), ed. Vincent C. Müller
(Springer 2018), 259-272. https://doi.org/10.1007/978-3-319-96448-5_29. I refer to
the online version of this publication.
4 Darius-Aurel Frank, Polymeros Chrysochou, Panagiotis Mitkidis and Dan Ariely,
“Human Decision-Making Biases in the Moral Dilemmas of Autonomous

Vehicles,” Sci Rep, No 9, 13080 (September 2019): 2.
5
Noah Goodall, “Ethical Decision Making during Automated Vehicle Crashes,”
Transp. Res. Record J., Transp. Res. Board 2424, No.1 (January 2014): 58.
6 Goodall, “Ethical Decision Making during Automated Vehicle Crashes,” 59.
7 Goodall, “Ethical Decision Making during Automated Vehicle Crashes,” 60.
8
NHTSA’s classification of AVs includes the following levels: no automation
(Level 0), driver assistance (Level 1), partial automation (Level 2), conditional
automation (Level 3), high automation (Level 4) and full automation (Level 5).
12 Chapter II
that could guide the machines’ behavior”.9 The largest project of how one
can “train” AI morality in such situations is so-called Moral Machine
experiment. It is an online experimental platform that is designed to
examine the moral dilemmas faced by AVs. It collects data of millions of
moral decisions for the purposes of training machine-learning algorithms.10
Similar to the methodological pitfalls in objectifying the moral outcomes of

the trolley dilemma, these regarding the Moral Machine experiment depend
upon who decides for whom under what circumstances. Therefore, the moral
outcomes can be evaluated by taking into account the plurality of the
intersecting perspectives (passenger, pedestrian, observer), as well as the
decision-making modes (deliberate, intuitive).11
The aforementioned specifications show that if one wants to find objective

and morally justifiable solutions to the AV scenarios, one should constructively
evaluate the role of different human predispositions in the process of moral
decision-making. Recognizing the role of biases is of crucial importance for
the AV manufacturers since “sourcing people’s moral preferences on a large
scale requires developing a standardized and reliable instrument that
actually controls for participants’ perspective and decision-making mode”.12
Structure
The main objective of this paper is to demonstrate why finding some
potential solutions to the moral design problem and Moral Machine
experiment requires one to recognize the challenges in building AVs as
moral rather than purely computational challenges. For the purposes of
exemplifying the latter, I tackle the benefits and disadvantages of two types
of AVs projects, viz. the Rawlsian collision algorithm, which displays a
9 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the

Moral Dilemmas of Autonomous Vehicles,” 1.



contractualist project and some of Frank et al.’s thought experiments which

represent utilitarian and deontological scenarios.13
The Section “A contractualist prospect for AVs” is devoted to the role of

the Rawlsian collision algorithm. In addition to the investigation of
Keeling’s criticism of this algorithm, in Section “Rawlsian collision
algorithm” I analyze why the way in which Leben elaborates upon Rawls’
theory of the original position and the maximin rule necessitates one to
rethink the role of value-of-life heuristics for the worst-off people. I also
explore how such an analysis can contribute to revealing the diversity of
core values: specifically, the values of survival probability and survival.
These values are examined as related to the computation of life and death in
AV accidents, as represented in Keeling’s three scenarios. The scenarios are
discussed in Section “Exemplifying Rawlsian collision algorithm”. In
Section “The role of biases”, I aim to explore the impact of people’s biased
moral preferences upon the evaluation of survival probabilities.
In Section “Building “utilitarian” AVs”, I analyze some Frank et al.’s thought

experiments regarding the complicated use of AVs. In Section “The role of
experimental ethics”, special attention is paid to the challenges posed by
evaluating the results through the methods of experimental ethics.
Consequently, in Section “Some “hybrid” moral explanations”, I investigate
why the difficulties in justifying moral autonomy of AVs are driven by the
way in which Green et al.’s exploration makes room for two triplets–these of
emotions–intuitive decision-making–deontological decision-making and
cognitive knowledge–deliberate decision-making–utilitarian decision-making.
The objective is to demonstrate why the exaggerated trust in the triplets may
trigger the misrecognition of a solution to one-versus-many case as a
utilitarian solution, while it can be driven by purely deontological motivation.
For the purposes of revealing the reasons behind the conflation of
deontological and utilitarian decisions within the AV scenarios, I also
examine the role of what I call utilitarian explanatory bias.
13 The choice of projects demonstrates how both theoretical ethical approaches

(these adopted in Rawlsian algorithm) and empirical ethical approaches (these
incorporated into Frank et al.’s experiments) require further elaboration.
14 Chapter II
A Contractualist Prospect for AVs

Rawlsian Collision Algorithm
The moral design problem inherits some of the moral concerns about the
trolley dilemma, when examined from a utilitarian perspective. That is why
one should look to adopt another approach. An illuminative example of such
an approach is found in Rawls’ theory of justice. It is elaborated upon by
Leben into so-called Rawlsian collision algorithm.
Leben’s “contractualist” answer to the moral design problem is grounded

into two main ideas borrowed from Rawls–these of the original position and
the maximin rule.14 The original position is “a hypothetical situation in
which representative citizens decide on principles of justice to regulate the
basic structure of society from a list of alternatives”.15 Each party represents
the interests of a sub-group of citizens and all citizens have a
representative.16 The parties in the original position are supposed to decide
from the perspective of so-called veil of ignorance. This is a situation where
“no one knows his place in society, his class, position, or social status; nor
does he know his fortune in the distribution of natural assets, his strength,
intelligence, and the like”.17
For the purposes of maintaining the objective foundations of justice, Rawls

introduces so-called maximin decision procedure. The gist of the latter
concerns the selection of principles, which provide “the greatest allocation
of primary goods to the worst-off citizens”.18 The point is some minimal set
14 Keeling, “Against Leben’s Rawlsian Collision Algorithm for Autonomous
Vehicles,” 4.
15 John Rawls, Theory of Justice (Cambridge, Mass.: The Belknap Press of Harvard
University Press, 1971), 122. Keeling, “Against Leben’s Rawlsian Collision

Algorithm for Autonomous Vehicles,” 2.
16 Keeling, “Against Leben’s Rawlsian Collision Algorithm for Autonomous
Vehicles.”, 2.
17 Rawls, Theory of Justice, 137. Keeling, ”Against Leben’s Rawlsian Collision

18 Rawls, Theory of Justice, 150-161. Keeling, ”Against Leben’s Rawlsian Collision

of rights, liberties and opportunities for the worst-off citizens to be

guaranteed.19
Regarding the maximin rule, Leben modifies it due to the specificity of

survival probabilities. As Keeling points out, the “iterated form of maximin
described by Leben is called leximin”.20 The leximin rule resembles the
maximin rule, while comparing the survival probabilities of the worst-off
person on each alternative. However, the leximin can randomize between
two or more alternatives having identical profiles of survival probabilities.21
In other words, the leximin can compare the second-lowest survival
probabilities with the remaining alternatives. By doing so, it can select the
highest survival probability to the second worst-off person.
Such a conceptualization raises some significant challenges because the

algorithms do not take into account the moral agents’ value-of-life
heuristics. These agents are recognized as belonging to the group of the
worst-off people just because their life is at stake.
Furthermore, such algorithms rely upon one formalized in a moral sense

presumption, namely, that survival is the highest good for all group
members. Certainly, no one questions the fulfilled probability for survival
as being high good in itself. However, the issue is that there might be
representatives of the worst-off group who prefer to die rather than
surviving with debilitating injuries.22
Leben himself is aware of this challenge, admitting that some non-fatal

injuries might be evaluated as equivalent or worse than fatal injuries.23 The
problem is that when a lifelong debilitating injury is set versus fatal injury
19 Keeling, ”Against Leben’s Rawlsian Collision Algorithm for Autonomous

Vehicles,” 3.
Vehicles,”, 4, Note 3.
Vehicles,” 5.
Vehicles,” 10.
23 Derek Leben, “A Rawlsian Algorithm for Autonomous Vehicles,” Ethics Inf
Technol 19, No. 2 (March 2017): 111.
16 Chapter II
within the Rawlsian collision algorithm, the fatal injury is given priority in
making the corresponding decision.
The strive to expand the scope of the maximin rule by introducing the
leximin rule requires a reevaluation of Rawls’ idea of life project. The
leximin rule makes room for comparing the second-lowest survival
probabilities on the remaining alternatives, but does not shed light upon
whether it might be “more just” for the worst-off person or people to die
than to suffer debilitating injuries. In turn, this specification necessitates one
to reconsider Rawls’ original position. One should also analyze how the veil
of ignorance should be elaborated upon so that the graduation of justice can
meet the requirements of an internally differentiated group of worst-off
people.
As Keeling points out, Rawls does not assume the application of the
maximin rule as universally valid.24 He clearly denies its application as a
general principle of rational decisions in the case of risk and uncertainty.
This clarification puts in question the extrapolation of the maximin rule to
that of the leximin, when survival probabilities are at stake. One of the
reasons is that when such probabilities are tested, the parties reaching an
agreement should not be indifferent to their own and others’ life projects. A
specification that contradicts the requirements set by the veil of ignorance.
Generally speaking, the moral challenge is to reveal why Rawlsian collision

algorithm can only address the moral design problem, but cannot solve it.
Keeling describes the gist of the problem by saying that Leben’s answer
concerns not a set of moral principles, but how one builds an algorithm
based upon some principles.25 Certainly, an algorithm grounded in
contractualist principles resists some objections against potential utilitarian
collision algorithms.26 However, I would argue that in the strive for avoiding
utilitarian relativism, one revives some crucial moral concerns.

Vehicles,” 8.
Vehicles,” 5.
Vehicles,” 5.
Correspondingly to the pitfalls of utilitarian moral math, one may face a

questionable contractualist moral math, which “threatens” moral evaluation
with its formal regulations.
Exemplifying Rawlsian collision algorithm

Keeling describes three challenges which should be overcome if Leben’s
answer to the moral design problem is proved as satisfactory.27
Scenario 1
The AV can swerve left or right. If the AV swerves left, there is a 0% chance
that its passenger will sustain a fatal injury and a 100% chance that its
passenger will sustain a lifelong debilitating injury. If the AV swerves right,
there is a 1% chance that its passenger will sustain a fatal injury and a 99%
chance that its passenger will remain unharmed.
According to Keeling, Leben’s algorithm chooses to swerve left because it

gives the passenger the greatest survival probability.28 Certainly, dividing
rational preferences into strict and weak preferences necessitates the
definition of the preferences to survival as strict preferences and those to
non-fatal injuries–as weak preferences. However, regardless of the fact that
the preference for survival is considered a strict preference in logical terms,
it may turn out that it is a weak preference in moral terms. Extrapolating
Keeling’s concern about the selection of an alternative, which is not in the
passenger’s rational self-interest,29 I would argue that the more serious
problem is when the programming is not in the passenger’s moral self-
interest either.

Vehicles,” 9.
Vehicles,” 10.
Vehicles,” 10-11.
18 Chapter II
Scenario 2
Keeling’s main concern about the second scenario is that the maximin rule
gives “undue weight to the moral claims of the worst-off”.30
The AV can swerve left or right. If the AV swerves left, there is a 100%
chance that its passenger will die, and twenty nearby pedestrians will be
unharmed. If the driverless car swerves right, there is a 99% chance that its
passenger will die, and a 100% chance that twenty nearby pedestrians will
receive lifelong debilitating injuries.
Rawlsian algorithm selects the right swerve regardless of how many

pedestrians will receive lifelong debilitating injuries.31 Leben argues that he
would always prefer to be one of the injured pedestrians claiming that such
scenarios are unlikely to arise.32 This argument is relevantly criticized by
Keeling who claims that the low probability does not make the moral
concerns of the pedestrians less important.33
Going back to Rawls’ theory of the original position, it is apparent that

Leben’s assumption is grounded in the misinterpretation of the veil of
ignorance. In the second scenario, the parties do not choose a principle of
justice because it can provide an objective moral treatment to the group of
the worst-off people, but because they want to be fairly treated if/when they
occasionally fall into that group. Thus, the requirements of having objective
knowledge in the original position and the maximin rule are not fulfilled.
In addition to Keeling’s well-formulated concern that a personal preference

to a non-fatal but debilitating injury is “not a good moral reason” to inflict
a large number of injuries to prevent a single death,34 one should take into
Vehicles,” 11.
Vehicles,” 11.
32 Leben, “A Rawlsian Algorithm for Autonomous Vehicles,” 114; Keeling,
“Against Leben’s Rawlsian Collision Algorithm for Autonomous Vehicles,” 11.

Vehicles,” 12.
Vehicles,” 12.
consideration the problems of moral decisions’ quantification. Swerving

right, as Leben suggests, is not necessarily a morally acceptable option. This
is possible only if both the decision-makers and those belonging to the
worst-off group evaluate the survival as being the highest good.
That is why I would argue that the problems with the second scenario do not
derive from “the undue weight” to the moral claims of the worst-off people,
but rather from the fact that the due weight is not relevantly graduated in
moral terms. The lack of such a graduation affects the way in which the
group of the worst-off people is determined.
Scenario 3
Keeling points out that there is a scenario which includes an algorithm that
assigns a higher survival probability to the worst-off people than Leben’s
algorithm. This is the greatest equal chance algorithm.35
The AV can swerve left or swerve right. If the AV swerves left, there is a 0%
chance that Anne will survive, and a 70% chance that Bob will survive. If
the AV swerves right, there is a 1% chance that Bob will survive, and a 60%
chance that Anne will survive.
Leben’s algorithm programs the AV to swerve right because it assigns a

survival probability of 1% to the worst-off party.36 The third scenario brings
us back to the moral concerns about the one-versus-one case. When we have
to decide who should die, taking into account that there are only two persons
involved, the decision cannot be made on the basis of the people’s number.
One should know who these people are, as well as what their life projects
look like. Otherwise, one cannot make an informed decision in moral terms.
This difficulty is not overcome by Keeling’s algorithm of the greatest equal

chances either. Even if the AV is programmed to construct a “weighted
lottery between the alternatives, where the weightings are fixed to ensure
Vehicles,” 12.
Vehicles,” 12.
20 Chapter II
that the affected parties receive the greatest equal survival probabilities”,37 the
following problem occurs. Precising the survival probability of 32.6%, which
is certainly greater than 1%, is only the first step in resolving the moral
dilemma. The precision of survival probabilities in programming the greatest
equal chances does not trigger unquestionable moral consequences for the
affected parties. For instance, computing the greatest equal chances for
survival does not shed light upon the case when Anne, who could be a mother
of three kids, or when Benn, who could be a researcher able to find a cure for
cancer, should be sacrificed.38
Therefore, even if the most precise algorithm is elaborated upon, this

algorithm does not make the moral concerns less significant. They derive
not from the computation of life and death, but from life and death as such.
The Role of Biases

The analysis of the Rawlsian collision algorithm shows that the challenges
in building AVs derive from people’s moral decisions. The crucial role of
biases is evident in the way in which the respondents give preference to
saving the life of the pedestrian or that of the passenger(s) depending on
whether their personal perspective is made salient or not.39
While analyzing and extrapolating the findings of the Moral Machine

experiment, one should keep in mind that the Rawlsian collision algorithm
explicitly avoids the recognition of the personal perspectives of the
passenger, the pedestrian and the observer for the sake of achieving an
Vehicles,” 12.
38
Having argued that there are some collisions in which the greatest equal chances
algorithm is preferred to Leben’s algorithm” (Keeling, ”Against Leben’s Rawlsian
Collision Algorithm for Autonomous Vehicles,” 13), Keeling elaborates upon his
view saying that the greatest equal chances algorithm is “not great” either (Keeling,
“The Ethics of Automated Vehicles,” 108). The reason is that it depends upon the
ties between the affected parties, as well as assuming that each person has an equal
moral claim to be saved (Keeling, “The Ethics of Automated Vehicles,” 108). These
conditions are not satisfied in AV collisions.

optimal objectivity by applying the veil of ignorance. The idea is that the
self-preserving intentions should be reduced to a minimum so that the group
of the worst-off people can be determined in the most objective manner.
In this context, a detailed examination of the human decision-making biases

within the Rawlsian collision algorithm can contribute to firstly, limiting the
role of the parties’ dominating self-preservation attitudes and secondly,
demonstrating why neither the group of the decision-makers nor that of the
addressees of the decisions are homogenous groups of moral agents. The
second clarification, which concerns value-of-life heuristics, can make
room for evaluating the heuristics in question in positive terms as well.
Regarding the future development of the Rawlsian collision algorithm

within the framework of the Moral Machine experiment, one may
investigate its effects if a graduated social norm violation is examined as a
part of the decision-making process. Frank et al. provide a thought experiment
(Study 6) assuming that the pedestrian’s social norm violation results in
significant likelihood of being sacrificed.40 The graduation of the violated
norm varying from a low norm violation (when the pedestrian walked in a
street with no signs, traffic signals, or crosswalks), going through a control
condition (when the pedestrian walked in the crosswalk) and ending up with
a high norm violation (when the pedestrian jaywalked at a red light)41 sets
the question of coupling the issue of responsibility with that of guilt.42
If the pedestrians in the Rawlsian collision algorithm are placed in the worst-
off group due to the objectivity of the high norm violation, does it mean that
we can deprive them of the right to survive? Furthermore, do we have the

Moral Dilemmas of Autonomous
Vehicles,” 11.

42 For the different evaluations of whether or not the jaywalker’s awareness of
undertaking a risk of death or a serious

harm assumes that they tacitly consent to being harmed or killed, see Keeling, “The
Ethics of Automated Vehicles,” 127-134.
22 Chapter II
right to deprive them of survival probability because we assume that it is

their fault to fall into that group?
Elaborating upon this approach, I argue that Rawls’ original position should
be modified. If the parties agree to sacrifice themselves, in case they are
norm violators, denying the survival probability by default is the only just
decision. Certainly, such a line of thought reaches a dead end not only with
respect to Rawls’ theory of justice.
On the other hand, if the parties agree to sacrifice the pedestrians because
they believe that the pedestrians are guilty, such an agreement hardly can be
called moral at all. It questions the moral design problem by compromising
the initial status of the pedestrians. The moral gist of the dilemma is who
has the moral right to decide on behalf of others for their own sake so that
one can avoid the implications of moral arbitrariness.
Building “Utilitarian” AVs

The Role of Experimental Ethics
Regarding the implications of moral agents’ preferences, Bonnefon, Shariff

and Rahwan43 argue that the participants in the Moral Machine experiment
favor “a utilitarian moral doctrine that minimizes the total casualties in
potentially fatal accidents, but they simultaneously report preferring an
autonomous vehicle that is preprogrammed to protect themselves and their
families over the lives of others”.44 When the participants in the thought
experiments think about the results of the dilemmas for the greater good of
society, they seem to employ a utilitarian moral doctrine. Consequently,
when they consider themselves and their loved ones, the participants show
43Jean-François Bonnefon, Azim Shariff and Iyad Rahwan, “The Social Dilemma of
Autonomous Vehicles,” Science, No. 352 (June 2016): 1573-1576.
44 Azim Shariff, Jean-François Bonnefon and Iyad Rahwan, “Psychological Roadblocks
to the Adoption of Self-driving Vehicles,” Nature Human Behavior, No. 1

(September 2017): 694-696.. Frank, Chrysochou, Mitkidis and Ariely, “Human
Decision-Making Biases in the Moral Dilemmas of Autonomous Vehicles,” 1.
preferences towards a deontological moral doctrine that rejects the idea of

sacrificing the passengers.45
Frank et al. elaborate upon the aforementioned correlations by conducting

some experiments in which they trace the relations between the choice of
perspectives (passenger, pedestrian, observer), the decision-making mode
(deliberate, intuitive) and the different conditions, which can affect the
participants’ biases (such as whether or not there is a child among the
passengers, whether or not the passenger sits in the front sit etc.). The
clarification of these conditions is of crucial importance when the dilemma
is not one-versus-many, as in the classical trolley dilemma, but one-versus-
one.
Before tackling the reasons behind the preference to a utilitarian over

deontological moral doctrine or vice versa, one should keep in mind that
such a preference is justified within the framework of experimental ethics
whose results are limited in terms of the number of respondents and target
groups.
For instance, the approval of utilitarian AVs is achieved by conducting six

online MTurk studies. As Bonnefon et al. argue, the studies in question are
considered as largely reliable, although the respondents are “not necessarily
representative of the US population”.46
The theoretical outcome of such investigations is that one should carefully

examine the results of experimental ethics since it sets some particular
objectives, which are not necessarily representative for the general diversity
of moral practice, nor are they representative for the moral motivation of all
moral agents. Even if a given group of respondents gives preference to
utilitarian decisions when cognitive resources are actively used, it does not
follow firstly, that the decisions are utilitarian only due to the reference to
45 David Rand, “Cooperation, Fast and Slow: Meta-Analytic Evidence for a Theory
of Social Heuristics and Self-Interested Deliberation,” Psychological Science, 27,
No. 9 (September 2016): 1192-1206. Frank, Chrysochou, Mitkidis and Ariely,
“Human Decision-Making Biases in the Moral Dilemmas of Autonomous
Vehicles,” 1.
46 Bonnefon,Shariff and Rahwan, “The Social Dilemma of Autonomous Vehicles,”
1575.
24 Chapter II
the resources in question and secondly, that every single time when
cognitive resources are mobilized in making decisions, these decisions are
necessarily utilitarian.
Some “Hybrid” Moral Explanations

Frank et al. conduct a study (Study 3)47 on the biased moral preferences to
a utilitarian or deontological moral doctrine. The hypothesis of prioritizing
moral utilitarianism is tested by adding a second passenger, while
maintaining the number of pedestrians (one) in the crosswalk. If people
employ a utilitarian doctrine, they should favor the saving of the two
passengers at the expense of the pedestrian. In case of no change, the result
would suggest that people employ a deontological doctrine due to the
assumption that it is not morally acceptable to sacrifice the pedestrian.48
The findings support the hypothesis that people employ a utilitarian moral
doctrine in a deliberate decision-making mode, while people in an intuitive
decision-making mode rely upon the more accessible deontological
doctrine.49 This means that when the participants in the thought experiment
react spontaneously, they decide to save the pedestrian. As a reason for that
Frank et al. point out the role of a culturally embedded bias, namely, that
US citizens are taught that pedestrians on public roads may not be hurt by
drivers.50 In turn, when the participants carefully think about the situation,
they prefer to sacrifice the pedestrian for the sake of the two passengers
since thus they can maximize the well-being of the majority. However, the
general trend, as demonstrated by Study 3, is that the prevalent choice of the
participants remains in favor of the single pedestrian. In this context, Frank
et al. ask the question of what degree of utility trade-off would be necessary


Moral Dilemmas of Autonomous Vehicles,” 7. See also Joshua D. Greene et al.,

“Cognitive Load Selectively Interferes with Utilitarian Moral Judgment,
“ Cognition, 107, No. 3 (June 2008): 1144-1154.

for people to prefer that an AV harms an innocent pedestrian, as well as

whether there will be a difference between the deliberate and intuitive
decisions.51
In Study 4, one more control variable is added, namely, that of age. Its role
is exemplified by the specification that one of the passengers is a child
sitting in the back sit. Frank et al. clarified that this variable concerns the
value-of-life heuristics, which is typical for Western cultures. The life of a
younger person is valued over that of an older person.52 Surprisingly,
participants’ decisions are almost identical to these in Study 3–there is no
significant increase in the likelihood of sacrificing the pedestrian. However,
the difference becomes obvious, when comparing the passenger and the
pedestrian perspectives. The likelihood of people’s intuitive decisions to
sacrifice the pedestrian in the passenger perspective is four times higher than
in the pedestrian perspective condition.53 The investigation shows that
people are less protective of the child in the pedestrian condition than in the
passenger and control conditions.54
In this context, I would argue that the main problem with the ethical
explanatory instrumentarium concerns firstly, the reasons behind relating
deliberate decision-making to utilitarian decisions and, consequently,
intuitive decision-making to deontological decisions. Secondly, one should
keep in mind the specification that intuitivism and deontology are mutually
exclusive in moral terms. That is why I raise a hypothesis that the biggest
complications regarding people’s biased moral preferences derive from
narrowing the role of emotions to intuitive and deontological decision-

Moral Dilemmas of Autonomous Vehicles,” 7


Moral Dilemmas of Autonomous

Vehicles,” 9.
26 Chapter II
making processes, while ascribing the use of cognitive knowledge to

deliberate decision-making as a matter of utilitarian decision-making.
Moral utilitarianism does not exclude the role of moral feelings. The latter
can play not only a negative role, as is in the case with hedonism, but also a
positive one. Moral feelings such as empathy and sympathy contribute to
the maximization of the collective well-being. In turn, the role of emotions
is explicitly neglected in deontological ethical projects such as Kant’s
ethics.
Judging by the results of Studies 3 and 4, I argue that that there are two
questionable points in favoring the role of deontological ethical reasons,
when one shows a high likelihood to avoid the sacrifice of the passenger.
Firstly, emotions, which make the Self identify with the passenger as other,
triggering the self-projective bias in Self’s favor, can hardly be called a
reason for a deontological ethical stance. In other words, the result seems
deontologically acceptable due to the fact that one survival probability is
ascribed a higher moral value. However, the motivation for that, which is
intuitively triggered by the self-preservation, could be egoistic, and nothing
to do with deontological ethics.
Secondly, if an intuitive decision-making based upon emotions is

considered as typical for a deontological decision-making, it would mean
that moral intuitivism will be wrongly reduced to a formalist approach of a
Kantian type. Consequently, it would mean that issues concerning the
maximization of well-being and that of survival probabilities in the field of
AVs are deprived of any emotional moral engagement whatsoever.
What are the particular consequences of the aforementioned specifications

to the utilitarian AV scenarios? Study 3 also shows that the number of the
sacrificed pedestrians increases with the number of the passengers.55 There
is nothing unusual, except for the tendency from the pedestrian perspective.
For the first time (compared to Studies 1 and 2), the focus is shifted toward
55
Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the
sparing the passengers.56 In the intuitive condition, only 4.3% of the

respondents chose to sacrifice the pedestrian, while in the deliberate
condition, that percentage increased up to 60%.57
The simplest explanation is that the more respondents think, the more they
are inclined to think altruistically. The respondents are supposed to be more
zealous of sacrificing themselves for the sake of preserving the survival
probability for the rest. Such an explanation, however, raises some
methodological concerns. Certainly, the act of self-sacrifice is emotionally
entangled, which means that it cannot be examined as a use of more
cognitive resources.58 It concerns the participants’ self-projection ability to
put themselves into others’ shoes and feel empathy for them by evaluating
their value-of-life heuristics.
On the other hand, the percentage increase can be interpreted differently.

The data of 60% can be refracted through the lens of Rawlsian collision
algorithm if Rawls’ original position is taken into account. Then, the high
percentage can be examined as a result of one’s willingness to provide a just
decision for the worst-off group of people, which, in this case, is recognized
as that of the passengers (having a more complicated status). Therefore, one-
versus-many case is not necessarily subjectable to moral utilitarian
interpretations since the number prevalence can be a factor for moral
evaluations within both utilitarian and contractualist theories.
Practically speaking, this means that from the perspective of Rawlsian

collision algorithm, the number of the passengers also plays a role, but the
reasons are different. While in the utilitarian mode, the quantity is
recognized as a criterion for the maximization of the survival probabilities,



28 Chapter II
in the contractualist mode, it is used for delineating the group of the worst-
off people.59
Some of Frank et al.’s findings also support the thesis that sacrificing one
person for the sake of saving many is not necessarily a moral utilitarian
outcome. This is well-demonstrated by the increase in the number of control
variables in Study 7. The results illustrate that the possession of a driver’s
license, higher education, knowledge about AVs etc. increases the
likelihood of sacrificing the pedestrian.60 The attitudes towards sacrificing
a single person for the sake of saving many will influence the significant
increase in the likelihood of sacrificing the pedestrian.61 However, this is
not necessarily the main reason for such a sacrifice. Otherwise, it would
have meant that highly-educated people, with a high economic standard, are
more inclined to promote utilitarian moral values. Drawing such a
conclusion is problematic in many respects. For instance, the respondents
may simply identify “more easily”62 with the passengers than with the
pedestrian due to their own background of being active car users.
The “easier” self-projection is theoretically explained by the similarities

between the passenger and the control conditions. If the observers’ self-
projection is similar to that of the passengers, the results are expectedly in
favor of the passengers. Thus, the maximization of the collective well-being
is encouraged when the passengers represent the social majority. This can
happen regardless of the observers’ motivation, which may be driven not by
utilitarian, but by some other values. If the similarities in the experience and
the corresponding self-projective biases were a necessary and sufficient
condition of solving the utilitarian AV dilemmas, it would have meant that
those who take the pedestrian’s stance are egotists giving priority to their
59 This interpretation supports Keeling’s concerns that Bonnefon et al.’s research can
solve the moral design problem by using empirical methods (Keeling, “The Ethics
of Automated Vehicles”, 35).



own individual survival. Drawing such a conclusion is highly exaggerated

as well.
Based upon the aforementioned investigations, I argue that respondents’

preference for a utilitarian explanation of the pedestrian’s sacrifice displays
a strong predisposition to a utilitarian explanatory bias. If the observers, who
more easily identify with the passengers due to their deliberate decision of
maximizing the collective survival probabilities, are guided merely by
utilitarian morality, one cannot explain why there is such a strong attitude
against the sacrifice of the pedestrian. Certainly, this attitude has much to
do with the strong normative validity of the idea of the innocent pedestrian.
However, since the AVs in the experiments are supposed to work without
passengers’ intervention, the presumption of the passengers’ innocence
should remain untouched as well.
Conclusion
By examining the moral challenges in building AVs, I argue that the pitfalls
deriving from the utilitarian implications of the one-versus-many case gain
new strength. For the purposes of demonstrating the origin of some
problems with the design of a universal moral code for AVs, I have
examined two types of projects. The first one is a contractualist project,
namely, Rawlsian collision algorithm formulated by Leben, while the
second project is underlined by the findings of some utilitarian and
deontological AV thought experiments, as conducted by Frank et al.
Regarding the Rawlsian collision algorithm, I draw the conclusion that in

addition to the necessity of critically rethinking Leben’s reasons behind
borrowing Rawls’ concepts of the original position, veil of ignorance and
maximin rule, one should also focus upon the neglected value-of-life
heuristics. This suggestion is based upon the assumption that only thus can
one understand the complexity of moral decisions about survival
probabilities.
Concerning the elaboration of the original position and the maximin rule, I
argue that tackling their reception is of crucial importance for understanding
the problematic aspects of Rawlsian collision algorithm. As such a
30 Chapter II
questionable issue, I point out the lack of some knowledge about value-of-
life heuristics in the original position, which is initially denied by both
Rawls and Leben. However, such heuristics affects the application of the
leximin rule. Specifically, if one does not know whether the worst-off
person prefers to die rather than suffering debilitating injuries, one cannot
make a just decision for that other. The impact of the blurred boundaries
between the values of survival probabilities and survival upon the leximin
rule shows that the latter might be no longer considered as just. That is why
I argue that elaborating upon the potential moral efficiency of Rawlsian
collision algorithm requires Rawls’ idea of the veil of ignorance to be
modified. It should address the enrichment of knowledge about the life
projects of all parties.
Having analyzed the three examples, which Keeling sets as a test for
Leben’s collision algorithm, I clarify why they face some limitations in
justifying the values of survival probabilities. Regarding the first scenario,
I point out the complexity of the logical distinction between the preferences
to survival as strict preferences and those to non-fatal injuries as weak
preferences. However, providing a logical distinction is insufficient for cases
when a rational preference to non-fatal injuries can be considered as a strict
preference in moral terms.
In addition to Keeling’s criticism of Leben’s use of the maximin rule, which

puts undue weight to the moral claims of the worst-off people, I also argue
that Leben’s own preference to be in the shoes of the injured pedestrian is
grounded in the misinterpretation of the veil of ignorance. In this second
scenario, it is demonstrated how the parties choose a principle as just
because they want to be fairly treated if they occasionally fall into the group
of the worst-off people.
The third scenario, which represents Keeling’s own suggestion for the
greatest equal chance algorithm, contributes to increasing the survival rate,
but does not provide moral criteria of differentiation for cases such as that
of one-versus-one.
In this context, I argue that an alternative in revealing the different moral

values ascribed to survival probabilities and survival as such can be found
by examining the role of people’s biased moral preferences. A detailed

examination of the impact of biases within Rawlsian collision algorithm can
contribute to firstly, limiting the role of the parties’ self-projective biases
(specifically, their self-preservation attitudes) and secondly, demonstrating
why neither the group of the decision-makers nor those of the addressees of
the decisions are homogenous groups of moral agents.
If one examines the impact of the pedestrian’s social norm violation upon
the personal perspective of the observer within Rawlsian collision
algorithm, there are two significant problems, at least. If the deciding parties
agree to sacrifice themselves, in case they are violators, they should initially
deny the survival probability for themselves as the only just decision.
On the other hand, if the parties agree to sacrifice the pedestrians because
they believe that the pedestrians are guilty, such an agreement cannot be
called moral at all. The moral gist of the dilemma is who has the moral right
to decide on behalf of others for their own sake so that one can avoid the
implications of moral arbitrariness.
Regardless of the fact that the potential solutions to the trolley dilemma go
beyond the utilitarian explanatory framework, utilitarian decision-making
plays a crucial role in understanding the challenges in building a universal
moral code for AVs. While tackling the implications of the decision-making
in question, one should keep in mind that it is determined as such due to the
limited methods of experimental ethics, which may encourage the
recognition of some ungrounded ethical generalizations.
Analyzing Frank et al.’s findings, as displayed in Studies 3 and 4, I reach

the conclusion that the main problem with the ethical explanatory
instrumentarium concerns firstly, the reasons behind relating deliberate
decision-making to utilitarian moral decisions and consequently, intuitive
decision-making to deontological moral decisions. Secondly, one should
reconsider the relation between intuitivism and deontology, because they
are mutually exclusive in moral terms.
The analysis of Frank et al.’s findings shows that even if there are no formal
reasons to reject the statement that the high likelihood of sacrificing the
passengers is driven by deontological ethical arguments, the emotion of self-
32 Chapter II
preservation cannot be recognized as a positive moral feeling, which

grounds a deontological decision-making.
I draw the conclusion that the relevant search for a universal moral code for
AVs requires the reconsideration of the triplet of emotions–intuitive
decision-making–deontological decision-making and that of cognitive
knowledge–deliberate decision-making–utilitarian decision-making. Such a
clarification reveals why the sacrifice of one person for the sake of many is
not necessarily a moral utilitarian outcome, although it can meet the formal
utilitarian requirements of solving the one-versus-many case.
If an intuitive decision-making based upon emotions is considered as typical

of deontological decision-making, it would mean that moral intuitivism is
paradoxically reduced to a formalist approach from a Kantian type.
Consequently, it would mean that issues concerning the maximization of
well-being and that of survival probabilities in the field of AVs are initially
unrelated to the process of emotional moral engagement.
The respondents’ tendency to give preference to a utilitarian over the

deontological interpretation of the one-versus-many case can be explained
with a stronger influence of a utilitarian explanatory bias. The impact of this
bias is clearly demonstrated when the thought experiments assume that the
pedestrian is guilty of the potential AV accident. However, both the
passengers and pedestrians are equally innocent in a sense, because the
thought experiments assume that the vehicle is designed to be autonomous.
Comparing and contrasting the application of deontological and utilitarian

decision-making principles to the AV design shows that some “hybrid”
moral models should be adopted. Elaborating upon such a line of thought
means that one can avoid the recognition of the universal moral code as an
absolute code by giving preference to a set of codes, which are mutually
complementary. Adopting this approach would contribute to improving the
options of AVs’ moral self-update by regulating the balance between their
autonomy and morality, as much as possible.
References
Bonnefon, Jean-François, Azim Shariff, and Iyad Rahwan. “The Social
Dilemma of Autonomous Vehicles.” Science, No. 352 (2016): 1573-
1576. https://doi. org/ 10.1126/science.aaf2654.
Foot, Philippa. “The Problem of Abortion and the Doctrine of the Double
Effect.” Oxford Review, No. 5 (1967): 5-15.
Frank, Darius-Aurel, Polymeros Chrysochou, Panagiotis Mitkidis, and Dan
Ariely. “Human Decision-Making Biases in the Moral Dilemmas of
Autonomous Vehicles.” Sci Rep, No. 9, 13080 (2019).
https://doi.org/10.1038/s41598-019-49411-7.
Goodall, Noah. “Ethical Decision Making during Automated Vehicle
Crashes.” Transp. Res. Record J., Transp. Res. Board 2424, No. 1
(2014): 58-65. https://doi.org/10.3141/2424-07.
Greene, Joshua D., Silviya A. Morelli, Kelly Lowenberg, Leigh E. Nystrom,
and Jonathan D. Cohen. “Cognitive Load Selectively Interferes with
Utilitarian Moral Judgment.” Cognition, 107, No. 3 (2008): 1144-1154.
https://doi.org/10.1016/j.cognition.2007.11.004.
Keeling, Geoff. “Commentary: Using Virtual Reality to Assess Ethical
Decisions in Road Traffic Scenarios: Applicability of Value-of-Life-
Based Models and Influences of Time Pressure.” Front. Behav.
Neurosci, 11, 247 (2017). https://doi.org/10.3389/fnbeh.2017.00247.
PMID: 29311864; PMCID: PMC5733039.
Keeling, Geoff. 2018. “Against Leben’s Rawlsian Collision Algorithm for
Autonomous Vehicles.” In Philosophy and Theory of Artificial Intelligence
2017 (Studies in Applied Philosophy, Epistemology and Rational Ethics.
44), edited by Vincent C. Müller, 259-272. (Leeds: Springer 2018).
https://doi.org/10.1007/978-3-319-96448-5_29.
Keeling, Geoff. “The Ethics of Automated Vehicles.” PhD thesis.,
University of Bristol, 2020.
Leben, Derek. “A Rawlsian Algorithm for Autonomous Vehicles.” Ethics
Inf Technol 19, No. 2 (2017): 107-115. https://doi.org/10.1007/s10676-
017-9419-3.
34 Chapter II
Rand, David G. “Cooperation, Fast and Slow: Meta-Analytic Evidence for

a Theory of Social Heuristics and Self-Interested Deliberation.”
Psychological Science, 27, No. 9 (2016): 1192-1206.
https://doi.org/10.1177/0956797616654455.
Rawls, John. 1971. Theory of Justice. Cambridge, Mass.: The Belknap Press
of Harvard University Press.
Shariff, Azim, Jean-François Bonnefon, and Iyad Rahwan. “Psychological
Roadblocks to the Adoption of Self-driving Vehicles.” Nature Human
Behavior No. 1 (2017): 694-696. https://doi.org./10.1038/s41562-017-
0202-6.
CHAPTER III
ETHICAL CHALLENGES TO ARTIFICIAL

INTELLIGENCE IN THE CONTEXT
OF PANDEMIC AND AFTERWARDS
IVA GEORGIEVA
Introduction
The ethical consequences of the work done in the field of Artificial
Intelligence (AI) in the current pandemic world differ extensively from
those that were discussed in the previous times before the outbreak of
COVID-19. The focus on the nature of the events happening worldwide due
to the outbreak of the virus led to various ripple effects on the notions
directed toward science and technology. It also, however, provokes the
occurrence of extreme phenomena such as conspiracy theories, fake news,
anti-vaccination movements, deep fakes, and similar ones, just to name a
few. It is now more than ever important to restore the trust in science and
deploy the means of technology in service of people’s health and wellbeing
and to prove the ethical foundations in any actions taken toward serving
these goals with the available and new technological endeavors. However,
exactly at this point, science, technology, and AI in particular might seem
even more threatening as one of the strongest traumatic results of the
pandemic is the one that questions the ability of humankind to survive and
undermines the innate ability to believe that we are not so existentially
vulnerable and it is important not to surrender to ideas that we might be a
subject to extinction.
Even though technology is not directly held responsible for the current state
of affairs, the theories about the origin of the pandemic, as well as fears of
malicious use of various technological means cause increased distrust in the
36 Chapter III
deployment of sophisticated means to fight the virus. In this way, fears of

uncontrollable forces, including such ones as AI and robots, might seem
taken out of science fiction, and still fit too well into the irrational beliefs of
the exhausted people amid the constant fight as a trait of human nature. That
is why, the necessity for ethical and social acceptance and the ensuring of
the reliability of the technological means meet those standards in the recent
uses of technology and the defending of its particular means is an important
work of the scientists1 along with other concrete scientific achievements that
aim to help and clear rather than further mystify the role of technology in
the fight with the coronavirus.
AI Before and After

The image of technology and AI and robotics in particular from the pre-
pandemic era was more of a result from a mixture of science-fiction dreams
and creative endeavors of advanced societies such as the Japanese one for
example.2 However, the needs of the current times are different and require
fast and focused response to secure areas of life that are directly threatened
by the pandemic – such as medicine, work, and education, and affect
lifestyle in general by deepening the need for more sophisticated
applications of technology, and AI in particular, in healthcare.3 For
example, before the COVID-19 times it was consequently discussed issue
to utilize AI in full replacement of doctors, who raised various ethical issues
and fears of depersonalization of medical work4 that of course, would not
1 Iva Georgieva, Elisabeth Beaunoyer and Matthieu J. Guitton. “Ensuring Social

Acceptability of Technological Tracking in the COVID-19 Context,” Computers in
Human Behavior 116 (March 2021): 106639.
2 Peter Kahn et al., “Human Creativity Can Be Facilitated through Interacting with
a Social Robot.” In 2016 11th ACM/IEEE International Conference on Human-Robot

Interaction (HRI), published by IEEE (Christchurch, New Zeeland, March 7-10
2016), 173-180.
3 Amelia Fiske et al., “Your Robot Therapist Will See You Now: Ethical
Implications of Embodied Artificial Intelligence in Psychiatry, Psychology, and

Psychotherapy,” Journal of Medical Internet Research 21, No. 5 (May 2019):
e13216.
4 Mark Arnold and Ian Kerridge, “Accelerating the De-personalization of Medicine:
The Ethical Toxicities of COVID-19,”Journal of Bioethical Inquiry 17, No. 4

(August 2020): 815-821.
Ethical Challenges to Artificial Intelligence in the Context 37
of Pandemic and Afterwards
dissolve in the current situation,5 as now it would be much better justifiable

to consider more than aiding and rather to accept generally replacing
medical personnel in dangerous environments due to the pandemic.6 Many
concerns have shifted in a similar way as a result of the changed way of
living and perception of technology. However, bridging the scientific
rationale with the common-sense concerns is crucial as this new type of
normal way of living requires acceptance of a state of affairs that is different
from those imagined and discussed before.
Areas of AI application directly affected and rapidly changed due to the

pandemic are for example telemedicine, tracking technology, vaccine
development, and digital health in general 7. In particular, such tools as
health chatbots (e.g. Babylon Health) 8would be a solution for the needs of
communication when face-to-face contact is avoided, and more and more
artificial agents and serious games would become means to address issues
that were otherwise given to practitioners and preferred as traditional in-
person practices.9 AI creation aims to disrupt healthcare in order to redesign
it and now the chance to do so is more than evident.10 Besides obvious
5 Taoran Liu et al., “Patients’ Preferences for Artificial Intelligence Applications

versus Clinicians in Disease Diagnosis During the SARS-CoV-2 Pandemic in China:
Discrete Choice Experiment.” Journal of Medical Internet Research 23, No. 2
(February 2021): e22841.
6 Sonu Bhaskar et al., “Designing Futuristic Telemedicine Using Artificial
Intelligence and Robotics in the COVID-19 Era.” Frontiers in Public Health 8

(November 2020): 708.
7 Deepak Jakhar and Ishmeet Kaur, “Current Applications of Artificial Intelligence
for COVIDဨ19,” Dermatologic Therapy (May 2020). Tim Robbins et al., “COVID-
19: A New Digital Dawn?” Digital Health (April 2020). Abu Sufian et al., “Insights
of Artificial Intelligence to Stop Spread of Covid-19.” In Big Data Analytics and
Artificial Intelligence against COVID-19: Innovation Vision and Approach, eds.
Aboul-Ella Hassainen, Nilanjan Dey and Sally Elghamrawy. (Springer, Cham:
Springer, 2020), 177-190.
8 Babylon Health, accessed June 12, 2022, https://www.babylonhealth.com.
9 John McGreevy et al., “Clinical, Legal, and Ethical Aspects of Artificial
Intelligence–assisted Conversational Agents in Health Care,” Jama 324, No. 6

(August 2020): 552-553.
10 Julia M. Puaschunder et al., “The Future of Artificial Intelligence in International
Healthcare: An Index.” In Proceedings of the 17th International RAIS Conference on

Social Sciences and Humanities (Scientia Moralitas Research Institute, 2020), 19-
36.
38 Chapter III
answers to the already discussed questions such as what is the artificial agent
that finds updated meaning in the new context of the pandemic, the issue of
heightened cyber-security threats in the midst of a worldwide situation
challenged with the sense of authenticity, should change the way these
perceptions are analyzed by specialists and accepted by society.11
Moreover, digital communication leads to information overload and

phenomena such as “Zoom fatigue”12 that lead to increased necessity for
artificial agents to replace or aid the human-computer interaction (HCI) and
the technologically aided communication means. However, this also leads
to issues about surveillance and lack of privacy as we choose our own
trackers from the ones enforced by the pandemic to the ones we have already
accepted with not much questioning, such as the health-related apps for
example – and it is difficult to see the true difference and the transparency
of the changing processes behind these phenomena of shifting needs for
technological support.13 All these current effects put enormous pressure and
toll on mental health, as well as pose ethical questions that are difficult to
address while events continue to unfold and decisions are yet hard to make
and, at the same time, already delayed. The difficult balance between
technology and humanity has always been an issue with the advance in this
field, and now more than ever it seems we are facing a shift in the necessity
for technological presence in the world.14
Shifts in AI Ethics due to the Pandemic

Looking through the development of technologies including AI to address
the pandemic might give some preliminary answers to questions, such as
11 Menaka Muthuppalaniappan and Kerrie Stevenson, “Healthcare Cyber-Attacks

and the COVID-19 Pandemic: An Urgent Threat to Global Health, “International
Journal for Quality in Health Care” 33, No. 1 (February 2021).
12 Jeremy N. Bailenson, “Nonverbal Overload: A Theoretical Argument for the
Causes of Zoom Fatigue, “Technology, Mind, and Behavior” 2, No. 1 (February

2021).
13 David Leslie, “Tackling COVID-19 through Responsible AI Innovation: Five
Steps in the Right Direction,” Harvard Data Science Review (April 2020).
14 Paul M. Leonardi, “COVIDဨ19 and the New Technologies of Organizing: Digital
Exhaust, Digital Footprints, and Artificial Intelligence in the Wake of Remote

Work,” Journal of Management Studies (October 2020).
which areas are most important to clarify. For example, interactive

monitoring maps were the first applications of AI during the COVID
outbreak, later new technologies for faster detection and vaccine
development were included while ethical issues were not sufficiently
considered as quick response was the main goal.15 AI and robots were
introduced to fight fatigue in hospitals,16 while people were developing
various adverse symptoms due to prolonged social distancing and effects
such as phobias began to emerge that obviously would need separate tools
to address.17 Now, as much more time passed than initially expected, the
situation has not changed much, and the settling of the “new normal”
requires new tools to address long-term and delayed rather than the initial
and quickly dissolved problems that the pandemic introduced to our lives.
It is still unclear in what way the main pillars of life such as healthcare, work
and education would have to change, and in what way people will be able
to cope with the necessity for permanent change or for the acceptance of the
constant change of affairs, which in fact is worse as a result and would
require a profound understanding of the ethical consequences of the
technological innovation related to this.18
The “infodemic” that also resulted exponentially from the specifics of the
pandemic could be addressed with the means of AI. This term refers to the
widespread misinformation that circulates through social media and creates
divisions between groups of people, nations and cultures and consequently
deepens the negative effects of the pandemic. Issues related to the ethical
and social acceptability of tracking technologies are just one side of the
irrational fears about the technological aids, on the other side there is a real
15 Asaf Tzachor, Jess Whittlestone and Lalitha Sundaram, “Artificial Intelligence in

a Crisis Needs Ethics with Urgency,” Nature Machine Intelligence 2, No. 7 (June
2020): 365-366.
16 Ajmal Zemmar, Andres M. Lozano and Bradley J. Nelson, “The Rise of Robots
in Surgical Environments during COVID-19,” Nature Machine Intelligence 2, No.

10 (October 2020): 566-572.
17 Gopi Battineni, Nalin Chintalapudi and Francesco Amenta, “AI Chatbot Design
during an Epidemic Like the Novel Coronavirus,” Healthcare, vol. 8, No. 2 (October
2020): 154.
18 Sara Gerke, Timo Minssen and Glenn Cohen., “Ethical and Legal Challenges of
Artificial Intelligence-driven Healthcare,” Artificial Intelligence in Healthcare (June

2020): 295-336.
40 Chapter III
danger of privacy breaches, online hoaxes and malicious software that might
significantly endanger the users while providing untrue information.19
Detecting these will again require technological means and possibly relying
on AI, and that is why trust becomes such a twofold phenomenon now – a
problem and a solution, and even more complex than that.20 Concerns about
the realization the such methods, which help mitigating the problems
resulting from technology require technology itself, are real and create
necessity for a separate response in relation to the current state of affairs of
ethics of technology and the increasingly leading role of AI in its application
in the context of the pandemic.
However, AI in connection with virtual reality (VR) for example, can

become the real disruptive technology in healthcare. From creating
intelligent virtual environments to building intelligent artificial agents,21 the
freedom of giving new possibilities pairs with the responsibility for offering
greater ethical impact 22 and so can build trust in the users. In relation to the
consideration of the double-edged impact of technology, defending the idea
that AI cannot fully replace medical personnel where high skills like
empathy and ethical considerations are needed, would rather propose the
use of the technology in the mundane and mechanical tasks. Still, the
possibility of creating a completely artificial counterpart in communication
where human beings might fail seems considerably promising and
necessary, especially in the current context of the pandemic, as exhaustion
and lack of motivation take their toll. When it is possible to rely on a well-
19 Harbind Kharod and Israel Simmons, “How to Fight an Infodemic: The Four
Pillars of Infodemic Management,” Journal of Medical Internet Research 22, No. 6
(2020): e21820.
20 Muhammad Mustafa Kamal, “The Triple-edged Sword of COVID-19:
Understanding the Use of Digital Technologies and the Impact of Productive,

Disruptive, and Destructive Nature of the Pandemic,” Information Systems
Management 37, No. 4 (September 2020): 310-317.
21 Michael Luck and Ruth Aylett, “Applying Artificial Intelligence to Virtual
Reality: Intelligent Virtual Environments,” Applied Artificial Intelligence 14, No. 1

(November 2000): 3-32. Vladimir M. Petroviü, “Artificial Intelligence and Virtual
Worlds–Toward Human-level AI Agents,” IEEE Access 6 (July 2018): 39976-
39988.
22 Ben Kenwright, “Virtual Reality: Ethical Challenges and Dangers [Opinion],”
IEEE Technology and Society Magazine 37, No. 4 (December 2018): 20-25.
developed virtual partner that serves its function, being it the care of an
elderly person who lacks contact with relatives, or a helper of kids who need
to learn from a different source than their teachers or overly busy parents in
the new era of virtual meetings and online education, seems like a refreshing
and promising alternative.23 Nevertheless, rethinking the way things function
became necessity and redesigning the means of achieving our goals is
necessary to answer this challenge and be prepared for the future.
It is without a doubt that digital health assistants and medical chatbots, e.g.
from sophisticated ones meant for visually impaired persons to offer more
freedom and better life to more common ones such as the period tracking
apps that also remind of birth control pills to women, they can all provide
significant changes in the facilitation of everyday tasks, especially when
people are overwhelmed with the response to more daunting necessities
such as prevention and treatment of a viral infection. Of course, the most
common application of such smart advisors will be the detection and
solution of various symptoms, related or not to the pandemic, that might
challenge the integrity of people’s lifestyles and threaten their health.24
Other areas of life that suffer strongly from the pandemic are more subtle,
namely the psychological changes resulting in decreased motivation and
creativity 25that can be addressed again with the help of AI and VR in areas
that are inevitably changed due to the pandemic, such as tourism for
example.26 The possibility of offering gamification in addition to the
intelligent assistants turns the interest toward serious games that again can
23 Hui Luan et al., “Challenges and Future Directions of Big Data and Artificial
Intelligence in Education,” Frontiers in Psychology 11 (October 2020).
24 Jingwen Zhang et al., “Artificial Intelligence Chatbot Behavior Change Model for
Designing Artificial Intelligence Chatbots to Promote Physical Activity and a

Healthy Diet,” Journal of Medical Internet Research Vol. 22, No. 9 (September
2020): e22845. Nicola Luigi Bragazzi et al., “How Big Data and Artificial
Intelligence Can Help Better Manage the COVID-19 Pandemic,” International
Journal of Environmental Research and Public Health 17, No. 9 (May 2020): 3176.
25 Giuseppe Riva et al., “Surviving COVID-19: The Neuroscience of Smart Working
and Distance Learning,” Cyberpsychology, Behavior, and Social Networking 24,

No. 2 (February 2021): 79-85.
26 Andrei O.J. Kwok and Sharon G.M. Koh, “COVID-19 and Extended Reality
(XR),” Current Issues in Tourism 24, No. 14 (July 2020): 1-6.

42 Chapter III
connect AI and VR and offer a new platform for treating patients with
different kinds of suffering (both purely physical and subtly psychological) and
serve as a training tool for education, prevention and building of resilience
in the context of the increasingly challenging contemporary life.27
Nevertheless, the above applications of AI technology go deep into personal

lives and hence create stronger concerns in relation to privacy, data
collection and overall fears for possible control of human life. Decisions
about why and how the personal information is used and stored are still
taken by the specific commercial entities rather than governments or health
agencies. It is necessary to consider these issues before accepting
technology on such a level and still the time is insufficient to extensively
consider all the factors and make the necessary decision in a well-thought
and debated ethical context.28 Again, here the human factor is so important
as no technology can offer a better solution than the complex consideration
of historical, political and even more, philosophical premises on the
possibility of giving such power to technological artefacts. That is why even
though the situation is so confusing, acting toward common decisions and
goals in relation to AI, healthcare and wellbeing in the context of the
pandemic world is more than necessary now.
From IoT solutions29 that are needed more than ever in the everyday life
now to other specific for this time issues such as the analysis of anti-
vaccination movements,30 the detection of misinformation and panic-
induced/inducing information online,31 as well as more global problems
27 Giuseppe Riva et al., “Positive Technology and COVID-19,” Cyberpsychology,

Behavior, and Social Networking 23, No. 9 (September 2020): 581-587.
28 Stephen Cave et al., “Using AI Ethically to Tackle Covid-19,” BMJ 372 (March
2021).
29 Musa Ndiaye et al., “IoT in the Wake of COVID-19: A Survey on Contributions,
Challenges and Evolution,” IEEE Access 8 (October 2020): 186821-186839.

30 Amir Hussain et al., “Artificial Intelligence–enabled Analysis of Public Attitudes
on Facebook and Twitter toward COVID-19 Vaccines in the United Kingdom and
the United States: Observational Study,” Journal of Medical Internet Research 23,
No. 4 (April 2021): e26627.
31 Jim Samuel et al., “Covid-19 Public Sentiment Insights and Machine Learning for
Tweets Classification,” Information 11, No. 6 (June 2020): 314.

such as the “infodemic” turning into a “digital pandemic”,32 these tasks can
all be handled by AI. It is a matter of collective responsibility33 to also
realize that there are the previously mentioned here long-term effects and
the general double-edged sword effect of technology, along with more
subtle ones that actually affect populations on a very deep level, e.g., the
problem with digital inequalities 34that accumulate and create a pandemic
of separate pandemic-related issues.
The Future Is Now, and Still Yet to Come

AI can become the main tool for the future prevention of pandemics and
related to them global issues35 but its role should be clarified and defended
as well as precisely regulated. Moreover, the ideas of how AI can be utilized
are somehow confined in strictly non-human types of activities and lack a
sense of freedom and imagination. AI should not be the cold and threatening
force that we can put to work in the desired areas, it should be perceived as
our own creation that is aiding force standing next to human efforts in areas
that need development.
For example, as studies show a steep decline in creativity due to technology

impact from an early age, it is the same research that offers that AI can be a
“companion” of man for more creative endeavors.36 The pandemic has
shown how creative industries such as theater and basic foundations of life
such as education might be completely transformed with digital, interactive
32 Charalambos Tsekeris and Yannis Mastrogeorgiou, “Contextualising COVID-19

as a Digital Pandemic,” Homo Virtualis, 3(2) (December 2020): 1–14.
33 Seumas Miller and Marcus Smith, “Ethics, Public Health and Technology
Responses to COVIDဨ19,” Bioethics Vo. 35, No. 4 (February 2021): 366-371.

34 Marco Marabelli, Emmanuelle Vaast and Lydia Li, “Preventing the Digital Scars
of COVID,” European Journal of Information Systems (2021).

35 Puaschunder, Julia. “The Future of Artificial Intelligence in International
Healthcare: An Index.” In Proceedings of the 17th International RAIS Conference on

Social Sciences and Humanities (Scientia Moralitas Research Institute, 2020), 19-
36.
36 Zaccolo, Sandro. “Artificial Intelligence as a Creativity Companion,” accessed
July 8, 2022, http://openresearch.ocadu.ca/id/eprint/3167/.

44 Chapter III
and more creative methods of communication.37 It has been a long-term

effort to export creative media in VR, for example, and now as the physical
distancing and other factors keep performers away, it is the virtual medium
that can bridge entities and actors together and even spice them up with the
help of AI.
To illustrate this, a connection between theater, VR and AI in the context of

cognitive science can work to provide “sense making” between the subject
and the system.38 VR can be aided by AI in utilizing virtual environments
for creating flexible to the desired treatment results mediums that match the
purpose of a psychological therapy.39 The notion that AI in VR can produce
powerful immersive stories that are aided by virtual characters designed to
specific needs is not new.40 Embodiment of avatars in them might bring
about necessary results that can help people achieve things otherwise
impossible due to many restrictions, and these are now increased in number
by the overall pandemic situation worldwide. While VR application of AI
might seem so compelling, it might also sound scary and even escapist
solution to an overly digitalized world.41 The subtle balance between
answering needs and leading to issues is a truly important task and that is
where the requirement of ethically considered and approved means should
come.
37 Mariana-Daniela González-Zamar and Emilio Abad-Segura, “Implications of

Virtual Reality in Arts Education: Research Analysis in the Context of Higher
Education,” Education Sciences 10, No. 9 (August 2020): 225.
38 Pierre DeLoor et al., “Connecting Theater and Virtual Reality with Cognitive
Sciences.” In Virtual Reality International Conference (Laval, France, Apr 2010),

221-225.
39 Iva Georgieva and Georgi V. Georgiev, “Reconstructing Personal Stories in
Virtual Reality as a Mechanism to Recover the Self,” International Journal of

Environmental Research and Public Health 17, No. 1 (March 2020): 26.
40 McLellan, Hilary. “Magical Stories: Blending Virtual Reality and Artificial
Intelligence.” In Imagery and Visual Literacy: Selected Readings from the Annual
Conference of the International Visual Literacy Association (Tempre, Arizona,
October 12–16, 1994).
41 Rana Saeed Al-Maroof et al., “Fear from COVID-19 and Technology Adoption:
The Impact of Google Meet during Coronavirus Pandemic,” Interactive Learning

Environments (October 2020): 1-16.
As stressed here before, it is the technology that intrinsically holds the

notion of seeming threatening creation and altogether an ultimate solution.
In the era of challenges to the human health, it seems that what was made
by the man can be the cure for the suffering caused as well.42 The toll of the
prolonged exposure to stress that has now becoming habituated as a “new
normal” way of living is causing a pandemic of other challenging effects on
human nature that are yet to be detected. It is important to address this
lingering avalanche of plethora of problems that might surface when life
starts to get back to normal or as it seems, as they never truly return as they
were before, to remain as safe and well as possible afterwards. The
possibility to seek help from technological innovation and AI in connection
with VR in particular, might help the person to experience new ways of
processing adverse events and seek growth among them.43 Seeing events in
such a new light might offer answers to how to exit a situation that has lasted
for too long already.
Conclusion
The aftermath of the pandemic is still unpredictable, but the new world after
it will and at the same time will not be so globalized or that much depending
on technology. Divisions and contradictions become stronger, however the
need for smarter solutions also emerges above previous needs. It is a matter
of devoted scientific work to determine which is important to stay and hold
and which is no longer serving, but it is undoubtedly sure that the
preservation of what is humane and what is ensuring the wellbeing of
humankind remains the greater goal of them all.
42 Krešimir ûosiü et al., “Impact of Human Disasters and COVID-19 Pandemic on

Mental Health: Potential of Digital Psychiatry,” Psychiatria Danubina 32, No. 1
(Spring 2020): 25-31.
43 Iva Georgieva and Georgi V. Georgiev. “Redesign Me: Virtual Reality Experience
of the Line of Life and Its Connection to a Healthier Self, “ Behavioral Sciences 9,
no. 11 (November: 2019): 111.
46 Chapter III
References
Al-Maroof, Rana Saeed, Said A Salloum, Aboul Ella Hassanien, and Khaled
Shaalan. “Fear from COVID-19 and Technology Adoption: The Impact
of Google Meet during Coronavirus Pandemic.” Interactive Learning
Environments (2020): 1-16.
https://www.tandfonline.com/doi/full/10.1080/10494820.2020.1830121.
Arnold, Mark, and Ian Kerridge. “Accelerating the De-personalization of
Medicine: The Ethical Toxicities of COVID-19.” Journal of Bioethical
Inquiry 17, No. 4 (2020): 815-821.
https://link.springer.com/article/10.1007/s11673-020-10026-7.
Babylon Health. Accessed June 12, 2022. https://www.babylonhealth.com.
Bailenson, Jeremy N. “Nonverbal Overload: A Theoretical Argument for
the Causes of Zoom Fatigue.” Technology, Mind, and Behavior” 2, No.
1 (2021). https://tmb.apaopen.org/pub/nonverbal-overload/release/2
Battineni, Gopi, Nalini Chintalapudi, and Francesco Amenta. “AI Chatbot
Design during an Epidemic Like the Novel Coronavirus.” Healthcare,
vol. 8, No. 2 (2020): 154. https://doi.org/10.3390/healthcare8020154.
Bhaskar, Sonu, Sian Bradley, Sateesh Sakhamuri, Sebastian Moguilner,
Vijay Kumar Chattu, Shawna Pandya, Starr Schroeder et al. “Designing
Futuristic Telemedicine Using Artificial Intelligence and Robotics in the
COVID-19 Era.” Frontiers in Public Health 8 (2020): 708.
https://org.doi/10.3389/fpubh.2020.556789.
Bragazzi, Nicola Luigi, Haijiang Dai, Giovanni Damiani, Masoud
Behzadifar, Mariano Martini, and Jianhong Wu. “How Big Data and
Artificial Intelligence Can Help Better Manage the COVID-19
Pandemic.” International Journal of Environmental Research and
Public Health 17, No. 9 (2020): 3176.
https://pubmed.ncbi.nlm.nih.gov/32370204/.
Cave, Stephen, Jess Whittlestone, Rune Nyrup, and Rafael A. Calvo. “Using
AI Ethically to Tackle Covid-19.” BMJ 372 (2021).
https:/doi:org/https://doi.org/10.1136/bmj.n364.
ûosiü, Krešimir, Siniša Popoviü, Marko Šarlija, and Ivan Kesedžiü. “Impact
of Human Disasters and COVID-19 Pandemic on Mental Health:
Potential of Digital Psychiatry.” Psychiatria Danubina 32, No. 1 (2020):
25-31. https://pubmed.ncbi.nlm.nih.gov/32303026/
De Loor, Pierre, Charlie Windelschmidt, Karen Martinaud, and Vincent

Cabioch. “Connecting Theater and Virtual Reality with Cognitive
Sciences.” In Virtual Reality International Conference, 221-225. (Laval,
France, Apr 2010).
Fiske, Amelia, Peter Henningsen, and Alena Buyx. “Your Robot Therapist
Will See You Now: Ethical Implications of Embodied Artificial
Intelligence in Psychiatry, Psychology, and Psychotherapy.” Journal of
Medical Internet Research 21, No. 5 (2019): e13216.
https://www.jmir.org/2019/5/e13216/.
Georgieva, Iva, and Georgi V. Georgiev. “Redesign Me: Virtual Reality
Experience of the Line of Life and Its Connection to a Healthier Self.”
Behavioral Sciences 9, No. 11 (2019): 111.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6912767/.
Georgieva, Iva, and Georgi V. Georgiev. “Reconstructing Personal Stories
in Virtual Reality as a Mechanism to Recover the Self.” International
Journal of Environmental Research and Public Health 17, No. 1 (2020):
26. https://doi.org/10.3390/ijerph17010026.
Georgieva, Iva, Elisabeth Beaunoyer, and Matthieu J. Guitton. “Ensuring
Social Acceptability of Technological Tracking in the COVID-19
Context.” Computers in Human Behavior 116 (2021): 106639.
https://www.sciencedirect.com/science/article/pii/S0747563220303861
Gerke, Sara, Timo Minssen, and Glenn Cohen. “Ethical and Legal
Challenges of Artificial Intelligence-driven Healthcare.” Artificial
Intelligence in Healthcare (2020): 295-336.
https://doi.org/10.1016/B978-0-12-818438-7.00012-5.
González-Zamar, Mariana-Daniela, and Emilio Abad-Segura. “Implications
of Virtual Reality in Arts Education: Research Analysis in the Context
of Higher Education.” Education Sciences 10, No. 9 (2020): 225.
https://www.mdpi.com/2227-7102/10/9/225.
Hussain, Amir, Ahsen Tahir, Zain Hussain, Zakariya Sheikh, Mandar
Gogate, Kia Dashtipour, Azhar Ali, and Aziz Sheikh. “Artificial
Intelligence–enabled Analysis of Public Attitudes on Facebook and
Twitter toward COVID-19 Vaccines in the United Kingdom and the
United States: Observational Study.” Journal of Medical Internet
Research 23, No. 4 (2021): e26627.
https://pubmed.ncbi.nlm.nih.gov/33724919/.
48 Chapter III
Jakhar, Deepak, and Ishmeet Kaur. “Current Applications of Artificial

Intelligence for COVIDဨ19.” Dermatologic Therapy (2020).
https://onlinelibrary.wiley.com/doi/full/10.1111/dth.13654.
Kahn, Peter H., Takayuki Kanda, Hiroshi Ishiguro, Brian T. Gill, Solace
Shen, Jolina H. Ruckert, and Heather E. Gary. “Human Creativity Can
Be Facilitated through Interacting with a Social Robot.” In 2016 11th
ACM/IEEE International Conference on Human-Robot Interaction
(HRI), published by IEEE, 173-180 (Christchurch, New Zeeland, March
7-10 2016).
Kamal, Muhammad Mustafa. “The Triple-edged Sword of COVID-19:
Understanding the Use of Digital Technologies and the Impact of
Productive, Disruptive, and Destructive Nature of the Pandemic.”
Information Systems Management 37, No. 4 (2020): 310-317.
https://doi.org/10.1080/10580530.2020.1820634.
Kenwright, Ben. “Virtual Reality: Ethical Challenges and Dangers
[Opinion].” IEEE Technology and Society Magazine 37, No. 4 (2018):
20-25. https://ieeexplore.ieee.org/document/8558774.
Kharod, Harbind, and Israel Simmons. “How to Fight an Infodemic: The
Four Pillars of Infodemic Management.” Journal of Medical Internet
Research 22, No. 6 (2020): e21820.
Kwok, Andrei O.J., and Sharon G.M. Koh. “COVID-19 and Extended
Reality (XR).” Current Issues in Tourism 24, No. 14 (2020): 1-6.
https://doi.org/10.1080/13683500.2020.1798896.
Leonardi, Paul M. “COVIDဨ19 and the New Technologies of Organizing:
Digital Exhaust, Digital Footprints, and Artificial Intelligence in the
Wake of Remote Work.” Journal of Management Studies (2020).
https://onlinelibrary.wiley.com/doi/10.1111/joms.12648.
Leslie, David. “Tackling COVID-19 through Responsible AI Innovation:
Five Steps in the Right Direction.” Harvard Data Science Review
(2020). https://hdsr.mitpress.mit.edu/pub/as1p81um/release/4.
Liu, Taoran, Winghei Tsang, Fengqiu Huang, Oi Ying Lau, Yanhui Chen,
Jie Sheng, Yiwei Guo, Babatunde Akinwunmi et al. “Patients’
Preferences for Artificial Intelligence Applications versus Clinicians in
Disease Diagnosis during the SARS-CoV-2 Pandemic in China:
Discrete Choice Experiment.” Journal of Medical Internet Research 23,
No. 2 (2021): e22841. https://www.jmir.org/2021/2/e22841/.
Luan, Hui, Peter Geczy, Hollis Lai, Janice Gobert, Stephen J.H. Yang,
Hiroaki Ogata, Jacky Baltes et al. “Challenges and Future Directions of
Big Data and Artificial Intelligence in Education.” Frontiers in
Psychology 11 (2020). https://pubmed.ncbi.nlm.nih.gov/33192896/.
Luck, Michael, and Ruth Aylett. “Applying Artificial Intelligence to Virtual
Reality: Intelligent Virtual Environments.” Applied Artificial
Intelligence 14, No. 1 (2000): 3-32.
https://doi.org/10.1080/088395100117142.
Marabelli, Marco, Emmanuelle Vaast, and Lydia Li. “Preventing the Digital
Scars of COVID.” European Journal of Information Systems (2021).
https://dx.doi.org/10.2139/ssrn.3755050.
McGreevey, John D., C. William Hanson, and Ross Koppel. “Clinical,
Legal, and Ethical Aspects of Artificial Intelligence–assisted
Conversational Agents in Health Care.” Jama 324, No. 6 (2020): 552-
553. https://pubmed.ncbi.nlm.nih.gov/32706386/.
McLellan, Hilary. 1995. “Magical Stories: Blending Virtual Reality and
Artificial Intelligence.” In Imagery and Visual Literacy: Selected
Readings from the Annual Conference of the International Visual
Literacy Association (Tempre, Arizona, October 12–16, 1994).
Miller, Seumas, and Marcus Smith. “Ethics, Public Health and Technology
Responses to COVIDဨ19.” Bioethics Vo. 35, No. 4 (2021): 366-371.
https://doi.org/10.1111/bioe.12856.
Muthuppalaniappan, Menaka, and Kerrie Stevenson. “Healthcare Cyber-
Attacks and the COVID-19 Pandemic: An Urgent Threat to Global
Health.” International Journal for Quality in Health Care” 33, No. 1
(2021). https://pubmed.ncbi.nlm.nih.gov/33351134/.
Ndiaye, Musa, Stephen S Oyewobi, Adnan M. Abu-Mahfouz, Gerhard P.
Hancke, Anish M, Kurien, and Karim Djouani. “IoT in the Wake of
COVID-19: A Survey on Contributions, Challenges and Evolution.”
IEEE Access 8 (2020): 186821-186839.
https://ieeexplore.ieee.org/document/9220109.
Petroviü, Vladimir M. “Artificial Intelligence and Virtual Worlds–Toward
Human-level AI Agents.” IEEE Access 6 (2018): 39976-39988.
https://ieeexplore.ieee.org/document/8410872.
Puaschunder, Julia M. “The Future of Artificial Intelligence in International
Healthcare: An Index.” In Proceedings of the 17th International RAIS
50 Chapter III
Conference on Social Sciences and Humanities, 19-36. (Scientia

Moralitas Research Institute, 2020).
Puaschunder, Julia M., Josef Mantl, and Bernd Plank. “Medicine of the
Future: The Power of Artificial Intelligence (AI) and Big Data in
Healthcare.” RAIS Journal for Social Sciences, Vol. 4, No. 1 (2020): 1-
8. http:/doi.org/10.2139/ssrn.3662010.
Riva, Giuseppe, Fabrizia Mantovani, and Brenda K. Wiederhold. “Positive
Technology and COVID-19.” Cyberpsychology, Behavior, and Social
Networking 23, No. 9 (2020): 581-587.
https://doi.org/10.1089/cyber.2020.29194.gri.
Riva, Giuseppe, Brenda K Wiederhold, and Fabrizia Mantovani. “Surviving
COVID-19: The Neuroscience of Smart Working and Distance
Learning.” Cyberpsychology, Behavior, and Social Networking 24, No.
2 (2021): 79-85.
https://doi.org/10.1089/cyber.2021.0009.
Robbins, Tim, Sarah Hudson, Pijush Ray, Sailesh Sankar, Kiran Patel,
Harpal Randeva, and Theodoros N. Arvanitis. “COVID-19: A New
Digital Dawn?” Digital Health (2020).
https://doi.org/10.1177/2055207620920083.
Samuel, Jim, G. G. Ali, Md Rahman, Ek Esawi, and Yana Samuel. “Covid-
19 Public Sentiment Insights and Machine Learning for Tweets
Classification.” Information 11, No. 6 (2020): 314.
https://doi.org/10.3390/info11060314.
Sufian, Abu, Dharm Singh Jat, and Anuradha Banerjee. “Insights of
Artificial Intelligence to Stop Spread of Covid-19.” In Big Data
Analytics and Artificial Intelligence against COVID-19: Innovation
Vision and Approach, edited by Aboul-Ella Hassainen, Nilanjan Dey
and Sally Elghamrawy, 177-190. (Springer, Cham: Springer, 2020).
Tsekeris, Charalambos, and Yannis Mastrogeorgiou. “Contextualising
COVID-19 as a Digital Pandemic.” Homo Virtualis, 3(2) (2020): 1–14.
https://doi.org/10.12681/homvir.25445.
Tzachor, Asaf, Jess Whittlestone, and Lalitha Sundaram. “Artificial
Intelligence in a Crisis Needs Ethics with Urgency.” Nature Machine
Intelligence 2, No. 7 (2020): 365-366.
https://www.nature.com/articles/s42256-020-0195-0.
Zaccolo, Sandro. “Artificial Intelligence as a Creativity Companion.”

Accessed July 8, 2022. http://openresearch.ocadu.ca/id/eprint/3167/.
Zemmar, Ajmal, Andres M., Lozano, and Bradley J. Nelson. “The Rise of
Robots in Surgical Environments during COVID-19.” Nature Machine
Intelligence 2, No. 10 (2020): 566-572.
https://www.nature.com/articles/s42256-020-00238-2.
Zhang, Jingwen, Yoo Jung Oh, Patrick Lange, Zhou Yu, and Yoshimi
Fukuoka. “Artificial Intelligence Chatbot Behavior Change Model for
Designing Artificial Intelligence Chatbots to Promote Physical Activity
and a Healthy Diet.” Journal of Medical Internet Research Vol. 22, No.
9 (2020): e22845. https://www.jmir.org/2020/9/e22845/.
CHAPTER IV
THE AI-RUN ECONOMY:

SOME ETHICAL ISSUES
ANTON GERUNOV
Introduction
The increased automation of human activity essentially means that a
plethora of activities that used to be performed by sapient and conscious
human beings are now gradually transferred to machines, or more precisely,
to artificial intelligence agents. The most visible examples of such automation
are phenomena like self-driving cars and autonomous weapons systems. It
is, therefore, little surprise that the ethical issues connected to those specific
applications are gaining increasing prominence. Those concerns are clearly
exemplified in the journal Nature’s specific ethics of AI invited
commentaries. There Stuart Russell1 strongly warns of the Lethal
Autonomous Weapons Systems (LAWS), while Veloso2 urges to find
complementarities and synergies between humans and AI.
These specific instances of AI applications may, however, be misleading as

to the overall ethical implications of this emerging technology. This paper
will thus take a broader perspective and try to elucidate the implications of
a full-fledged transformation of the economy by AI. The trends for ever
larger use of automated business decision-making leveraging big data are
1 Stuart Russell, “Take a Stand on AI Weapons,” Nature, 521 (7553) (May 2015):
415-416.
2 Veloso, Manuela, “Embrace a Robot-Human World,” Nature, 521 (7553) (May
2015): 416-418.
The AI-Run Economy 53
quite obvious at this point3 but the key question remains as to what would
be the implications of marginalizing or even almost completely eliminating
human actors from a majority of economic choices. Apart from the obvious
individual consequences, such a move is likely to generate systemic
consequences that can be aptly explored via means of comprehensive
simulations. Our research aims to outline such an approach by constructing
two very simple model economies–a human-run and an AI-run one and
comparing and contrasting their performance. Based on this simulation
experiment, we further analyze some ethical issues that will arise as
business automation and AI-driven decision-making become ubiquitous.
Literature Review
The increasing abilities to process potentially sensitive information together
with the explosion of data availability and the rapidly decreasing costs of
computing have enabled unprecedented mass data processing with
potentially large implications for humans. Such data processing – either on
its own, or as enabling AI creates a plethora of ethical issues that have
increased in complexity over the past decades. In the era before autonomous
decision-making by machines (so-called domain AI) ethical issues revolved
mainly around data and included issues such as the privacy, accuracy,
property, and accessibility of information.4 A key issue here was how
personal information is protected in such a way so as not to cause harm to
individuals, and what its overall governance (and thus power) structure is.
This early thinking on data ethics presupposes human agency in the use of
information and so the key question is who possesses the data and has the
right to profit from it. A natural early question of data ethics is about the
locus of ownership and the power asymmetries surrounding it. This is then
still a human-centric version of ethics.
3 Hsinchun Chen, Roger H. Chiang and Veda C. Storey, “Business Intelligence and
Analytics: From Big Data to Big Impact,” MIS Quarterly, 36(4) (December 2012):
1165-1188.
4 Mason, Richard O. “Four Ethical Issues of the Information Age.” In Computer
Ethics, ed. John Weckert (London: Routledge, 2017), 41-48.

54 Chapter IV
Technological developments meant that the agency in decision-making has

been slowly moving from humans to machines, or AI. Following Bryson
and Winfield,5 we define Artificial Intelligence (AI) as any digital artefact
displaying one or more of the following properties:
Ɣ The capacity to perceive contexts for actions;

Ɣ The capacity to act;
Ɣ The capacity to associate context with action.
A note of clarification is in order here. When using the term “AI” the usual
connotation entails domain AI, where a decision-making algorithm can
perform domain-specific activities, replacing or augmenting the human
agent. The broader term “Artificial General Intelligence”, or AGI, has come
to describe the “real” all-encompassing multi-domain intelligence that will
make machines both sentient and sapient.6 The former is already a fact, the
latter is in the research phase and will likely need several decades before it
is fully developed.
Rogozea7 spells a number of necessary characteristics for such AI services.

They need to be available, affordable, accountable, compatible,
comprehensible and comprehensive. Those characteristics can be viewed as
being the foundations for ethical AI, but they neither describe it, nor can be
used to judge whether decisions made were indeed ethical in their very
nature. Those features merely enable an ethical inspection of the AI system.
The introduction of actual machines with agency and potentially destructive
capability accelerated the ethical discussion. In 2005, the project Euronet
5 Joanna Bryson and Alan F.T. Winfield, “Standardizing Ethical Design for Artificial
Intelligence and Autonomous Systems,” Computer, 50 (5) (May 2017): 116-119.

6 Nick Boström and Eliezer Yudkowsky, “The Ethics of Artificial Intelligence.” In
The Cambridge Handbook of Artificial Intelligence, eds. Keith Frankish and

William M. Ramsey (New York: Cambridge University Press, 2014), 316-334.
7 Rogozea, Liliana. “Towards Ethical Aspects on Artificial Intelligence.” In
Proceedings of the 8th WSEAS International Conference on Artificial Intelligence,

Knowledge Engineering and Data Bases (World Scientific and Engineering
Academy and Society (WSEAS), 2009), 507-512.
Roboethics Atelier8 aimed to create the first comprehensive ethical guidance

for users, manufacturers and policy-makers in the domain. Some of the
major recommendations include
Ɣ Safety – the robot needs to be safe by design but also include an

override mechanism by a human operator;
Ɣ Security – the level of information and physical security of the robot
needs to be sufficiently high to avoid malicious attacks;
Ɣ Traceability – the robot will need to have a logging and audit system
to record their activities;
Ɣ Identifiability – the robots should have unique identifiers;
Ɣ Privacy – sensitive personal data of humans need to be protected.
Research in this domain continues activity.9 Another major spur for the
ethics discussions was the development and introduction of self-driving cars
and their potential deleterious impact on human beings.10 In this context
Etzioni and Etzioni11 underline that there are two major ways to introduce
ethical aspects to AI-driven decision-makers: top-down and bottom-up.
Top-down approaches include “teaching” a system of ethics to the machine,
while bottom-up approaches involve learning from a large number of actual
human decisions in such a way that the AI can reconstruct a system of ethics.
Etzioni and Etzioni12 argue that both approaches are unfeasible and probably
unnecessary as the locus of ethics needs to remain in humans either through
their preferences or through legislated norms.
8 Veruggio, Gianmarco. “The Euron Roboethics Roadmap.” In 2006 6th IEEE-RAS

International Conference on Humanoid Robots (Genoa, December 4-6, 2006), 612-
617.
9 Wendell Wallach and Colin Allen, Moral Machines: Teaching Robots Right from
Wrong (Oxford: Oxford University Press, 2008). Patrick Lin, Keith Abney and
George A. Bekey, eds. Robot Ethics. The Ethical and Social Implications of Robotics
(Cambridge, Massachusetts. London, England: The MIT Press, 2012).
10 Jean-François Bonnefon, Azim Shariff and Iyad Rahwan, “The Social Dilemma
of Autonomous Vehicles,” Science, No. 352 (June 2016): 1573-1576.

11 Amitai Etzioni and Oren Etzioni, “Incorporating Ethics into Artificial Intelligence,”
The Journal of Ethics, 21(4) (March 2017): 403-418.

12 Etzioni and Etzioni, “Incorporating Ethics into Artificial Intelligence”.
56 Chapter IV
While this may well be true in well-regulated domains, it seems unrealistic

in others, such as defense and security. Russell13 strongly warns of the
technological sophistication of autonomous lethal weapons systems and
their ability to target human beings without explicit operator control. This
wave of technological innovation has further spurred important ethical
debates.
Boström and Yudkowski14 argue for a more comprehensive approach to AI

ethics, claiming that AI needs to be subjected to the same criteria that are
used for human actors performing social functions. More precisely those are
Ɣ Responsibility – the AI needs to act in a judicious way so as to fairly

make decisions for the benefit of the greater good;
Ɣ Transparency – the AI needs to make decisions in such a way that
they can be understood by human agents, or even rationalized by
humans’
Ɣ Auditability – all activities of the AI need to leave clear traces (e.g.
logs) so that they can be reviewed and audited if deemed necessary
by human agents;
Ɣ Incorruptibility – the algorithms need to be designed in such a way
that they cannot be easily compromised by careless or malicious
operators or other actors;
Ɣ Predictability – the AI needs to produce reliable results, i.e., to
produce similar output given a set of inputs in order to ensure
stability of decision-making;
Ɣ Avoidance of harm to innocent humans – the aspect that has naturally
attracted most attention is the safety of AI. It needs to be designed in
such a way that no harm or unintended negative effects are brought
to bear on humans in contact with the AI.
Torresen15 underlines the important point that even under strict rules for AI,
some resultant harm may be difficult to predict or avoid, thus further
13 Russell, “Take a Stand on AI Weapons”.

14 Boström and Yudkowsky, “The Ethics of Artificial Intelligence”.
15 Jim Torresen, “A Review of Future and Ethical Perspectives of Robotics and AI,”
Frontiers in Robotics and AI, 4, 75 (January 2018).

complicating the ethical analysis of AI decision-making. In this paper we

aim to show how those overarching principles of ethics can be applied to a
hypothetical centralized AI-driven economic decision-maker.
Simulation Results
Two simulations are run to better compare and contrast two types of
economies. The former is a realistic human-driven economy, where a large
number of heterogeneous human agents with bounded rationality make
sometimes suboptimal decisions. Those agents are modeled in a
behaviorally realistic fashion by drawing major insights from the fruitful
research in the field of behavioral economics. While this economy does
exhibit a tendency towards full resource utilization, it is hampered by
imperfections in markets and individual decisions (inertia, habit formation,
imperfect markets, suboptimal decisions, etc.)
The latter economy assumes a single centralized, perfectly rational

optimizing AI taking care of the entire economy. We assume that the AI
economy is programmed in such a way that it exhibits economically rational
optimizing behavior so that all productive assets are fully utilized at any
given time. This means that the difference between the optimal or potential
production (so-called output gap) tends to converge to zero. This section
presents the overall structure of the two economies, presents key
assumptions in their modeling, and outlines the main results from 10,000
simulated periods of each.
Overall Economic Structure

First, we model the overall economic structure by separately identifying the
behavior of two aggregated groups of actors – the producers (i.e. the
aggregate supply), and the consumers (i.e. the aggregate demand). We use
a standard set of equations to model economic dynamics by taking recourse
to the canonical New Keynesian DSGE model.16 The economic is thus
16 Jordi Gali, Monetary Policy, Inflation and the Business Cycle (Princeton:
Princeton University Press, 2008). Carl Walsh, Monetary Theory and Policy (US:
MIT Press, 2003).
58 Chapter IV
described by forward-looking aggregate demand (equation 1) and aggregate

supply (equation 2):
The aggregate demand equation (1) shows how demand depends on

previous income (yt-1), on the levels of expected prices (ʌt+1), on interest
rates (r), and on future expectations of growth (yt+1). In addition, there is a
random shock component (İt) taking part of unexpected endogenous or
exogenous events. The forward-looking components of this equation
(expectations) are the key behavioral input into the determination of
production. Similarly, the aggregate supply equation (2) is modeled as
dependent on current levels of income (yt), previous price level (ʌt-1), and
future expectations about pricing (ʌt+1). And the error term (Șt) is added to
account for potential unexpected shocks to the supply side of the economy.
Again, the forward-looking components are the main behavioral input into
the equation.
To close the system, we postulate a process for the interest rates, following
a standard Taylor rule.17 Essentially, this rule states that the Central Bank
sets the economy-wide interest rates taking reference to the current prices
(ʌt) in order to meet its inflation targets, to the current input (yt) in order to
avoid excessive harm to the real economy, and is bound by previous interest
(rt-1) rates both through decisions and through efforts to implement interest
rate smoothing. This equation has no clear behavioral component and can
be implemented by both human actors (as it is currently the case) or by
algorithms (in a possible future development):
Despite being quite simple, those three key equations present the mainstay
of the modeled economy and provide for relatively realistic overall
dynamics.
17
John B. Taylor, Macroeconomic Policy in a World Economy: From Econometric
Design to Practical Operation (New York: W.W. Norton, 1993).
A Decentralized Economy with Heterogeneous Irrational

Agents
The first simulation revolves around modeling a number of autonomous
heterogeneous agents that show signs of irrationality in their decision-
making patterns. Previous research18 has shown that humans follow a
number of predictable behavior patterns when confronted with economic
decisions. Most notably, they default to habit, try to maximize their utility,
or mimic others’ decisions (herd behavior). In addition to that, they tend to
evolutionary change their behavior in response to previous mistakes, but
this adjustment remains imperfect and is bound by habit and inertia.
We thus model human expectations using a fixed set of the following

heuristics:
In this case H1 corresponds to rationality, whereby humans try to make the

best possible forecast; H2–to habit default, where human merely adopt their
previous forecasts as the new expectation; and H–to herding, where humans
just adopt the expectation that is most prevalent in the economy.
Furthermore, we model the instability of human behavior by allowing

expectations to vary, thus letting individuals switch between heuristics with
a given probability (p) using a logit model. Equation (7) presents the logit
switching model for output:
18Gerunov, Anton. “Experimental Study of Economic Expectations.” In Yearbook

of St Kliment Ohridski University of Sofia, 14(1) (Sofia: Sofia University Press,
Faculty of Economics and Business Administration, 2016), 103-130.
60 Chapter IV
In order to model even more realistically the effects of avoidance of

cognitive dissonance and bounded rationality,19 we also include a lagged
term in the switch mechanism, thus reaching:
Concluding, equation 8 gives the probabilities of agents choosing either of

the three main heuristics for expectation formation. The overall economy-
wide expectations are thus formed as a weighted average of individual
expectations in accordance to individual choice probabilities, or:
This system of equations (4) through (9) aims to realistically model uniquely
human behavior and allow us to see how it influences observed economic
dynamics. This is done in exactly the same fashion for inflation as it is
shown for output in eq. (4)-(9). In short, here we model economic
expectations and decisions as dependent on irrational human impulses and
show how this irrationality spills into economy-wide inertia and sub-
optimality.
A Centralized Economy with a Single Rational Decision-Maker

The polar opposite of having a decentralize human-run economy is to have
a centralized machine-run one. In this case, we expect decisions to be taken
rationally, following the tenets of utility, profit, and welfare maximization
across the system of interconnected markets. This means that expectations
will be set rationally, and current decisions will be based on two key
components: the totality of currently available information (Ĭt,1), and an
accurate and unbiased forecast of the future. We model this by setting the
following expectation formation mechanism for the rational AI decision-
makers (for output gap and inflation, respectively):
19Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and
Giroux, 2011).
We hypothesize that the AI is not bound by irrational habits and will have
no incentive to switch away from the optimal decision-making. The fact that
we are dealing with a single unified decision-maker also means that there is
no need for an aggregation mechanism for disparate individual expectations.
Due to the long-term trend of the economy to drift toward its potential
production, the rational expectations for output and inflation would be set
to zero plus a random shock due to unforeseen circumstances.
Simulation Results
To simulate the two model economies, we use standard numeric values for
the parameters as per the literature.20 Those are presented in Table IV.1 and
can be shown to generate realistic economic dynamics.21 Each economy is
then simulated over 10,000 periods and its overall statistical properties are
shown and investigated. Detailed results for inflation and output are
presented in Table IV.2.
Table IV.1: Model Parametrization
Both simulated economies are designed in such a way that they tend to
converge to full utilization of production factors (or potential output). Thus,
it is not surprising that both of them have a mean output gap that is not
significantly different from zero. This is also true for the corresponding
20
Gali, Monetary Policy, Inflation and the Business Cycle, 58.
21Anton Gerunov, Macroeconomic Modeling: Modern Approaches (Sofia: Sofia
University Press, 2015).
62 Chapter IV
aggregate of prices. Over the long run, both human and AI-run economies
tend to stabilize in the absence of large external shock.
Table IV.2: Statistical Properties of the Two Simulated Economies
However, the key factor to note here is that the AI economy produces very
different dynamics in this process. The human-run economy has both
pronounced peaks and deep recessions in growth, with a total range over
15% between the highest and the lowest points. This is the typical economic
cycle of boom and bust. In contrast, the AI-run economy has an extremely
limited range of the gap–only 5.1%. This is also reflected in the standard
deviations–the human-run economy features standard deviations for output
that are three times as large as those generated by the simulated rational
decision-maker. The proportions are similar when it comes to the price
dynamics. The range of the human-run economy is about 2.5 times larger
than that for the AI one, and the standard deviations are two times greater.
Those results can be easily followed in Figure IV.1.
Figure IV.1: Output Gap in Two Simulated Economies
It is straightforward to identify the differences in economic dynamics by

following the output gap over time (see Figure IV.2). In the AI-run
economy, there are virtually no dramatic drops in activity, and there are no
upsurges. The trend in production is smooth, thus eliminating transition and
adaptation costs for different automatic and human economic agents. Those
fluctuations persist in the human-run economy and impose possibly large

overall costs on social welfare. The trends in inflation follow very similar
dynamics.
Figure IV.2: Output Gap Dynamics in Two Simulated Economies
In short, the AI-run economy features much lower volatility, greater short-
run stability, and has virtually eliminated violent economic crises and
possible hyperinflation risks. We will use those economic simulations as
starting points for investigating possible ethical issues in case such a
transformation takes place.
Ethical Issues
Leveraging the Boström and Yudkowski’s framework,22 we investigate the
ethical aspects of transforming the economy from a decentralized boundedly
rational system, run by humans into a centralized rational system, run by

64 Chapter IV
AI. The following aspects are evaluated toward this end: responsibility,
transparency, auditability, incorruptibility, predictability, avoidance of harm.
The responsibility of the system refers to its ability to make fair and
responsible decisions in pursuit of a beneficial goal. Depending on how the
AI is implemented, we may observe different facets. In case the AI decision-
maker is programmed top-down with some pre-defined objective functions
that are to be optimized, the AI can only be expected to take limited
responsibility for its narrowly defined objectives. If the AI is instead trained
in a bottom-up fashion, it should have a holistic understanding of its
responsibility. At any rate, the ultimate rationale for the AI decisions and
actions needs to be clear and will likely converge to some type of
maximization of individual and group welfare. The end deliberation of
whether the AI is responsible may need to be taken after extensive analysis
is conducted.
The transparency of the algorithm will again depend on its implementation.

Detailed pre-programming or recourse to simple machine learning methods
such as decision trees or even random forests will greatly enhance the
transparency of decision-making. On the other hand, in the more likely
scenario-sophisticated methods such as deep neural networks are used, then
the transparency of the algorithm will likely be low and difficult to
comprehend even by domain experts. This outlines a major trade-off in
leveraging AI for business automation – transparency might have to be
sacrificed to obtain efficiency. Such issues are unlikely to be solved
computationally by identifying some optimal trade-off.
The auditability of the AI is yet another issue at hand. With the automation
of decisions and under current best practices and extant trends for logging,
this seems to be a relatively minor issue. It is highly likely that a large part
(if not all) of the actions and decisions of the AI are to be meticulously
documented and available for further investigation. A possible concern here
may be tampering with those logs, but some emerging technologies such as
the blockchain can prevent that as well. Overall, auditability seems to be of
limited concern.
The incorruptibility of the system is intimately connected to its

transparency. The main goal here is to prevent the misuse of the system
through its internal actions or through the action of outside agents. The
incorruptibility is difficult to assess as there is hardly a way to decide
whether a series of decisions is the result of the original process or a
corrupted one. This is particularly true in our current case for a series of
complex economic decisions that aim to align a set of incentives in such a
way that the medium-term growth trend targets the elimination of a
synthetically calculated gap. Thus, the incorruptibility can hardly be
ascertained through the AI’s outcome but rather needs to be checked by
investigating the process that generated it. Hence, transparency is needed.
The predictability of the AI is another pivotal point to consider. This key

ethical issue supposes that the stability needed for a complex social or
economic system to function must be predicated on a certain degree of
consistency, or predictability, of decision-making. In the case of the AI-run
economy, we conjecture that there may be a high degree of predictability of
outcomes, and a significantly lower degree of predictability of the process.
Regardless of whether the AI’s objective is pre-programmed in a top-down
approach or learned through bottom-up, it is highly likely that its resulting
actions will tend to maximize profit for firms, utility for consumers, and
overall social welfare within some given constraints. Moreover, we expect
a certain stabilization behavior as shown in the simulations presented here,
as extreme volatility tends to impose costs that are rationally avoided. In
terms of means of achieving all of this, the AI will probably exhibit less
predictability given the domain complexity.
The final, and probably most controversial, ethical issue connected to a

potential AI-run economy is the algorithm’s ability to inflict harm upon
innocent humans. While this is usually connected to inflicting easily
detectable physical harm, e.g., in the case of self-driving cars of LAWS, the
AI-driven economic decision-maker has the capacity to inflict more
insidious damage. As a result of some production or supply decisions,
humans may be left with insufficient goods or services, thus endangering
their health, physiological or emotional well-being. This subtle damage can
be amplified at scale, leading to a serious decrease in social welfare. More
directly, the economic AI may inflict damage by supplying goods that are
66 Chapter IV
demanded by consumers but are detrimental to them, such as weapons,

drugs, or excessive amounts of alcohol. The AI-driven economy has yet
another subtle property. It serves to greatly smoothen the economic cycle,
thus depriving the human agents of both the spur of difficulty in downturns
and the elation of exuberance in economic upturns. This realignment of
incentives may lead to critical repercussions on the consumer behavior and
the capacity for innovation. While those effects are barely studied, Sidhu
and Deletraz23 show that excessive comfort has negative implications on
entrepreneurial attitudes and risk tolerance. This may easily translate into a
lower capacity for innovation. The harm in automating decision-making
may thus not be physical pain but rather the stifling of human creativity.
Discussion
The two simulated economies presented here serve as useful vantage points
for applying a set of ethical criteria toward a hypothetical AI economic
decision-maker. It seems that algorithmic responsibility and transparency
will crucially depend on the specific implementation choices. In the
example presented, both conditions were satisfied as responsibility was
ensured through clear optimization behavior, while transparency stemmed
from a nearly defined set of equations used by the AI. In a more realistic
setting that leverages complex machine learning algorithms such as deep
neural networks, those two properties are far from guaranteed. The
auditability property of AI seems to be the least concern of all since
extensive logging allows for a clear audit trail enabling ex-post forensics (if
needed).
AI’s incorruptibility, on the other hand, seems to be particularly difficult to

assess in situations where the AI decision-maker is responsible for complex
decisions in a non-linear uncertain environment such as the economy. Such
cases call for process inspection rather than for evaluation of outcomes. This
property is tightly connected to auditability and transparency. If the AI
23Sidhu, Ikhlaq and Paris Deletraz. “Effect of Comfort Zone on Entrepreneurship

Potential, Innovation Culture, and Career Satisfaction.” In 122nd ASEE Annual
Conference & Exposition. Making Value of Society (Seattle, WA, June 14-17, 2015),
1770-1783.
algorithm is both transparent and auditable, then the human agent will find
it much easier to ascertain whether the AI has been corrupted or is
malfunctioning. Predictability also relies heavily on transparency, but it
seems that outcomes will be easier to predict while the process used to reach
them will not be. This is particularly true if the AI is trained using
sophisticated black-box methods. Finally, the possibility of inflicting
unforeseen and possibly undetectable damage is quite high in the case of
economic decision-making by AI. Given the high complexity of the
environment and since there are no discernible human losses, a possible
suboptimal, malicious, or malfunctioning AI may remain undetected for an
extended period of time. This holds particularly true for intangible damages,
such as incentive misalignment and decreasing innovative and creative
potential.
It is clear that while some of the ethical requirements align neatly with
economic imperatives (e.g. responsibility and effectiveness), others diverge
quite dramatically (e.g. transparency and efficiency). Thus, the automation
of economic and business decision-making will pose substantive moral and
ethical questions and will necessitate significant tradeoffs between the
desirable properties of the AI. We venture to propose that in order to achieve
an optimal balance between objective constraints, large-scale and high-risk
Artificial Intelligence algorithms need to go through a formal assessment
and approval procedure. This may be in the form of an AI Impact
Assessment that is conducted to answer the complex and interconnected
ethical questions posed by complex AI algorithms. This Impact Assessment
needs to balance the economic and business needs of producers and
consumers against the safety and ethical concerns of broader society. While
such an assessment does not need to be conducted by a state authority and
can be relegated to alternative providers, its results may have to achieve a
minimum amount of consensus among stakeholders before the evaluated AI
is put into production. Such a rigorous multi-stakeholder approach will
enable humankind to ensure that AI will end up creating more benefits than
inflicting unsuspected harm.
68 Chapter IV
Conclusion
This paper focused its attention on an area of AI decision-making that is still
not substantially researched–the ethical issues stemming from the
automation of economic and business decisions. To this end, we have
presented and contrasted two model economies and shown how an AI-
driven economy dramatically differs from the current one. Its new properties
raise a set of ethical questions that were formally addressed by using the
framework topic, as summarized by Boström and Yudkowski.24 While some
of the issues necessitate concrete implementation to be fully resolved, we
reached some preliminary conclusion that outline that a number of difficult
tradeoffs may need to be made when AIs are comprehensively put into
production and service. Our proposal is to define a rigorous assessment
process with large-scale stakeholder involvement that ensures the beneficial
utilization of the upcoming Artificial Intelligence algorithms.
References
Bonnefon, Jean-François, Azim Shariff, and Iyad Rahwan. “The Social
Dilemma of Autonomous Vehicles.” Science, No. 352 (2016): 1573-
1576. https://doi. org/ 10.1126/science.aaf2654.
Boström, Nick, and Eliezer Yudkowsky. “The Ethics of Artificial
Intelligence.” In The Cambridge Handbook of Artificial Intelligence,
edited by Keith Frankish and William M. Ramsey, 316-334. (New York:
Cambridge University Press, 2014).
Bryson, Joanna, and Alan F.T. Winfield. “Standardizing Ethical Design for
Artificial Intelligence and Autonomous Systems.” Computer, 50 (5)
(2017): 116-119. http://doi.org/10.1109/MC.2017.154.
Chen, Hsinchun, Roger H. Chiang, and Veda C. Storey. “Business
Intelligence and Analytics: From Big Data to Big Impact.” MIS
Quarterly, 36(4) (2012): 1165-1188.
Etzioni, Amitai, and Oren Etzioni. “Incorporating Ethics into Artificial
Intelligence.” The Journal of Ethics, 21(4) (2017): 403-418.
https://doi.org/10.1007/s10892-017-9252-2.

Gali, Jordi. 2008. Monetary Policy, Inflation and the Business Cycle.
Princeton: Princeton University Press.
Gerunov, Anton. 2015. Macroeconomic Modeling: Modern Approaches.
Sofia: Sofia University Press.
Gerunov, Anton. “Experimental Study of Economic Expectations.” In
Yearbook of St Kliment Ohridski University of Sofia, 14(1), 103-130
(Sofia: Sofia University Press, Faculty of Economics and Business
Administration, 2016).
Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar,
Straus and Giroux.
Lin, Patrick, Keith Abney, and George A. Bekey, eds. 2012. Robot Ethics.
The Ethical and Social Implications of Robotics. Cambridge,
Massachusetts. London, England: The MIT Press.
Mason, Richard O. “Four Ethical Issues of the Information Age.” In
Computer Ethics, edited by John Weckert, 41-48. (London: Routledge,
2017).
Rogozea, Liliana. “Towards Ethical Aspects on Artificial Intelligence.” In
Proceedings of the 8th WSEAS International Conference on Artificial
Intelligence, Knowledge Engineering and Data Bases, 507-512. (World
Scientific and Engineering Academy and Society (WSEAS), 2009).
Russell, Stuart. “Take a Stand on AI Weapons.” Nature, 521 (7553) (2015):
415-416.
Sidhu, Ikhlaq, and Paris Deletraz. “Effect of Comfort Zone on
Entrepreneurship Potential, Innovation Culture, and Career Satisfaction.”
In 122nd ASEE Annual Conference and Exposition. Making Value of
Society, 1770-1783 (Seattle, WA, June 14-17, 2015).
Taylor, John B. 1993. Macroeconomic Policy in a World Economy: From
Econometric Design to Practical Operation. New York: W.W. Norton.
Torresen, Jim. “A Review of Future and Ethical Perspectives of Robotics
and AI.” Frontiers in Robotics and AI, 4, 75 (2018).
https://doi/org/10.3389/frobt.2017.00075.
Veloso, Manuela. “Embrace a Robot-Human World.” Nature, 521 (7553)
(2015): 416-418.
70 Chapter IV
Veruggio, Gianmarco. “The Euron Roboethics Roadmap.” In 2006 6th

IEEE-RAS International Conference on Humanoid Robots, 612-617.
(Genoa, December 4-6, 2006). Accessed July 7, 2022.
https://www3.nd.edu/~rbarger/ethics-roadmap.pdf.
Wallach, Wendell, and Colin Allen. 2008. Moral Machines: Teaching
Robots Right from Wrong. Oxford: Oxford University Press.
Walsh, Carl. 2003. Monetary Theory and Policy. US: MIT Press.
CHAPTER V
SCRATCH MY BACK
& I WILL SCRATCH YOURS:
PUTTING GAME THEORY AND ARTIFICIAL
INTELLIGENCE ON A CONVERGING PATH
BEYOND COMPUTATIONAL CAPABILITIES
BORIS GUROV
Game Theory Theorization and Possible Practical

Applications and Solutions
What are the options for optimizing the use of congested highways to the
seaside and back during weekends? Is there a way to limit increased
individual rational spending during periods of inflation, which triggers the
aggravation of the phenomenon and paves the way for hyperinflation? How
do two concurrent firms overcome the competition problem and collaborate
instead in the creation of a new technical standard that will ensure their
dominant position in the market? All of the above, without the intervention
of a centralized authority. Those are some questions that game theory seeks
answers to. Why would this type of problematic and analytical focus be of
use for the development of AI? And vice versa.
It is very tempting to declare that game theory was once a very promising
research field, which reached its analytical limits and provided some good
insight into decision making under uncertainty, and gave some theoretical
awareness of the complex links between autonomous and free individual
decision-making and the harmonious functioning of the collective, or the
group. The problem with such a position lies in the fact that game theory is
the sole analytical tool available to us for bridging the gap between
72 Chapter V
individual strategical behavior and reasoning in terms of group and group

effects. Game theory might be highly formalized and offer complex and
counterintuitive solutions, but the fact is that it represents the only path
towards thinking and alleviating the tension between individual rationality
(agent optimization) and collectively rational outcomes (social optimality).
What is the best possible approach for a fleet of military ships to engage an
enemy fleet depending on its goals (destroy the enemy, inflict heavy
damage, hold them off, limit its own losses)?1 How do ships coordinate
themselves and how do they cooperate with each other? Is it better that there
is a centralized top-down decision-making mechanism in place, or the
overall performance will gain from different ships in the fleet benefitting
from some form of autonomy?
There is a myriad of problems and layers of overlapping complex non-linear

effects to be considered even before starting to answer such a question. Even
if we eliminate the idea that we do have battleships with human crews and
we are dealing with drones instead, which will reduce all ethical issues to
resource management, the complexity of the problem remains impressive.
What game theory provides for proposing a practical solution for such
problems are exquisite models and quantification of this type of
interrelations/interaction, quantifications that can produce a virtually
unlimited amount of input for AI data hungry algorithms. Another path to
be explored is that, by definition, those models are rooted in a decentralized
decision-making optics. Given the recent advances of AI toward
decentralization, for example, in the field of advanced architecture, GT
could prove an extremely useful tool for shifting the focus from individually
optimizing and adapting algorithms (deep learning on the individual
algorithmic level) towards the creation of smart networks where different
algorithms cooperate and coordinate their answers to systemic changes and
disequilibria via common knowledge. Pushing the research agenda in the
domain of AI in this direction may be an ambitious endeavor, but the
1 The history of Naval warfare testifies for the very heterogeneous goals of fleet
commanders in different contexts. Of course, classical game theory is unable to
explore the interdependency of such heterogeneous logics. Generally, it will assume
the goal to be full-scale maximum damage to the opponent type of interaction.
Scratch My Back & I Will Scratch Yours 73
potential benefits are obvious. Last, but not least, exploring game-
theoretical models via AI has the potential, if it goes beyond the usual
omission of theoretical discussions, characteristic of the field (a tendency,
which was rather present even in the 1980s and 1990s in the domain of
evolutionary GT), to reconceptualize our understanding and analytical
procedures of rationality. Bluntly said, engineers should start consulting
more often the literature of economists.
Individual Optimization, Pareto-optimality, Common

Knowledge. General Features of Static Gamer Theory–
Historical Perspective
Historically, GT was confronted with a decisive crossroad from its
beginnings. From 1921 onward (1921-1928), French mathematician Emile
Borel published a series of articles on the ways to formalize and study social
situations and interactions in the form of games. What is crucial about Borel
is that he considered that game analysis should be carried out from the
standpoint of players.2 It is not a position that has gained popularity. Most
specialists consider John Von Neumann as the real founding father of GT.
He had a drastically different conception of how to abord games.
In his 1928, “Theory of Parlor Games”,3 he profoundly marked the future

development of GT by defining the task of the analyst as one of finding a
mathematical solution to the game. The important point here is that Von
Neuman’s formalist-theoretical vision aimed at a perfect mathematical
solution4 prevailed over the psychological-empiricist search for individual
2 Emile Borel,”La théorie des jeux et les équations à noyau symétrique gauche.” In
Comptes rendus de l’Académie des sciences, Vol. 173 (Paris: Académie des
Sciences Morales et Politiques), 1304–1308.
3 There are two separate translations of the original text from German (“Zur Theorie
der Gesellschaftsspiele,” Matematische Annalen, Vol. 100 (1928): 295-320).

“Theory of Parlor Games” is in our humble opinion the better title, but the fact is
that among Anglo-Saxon specialists it is the other version that is by far the dominant
reference: John von Neuman, “On the Theory of Games of Strategy.” In
Contributions to the Theory of Games, Vol. 4, eds. Albert Tucker and Duncan Luce
(Princeton: Princeton University Press, 1959), 13-42.
4 Christian Schmidt makes a very convincing case for the unintended consequences
for GT of the impact that Von Neuman had on its research program by subordinating
74 Chapter V
strategy formation of Borel. Von Neuman started with the analysis of

bilateral games (interactions) and forged a considerable part of the
conceptual tools of modern GT (strategy, outcome, payoff, player, etc.).
Game theory took a sharp turn toward economics with the publication of
Theory of Games and Economic Behavior (1944) in which Von Neuman
collaborated with Oscar Montgerstern and they laid the foundations for
expected utility theory and anchored in a definitive manner GT as a
powerful hypothetico-deductive tool. The subsequent development of the
domain was solely based on formalized mathematical procedures using as
premises individual self-interested rationality of agents as a theoretical
definition and exploring the effects on the aggregated level of the system of
players. Probably the most famous mixed-motive (there are possibilities
both for cooperation and conflict) bilateral game is the Prisoner’s dilemma.
Below is its classical matrix of payments.
Fig. 1. Prisoner’s dilemma
axiomatically the rationality of players to the solution of the game, which in turn
becomes defining for the definition of the rationality of players. Cf. Schmidt,
Christian “From the ‘Standards of Behaving’ to the ‘Theory of Social Situations’. A
Contribution of Game Theory to the Understanding of Institutions.” In Knowledge,
Social Institutions, and the Division of Labor, eds. Pier Luigi Porta, Roberto
Scazzieri and Andrew Skinner (Cheltenham: Edward Elgar Publishing, 2001), 153-
168.
Prisoner’s dilemma (PD) erupted on the front scene of decision-making in

the 1950s5 and is the most famous and discussed strategic interaction, with
thousands of publications exclusively dedicated to it. In its classical version,
the PD puts in an interaction with two outlaws arrested by the police. They
are presented before a judge who can charge them with minor offenses (each
one will go to prison for 2 years for illegal arms possession) or try to convict
them of a major crime (armed robbery) for which the prosecution lacks
evidence, however. The judge uses a demonic logic: each detainee is given
the choice of either testifying against his/her accomplice (defect towards
him/her), with the explanation that if they both denounce each other, they
will go for 5 years in prison, and in the event of one of them confessing and
the other not, the witness for the prosecution is set free and the other will
serve the maximum possible sentence: 10 years in prison. The problem with
the PD is that whatever the other party does, it is preferable for ego to defect.
In this sense, from the standpoint of individual rationality, there is no
dilemma whatsoever. Communication and coordination do not change
anything: whatever the other party does it is still preferable for each player
to confess and defect. The decision-making process being simultaneous or
sequential, also does not affect the outcome: in the PD, defection is a strictly
dominant strategy. The force of attraction of the PD comes from the
extreme simplicity with which it puts in perspective the obstacles for self-
interested, instrumentally rational, egoistic individuals to benefit from
mutually advantageous cooperation. PD became the ensign for the tension
between individual and group rationality.6 Among the 78 (2 X 2)
strategically non-equivalent games (2 players and 2 strategies per player),
5 It is the Rand Corporation who decided to finance the pioneers Flood and Dresher
to test “Nash solutions” for non-cooperative games to support the US nuclear policy
and doctrine. Cf Marvin Flood, Some Experimental Games. Research Memorandum
RM–789 (Santa Monica, Cal.: Rand Corporation, 1952). Flood and Dresher are the
first who have formalized the PD. Often it is the name of Albert Tucker, which is
mistakenly associated with the primacy as to putting the PD in the centre of interest
of GT. It is somewhat ironic that Tucker, who is a mathematician and who was the
Ph.D. advisor of non-other than John Nash (popularized by actor Russell Crowe in
the biographical drama film A Beautiful Mind) is credited with putting together the
dramatic literary narrative that accompanies the model.
6
Rapoport, Anatol. “Prisoner’s Dilemma-Recollections and Observations.” In Game
Theory as a Theory of Conflict Resolution, ed. Anatol Rapoport (Dordrecht: D.
Reidel Publishing Company, 1974), 17-34.
76 Chapter V
which are present in Rapoport and Guyer’s classification,7 the PD is unique

in its characteristics. It is the only one, which has a unique and stable
equilibrium,8 which is Pareto-deficient (or Pareto-non optimal)9. Mutual
defection is the only equilibrium in both pure and mixed strategies and
defection is the dominant strategy for both players. Simultaneous and
sequential playing both lead to the collectively unacceptable outcome. If it
is a one-shot game, there is no room for cooperation.
A point of special importance, especially in the light of bringing together

GT and AI, resides in the direction that game theorists chose in their efforts
to deal with the complexity of the problems posed by the interaction of
players. It is a hypothesis (complete information)10 emitted in a rudimentary
form for the first time by Von Neumann and Montgerstern,11 refined by John
Nash12 and after some empirical tests and experimentation by Thomas
Schelling,13 it became one of the pillars of David Lewis’s theory of social
7 Anatol Rapoport and Melvin J. Guyer, “A Taxonomy of 2 X 2 Games,” General

Systems, Vol. 11 (1966): 203-214. Cf. also Anatol Rapoport and Melvin J. Guyer
and David Gordon, The 2 X 2 Game (Ann Arbor: The University of Michigan Press,
1976).
8 Nash equilibrium (John Nash is Noble Laureate in economics in 1994 and Abel
Prize winner in 2015) represents the outcome of the game in which the strategy of
each player is the best possible response to the strategy chosen by the other player.
It is also called “no-regret situation” given the fact that after the game is played no
ulterior modification of any player’s strategy could provide him/her with a better
payoff.
9 In contemporary economics a situation is Pareto-optimal if there is no other social
situation theoretically conceivable that can ameliorate the resource situation of at

least one individual, without deteriorating that of another one. The process of
switching from a non-optimal to optimal situation being called Pareto-amelioration.
10
The concept of complete information considers that game situations are such that
“each player is fully aware of the rules of the game and the utility functions of each
of the players” [Duncan R. Luce and Howard Raiffa, Games and Decisions:
Introduction and Critical Survey (Wiley, 1957)].
11 John von Neuman and Oscar Montgerstern, Theory of Games and Economic
Behavior (Princeton: Princeton University Press, 1944).

12 John F. Nash, “Equilibrium Point in N-person Games,” Proceedings of the
National Academy of Science, No. 36 (1950): 48-49.

13 Thomas Schelling, The Strategy of Conflict (Cambridge, MA: Harvard University
Press, 1960).
conventions.14 Lewis developed the first systematic account of the concept

of common knowledge,15 which is crucial for the correct understanding of
orthodox game theory It is through common knowledge that game theorists
chose to overcome the indeterminacy problem. Generally, common
knowledge is explained as an assumption for the complete specification of
the model, completing the notion of complete information (each player
knows the rule of the game and each other utility functions) by making
players additionally required to be aware of this fact; players must be aware
of the awareness of the other players regarding the rules and each other’s
utility functions. Furthermore, each player must be aware that each player
is aware that all other players are aware, and so on. To put it in a nutshell,
the common knowledge serves to smash the vicious circle of rational
anticipation. The principle is especially useful in games (unlike the PD) in
which there is no equilibrium (the assurance game) or there are multiple
concurrent equilibria (the dating game), and its main purpose is coordination
between players and avoiding sub-optimal outcomes. Given the fact that
players are rational, they anticipate other players being rational and on the
collective level, they all opt for the solution of the game: the Nash
equilibrium, which is the demonstration of their rationality. In fact, it is not
the rationality of players that leads to the solution of the game, but the
solution of the game that brings out the rationality of players. Common
knowledge assures the perfect uniformization of the point of view of players
and the game theorists and as Schmidt16 and Hedoin17 rightly point out, there
are enough elements in the theoretical and analytical backbone of game
theory to strongly question its automatic affiliation with micro/individualistic
explanations. Both authors point to common knowledge as a vivid
illustration that the logic, which operates game-theoretical models, implies
14 David K. Lewis, Convention: A Philosophical Study (Oxford: Blackwell
Publishing, 1969).
15 Cyril Hedoin, “A Framework for Community-based Salience: Common Knowledge,
Common Understanding and Community Membership,” Economics and

Philosophy, 30 (2014): 366.
16 Schmidt, “From the ‘Standards of Behaving’ to the ‘Theory of Social Situations’.
A Contribution of Game Theory to the Understanding of Institutions”.

17
Cyril Hedoin, “Linking Institutions to Economic Performance: The Role of
Macro-Structures in Micro-Explanations,” Journal of Institutional Economics, 8(3)
(2012): 327-349.
78 Chapter V
to a very large extent the pre-existence of macrostructures that are directing

the interaction between individual agents. Common knowledge is interesting
from our perspective because it provides a way to vastly improve the
interactional capacities of isolated agents (or strategies, or players, or
algorithms). In fact, in many game-theoretical models, the postulate of
common knowledge is a bridge between individual rationality and common
rationality. Even if in its classical version, the universe of common
knowledge is an outdated and exhausted analytically hypothetico-deductive
common space of interactional logic, there are no obstacles to transposing
this interactional model to have meaning for smart, heterogenic, and
adaptative strategies/players/algorithms/ or even individual single-agent
neural networks and why not, even it is probably overambitious, multiagent
deep neural network.18
Two more remarks on the PD are of order:
When the PD is played like a one-shot game (interaction is unique), there is

no room for cooperation. However, if the game (interaction) is iterated, then
there is the possibility of switching toward the Pareto-optimal outcome and
cooperation.
Isn’t bilateral optics especially constraining regarding the use of GT as a

tool to collaborate with AI? Bilateral games in GT are by far more
renowned, and when future interactions and heterogenic strategical
populations have introduced a myriad of endless bilateral interactions
generate an astonishing amount of data and potential input for the data-
hungry algorithms. Moreover, the classical problems of collective action
(the n-person prisoner’s dilemma) are treated in GT with much of the same
conceptual framework and even though there are some differences in the
hypothetico-deductive treatment of models, those divergencies fade away
18When doing the preliminary research for this far-fetched idea I stumbled onto a
piece by Elena Nisioty, “In need of evolution: game theory and AI” MAY 12,
2018/#MACHINE LEARNING [“In Need of Evolution: Game Theory and AI”,
accessed June 1, 2021, https://www.freecodecamp.org/news/game-theory-and-ai-
where-it-all-started-and-where-it-should-all-stop-82f7bd53a3b4/ “MAY 12, 2018/
#MACHINE LEARNING], which was a factor to pursue in such direction and where
very similar ideas, concerning the opportunities for the development of supra-
individual neural networks are put to the front.
in the optics of adaptation, learning and especially reinforcement learning

algorithms.
Evolutionary Game Theory, Computer Assisted

Simulations, Reinforcement Learning,
Deep Learning, Machine Learning
What computer-assisted simulations (CAS) as an early precursor to the
introduction of AI brought to the field of GT was the possibility of the
omniscient, omni-calculating rational, but somewhat dorkish economic agent
(ultimately repeating eternally the same choice in the same circumstance)19
with evolving, adapting and learning players (strategies/algorithms). That
shifted almost immediately the analytical focus from exploring the faith of
individual strategies to investigating the outcome and processes on the level
of the system of players. That could be players playing in a multitude of
bilateral PDs or players involved in a collective action problem (or
coalition-building problems) in an n-person PD. It also shifted the balance
from optimal individual strategies to strategies that perform well
(satisfaction logic replaces individual maximization, limited rationality
replaces omniscient economic rationality) in different strategical environments
and with different sets of other strategies. Interactional capacities, which
were not a topic of interest in deductive GT,20 became a center of attention.
The shift of interest toward evolutionary game theory (EGT) started as a

process at the end of the 1960s with the sustained work in economics and
experimental psychology of pioneers like Herbert Simon and James March21
who were trying to formulate an alternative to the strict economic definition
19 As Vilfredo Pareto put it already in 1900: “From the point of view of choices, the
Homo oeconomicus becomes a machine to make these choices, which given the
circumstance, always makes the same choice”, cited by Bruni and Guala. Cf. Bruni,
Luigino and Francesco Gualo. “Pareto’s Theory of Choice from the Cours to the
Traittato. Utility, Idealization and Concrete Deductive Method.” In Pareto
aujourd’hui, coll. Sociologies, ed. Bouvier Alban. (Paris: P.U.F., 1999), 118.
20 Except from experimental psychologists who used GT models (mostly PD) and
conducted a massive, but somewhat chaotic research on the topic.

21 James G. March and Herbert Simon, Organizations (New York: Dover
Publications, Inc., 1958).

80 Chapter V
of rationality. They jointly formulated the theory of bounded rationality (as

early as 1958) unsatisfied with the empirical irrelevance of the orthodox
economic agent and started working on finding analytical procedures that
could reflect this theoretical reorientation. Very quickly, over the next years,
they realized that eliminating the consequential program of strict economic
rationality comes at a heavy cost: ipso facto it implies the abolition of the
perfect analytical toolkit of the economic model and the use of mathematical
deduction. Without going into a detailed historical account, it suffices to say
that from that point onward, the search for such an alternative had a
profound modifying impact on the research in the field of decision-making
analysis. And it paved the way for the search for an alternative analytical
logic: computer-assisted simulations, the use of Bayesian rationality,
probabilistic strategies, and, at the end of the process, AI made its entrance
into the field of GT. The most important consequence, however, was that
giving up on strict economic rationality became a legitimate act, even to
some extent among economists. It is a short 1973 article (just over 3 pages)22
published in “Nature” by the British-American duo John Maynard Smith
(evolutionary biologist and geneticist) and George R. Price (population
geneticist) that marks the abrupt switch of analytical optics and is associated
with being the trigger of the boom of evolutionary GT. It is this paper that
is unanimously considered the basis for the effervescence of EGT. The
player and his strive for optimality are no longer the centers of attention. It
is the collective stability of strategies and their long run fitness in changing
and dynamic environments that become the main preoccupation. This shift
originated in the conceptual frame of evolutionary biology and introduced
a whole new conceptual area in GT bringing to the front concepts like
evolutionary stable strategy, collective stability, evolutionary equilibrium
selection, strategic selection, strategic replication, random mutations, etc.
22 John M. Smith and George R. Price, “The Logic of Animal Conflict,” Nature 246
(1973): 15–18.
Fig. 2 Principles of the use of replicator dynamics in iterated bilateral games

with heterogeneous strategical population
Despite some resistance from mainstream game-theorist, this turn was

welcomed and relatively quickly adopted by the research community. Pure
imports for biology with a Darwinist perspective were very quickly
introduced and ecological (replicator dynamics – operating at the
populational level) and evolution-based (genetic algorithm – operating at
the individual level) simulations became common practice in the field.
From an initial set of agents (strategies) the strategic interactions are
simulated in a process that favors agents/strategies that procure relatively
better results and penalizes those that perform badly Despite the exact
parameters chosen by the researcher, the process closely resembles the logic
of the survival of the fittest. With each new generation of agents/strategies,
the replicator rules determine the composition of the new population (see
Fig. 2 above). Controlled or random mutations can also be introduced and
thrown into the mix. The center of attention is now shifted toward both the
evolution of individually successful strategies and the collective faith of the
population and its strategic composition. And their mutual dependency.
82 Chapter V
The major novelties for GT from the introduction of CAS were:
ৼ GT shifted from a hypothetico-deductive to a much more experimental

analytical logic.
ৼ A dynamic, changing environment replaced the static framework of
classical GT.
ৼ The static equilibrium optic was replaced with analysis in terms of
processes, patterns, and mechanisms.
ৼ CAS simulations put the spotlight on the feedback loop between
individual strategic characteristics (composition of the strategic
population) on the micro-level and/or the emerging/emergent structural
factors and effects on the macro level.
ৼ Classic GT models were definitively anchored in interactions between
strategically homogeneous actors (rational agents). CAS introduced the
possibility to run and simulate interactions between players with
different decisional logics (strategical heterogeneity) and explore the
systemic consequences.
ৼ Multi-agent modeling, which is intimately linked to the progress of AI
was an inextricable part of the spectacular development of evolutive GT
in the 1980’s.
ৼ CAS simulations introduced the possibility for researchers to manipulate
the experimental setting and introduce controlled manipulations of the
environment, time, events. etc. This is an important shift, especially in
the optics of adapting, evolving, and learning strategies.
ৼ CAS simulations greatly improved the empiric relevance of GT. The
spectacularly accrued capabilities for specifying the experimental
conditions offered the possibility to formulate conditional interactional
patterns of the type that if the agents engaged in a specific configuration
of their interactions behave in a certain manner, we could expect the
subsequent effects on the individual level and the following emerging
effects on the systemic level.
ৼ CAS also had implications for Rational Choice Theory. It showed the
discrepancy between the premises of the analytical model of perfect
rationality (perfect information, unlimited capacities for information
treatment, stable and hierarchized preferences, etc.)23 and the real

conditions for decision making. The first modification consisted in the
fact that several authors opted to think of strategies as a form of
intelligent adaptation to context and not as a constant optimization.24
The second, and even more important one, was to shift the focus towards
learning processes and capacities (the exploration of Bayesian
rationality in the first stages),25 and the subsequent introduction of the
genetic algorithm (GA),26 which introduced the optics of adaptation.
23 The economic agent must execute the optimal choice in any theoretically
conceivable situation, thus maximising his/her utility function.

24 Robert Axelrod and Michael D. Cohen, “Coping with Complexity: The Adaptive
Value of Changing Utility,” The American Economic Review, Vol. 74, No. 1 (March
1974): 30-42. John H. Holland, Adaptation in Natural and Artificial Systems
(Michigan: The University of Michigan Press, 1975). Richard H. Day and Theodore
Groves, eds. Adaptive Economic Models (New York: Academic Press, 1975).
Patrick D. Nolan, “External Selection and Adaptive Change: Alternative Models of
Sociocultural Evolution,” Sociological Theory, Vol. 2, (1984): 117-139. Robert
Axelrod, The Complexity of Cooperation: Agent-Based Models of Competition and
Collaboration (Princeton, N.J.: Princeton University Press, 1997).
25 Richard M. Cyert and Morris DeGroot, “Bayesian Analysis and Duopoly Theory,”
The Journal of Political Economy Vol. 78, No. 5 (September–October 1970): 1168-
1184. Richard M. Cyert and Morris DeGroot, “An Analysis of Cooperation and
Learning in a Duopoly Context,” The American Economic Review, Vol. 63, No. 1
(March 1973): 24-37. Richard M. Cyert and Morris DeGroot, “Rational
Expectations and Bayesian Analysis,” The Journal of Political Economy, Vol. 82,
No. 3 (May–June 1974): 521-536. Robert Denhardt and Philip Jeffers, “Social
Learning and Economic Behavior: The Process of Economic Socialization,” The
American Journal of Economics and Sociology, Vol. 30, No 2 (April: 1971): 113-
125. Day and Groves, Adaptive Economic Models. John Harsanyi, “Bayesian Theory
and Utilitarian Ethics,” The American Economic Review. Papers and Proceedings
of the Ninetieth Meeting of the American Economic Association, Vol. 68, No. 2 (May
1978): 223-228.
26
The GA, very roughly, is functioning by safeguarding interactional information
that procured relative success to the strategy and discarding information that didn’t
for determining better strategic behavior in future interactions (Cf. Holland,
Adaptation in Natural and Artificial Systems. John H. Holland, “Adaptive
Algorithms for Discovering and Using General Patterns in Growing Knowledge
Bases,” International Journal of Policy Analysis and Information Systems, No. 4
(1980): 245-268. John H. Holland, “Genetic Algorithms,” Scientific American, 267,
No. 7 (July 1992): 44-50. David. E. Goldberg, Genetic Algorithms in Search,
Optimization, and Machine Learning (Reading, Mass.: Addison-Wesley Pub. Co.,
1989). Rick L. Riolo, “The Amateur Scientist: Survival of the Fittest Bits,” Scientific
84 Chapter V
The EGT even produced a best-seller and a star author. Robert Axelrod’s
The Evolution of Cooperation (1983)27 became one of the most read,
translated (20 + languages), and quoted scientific books. It is an excellent
starting point for everybody interested in the problemɚtic. It showed in a
rather convincing manner that from the moment the interaction in a PD has
a future (iterated games) and there is room for a decision-making autonomy
outside of the box (strategical heterogeneity), mutual general defection is no
longer a pole of universal attraction. Nevertheless, a warning regarding
Axelrod’s conclusions is of order. The hasty supposition of Axelrod that the
simple reciprocal imitation of the strategy of the other player (the TIT-FOR-
TAT strategy)28 is the best possible strategy, and that, gradually it becomes
an evolutionary stable outcome on the level of the population (that the
population will evolve towards an all TFT composition and no one will have
the incentive to deviate from that collective equilibrium) is flawed. Kenneth
Binmore, one of the leading game-theorists during the last 40 years, went
so far as to devote a whole chapter to understanding the impact of Axelrod’s
work. He labeled the whole interest generated by the work of Axelrod and
the hasty acceptance of his conclusions the “TIT FOR TAT bubble”.29
Binmore is critical of the impact of Axelrod’s work and especially of his
method of merging easily the borders between classical GT, CAS and quasi-
experimentation, theory construction, and the empiric reality. The term is
also an important reminder of the effects that taking methodological
shortcuts has in the domain of the use of AI: by accepting Axelrod’s
conclusions as firm theoretical foundations and proven decision-making
patterns number of researchers in the field started integrating his theory of
cooperative behavior in their own experiments. This conclusion is valid
even to this day and in our understanding, is one of the determining factors
for grounding the use of AI in the optic of individual learning/adaptation
and the definition of the “work” of agents as understanding, mapping, and
American, Vol. 267 (1) (1992): 114-117. This process is coupled with the
maintenance of a general strategical line, which is rectified via the input of new
interactional information.
27 Robert Axelrod, The Evolution of Cooperation (New York: Basic Books, 1984).
28 Cooperate when the other party cooperates, defect to defection and never start an
interaction by a defection.
29 Kenneth Binmore, Playing Fair: Game Theory and the Social Contract, Vol. 1
(Cambridge, Mass.: MIT Press, 1994), 194.

detecting the properties of the environment (its strategical composition and

structural properties) and the formulation of a corresponding line of
strategic behavior. Learning is a tool to reach coordination with others in
multiagent environments.30 There is a variety of multiagent learning
techniques proposed with the goal to coordinate interactions on various
solutions and outcomes (e.g., Nash equilibrium, specific collective
equilibria, Pareto-optimality, etc.) in different settings, e.g., minimax Q-
learning,31 Nash Q-learning,32 and Conditional Joint Action Learning,33
quantum computation conditional strategies,34 to name just a few. Game
theory is currently investing in the exploration of settings where multiple
players learn through reinforcement, an area called multi-agent reinforcement
learning (MARL). At the price of a considerable simplification, we can
speculate that multi-agent learning is a rather complexified form of single
agent learning in which additional neural networks are made available and
“plugged” into the agent. The progress in this field is undeniable; but what
is important is that ontologically speaking MARL is still a form of complex
individual learning. A very complex and still complexifying form of
learning, but clearly individual learning. Yet we need more if our choice is
to extract more lessons from GT. We need more than strictly individual
optimization for comprehending how to prevent pervasive collective effects
and how to efficiently channel individual contributions toward the
realization of common goals. Enough of rationality! Let’s talk some ethics
… Promoting “The Collective” (if we use the Borg terminology from Star
Trek) can be seen as a vicious form of totalitarian menace aiming to subdue
individuality to the good of the whole. There are four major practical
30 Peter Stone and Manuela Veloso, “Multiagent Systems: A Survey from a Machine
Learning Perspective,” Autonomous Robots, 8(3) (2000): 345–383.
31 Littman, Michael. “Markov Games as a Framework for Multi-agent Reinforcement
Learning.” In Proceedings of the Eleventh International Conference on Machine

Learning (San Francisco: Morgan Kaufmann, July 1994), 157–163.
32 Junling Hu and Michael P. Wellman, “Nash Q-learning for General-sum
Stochastic Games,” Journal of Machine Learning Research, Vol. 4 (Nov 2003):

1039–1069.
33 Dipyaman Banerjee and Sandip Sen, “Reaching Pareto-optimality in Prisoner’s
Dilemma Using Conditional Joint Action Learning,” Autonomous Agents and Multi-
Agent Systems 15(1) (2007): 91–108.
34 Konstantinos Giannakis et al., “Quantum Conditional Strategies and Automata for
Prisoners’Dilemmata under the EWL Scheme,” Appl. Sci. 9(13) (2019): 2635.
86 Chapter V
consequences that we foresee from pursuing the type of research agenda we

promoted here, and we will briefly examine their ethical implications:
1. Upgrading the multi-agent reinforcement learning at the procedural

level via the use of GT models (for machines): there are no true
ethical issues here. Machine learning is constantly refining and
improving itself as a process, regardless of its collaboration with GT.
GT is a research and refining path amongst others.
2. Upgrading and refining the analytical capacities in the exploration
of GT models and increased understanding of the implications for
the empirical world (both for human and machines): again, no ethical
issues here. Learning GT and using its practical implications is not
an ethical issue per se. Moreover, if I am responsible for negotiating
an important peace treaty ending a bloody conflict, I will strongly
prefer that the representative of the other party knows GT too.
3. Increased capacity to boost and/or prevent cooperation (for
machines and humans): This is a classical ethical problem that could
be found in the literature surrounding cooperation. If we go back to
authors like Hobbes, Hume, Rousseau, and Smith (and engineers
testing algorithms via GT also should do the same) we will discover
that the ethical dimensions of the contradicting realities of a
harmonious social order and individual liberty and autonomy are
present in social and political philosophy from its debuts. Authors
like Kenneth Arrow,35 Radiyna Braithwaite,36 Garry Runciman and
Amartya Sen,37 Amartya Sen,38 John Rawls,39 Robert Nozick,40
35 Kenneth J. Arrow, Social Choice and Individual Values (New York: Wiley, 1951).
36
Radiyna B. Braithwaite, Theory of Games as a Tool for the Moral Philosopher
(Cambridge: Cambridge University Press, 1955).
37 Garry W. Runciman and Amartya K. Sen, “Games, Justice and the General Will,”
Mind, New Series, Vol. 74, No. 296 (October 1965): 554-562.
38 Amartya K. Sen, Ethique et Economie (Paris: Presses Universitaires de France,
2001).
39 John. Rawls, “Justice as Fairness,” Philosophical Review, Vol. 67 (April 1958):
164-194. John. Rawls, A Theory of Justice (Cambridge, Mass.: Belknap Press of

Harvard University Press, 1971).
40 Robert Nozik. Anarchy, State and Utopia (New York: Basic Books, 1974).
David Gauthier,41 and Gregory Kavka42 continued with that

problematic discussing the ethical dimensions of the capacity for
promoting/preventing cooperation but none of them declared that
such knowledge could be unethical. At the end of the day,
understanding those processes is beneficial for autonomous free
individuals. Ethical issues arise when we are dealing with a central
authority that promotes/prevent cooperation. It becomes, yet again,
a question of good or bad governance. Ignoring purposefully those
mechanisms is not really a safeguard against bad governance.
4. AI capacity to analyze, detect and prevent negative “unintended
consequences” (perverted effects) in complex interactions – (machines
and humans): the fact that people often, acting in accordance with
rational individual reasons, could trigger collective consequences
and phenomena that are unintended, undesirable and negative and
non-rational for everybody is a theme that is present in social thought
from the times of Weber and Simmel and explicitly formulated
research agenda by Popper and von Hayek and later by Raymond
Boudon.43 All those authors consider individual freedom as a
cornerstone value, yet all of them strived to limit and counter
“perverted effects”. There is nothing paradoxical in such a position.
As thinkers that openly promote individual freedom, they are
perfectly aware of one of the major consequences of its central stage
place and growing importance in our societies: the risk of accrued
undesirable and unintended negative social effects.
As you might already have suspected the author of this article is not an
expert in AI (despite a Ph.D. involving CAS), but rather in decision-making.
The present text is an effort to persuade specialists in AI to consider the
possibility that even though GT proved to be an extremely fruitful lab field
41 David Gauthier, “David Hume, Contractarian,” The Philosophical Review, Vol.
88, No. 1, (1979): 3-38. David Gauthier, Morals by Agreement (Oxford: Clarendon,
1986).
42 Gregory Kavka, Hobbesian Moral and Political Theory (Princeton: Princeton
University Press, 1986).

43 Raymond Boudon, Effets pervers et ordre social (Paris: Presses Universitaire de
France, 1977).
88 Chapter V
for the progress of different aspects of AI (algorithm refinement, learning

procedures, deep learning, etc.) it is grant time to decide if the area of GT is
something more than a test tube for specific algorithms and a tool for
optimization of resources under constraints. Since GT is much more for
numerous researchers in economics, sociology, political science,
philosophy, decision making, biology and many other fields of knowledge,
the question of if and how much AI is going to help affront the paradoxes
of rationality is unavoidable. For us, that haven’t given up on exploring the
effects of rationality (though its individual analytical effects and
implications in its classical hypothetic-deductive form maybe on the board
of exhaustion) the possible shift of optics from individually optimizing and
adapting algorithms (deep learning on the individual algorithmic level)
toward the creation of smart networks where different algorithms cooperate
and coordinate their answers to systemic changes and disequilibria via a
pool of common knowledge (yes …. “collective” common knowledge)44 is
a logical and pressingly needed next step. Pushing the research agenda in
the domain in this direction may be an ambitious and overcomplex
endeavor, but the potential benefits are obvious, and they are far more
reaching than the strict local advances of AI.
References
Arrow, Kenneth J. 1951. Social Choice and Individual Values. New York:
Wiley.
Axelrod, Robert. 1984. The Evolution of Cooperation. New York: Basic
Books.
Axelrod, Robert. 1997. The Complexity of Cooperation: Agent-Based
Models of Competition and Collaboration. Princeton, N.J.: Princeton
University Press.
Axelrod, Robert, and Michael D. Cohen. “Coping with Complexity: The
Adaptive Value of Changing Utility.” The American Economic Review,
Vol. 74, No. 1 (1974): 30-42.
44 Cf. Nisioti, “In Need of Evolution: Game Theory and AI”.

Banerjee, Dipyaman, and Sandip Sen. “Reaching Pareto-optimality in

Prisoner’s Dilemma Using Conditional Joint Action Learning.”
Autonomous Agents and Multi-Agent Systems 15(1) (2007): 91–108.
Binmore, Kenneth. 1994. Playing Fair: Game Theory and the Social
Contract, Vol. 1. Cambridge, Mass.: MIT Press.
Borel, Emile. “La théorie des jeux et les équations à noyau symétrique
gauche.” In Comptes rendus de l’Académie des sciences, Vol. 173,
1304–1308. (Paris: Académie des Sciences Morales et Politiques).
Boudon, Raymond. 1977. Effets pervers et ordre social. Paris: Presses
Universitaire de France.
Braithwaite, Radiyna B. 1955. Theory of Games as a Tool for the Moral
Philosopher. Cambridge: Cambridge University Press.
Bruni, Luigino, and Francesco Gualo. “Pareto’s Theory of Choice from the
Cours to the Traittato. Utility, Idealization and Concrete Deductive
Method.” In Pareto aujourd’hui, coll. Sociologies, edited by Bouvier
Alban, 111-126. (Paris: P.U.F., 1999).
Cyert, Richard M., and Morris DeGroot. “Bayesian Analysis and Duopoly
Theory.” The Journal of Political Economy Vol. 78, No. 5 (1970): 1168-
1184.
Cyert, Richard M., and Morris DeGroot. “An Analysis of Cooperation and
Learning in a Duopoly Context.” The American Economic Review, Vol.
63, No. 1 (1973): 24-37.
Cyert, Richard M., and Morris DeGroot. “Rational Expectations and
Bayesian Analysis.” The Journal of Political Economy, Vol. 82, No. 3
(1974): 521-536.
Day, Richard H., and Theodore Groves, eds. 1975. Adaptive Economic
Models. New York: Academic Press.
Denhardt, Robert, and Philip Jeffers. “Social Learning and Economic
Behavior: The Process of Economic Socialization.” The American
Journal of Economics and Sociology Vol. 30, No 2 (1971): 113-125.
Flood, Marvin. 1952. Some Experimental Games. Research Memorandum
RM–789. Santa Monica, Cal.: Rand Corporation.
Gauthier, David. “David Hume, Contractarian.” The Philosophical Review,
Vol. 88, No. 1 (1979): 3-38.
Gauthier, David. 1986. Morals by Agreement. Oxford: Clarendon.
90 Chapter V
Giannakis, Konstantinos, Georigia Theocharoplou, Christos Papalitsas,

Sofia Fanarioti, and Theodore Andronikos. “Quantum Conditional
Strategies and Automata for Prisoners’ Dilemmata under the EWL
Scheme.” Appl. Sci. 9(13) (2019): 2635.
https://doi.org/10.3390/app9132635.
Goldberg, David. E. 1989. Genetic Algorithms in Search, Optimization, and
Machine Learning. Reading, Mass.: Addison-Wesley Pub. Co.
Harsanyi, John. “Bayesian Theory and Utilitarian Ethics.” The American
Economic Review. Papers and Proceedings of the Ninetieth Meeting of
the American Economic Association, Vol. 68, No. 2 (1978): 223-228.
Hedoin, Cyril. “Linking Institutions to Economic Performance: The Role of
Macro-Structures in Micro-Explanations.” Journal of Institutional
Economics, 8(3) (2012): 327-349.
Hedoin, Cyril. “A Framework for Community-based Salience: Common
Knowledge, Common Understanding and Community Membership.”
Economics and Philosophy, 30 (2014): 365-395.
Holland, John H. 1975. Adaptation in Natural and Artificial Systems.
Michigan: The University of Michigan Press.
Holland, John H. “Adaptive Algorithms for Discovering and Using General
Patterns in Growing Knowledge Bases.” International Journal of Policy
Analysis and Information Systems, No. 4 (1980): 245-268.
Holland, John H. 1992. “Genetic Algorithms.” Scientific American, 267,
No. 7 (1992): 44-50.
Hu, Junling, and Michael P. Wellman. “Nash Q-learning for General-sum
Stochastic Games.” Journal of Machine Learning Research, Vol. 4
(2003): 1039–1069.
Kavka, Gregory. 1986. Hobbesian Moral and Political Theory. Princeton:
Princeton University Press.
Lewis, David K. 1969 [2002]. Convention: A Philosophical Study. Oxford:
Blackwell Publishing.
Littman, Michael. “Markov Games as a Framework for Multi-agent
Reinforcement Learning”. In Proceedings of the Eleventh International
Conference on Machine Learning, 157–163. (San Francisco: Morgan
Kaufmann, July 1994).
Luce, Duncan R., and Howard Raiffa. 1957. Games and Decisions:
Introduction and Critical Survey. New York: Dover Publications, Inc.
March, James G. and Herbert Simon. 1958. Organizations. New York:

Dover Publications, Inc.
Merton, Robert K. “The Unanticipated Consequences of Purposive Social
Action.” American Sociological Review, 1 (6) (1936).
Nash, John F. “Equilibrium Point in N-person Games.” Proceedings of the
National Academy of Science, No. 36 (1950): 48-49.
Nisioti, Elena. “In Need of Evolution: Game Theory and AI”. Accessed
June 1, 2021. https://www.freecodecamp.org/news/game-theory-and-ai-
where-it-all-started-and-where-it-should-all-stop-82f7bd53a3b4/
“MAY 12, 2018/#MACHINE LEARNING
Nolan, Patrick D. “External Selection and Adaptive Change: Alternative
Models of Sociocultural Evolution.” Sociological Theory, Vol. 2 (1984):
117-139.
Nozik, Robert. 1974. Anarchy, State and Utopia. New York: Basic Books.
Rapoport, Anatol. “Prisoner’s Dilemma-Recollections and Observations.”
In Game Theory as a Theory of Conflict Resolution, edited by Anatol
Rapoport, 17-34. (Dordrecht: D. Reidel Publishing Company, 1974).
Rapoport, Anatol, and Melvin J. Guyer. “A Taxonomy of 2 X 2 Games.”
General Systems, Vol. 11 (1966): 203-214.
Rapoport, Anatol, Melvin J. Guyer, and David Gordon. 1976. The 2 X 2
Game. Ann Arbor: The University of Michigan Press.
Rawls, John. “Justice as Fairness.” Philosophical Review, Vol. 67 (1958):
164-194.
Rawls, John. 1971. A Theory of Justice. Cambridge, Mass.: Belknap Press
of Harvard University Press.
Riolo, Rick L. “The Amateur Scientist: Survival of the Fittest Bits.”
Scientific American, Vol. 267 (1) (1992): 114-117.
Runciman, Garry W., and Amartya K. Sen. “Games, Justice and the General
Will.” Mind, New Series, Vol. 74, No. 296 (1965): 554-562.
Schelling, Thomas. 1960. The Strategy of Conflict. Cambridge, MA:
Harvard University Press.
Schmidt, Christian. “From the ‘Standards of Behaving’ to the ‘Theory of
Social Situations’. A Contribution of Game Theory to the Understanding
of Institutions.” In Knowledge, Social Institutions, and the Division of
Labor, edited by Pier Luigi Porta, Roberto Scazzieri and Andrew
Skinner, 153-168. (Cheltenham: Edward Elgar Publishing, 2001).
92 Chapter V
Sen, Amartya, K. 2001. Ethique et Economie. Paris: Presses Universitaires

de France.
Smith, John M., and George R. Price. “The Logic of Animal Conflict.”
Nature 246 (1973): 15–18.
Stone, Peter and Manuela Veloso. “Multiagent Systems: A Survey from a
Machine Learning Perspective.” Autonomous Robots, 8(3) (2000): 345–
383. https://doi.org/10.1023/A:1008942012299.
Von Neuman, John. “Zur Theorie der Gesellschaftsspiele.” Matematische
Annalen, 100 (1928): 295-320.
Von Neuman, John. “On the Theory of Games of Strategy.” In
Contributions to the Theory of Games, Vol. 4, edited by Albert Tucker
and Duncan Luce, 13-42. (Princeton: Princeton University Press, 1959).
Von Neuman, John, and Oscar Montgerstern. 1944. Theory of Games and
Economic Behavior. Princeton: Princeton University Press.
CHAPTER VI
ARTIFICIAL INTELLIGENCE IN DEFENCE
TODOR DIMITROV
Introduction
Today, geopolitical competition is global and the race of technological
adoption is extremely important. One of the main issues is the acceptance,
integration and use of new technology in society. From Artificial
Intelligence to quantum and everything in between, governments are in a
race to leverage these technologies at scale and speed. The first adopter
advantage for emerging disruptive technology could not be more prevalent
in the world of geopolitics and deterrence. It is quite possible that the nations
that win this race may be those with the most flexible bureaucracy rather
than those with the best technology.
Analyzing the processes of transformation in technology and taking into

account the advantage of the military sphere to take priority of new
developments first, it is necessary to change the psychology of military
operations and issues of defense and security in general. The new
dimensions of the combat space, the philosophy of intelligent warfare and
the experience of military operations require the acquisition and use of a
complex, broad spectrum of capabilities.
The actuality of the theme comes from dynamics of the security

environment and the increasingly widespread use of AI in the area of
defense. The rapid development of intelligent machines in recent decades
has shaped their path and their increasing use in the field of defense. The
powerful economies allocate significant budgets to finance projects for the
development of AI in the military field, and their goal is to establish
94 Chapter VI
leadership in this area. The competition is certainly not advertised, but to a

large extent, an analogy can be made with the Cold War nuclear race.
The topic of AI has been widely discussed in the last few decades, but the
world has evolved, and nowadays there is a rapid technological
development in all areas of our society. Some of them constantly involve
new applications in the military and security affairs developing different
systems, based on AI. It is quite important to pay attention to the issues of
implementation of that new technology for defense capabilities because they
could impact the entire future world. The emerging and disrupting
technologies are expected to have a tremendous impact on the military
affairs and not exclusively for creating killing robots. These technologies
even do not have to be used only in lethal systems. According to the
specialists the main issues for defense sector that AI technologies could
improve are management, operational speed, precision, logistic, procurement,
command and control, communications, intelligence, situational awareness,
electronic warfare dominance, cybersecurity, sensors, integration and
interoperability, different types of analysis and voice recognition algorithms,
military medicine, etc. Also, it is critical that other factors such as ethics,
legality, privacy and human control be considered.
Before considering the different applications of Artificial Intelligence in

defense, it is necessary to analyze the capabilities and differences between
defense systems based on Narrow and General AI.
Narrow Artificial Intelligence in Defense

Modern opinions say that the Narrow or called by Kenneth Payne Tactical
AI, has relatively limited applications in a strictly profiled field.1 It can be
concluded that the advantages of this type of AI in the defense, especially
in the speed of decision-making, are likely to be decisive and will make a
large part of the systems currently used unnecessary. Furthermore, the
balance of power can shift dramatically depending on the pace of
development of AI. Offensive strategy is likely to dominate, as the speed of
1Kenneth Payne, Strategy Evolution and War–From Apes to Artificial Intelligence

(Washington DC: Georgetown University Press, 2018).
Artificial Intelligence in Defence 95
decision-making will provide a huge advantage to the first strike. This is

likely to lead to a reconsideration of some current concepts of automation
in military operations. For example, in his study, Frank Barnaby claims that
technology favors defensive action because remote-controlled weapon
systems are more effective at destroying attack platforms.2 Additionally, if
the defense itself is decentralized, it will be a difficult target for attack. But
with the development of technology, the ability to recognize objects on
modern artificial neural networks gives them a better chance to detect and
recognize defense systems and then attack them with concentrated fire or
other actions. It is likely that this development of technology will increase
the prospects for success in the preventive preemptive strike and will change
the current perception of the defensive dominance of nuclear weapons. The
ability to concentrate force quickly and accurately revives the offensive
logic mathematically based on the Lanchester square law. Control in this
area will become increasingly difficult given the speed with which tactical
AI decides how best to optimize its functions.
AI raises concerns about the relative balance of power, especially with

regard to offensive scenarios. The result could be that the dilemma of
maintaining the status quo would fuel an AI arms race and increase pressure
on the leading countries and companies. It is important to take this into
account before discussing the broad speculation about the extent to which
AI systems themselves form key areas of international relations - for
example, through the need for collective solutions to major social changes
in employment or through the impact on the boundary between the public
and private spheres. Both raise significant controversy over the appropriate
powers of states to monitor and control their citizens. And all this only if at
this stage we are talking about narrow AI or tactical weapon systems with
AI. 3
In order to better account for the nature of narrow AI, it is appropriate to

make some generalizations about it:
2 Frank Barnaby, The Automated Battlefield: New Technology in Modern Warfare

(Oxford: Oxford University Press, 1987).
3 Martin Ford, Rise of the Robots: Technology and the Threat of a Jobless Future
(New York: Basic Books, 2019), 229-248.

96 Chapter VI
Ɣ At this stage, there are mainly theoretical dimensions of that type of

AI. Intelligent machines are evolving rapidly, and even if broad
parameters of this speed are visible, such as image recognition, in
many other areas these parameters remain unclear and hazy;
Ɣ When using narrow AI, the logic of combat effectiveness is likely to
surpass the effects of traditional conflicts because the impact through
strike force is likely to be significantly larger and more short-term.
AI does not necessarily have immediate success in all areas of
combat. For example, land conflicts are much more difficult to
regulate and use AI than maneuvering in the space, air, or at sea. This
is due to a more complex and diverse environment in terms of human
presence and physical geography. Simultaneously, as long as war
remains a human endeavor, the morale of the personnel, who have
occasionally reversed the course of hostilities, has been greatly
reduced, and less well-equipped forces have succeeded against
nominally more powerful adversaries. Here, scenarios could be
complicated if this type of weapon is used by malicious non-state
groups. Nevertheless, the changes that will take place in the short
and medium term are likely to be significant;
Ɣ The application of AI in society changes a large number of existing
institutions and practices, and the Armed Forces system does not stay
away from this development. It is highly probable that AI will lead
to even more significant changes in Defense than nuclear weapons.
Autonomous, remote-controlled, nano- and other related technologies
will significantly decrease the number of personnel in the battlefield
and will require new skills from it. They will change the traditional
organization of the Armed Forces into separate areas of land, sea, air
and space because they will integrate platforms operating in all these
areas. Alternative staff differences may arise - perhaps organized by
combat security, such as intelligence, logistics, and others, or
perhaps by technical functions - engineer, computer specialist,
communications specialist, and others. They are likely to change
existing perceptions of the military profession, the requirements for
which will change significantly due to the increasing availability of
automated and remote-controlled platforms.
Ɣ There are certainly speculations on which countries will produce the

best narrow AI in Defense. The progress observed requires ongoing
research and the allocation of large budgets, as well as other
incentives to attract the best researchers, often foreign specialists.
The usual structural incentives for intellectual creativity are applied
- personal freedom, rule of law and guaranteed intellectual property
rights. AI research is concentrated in countries that already have
military power: United States, China, Russia, Great Britain, France
and others. The United States are projected to invest more than $6
billion in AI-related research-and-development projects in 2021,
while contract obligations are on pace to grow nearly 50 percent, to
$3 billion, relative to 2020, according to the forecast.4 China is also
betting on AI to strengthen its defense capabilities, and forecasts
suggest that the country will become a world leader in this field by
2030. The analysis shows that the market share of AI in the defense
industry is expected to reach $ 18.82 billion by 2025, with annual
growth of 14.75% from 2020 to 2025.5
Numerous limitations slow down the spread of AI technologies at this stage.

These are primarily hardware that could advance even faster with the
development of specialized architectures, such as neuromorphic chips or
quantum computing. Another limitation is the engineering expertise for
training AI with working algorithms. Intelligent machines have a lack of
motivation and for this training planned options are to develop an analogy
of “genetic algorithms” that can be combined to create a more efficient
result similar to human DNA.
4 Jon Harper, “Federal AI Spending to Top $6 Billion,” National Defense Magazine,
October 2, 2021,
https://www.nationaldefensemagazine.org/articles/2021/2/10/federal-ai-spending-
to-top-$6-billion.
5 Sebastian-Gabriel Popescu, “Artificial Intelligence in Naval Operations” (paper
presented at the International Scientific Conference Strategies XXI: Technologies

— Military Applications, Simulations and Resources, NDU „Carol I”, Bucharest,
2019).https://docplayer.net/145260428-National-defense-university-command-
and-staff-college-program-technologies-military-applications-simulation-and-
resources.html.
98 Chapter VI
A skeptical view of narrow AI shows that it can be seen as similar to earlier

information processing technologies. This is a good way to reduce, but not
eliminate, the available “fog of war”. The nature of war continues to be
filled with a great deal of uncertainty. The development of combat narrow
AI could further reduce friction on the battlefield without eliminating it
altogether. Taking into account the main qualities of AI for independence
and ability to improve performance through learning, it can be summarized
that the information processing will be improved significantly. The main
difference here is that AI has the ability to make technological decisions,
not just easier decisions. Another skeptical view predicts that AI will never
achieve the ability to deal in complex situations like continental land
conflicts, where the military and civilians mix in a single space. In such
environments, humans are certainly imperfect and unpredictable; also, the
specific situation is highly dependent on technological progress.
AI, used as a tactical weapon system, is able to transform the current

psychological basis for making quick decisions when using violence.
Intelligent machines will not be psychologically shocked by a surprise
enemy shelling or rapid maneuvers. They will not experience fatigue or
stress, as a ship's commander after a long watch or pressed by circumstances
headquarters. They will neither assess risk based on prospect theory by
calculating benefits and losses, nor make a subjective decision based on
anger, failure, or perceived insult, nor any of the multiple cognitive
principles people use in such a situation. But they will maneuver and
concentrate much faster than human controlled systems. Despite skepticism
and the security dilemma based on its unpredictability, the growing
involvement of AI in the armed struggle given its essential advantages is
inevitable. Regulation or banning of the development of AI combat systems
will be extremely complicated because AI is a decision-making technology,
not a specific weapon technology. Also, intelligent machines enter a wide
range of activities outside the field of defense, which makes them extremely
difficult to be limited with restrictive measures.
Artificial General Intelligence in the Field of Defense

Artificial General Intelligence (AGI) is capable of having a revolutionary
impact on military affairs. AGI has a more flexible and powerful intelligence
in each above aspect of narrow AI. This further expands the possibility for
discrepancies between human designers and the behavior of the final
product. This tremendous change will affect the balance of power between
states by increasing the relative strength of the attack, challenging the ability
of human commanders to exercise control after the start of hostilities and
behaving in ways that are in mutual conflict, go beyond human boundaries
and are difficult to predict. The problems could be exacerbated when it
comes to escalating military conflict, deterrence, or violence in AI use.
These aspects have so far been measured only by human psychology and,
eventually, in the future, a novel discipline, “machine psychology,” will
attempt a similar analysis of decision making, justified and performed by
machines.6
Tactical AI, as we have seen, will have a significant impact on the strategy,
even if it uses only the limited technologies available today. This type of
narrow AI is much more “basic” than the scheme outlined before the AGI.
It lacks the degree of cognitive flexibility to easily switch between tasks and
adapt quickly to new circumstances. It can be combined with another
domain-specific AI to create complex knowledge, as in the intelligent poker
machine or similar to the group “swarm intelligence”. In contrast to this
model, the AGI would be an integrated package capable of managing the
individual discrete tasks of composite AI and many other elements. In its
military form, this type of AI could integrate the activities of multiple nets
into a distributed, swarm-like system. It could concentrate power at all
levels of intensity. Of course, this AGI will retain the basic qualities of
narrow AI, such as pattern recognition, memory, and speed, which are easily
transformed into military power. Additionally, it is assumed that the overall
AI will be able to generate distinctive motivation that is not provided by its
designers. Combined with speed and the ability to escalate your efforts
sharply, this makes it dangerous and unpredictable.
The AGI will have various attributes: its nets will maneuver at speed, and it
will concentrate its efforts quickly, outrunning any adversary known to date
in military science. Additionally, it will be able to coordinate efforts across
6 Payne, Strategy Evolution and War–From Apes to Artificial Intelligence, 193.

100 Chapter VI
its full range of capabilities, the main impact of which would be the
management of existing parameters for escalation, violence and deterrence.
When considering the military aspects of AI, much attention is paid to the
synergy between humans and machine. With the increasing speed of
processes, people will not have the ability to stay in the cycle due to the lack
of speed. In such cases, it is good that they can at least remain in the
management chain and be able to intervene and stop actions that are already
under way, especially escalation actions.
The AGI does not need to be aware of what it is doing in a military conflict.
He qualifies as a flexible and adaptive intelligence, able to navigate in a
complex environment and achieve its goals. These qualities are especially
useful when it comes to combat. The integration of different types of
weapon systems with AGI will allow the machine to coordinate not only at
the tactical level, but also at the operational and possibly strategic level.
While tactical AI decisions may have operational and strategic effects in
certain situations, combat AGI will deliberately modulate its behavior to
prevail at higher levels of combat operations.
Aspects and Trends for the Use of AI in the Defense Field

Researchers consider two macro-revolutions in the development of defense
strategy. The first is related to evolutionary development, which gives
humans a different type of social knowledge. One of the key features of
prehistoric existence was the need for close cooperation in single-family
groups for surviving. This environment probably led to the development of
language. The abstract mathematical connection presented by Frederick
Lanchester suggests that larger groups win proportionately in battle, where
it is possible to concentrate power when all other elements are equal.
Dominic Johnson and Niall McKay use Lanchester's law to view and justify
human evolution as a product of conflict.7
7 Dominic Johnson and Niall MacKay, “Fight the Power: Lanchester’s Laws of
Combat in Human Evolution,” Journal Evolution and Human Behavior, Vol.36, No.
2 (March 2015): 152-163.
The second macro-revolution in strategy is currently underway and

happening as a result of the accelerated development of information
processing technologies. It challenges the biological basis of decision-
making. In human history, many other technologies and numerous weapon
systems have been described as revolutionary. They dramatically changed
the nature of the war, but these innovations and weapons require careful
thinking in every context about how better to use them in a strategy. They
certainly complicate the extraction of useful and lasting strategic concepts
throughout history. But at the most fundamental level, such an abstraction
is possible precisely because strategy is primarily a psychological activity,
and human psychology is evolving very slowly. Given this, Clausewitz's
triumph is relatively long in time. Later, when game theory came into play,
Thomas Schelling's analysis recalled that human psychology was not
limited to modeling rational choice theory. Emotion and heuristics are
products of our psychology, and while misperceptions and calculations can
be minimized, it is difficult to avoid situations such as the Cuban Missile
Crisis, which could drag humanity into a crisis of devastating proportions.
In the course of human evolution, a technology has already changed to some

extent the developed psychological basis of strategy, and that is writing.
According to current forecasts, AI will lead to an even more radical
transformation. The connection between the two is that both technologies
are changing our developed psychological ways of making decisions. The
development of writing, independently several times in human history,
provides a way to extract knowledge and expand information in time and
space, thus facilitating specialization. Writing allows for reflection on
strategy and catalyzes technological innovation. Information technology
was relatively limited until very recently in human history. As they
accelerated the development of computers, information technology created
an expectation for a radical change in strategy. This, however, was
premature. The information revolution in military affairs has changed the
parameters of the strategy in multiple respects. It strengthened situational
awareness and allowed greater speed, accuracy and concentration of impact.
The countries that have developed it have significantly increased their
fighting power. An example in this regard is the United States in Operation
Desert Storm in 1990-91 against the Iraqi army. But it is also important to
102 Chapter VI
take into account the fact that there are limitations, both in terms of available
technology and at the conceptual level, where the strategy, at this stage,
remains undoubtedly psychological.
All the world's smart munitions and network sensors cannot determine how
effective an operation in Afghanistan against insurgents and the radical
Taliban would be. At this stage in the development of military affairs,
unequivocally, every automated technique needed a man for selecting the
target or carry out some other decisive intervention. On the one hand, the
development of AI can be seen as a logical continuation of this ongoing
revolution in information processing. In this way, AI could be seen as
another tool for processing information about people. Claims for military
revolutions should be analyzed with skepticism because often the conditions
set for their introduction are too insignificant. Examples in this regard are
concepts that have come and gone at high speed in recent years, such as
hybrid warfare, the indirect approach, the comprehensive approach, the
fourth-generation war, effect-based operations, and much more. There is a
certain possibility that the revolution in strategy, as a consequence of AI,
will pass quickly and unnoticed. But it is important to stress that AI is more
than just another means of processing information. The friction of war and
luck will remain key elements of the strategy, but AI has enormous potential
to bring about a radical change in strategic issues. By making strategic
decisions on behalf of people, and even more so by performing them on
their own, based on calculations that are not within human limits, intelligent
machines can have far-reaching consequences. This could lead to the
transformation of the societies that rule them, to a change in the relative
power between them and to the evolution of the overall nature of future
conflicts. Although the current technologies have made great changes, AI is
much more radical because it changes the very basis of strategic decisions.
Machines do not use the cognitive heuristics that inform human decisions,
much less the conscious reflection we use to imagine the future or to predict
what the adversary may aim for. In such a situation, when the intelligent
machine wants to act at all costs to achieve the goals set by man, it is very
likely that part of the communication will be “lost in translation”.
Additionally, in the near future, AI actions will remain largely at the tactical
level, but the development of technology and competitive relationships will
lead to the introduction of machine intelligence at ever higher levels of

decision-making. AI itself is not a weapon technology, but above all a
decision-making process. The course of evolution will inevitably bring the
influence of AI to a strategic level. The strategy has evolved over the years
thanks to human evolution, but in the future, the main factor for it will
probably be increasingly machines and decresingly people.
Machine learning, artificial neural networks, biosynthetic hybrid elements

combining computers and living brain tissue are leading to more and more
scenarios for the future development of AI. This trend in technology
promises to blur the line between human and computer knowledge. Even
more dramatic changes are possible. Such as “mind fusion”, where thought
processes from one biological brain are transmitted to another, as this has
already been demonstrated in rats.8 These synthetic approaches to modified
human intelligence can be combined with others, such as “optogenetics”,
where light elicits reactions in genetically engineered neurons. Biotechnology,
including CRISPR (clustered regularly interspaced short palindromic repeats)
gene editing9, raises additional opportunities to change evolutionary
knowledge, arguing that it will be able to shape the ability to pay attention
or recall. The goal of the individual human mind to act in accordance with
external intervention is already obvious, and the way in which this can be
realized has not yet been specified. The technologies described above are
extremely suitable for creating a military force executing hierarchical
commands, but they ask ethical questions. They also challenge the essence
of what it means to be a human and even what it means to be an individual.
Applications and Perspectives in the Use of Artificial

Intelligence in Defense
There is a trend in the last few decades to incorporate more robotics and
autonomous systems into military forces world-wide. Artificial intelligence
and machine learning will allow these systems to tackle more challenging
8 Miguel Pais-Vieira et al., „A Brain-to-Brain Interface for Real-Time Sharing of

Sensorimotor Information,” Sci Rep 3, 1319 (February 2013).
9 Michael Le Page, “What is CRISPR? A Technology that Can Be Used to Edit
Genes,” New Scientist, November 12, 2020.

104 Chapter VI
tasks in a wider range of environments. Because of the comprehensive

nature of AI technology, non-state groups and individuals will also be able
to harness and use this technology. In combat operations, robots, swarms,
and autonomous systems have the potential to increase the pace of combat.
This is particularly the case for domains of machine-to machine interaction,
such as in cyberspace or the electromagnetic spectrum. AI could be used not
only to create more intelligent robotics, but also to power more advanced
sensors, communications, and other key enablers.
Following the development of AI in recent decades, it can be concluded that

its superiority over human intelligence in the field of a number of logic
games, such as chess, Asian Go and others, is obvious. Success in this type
of task, which is very similar to the structure of the battlefield, shows how
successful AI can be in such conditions. In games, the machine dominates
because people think much slower and make mistakes. Even the best of us
are not as consistent as intelligent machines. The collaboration between
human minds and AI can increase the potential of a combat and other
military systems to perform tasks more successfully.
AI continues to develop, presenting and increasing its applications in a

growing number of diverse defense systems. One of the key factors for
success in this direction will be the reassessment of the goals of the
algorithms for replacing human analysts and their activities. At this stage, it
is still obvious the necessity to have human intervention in the cycle in order
to realize the continuous evaluation and to guarantee the quality of the
output of the advanced models and algorithms. But according to experts, it
is not far away the moment when AI-led systems with some operational
autonomy will take part in hostilities. Probably maritime environment and
some other areas like space are particularly favorable for the initial
deployment of this type of system, as it identifies targets relatively easily
and without risk, as well as the relatively small presence of civilians.
AI is defined as a critical component of combat operations in modern theory

of combat operations. Compared to conventional systems, military
platforms and systems equipped with AI are capable to manage a
significantly larger volume of data, which leads to increased efficiency and
speed of decision-making. Research and development in this area are
growing in line with increased funding. This, in turn, leads to easier and
broader implementation of AI in the defense sector.
This type of intelligent machine is already in service in a number of high-

tech military services around the world. They and the resulting capabilities
can build and maintain combat space with software applications in different
types of operations, as well as build networks of interconnected military
platforms and systems.
The contribution of AI to military operations is present and will become

increasingly prevalent in all five military environments: land, air, sea, space
and cyberspace. In the medium term, the commands of military operations
and traditional defense systems are expected to be replaced by intelligent
systems that can be deployed online and updated through communication
links during the mission.
In analyzing global trends, available AI-based investments and programs

several main applications in the defense sector could be distinguished,
where this type of intelligent systems tends to prove its importance in the
medium term:
Battle Platforms
Armed forces from around the world are increasingly using AI in weapons
and other systems based on land, air, sea and space platforms. They perform
a variety of tasks: surveillance and reconnaissance, barrier patrol for defense
and maintenance of space/base, protection of forces, countering the mine
threat, anti-submarine warfare, reconnaissance, hydrographic surveys,
communication, information and navigation services, cargo delivery,
information operations, performing high-precision synchronized strikes and
others.
Remote underwater vehicles have been widely used in recent decades to

solve a wide range of tasks: surveillance and reconnaissance, mine search,
hull inspection, data collection on hydrometeorological conditions and
more. The tendency is for them to become more multifunctional and with
greater autonomy. For example, the US Department of Defense's Defense
106 Chapter VI
Research Agency (DARPA)10 is funding the development of a robotic

underwater system that is expected to be used to solve a wide range of tasks,
ranging from detecting underwater mines to conducting underwater
operations.11
Threat Monitoring and Situational Awareness

Situational awareness and combat space monitoring in operations depend to
a large extent on Intelligence, Surveillance and Reconnaissance (ISR)
systems. These systems are based on the principle of comparisons of
embedded algorithms and are used mainly for the acquisition and processing
of information for the maintenance of a number of combat activities and
decision-making. Unmanned platforms used for surveillance and
reconnaissance are increasingly based on AI. At this stage, they are
controlled remotely or sent on a predetermined route. Equipping these
systems with analytical machines assists defense personnel in monitoring
threats and improves situational awareness.
Unmanned aerial and land vehicles, surface platforms and submarines with
integrated AI can patrol large areas, identify potential threats and
automatically transmit information about these threats to the response
forces. The use of remote systems increases the security of the bases and
guarded objects, as well as the safety and efficiency of the personnel in a
combat situation or during a transition.
Small robotic sensors could be used to collect information, and AI-enabled

sensors and processing could help make better sense of that information.
Deep neural networks are already being used for image classification for
drone video feeds as part of the US Department of Defense’s Project Maven,
in order to help human operators to process the large volumes of data being
collected. While current AI methods lack the ability to translate this into an
understanding of the broader context, AI systems could be used to fuse data
from multiple intelligence sources and cue humans to items of interest. AI
10 DARPA is a government agency of the United States Department of Defense that
deals with the development of new technologies in the US military

11 “Selects Performers to Advance Unmanned Underwater Vehicle Project,” DARPA,
accessed February 05, 2021, https://www.darpa.mil/news-events/2021-02-05a.

systems can also be used to generate tailored spoofing attacks to counter

such sensors and processors.12
Data Processing
AI is particularly useful for quickly and efficiently processing large
databases in retrieving valuable information. Intelligent machines can assist
in the collection and analysis of information from different data sets, as well
as in the retrieval and processing of information from various sources. This
advanced analysis enables military personnel to recognize patterns and
derive correlations, significantly reducing the timing of individual stages.
For collection, the explosion of data that is occurring because of smart
devices, the Internet of Things, and human internet activity is a tremendous
source of potential information. This information would be impossible for
humans to manually process and understand, but AI tools can help analyze
connections between data, flag suspicious activity, spot trends, fuse
disparate elements of data, map networks, and predict future behavior.
Force Management
Starting with combat information and control systems, the defense programs
of a number of countries expand their scope in the planning and
implementing of AI in the command and control systems. These systems
are used by platforms on land, sea, air, space, as well as in the cyber domain.
The use of these types of systems leads to improved interaction between the
individual components and the functioning of the military system as a
whole. Simultaneously, they require significantly more limited maintenance
and minimal human intervention. AI will facilitate the management and
configuration of autonomous and high-speed platforms and weapons in the
implementation of joint attacks and automatic control. A typical example in
this regard is the use of swarm tactics, in which a large number of platforms
12 Cheryl Pellerin, “Project Maven to Deploy Computer Algorithms to War Zone by
Year’s End,” Defense Government, accessed July 21, 2017,

https://www.defense.gov/Explore/News/Article/Article/1254719/project-maven-to-
deploy-computer-algorithms-to-war-zone-by-years-end/.
108 Chapter VI
at relatively low cost, can attack an important and well-guarded enemy

target.
As the pace of battle accelerates and the volume and speed of information
eclipse the ability of human warfighters, AI will become increasingly
important for command and control. Autonomous systems that have been
delegated authority for certain actions can react at machine speed at the
battlefield’s edge without waiting for human approval. AI can also help
commanders process information faster, allowing them to better understand a
rapidly changing battlespace. Through automation, commanders can then
relay their orders to their forces – human or machine – faster and more
precisely. AI systems can also aid the military in a range of non-combat
support functions. One use of AI will be to help defense leaders better
understand their own forces. By analyzing large amounts of data, AI
systems may be able to predict stress on the force in various components:
when equipment requires maintenance; when programs are likely to face
cost overruns or schedule delays; and when service members are likely to
suffer degraded performance or physical or psychological injuries. Overall,
AI has tremendous potential to help defense leaders improve the readiness
of their own forces by assembling and fusing data and performing predictive
analysis so that problems can be addressed before they become critical. AI
is also is ripe for transforming traditional business processes within military
and other government organizations.13
Logistics
AI systems could play a crucial role in military logistics. The efficient
supply and transportation of goods, ammunition, weapons and personnel is
an essential factor for success in modern operations. The purpose of
providing what is needed, where and when you need it, can be accomplished
with the help of intelligent machines that synchronize the request, the
13 Singh Manvendra, “Is Predictive Analytics the Future? How AI Will Take It to
the Next Level”, accessed September 14, 2020,

https://levelup.gitconnected.com/is-predictive-analytics-the-future-how-ai-will-
take-it-to-the-next-level-4ee90a240f9b.
database and the chain provider with an automated design of systems for
loading and sending items.
Probabilistic modeling can reduce forecasting errors by 20 to 50 percent.

There are too many variables in a combat operation with which human
operators can predict real-time levels to some extent. AI can solve these
inaccuracies by making combat systems many times more effective.14
Targeting
Some AI systems are developed with the task of increasing the accuracy in
defining and prioritizing targets or target systems, as well as to determine
the adequate impact on them in a complex combat environment. They allow
the Defence Forces to gain an in-depth understanding of potential areas of
work by analyzing reports, documents, information channels and other
forms of unstructured information. In addition, the presence of AI in target
recognition systems improves the ability of these systems to identify the
position of their objects.
The capabilities of activated AI recognition systems include predictions of

enemy behavior based on probabilities, aggregation of weather and
environmental conditions, anticipation and signaling of potential supply line
difficulties or vulnerabilities, assessments of mission approaches, and
proposing actions to reduce the risk.
The machine learning of these types of systems is also used to detect, track
and investigate targets from the data received from the sensors. For
example, DARPA's Target Recognition and Adaptation in Contested
Environments (TRACE) program uses machine learning techniques to
14 Harald Bauer and Peter Breuer, “Smartening up with Artificial Intelligence (AI)
— What’s in It for Germany and Its Industrial Sector?”, accessed April 01, 2017,
https://www.mckinsey.com/~/media/mckinsey/industries/
semiconductors/our%20insights/smartening%20up%20with%20artificial%20intelli
gence/smartening-up-with-artificial-intelligence.ashx.
110 Chapter VI
automatically locate and identify targets and images using Synthetic-

Aperture Radar - SAR).15
Combat Simulations and Training

Simulation and training are a multidisciplinary field that connects systems
engineering, software engineering and computer science to build
computerized models to train personnel with the multilateral combat and
support systems used during various types of operations.
Investments in this type of application have increased significantly in recent

years. AI can actively participate in such systems to analyze procedures and
act in accordance with the identified adversary. In this way, the personnel
are trained in conditions close to combat without endangering their lives and
health. These types of systems allow you to easily redefine the tasks and
through the use of AI to build a more difficult combat environment, with
the ultimate goal to improve training without endangering people.
Evolutionary and reinforcement learning methods could be used to generate

new tactics in simulated environments, finding surprising solutions as they
have in other settings.
Electronic Warfare
AI systems could be used for electromagnetic spectrum dominance. They
have ability to generate novel methods of jamming and communications
through self-play, like AlphaGo Zero improving its game by playing itself.
For example, one AI system could try to send signals through a contested
electromagnetic environment while another system attempts to jam the
signal. Through these adversarial approaches, both systems could learn and
improve. In 2014 an US government agency DARPA held a Spectrum
Challenge with human players competing to send radio signals in a
contested environment. DARPA is now using machine learning to aid in
15 Ke Wang and Gong Zhang, “SAR Target Recognition via Meta-Learning and
Amortized Variational Inference”, accessed October 21, 2020,
https://www.mdpi.com/1424-8220/20/20/5966/pdf.
radio spectrum allocation, but this concept could also be applied to jamming
and creating jam-resistant signals.16
Decoys and Camouflages

Generative adversarial networks could be used to create militarily relevant
deep fakes for camouflage and decoys, and small robotic systems could be
used as expendable decoys. As the military incorporate more AI-enabled
sensors for data classification, spoofing attacks against such systems will be
increasingly relevant as well.
Cybersecurity
Defense systems are often vulnerable to cyberattacks, which can lead to loss
of classified information and damage to combat systems. This is why
significant efforts are being made to systems equipped with AI to
autonomously protect networks, computers, programs and data from any
kind of unauthorized access. In addition, supported AI systems for web
security can record the pattern of cyberattacks and develop counterattack
tools to deal with them.
The applications of AI in military operations are diverse and their number

will increase with the increase of investments in the field and the practical
imposition of this type of system in the defense sector.
There is an ongoing debate about the ethical aspect of the introduction of

intelligent systems in the armed struggle. The topic raises a variety of
questions, such as: What will happen if an adversary or terrorist organization
takes control of our remote platforms? Or if we do not produce these types
of systems ourselves, but buy them and the manufacturer has a hidden
ability to control them? We will find the answers to these questions in the
future, but at this stage the dynamic development of AI sector and the new
horizons in front of it are visible from a broad perspective.
16DARPA, “The Radio Frequency Spectrum + Machine Learning=A New Wave in

Radio Technology,” accessed August 11, 2017, https://www.darpa.mil/news-
events/2017-08-11a.
112 Chapter VI
Conclusion
In conclusion, the following key remarks can be made:
1. The applications of AI accelerate decision-making processes, minimize

errors, rapidly concentrate force and precision, could find their place in
more and more defense sectors, leading to the evolution of the overall nature
of future conflicts.
2. Human control in the military sphere will become increasingly difficult

given the speed with which intelligent machines optimize their functions.
3. There is concern about the possibility of a rapid change in the relative

balance of power, especially with regard to offensive scenarios. Combined
with the likelihood that the lethal outcome will be delegated entirely to AI
or that such technology falls into unscrupulous hands, it raises ethical
objections to the uncontrolled development of intelligent machines.
4. Maintaining the status quo fuels a race in AI armaments and increases the
pressure on leading nations.
Overall, artificial intelligence can help the military improve understanding,

predict behavior, develop novel solutions to problems, and execute tasks.
Some applications, such as the use of AI to enable autonomous weapons,
raise difficult legal, ethical, operational, and strategic questions. The
potential for automation to increase the pace of combat operations to the
point where humans have less control over the conduct of war raises
profound questions about humanity’s relationship with war, and even the
nature of war itself.
The importance of AI is due not so much to the technology itself as to the

processes and transformation of a number of aspects of defense. In this
context, the three main challenges arising from AI's entry are: how AI can
help achieve security and defense goals; how AI technologies can be
protected from attacks; how to ensure protection against malicious use. The
latter inevitably raises a number of serious issues of international law and
ethics. The importance of the informal discussions on emerging
technologies in the field of deadly autonomous weapons systems.
Expanding the role of AI can increase the benefits of the military system.
Not only does the integration of greater AI autonomy reduce casualty rates
of personnel, but such systems can adopt riskier tactics; target with greater
accuracy; and operate with greater endurance, range, and speed while
retaining a greater level of flexibility and mobility.
There is a need for regulation of the legal and ethical aspects that AI in the
defense sector will inevitably pose in the near future. In the process of
developing international humanitarian law, it is essential that states ensure
human control in decision making for the use of autonomous lethal weapons
systems.
Our new tools can make us smarter and can enable us to better understand
the military battlefield, our world and ourselves. Deep Blue didn’t
understand chess, or even know it was playing chess, but it played it very
well. We may not comprehend all the rules our machines invent, but we will
benefit from them nonetheless. Synergy between the human mind and AI
has tremendous potential to increase our defense capabilities and at this
moment, everything depends on us. The thought from the President of the
Future of Life Institute Max Tegmark, is exactly in that direction:
“Everything we love about civilization is a product of intelligence, so
amplifying our human intelligence with artificial intelligence has the
potential of helping civilization flourish like never before–as long as we
manage to keep the technology beneficial.”17
References
Barnaby, Frank. 1987. The Automated Battlefield: New Technology in
Modern Warfare. Oxford: Oxford University Press.
Bauer, Harald, and Peter Breuer. “Smartening up with Artificial Intelligence
(AI)—What’s in It for Germany and Its Industrial Sector?”. Accessed
April 01, 2017.
17 “Benefits and Risks of Artificial Intelligence”, Future of Life Institute, accessed

February 12, 2022, https://futureoflife.org/background/benefits-risks-of-artificial-
intelligence/.
114 Chapter VI
https://www.mckinsey.com/~/media/mckinsey/industries/semiconducto
rs/our%20insights/smartening%20up%20with%20artificial%20intellig
ence/smartening-up-with-artificial-intelligence.ashx.
DARPA. “The Radio Frequency Spectrum + Machine Learning=A New
Wave in Radio Technology”. Accessed August 11, 2017.
https://www.darpa.mil/news-events/2017-08-11a.
DARPA. “Selects Performers to Advance Unmanned Underwater Vehicle
Project”. Accessed February 05, 2021. https://www.darpa.mil/news-
events/2021-02-05a
Ford, Martin. 2019. Rise of the Robots: Technology and the Threat of a
Jobless Future. New York: Basic Books.
Future of Life Institute. “Benefits and Risks of Artificial Intelligence”.
Accessed February 12, 2022.
https://futureoflife.org/background/benefits-risks-of-artificial-
intelligence/.
Harper, Jon. “Federal AI Spending to Top $6 Billion.” National Defense
Magazine, October 2, 2021.
https://www.nationaldefensemagazine.org/articles/2021/2/10/federal-
ai-spending-to-top-$6-billion.
Johnson, Dominic, and Niall MacKay. “Fight the Power: Lanchester’s Laws
of Combat in Human Evolution.” Journal Evolution and Human
Behavior, Vol.36, No. 2 (2015): 152-163.
Le Page, Michael. “What is CRISPR? A Technology that Can Be Used to

Edit Genes.” New Scientist, November 12, 2020.
https://www.newscientist. com/term/what-is-crispr/.
Pais-Vieira, Miguel, Mikhail Lebedev, Carolina Kunicki, Jing Wang, and
Miguiel A. L. Nicolelis. „A Brain-to-Brain Interface for Real-Time
Sharing of Sensorimotor Information.” Sci Rep 3, 1319 (2013).
https://doi.org/10.1038/srep01319.
Payne, Kenneth. 2018. Strategy Evolution and War–From Apes to Artificial
Intelligence. Washington: Georgetown University Press.
Pellerin, Cheryl. “Project Maven to Deploy Computer Algorithms to War
Zone by Year’s End”. Accessed July 21, 2017.
https://www.defense.gov/Explore/News/Article/Article/1254719/proje
ct-maven-to-deploy-computer-algorithms-to-war-zone-by-years-end/.
Popescu, Sebastian-Gabriel.”Artificial Intelligence in Naval Operations.”

Paper presented at the International Scientific Conference Strategies
XXI: Technologies—Military Applications, Simulations and Resources,
NDU „Carol I”, Bucharest, 2019.
https://docplayer.net/145260428-National-defense-university-
command-and-staff-college-program-technologies-military-
applications-simulation-and-resources.html.
Singh, Manvendra. “Is Predictive Analytics the Future? How AI Will Take
It to the Next Level”. Accessed September 14, 2020.
https://levelup.gitconnected.com/is-predictive-analytics-the-future-
how-ai-will-take-it-to-the-next-level-4ee90a240f9b
Wang, Ke, and Gong Zhang. “SAR Target Recognition via Meta-Learning
and Amortized Variational Inference”. Accessed October 21, 2020.
https://www.mdpi.com/1424-8220/20/20/5966/pdf.
CHAPTER VII
PROCESS MINING WITH MACHINE LEARNING
NIKOLA SOTIROV
Introduction
The problem of predicting processes has been arising in frequency in recent
times due to many companies attempting to digitally transform and describe
the process traces within their systems optimally. The purpose of this project
is to tackle the problem by creation and comparison of different machine
learning techniques.1 As event logging has been part of the majority of the
industry sectors for many years, enough data can be collected and used to
create and test mathematical models, which will be trained to perform
pattern recognition, event and time of event predictors.2
Methodology
The project was developed and implemented by a team of five researchers.
The workflow of the project was segregated into the following subdivisions:
1. Exploratory data analysis on the dataset and generation of

visualizations of the general process flow in the context of the system
2. Event and time prediction strategy layout
3. Feature extraction for each model from the dataset
1 “Your First Steps into the World of Process Mining”, Software AG, accessed May
25, 2021,
https://www.softwareag.com/en_corporate/resources/asset/ebook/business-process-
transformation/first-steps-processmining.html?utm_source=bing&utm_medium=
cpc&utm_campaign=bt_aris_process-mining&utm_region=hq&utm_subcampaign
=stg1&utm_content=ebook_your-first-steps-into-process-mining_
stage1&msclkid=1a8e9abfd556137ca255f8c70261ec49.
2 Software AG, “Your First Steps into the World of Process Mining”.
Process Mining with Machine Learning 117
4. Training and testing the models

5. Plotting the results of each model and comparing them
The dataset used in the project is the process traces collection from 2012 in
the BPI challenge. The columns extracted and used from the dataset are the
following: case (a number, which indicates which trace the event belongs
to), event (the name of the process), startTime (the timestamp on which the
process occurred), completeTime (the timestamp of completion of the
event).3
lifecycle: Describes the event state transition, based on that

transition point of time in its lifecycle.
concept: The name of the event.
name
time: The time at which the event occurred.
timestamp
Case: The amount of money that was requested at the
AMOUNT particular event.
REQ The amount of money that was requested at the
particular event.
Figure 1: Data dictionary for the dataset BPI Challenge 2012
The dataset has the following columns in it: lifecycle: transition, concept:
name, time: timestamp, case: REG DATE, case: concept: name, case:
AMOUNT REQ. Each step will be explained in detail in this section. The
emphasis of the exploratory analysis was to visualize and understand the
overall process flow of the dataset and how much a process matters (via
looking at the transitions from process to process).
3“Nikola Sotirov. Github Project Repository”, accessed June 24, 2021,

https://github.com/ NickSot/process%5C_mining.
118 Chapter VII
Preliminary Analysis and Visualization of the Data Set
Figure 2: States and transitions 4
Figure 2 denotes the types of processes that occur in the system, and the
respective transitions from and to each other.
The more highlighted the transitions are in Figure 2, the larger their numbers
of occurrences are. The goal is to create and use algorithms that learn the
patterns of transitions between the processes and forecast where the next
transition will lead towards and at what timestamp that particular transition
will occur in the future with reasonably small errors.
In Figure 3 it is showcased what the frequency of each event type present in

the dataset is. This gives a notion of how skewed the dataset is towards
certain data points. In this case, the dataset is imbalanced, since it contains
some events much more than others. Fortunately, this does not introduce
problems in terms of the prediction, since the event types are static, and
follow the same flow patterns.
4 “Nikola Sotirov. Github Project Repository”.

Figure 3: Event occurrence frequencies 5
Event and Time Prediction Models

The project was segregated into two main parts-implementation of
predictors for the next event/process prediction, and prediction of the time
until the next process occurs.
As to the event prediction, two models were used and evaluated a random
forest classifier, and a neural network classifier. For each classifier, the
confusion matrices were plotted and compared. Accuracy has been the used
as the primary metric for both models. See section “Results”.
The time until the next event prediction was implemented by using a neural
network with a singular regression output neuron. R2 score and relative
absolute errors were used to evaluate the performance. See section
“Results”.
5 “Nikola Sotirov. Github Project Repository”

120 Chapter VII
Random Forest Classifier

A random forest is a set of decision trees, where all trees are run on random
samples of the data. In the end, the best viable solution is selected by voting.6
Figure 4: Random forest example 7
In this project, such a model is utilized to classify what the next event is
based on the previous event as input. Figure 4 showcases an example
random forest and its generalized workflow.8
A Neural Network for Next Process Prediction

A neural network is a type of data structure that consists of layers of nodes and
connections between those layers, where every node from one layer is connected
to all nodes in the next layer. The connections between the nodes/neurons are
altered mathematically with the purpose of reaching the result values in the
output nodes/neurons with minimized error with respect to an expected value.9
6 “Workflow of the Random Forest Classifier”, accessed May 18, 2022,
https://www.researchgate.net/figure/Workflow-random-forest-classifier_
fig3_343627992.
7 “Workflow of the Random Forest Classifier”
8
Workflow of the Random Forest Classifier”.
9 “Artificial Neural Networks”, accessed June 4, 2022, https://iq.opengenus.org/
artificial-neural-networks/.
Figure 5 represents a neural network with four hidden layers, one input and
one output layer.
Figure 5: Neural network example 10
A neural network was used to predict the next event, given an arbitrary state
of the system. The network has four layers, each with the following
sequence of neuron numbers in each layer: (128, 64, 48, 24) and respective
activation functions: (’relu’, ’relu’, ’relu’, ’softmax’). The optimizer for
adjusting the learning rate used is ADAM (Adaptive Moment Estimation).
The loss function used is Binary Cross-entropy.
A Neural Network for Time until Next Event Prediction

For the time until the next event prediction, a neural network with a
continuous output, which indicates the time until the next event occurs in
seconds, was used. This model contains four hidden layers with the
following sizes and activation functions: (128, 48, 48), (’relu’, ’relu’,
’relu’). The output layer contains one neuron in it with linear activation
since the value has to be continuous (a timestamp, represented as a float).
The loss function is the mean squared error of the expected and the actual
output of the network.
10 “Artificial Neural Networks”.

122 Chapter VII
Data Set Preprocessing and Feature Extraction

for the Models
Each model requires a set of attributes from the dataset in order to predict
the time of the next event or the next event itself. In this section it will be
described how the input data for each model is prepared.
Preprocessing for the Random Forest Model

An essential aspect to take into consideration when inputting categorical
independent variables in any model is to convert those inputs to one-hot
encoded vectors. This ensures that the model does not get confused as to the
statistical nature of the input. For example, the mean between Saturday and
Monday is not Thursday, which would have been the case if Monday was
labeled as ’1’, and Saturday as ’6’, assuming that the result is rounded up.
Preprocessing for the Neural Network Classifier

of the Next Event
The preprocessing required feature engineering in order to encapsulate the
correlations between the features and the dependent variable - the next event
type.
The following features were used as inputs for the neural network: concept:
name, previous event, lifecycle: transition, pp event, ppp event, p lifecycle:
transition, where previous event, pp event, ppp event, p lifecycle: transition
are the engineered features from the dataset.
previous event The event that occurred before the current event.
pp event The event type of the process that occurred before the
previous event.
ppp event The event type of the process that occurred before the
pp event.
p lifecycle: The state in which the previous event was at that
transition point of time in its life cycle.
Figure 6: Engineered feature dictionary for the neural network classifier

The input data are the data frame of those columns, converted to a matrix
(or tensor), which is then input to the model. The output layer [See Section
“Neural network for next process prediction”] contains 24 neurons, and the
activation is softmax, meaning that each neuron has a value between 0 and
1. The neuron where the value is the largest indicates which position in the
one-hot encoding of the event type contains a ’1’. Therefore, this is chosen
as the predicted event type of the next process.
Preprocessing for the Neural Network Regressor for the Time

until the Next Event
Previous-event The event that occurred before the current event.
pp event The event type of the process that occurred before

the previous event.
weekday The day of the week in which the process executes
p lifecycle: The state in which the previous event was at that

transition point of time in its life cycle.
Figure 7: Engineered input features for the
The data used as inputs for the time of event prediction model consist of the
following columns in the data set: concept: name, previous event, lifecycle:
transition, pp event, p lifecycle: transition, weekday. The subset of these
features that contains only the engineered ones is previous event, pp event,
p lifecycle: transition, weekday. Analogously, as in the case of the model in
Section “Preprocessing for the neural network classifier of the next event”,
the input data frame is converted to a matrix tensor that is then processed by
the network. The last layer’s neuron contains the timestamp value of the
result. That is then compared to the expected result, and the mean squared
error is calculated in order to minimize the error.11
11 See the Section “Results”.

124 Chapter VII
Training and Testing of the Models

The data had to be segregated into training and testing data, where 70
percent of it is training the models, and 30 percent is for testing the models’
performances. Since the data is temporal, the following approach was used
in order to divide the dataset without using future data to predict the past.
The dataset is ordered by the first event’s timestamp in every trace in an
ascending manner. Then a timestamp threshold is chosen to divide the
dataset into 70 and 30 percent for training and testing data, respectively.
Once it is split, separate .csv files are generated for each model, since all
models have their own input features. Each file contains the base features
of the dataset and the engineered ones.
Random Forest
The random forest is trained by preprocessing the data from the default data
set and then passing it to the model. Since the model cannot see the meaning
behind a string enumeration value, the training becomes much more
difficult, and inefficient. The training occurs in the following steps:
1. Separate the event columns into separate binary columns, where the
name of the column is the name of the concrete event, and the value
is whether the event is that or not by placing ’1’ and ’0’ respectively.
2. Drop the rows where the values are null.
3. Divide the set of dependent and independent variables - next event
and event, lifecycle: transition, respectively.
4. Train the model with the data.
The training step is implemented with the default method fit in the sklearn
library. The fitting is done by the bagging method, which means that the
dataset is fed to the forest in a random manner each time.
Neural Network Classifier

Training the neural network classifier was implemented by preprocessing
the data, and passing it to the model in five epochs, with a batch size of 512.
The split between training and testing data sets was 90 to 10 percent,
respectively. The reason for the test data reduction is to increase the amount
of training samples for the model, since the accuracy, while using only 70
percent as training data, was insufficient. The process of training and testing
the model was the following:
1. Subdivide the event columns into separate binary columns, where

the name of the column is the name of the concrete event, and the
value is whether the event is that or not by placing ’1’ and ’0’,
respectively.
2. Fill the values of the places where null values are present.
3. Train the model with the training data.
4. Test the model with the testing data.
The training procedure consists of the following:
• Forward propagation of errors
Inputting the training data as batches of tensors to the input layer, and
producing the results in the last layer, where each neuron contains the results
of the previous hidden layers, passed through a softmax activation. This
means that every neuron in the output layer contains a probability of being
active relatively to the others. So the highest probability neuron becomes
’1’, and the rest - ’0’. This layer of neurons linearized in to a 1-dimensional
vector, corresponds to the one-hot encoding of the event type that is
predicted. The binary cross-entropy function is used to calculate the
deviation from the actual result, produced by the network (aka the predicted
event) and the expected predicted event.
• Backward propagation of error
Knowing the error from the forward propagation phase, the neural network
can adjust the weights (the connections between the layers), such that the
error is minimized. The learning rate is variable as this model utilizes the
126 Chapter VII
ADAM algorithm for momentum gain of the learning rate. As soon as the
values of the weights are recalculated, the training procedure repeats until a
reasonably small error is reached.
Neural Network Regressor

The deep learning regression model was trained and tested in a similar
fashion to the classifier. The differences are primarily in the loss function
and the activation function.
In the forward propagation, the final layer contains one neuron with a linear
activation, hence making it appropriate to use a MSE (Mean Squared Error)
activation function.
Then the error is backwards propagated in the same manner, the only
difference being the adjustment of weights, since the learning rate
adaptation is now not achieved using the ADAM algorithm, but FTRL as it
reduces the loss much faster. In this case, FTRL does not introduce issues
regarding stability since the model is relatively shallow, and each layer
contains a relatively large enough number of neurons.
Results
This section explains the results of each model’s performance and opens a
discussion as to which model behaves best by comparing the metrics.
Random Forest
To describe the results of the random forest, a confusion matrix was
generated and plotted to describe the different metrics used to validate the
classifier. From Figure 8, one can conclude that for some of the events the
random forest generates a relatively high number of true positives, however,
for some the true positives are low in comparison to the false negatives and
the false positives.
Figure 8: Confusion matrix for the random forest’s predictions 12
The calculated accuracy for the random forest was 77.6%.
The method for deriving results for the neural network classifier was
identical to the random forest - a confusion matrix was generated. From
Figure 9, it can be seen that there is a much higher percentage of true
positives in the neural network predictions than in that of the random forest.

128 Chapter VII
Neural Network Classifier
Figure 9: Confusion matrix for the neural network classifier’s predictions13
An interesting observation to note is that the matrix is quite sparse, meaning

that the model is either predicting with high precision, but in some cases, it
is thoroughly convinced that the predicted event type is correct, although
the expected result is different. The calculated accuracy of the neural
network next event type classifier is 83%. That is a relatively high
percentage, considering the shallow structure of the neural network - it
contains only four hidden layers. However, the high accuracy could be
explained by the fact that the dataset contains patterns that are easily
predictable.

The neural network predictor of the next event type is better in comparison
to the random forest, as seen from 9 and by having compared the accuracy
of both models.
Neural Network Regressor
Figure 10: Timedelta prediction occurrences against real timedelta

occurrences.14
As the predictions of the regressor are not binary, but rather continuous, one
cannot generate a confusion matrix to visualize the accuracy and precision
of the model. Another method was used to visualize the model performance.
From Figure 10, one can infer that the predictions somewhat align with the
real timedeltas with some deviations, especially in the 0.5 to 2 seconds
prediction interval. That is due to the fact that the model is biased. However,
the predictions stabilize at the 3 to 5-second interval. The plot was done by
implementing a logarithmic scale in order to fit all the bars in one plot. The
calculated R2 score of the neural network timedelta predictor was 0.64. This
14 “Nikola Sotirov. Github Project Repository”

130 Chapter VII
means that the model is relatively accurate (predicting correctly in more than
60 percent of the cases), but definitely not as precise as the event type predictor.
Conclusion
In conclusion, the results indicate that for classifying the next event type,
the neural network prevails in performance, and in the context of prediction
of the time until the next event occurrence, a reasonable R2 score was
reached, even though the deep learning model was shallow.
Discussion
Although reasonable results were achieved, the models could further be
improved to fit the data better with increased accuracy and precision. For
instance, the deep learning model that classifies the next event type can be
improved by adding at least one LSTM layer that would capture the
temporal aspect of the data. As to the timedelta predictor, more layers could
be added with various activations. The number of epochs for both models
could be increased, which could mean improvements in the results.
References
“Artificial Neural Networks”. Accessed June 4, 2022.
https://iq.opengenus.org/ artificial-neural-networks/.
“Nikola Sotirov. Github Project Repository”. Accessed June 24, 2021.
https://github.com/ NickSot/process%5C_mining.
Software AG. “Your First Steps into the World of Process Mining”.
Accessed May 25, 2021. https://www.softwareag.com/en_corporate/
resources/asset/ebook/business-process-transformation/first-steps
processmining.html?utm_source=bing&utm_medium=cpc&utm_camp
aign=bt_aris_process-mining&utm_region=hq&utm_subcampaign=stg
1&utm_content=ebook_your-first-steps-into-process-mining_stage1
&msclkid=1a8e9abfd556137ca255f8c70261ec49.
“Workflow of the Random Forest Classifier”. Accessed May 18, 2022.
https://www.researchgate.net/figure/Workflow-random-forest-
classifier_ fig3_343627992.
CHAPTER VIII
APPROACHING THE ADVANCED

ARTIFICIAL INTELLIGENCE
ALEXANDER LAZAROV
Introduction
A philosophical discussion of how to behave when communicating with
autonomous, Advanced Artificial Intelligence (AAI) objects requires
consideration of the following questions:
What do intelligence, understanding, and autonomy mean?

What is the difference between data and information?
What is the philosophical essence of any intelligent procedure?
What is real and what is virtual in a data exchange process?
What are the differences between AI and robots?
What is AAI, and where does it fall within the framework of narrow
(specialized) AI and general artificial intelligence (AGI)?
Which issues should be considered in cases of human to AAI and AAI
to AAI communication?
Answers to the above questions will offer a discourse for both theoretical
analysis and future AI development. This paper will conclude that the
progress of AI’s analytic and synthetic capabilities—in some perspectives,
already close to human mental faculties—involve applied philosophy with
a never previously experienced framework and range. Therefore, in
designing the next generation AI—one intended to correspond to social
expectations—thinkers of all philosophical traditions can no longer draw a
strict borderline separating their wisdom from what science and engineering
have already demonstrated.
132 Chapter VIII
Intelligence, Understanding, Autonomy

Both psychology and philosophy have offered many descriptions of
intelligence. Most of these have met severe criticism, and a consensus has
never been reached. Thus, in addition to the constructive challenges, AI
designers also answer fundamental philosophical queries as they go about
their work. Perhaps, and amazing to many philosophers, some AI inventors
were very successful in managing this situation. Among these narratives on
intelligence, I see David Hanson’s concept as best serving AI development
and one that makes possible a bridge between humans and machines in a
philosophical discourse.1 Hanson claims that intelligence is a set of the
following:
An autonomous capacity to produce predictions about future events.

An openness to the external world that allows detection and exploration
of what is occurring.
Developing and conducting an appropriate reaction to oncoming
changes of state.
According to Hanson, understanding of events and processes emerges

whenever the intelligent bodies’ autonomously generated forecasts happen
indeed. Evidently, successful predictions numbers and complexity can serve
as criteria to assess and compare the characteristics of both biological and
automata intelligent performance. However, this analysis requires an
awareness of what autonomy means. As Luis Moniz Pereira outlines,
although there are many arguable descriptions of autonomy (as there are
with intelligence), today most experts would claim that autonomy simply
means the opposite of externally driven.2 Based on these notions,
intelligence is a capacity that belongs to living organisms, but also to some
human-made equipment, which by itself manages data identification,
collection, and processing that results in an independent informational
1 David Hanson, “Expanding the Design Domain of Humanoid Robot” (paper

presented at the ICCS Cognitive Science Conference, Special Session on Android
Science, Vancouver, USA, 2016).
2
Luis Moniz Pareira, “AI & Machine Ethics and Its Role in Society” (keynote
speech at the International Conference of Artificial Intelligence and Information,
Porto, Portugal, Dec. 5–7 2017).
Approaching the Advanced Artificial Intelligence 133
production. Comparing Hanson’s concept of AI to older ones illustrates that

the research path has shifted over several decades. In 1955, Allan Turing
foresaw AI’s emergence and suggested his famous Turing Test, thus,
offering a first attempt at describing the workspace of AI exploration,3
Later, John McCarthy, et al. suggested what became the classic definition
of AI.4 Positioning Mc Carthy’s claim next to Hanson’s illustrates that the
current conceptualization offers a more pragmatic perspective. This is
because the 1955 statement insisted that the core of the AI problem was to
make machines behave in ways that would be called intelligent if humans
acted in the same manner in similar circumstances, while the 2016 narrative
posits philosophical guidelines which can be applied in practice.
Alexander Wissner-Gross enhances Hanson’s view by stating that when

dealing with probability (the prediction of various scenarios), intelligent
bodies act to maximize the number of possible options that might occur in
the future.5 He holds that intelligence involves keeping the freedom of
action and producing choices of opportunities. Stuart J. Russell goes further
underscoring that contemporary AI is already successful in dealing with
predictions that it judges as uncertain at the time of their generation.6 Floridi
and Sanders also highlight that AI is a growing source of interactive,
autonomous, and self-learning actions that are informed, smart, autonomous,
and are able to perform morally relevant actions independently.7 According
to them, this is the fundamental ethical challenge posed by AI.
In my opinion, a crucial factor to add to these analyses is that the highest

intelligent performance is expressed not only in the passive estimation of
what is about to happen and to react appropriately, but it also includes a
3 Alan Turing, “Computing Machinery and Intelligence,” Mind: A Quarterly Review

of Psychology and Philosophy (October 1950): 433–460.
4 John McCarthy et al., “A Proposal for the Dartmouth Summer Research Project on
Artificial Intelligence, August 31, 1955,” AI Magazine 27, No. 4 (Winter 2006): 12.
5
Alex Wissner-Gross, “A New Equation for Intelligence.” TEDx-Beacon Street
Talk. November 2013, accessed July 3, 2022,
https://www.ted.com/talks/alex_wissner_gross_a_new_equation_for_intelligence.
6 Stewart J. Russell and Peter Norvig, Artificial Intelligence: A Modern Approach,
4th edition (Hoboken, N.J.: Pearson, 2021).

7 Luciano Floridi and J. W. Sanders, “On the Morality of Artificial Agents,” Minds
and Machines 14, No.3 (August 2004): 349–379.

134 Chapter VIII
focused activity intended to influence the future in a pre-targeted

perspective. All of us plan for tomorrow, next month, next year. We also
seek objectives like affecting global climate change, prolonging human life,
designing AI and our co-existence with it, etc. Therefore, if we have created
and implemented high-quality AI systems, we must assume that they will
perform a predictive function together with their manipulation of processes
that will impact their future too. In case of their success in this, a great
challenge immediately arises as to how our autonomously modeled future
and AI’s will relate to a common physical and social environment. This new
factor will strongly affect our traditional practices due to the encounter of
our interests with those of the new human-made intelligent bodies.
Moreover, we shall have to assess all ongoing events while considering
what the innovative automation is about to demonstrate. AI will do the same
regarding our actions.
The Difference between Data and Information

Both data and information have a binary nature that humans and digital
machines apply when examining the universe. The binary system is a clear
and logical mathematical method that has a philosophical background, just
as a mathematic equation considered from a philosophical standpoint is a
matter of bringing diverse concepts together.8
Contemporary philosophy claims that, unlike an inanimate object, a living

organism is capable of perceiving and investigating changes of state in
diverse ways. The higher an organism’s level of nervous system
development, the greater the variety of action assessments and option
choices it can conduct. However, without exception, the exploration of what
is external to the intelligent body results in an inclusive context that defines
the entity’s status in a single from a pair of polar contrast states—front or
back, big or small, static or moving, dangerous or not. Thus, all of us employ
8 Sylvain Duranton, “How Humans and AI Can Work Together to Create Better
Businesses,” accessed May 7, 2020,

https://www.ted.com/talks/sylvain_duranton_how_humans_and_ai_can_work_tog
ether_to_create_better_businesses?language=en.
binary opposites to sustain a picture and figure out its meaning no matter if
considering physical entities or abstract social issues like good and evil.
A peculiarity of our binary approach to the universe is that all of us find it

difficult to define a meaning unless we recognize the possible opposite. If
we cannot do that, confusion occurs and formulating a comprehensive
explanation for such problems often lasts for centuries without a conclusion.
For example, one can study the discussions of what happiness is, but since
the description of being happy cannot be considered as exactly the opposite
of unhappy, the matter remains unsettled. It is similar to harmony and
disharmony, etc. In other words, we simultaneously negate that what things
are cannot define what they are indeed. Perhaps, this mode of our mental
activity is predefined by, and relevant to our male/female nature. Some
thinkers hold that the physical and social reality is something quite different
from the representations that humans produce by applying the binary
approach. However, this issue falls beyond this text’s focus.
The binary system’s inventor is Leibniz, and a few centuries ago he

discussed it in The Monadology that experts highlight as a historic root of
all recent philosophical-informational analyses.9 For Leibniz, Monads are
the elementary building units of the universe and can only be found through
pure logic. He aimed to prove that all knowledge is composed of these
logical atoms or building blocks, and his claim has inspired many
philosophers. The Leibnizian school has influenced digital equipment
development as it stated that a mathematical system of symbolic logic
allows one to better explore the laws of human thought and reasoning
compared to spoken and written language. Apparently, this view has already
been confirmed, at least for writing and storing human ideas, because any
text editing computer program is doing what Leibniz proposed—it turns
thoughts into digital, mathematical, codes. Moreover, the newest data and
informational research imply the following consistent, logical concepts:
9 Gottfried Leibniz, The Monadology. Trans. Robert Latta (Oxford: Clarendon Press,
1898).
136 Chapter VIII
Whenever humans and AI operating bodies interact with the

environment, with one another or among themselves, a change of state
occurs.
Each distinguishable state is a bit of data. The binary symbols “1” and
“0” have just two states. They encode either a positive or negative
answer to any assessment: “1” signifies “yes” while “0” signifies “no”
in reply to a specific question.
Each distinguishable state is a bit of data. The binary symbols “1” and
“0” have just two states. They encode either a positive or negative
answer to any assessment: “1” signifies “yes” while “0” signifies “no”
in reply to a specific question.
The binary system is useful because it is the simplest data encoding

method as a binary “1” can represent any entity, while “0” indicates its
absence.
The number of distinguishable states that a system identifies and

memorizes is the amount of data that it can encode.
Binary system results are easy to conserve in memory for indefinitely

long periods. Moreover, any code can be easily transferred from one
place to another, carried by signals, generated in energetic circuits where
each pulse serves as “1” and its absence as “0.”
Significantly, a data bit is an elementary particle that can have only one of
two possible values and thus, it is one which can be used in digital
processing. I am not claiming that the same process happens in the human
brain. However, I do not exclude such an option. There is a similarity
because humans, and animals with higher nervous systems, and AI, have
the capability to memorize large quantities of data, as well as to generate
numerous links and references among these data. The higher the intelligent
body’s memory and data processing capability, the greater the number of
produced data links and references.
The philosophical essence of the digital bit is obvious: it allows the

intelligent actor external investigation and the translation of results into
digital codes, which can undergo computations. As Floridi assumes, data

are durable, reusable, quickly transportable, easily duplicable, and
simultaneously shareable without end.10 A few years ago, the theory of
Quantum Computing introduced the qubit, which made possible a
compressed “yes+no” processing up to the moment when a necessity arises
to provide decompression and have either the particular “yes” or “no”
involved in an issue’s conclusion. The qubit technology brought a
tremendous increase in processing speeds, described as thousands of times
faster than classical computation. However, it did not change the bits’
essence in a philosophical discourse, which highlights that:
There must be an intelligent body involved to formulate and ask a

question, and then to detect or assess whether the answer is a positive or
a negative one.
Material-energetic objects or processes, as well as any form of social

communication, are data carriers.
In other words, each intelligent body, no matter of its biological or

mechanical origin, can explore and synchronously synthesize new data.
Most often, data, and information do not refer to a single bit, but to
digital bit configurations (patterns) that emerge due to simultaneous
asking and answering large-scale sets of various questions. Thus, the
highest intelligence approaches difficult to understand and complex
issues.
All intelligent bodies can extract the data from any of its carriers, and
further deal with it by modeling.
Regarding humans, this activity relates to most mental faculties, but is

most brightly expressed in perception and imagination, while machines
perform computer modeling.
10 Luciano Floridi, “What the Near Future of Artificial Intelligence Could Be,”
Philosophy & Technology 32 (March 2019): 19,
https://link.springer.com/article/10.1007/s13347-019-00345-y.
138 Chapter VIII
Single digital bits as well as sets of bits move from sensors to brains and
artificial processing units and undergo grouping, configuration, and
structuring, which results in the generation of new digital constructions.
These results are conserved in memory in a hierarchical architecture.
Unlike the polar “yes” or “no” of a single data bit, a targeted set of
questions can lead to an unlimited variety of digital patterns. They
signify much more complex answers that can shift the pure positive or
negative statements to deep multi-layered conclusions. To illustrate,
applying digital bit calculation any computer-drawing program can
suggest a color palette that is richer in nuances than that which the
human eye can discern.
Importantly, the higher the level of intelligence, the more frequent the
inquiries and recognition of the environment via pattern applications. The
intelligent actor’s capacity, and experience to examine and learn predefine
the range and deepness of what he or it could achieve via an investigative
analytic approach. In other words, as Michelangelo has claimed, every block
of stone has a statue inside it, and it is the task of the sculptor to discover it.
At a later stage, the conserved bits and patterns can be recalled from memory
whenever needed and used for new analytic or synthetic processing. This in
turn leads to partial or total restructuring, reorganization, and rearrangement
as well as placing them in a set with new incoming data flows.
Simultaneously, depending on its memory and processing capacity, an
intelligent system may record a great number of both old and new digital
constructions (codes) combined with various links and co-relationships.
Importantly, these codes present a large variety of facts and dynamic events,
but also synthesize data such as conclusions, assessments, judgments, and
others that may affect future activities.
Without exception and based on the processing of what is available in its

memory that corresponds to the new incoming data flow, every intelligent
body (whether biological or human-made) can provide targeting and goal-
orientation in reaction to the observed environment. Such bodies can also
autonomously plan their activities within the environment. This ability
illustrates the difference between the philosophical essence of data and that
of information. To summarize, information is an intentional goal-oriented

data bit pattern that belongs to its intelligent creator. That creator can share
it by publishing it (making it accessible to other intelligent bodies) or
memorize it and store it internally. Significantly, if published, information
flows to other intelligent bodies) as data, and it is necessary to underscore
that there is a nonstop data-information-data pulsation, which is essential
and typical for all intelligent bodies’ communication. Expert analyses in this
field recognize this, but frequently, non-experts apply “information” to
characterize events that are only “data.”
The Essence of Intelligent Procedures

No matter what type of intelligent actor (human, animal, or machine) the
intelligent procedure in play is the same and follows four-steps:
Step 1 involves external data identification, acquisition from the carrier,

transfer to a processing unit, that encodes it, assesses it, and references it.
All this activity results in the production of data patterns and placing them
in memory as digitally recorded frameworks linked or related to others.
Importantly, these new data constructions belong only to their creator.
When discussing biological intelligent bodies, this intelligent procedure
step is termed perception and presentation building, while for machines,
this is data input. In either case, this phase results in creating a digital twin
of what has been explored.
Step 2 is the analysis of data records, followed by data synthesis—decision

making on how to act accordingly to circumstances recognized in advance.
Briefly, this is the stage of generating intentions and goal orientations. In
other words, this is the phase of creativity when entirely new and purely
private data patterns are born to drive subsequent intelligent actions.
Step 3 is the information production phase. This is the process of structuring

an approximate or precise plan of consecutive moves regarding how to reach
the target goals. First, this intelligent act requires stored data recall from
memory and an assessment of whether it is of sufficient quantity and quality
to serve the wanted informational production. Then, most often, a targeted
additional foreign data search is conducted to fill in any data gaps that were
140 Chapter VIII
found. Once the intelligent body recognizes that the new data acquisition
from the physical and social environment is enough and appropriate, the
new data search stops. Then, the data bits rearrangement, reconfiguration,
and reconstruction occur combined with re-judgement and re-assessment of
the new product’s applicative potential. The generative process ends when
a representation/model is built on what is expected to happen in case the
plan is put to practice. Thus, the in-form is created.
To illustrate steps one to three: most people know that Frankfurt, Paris, and
London are among the largest civil aviation centers in Europe. This is data
they have probably captured in an act of non-focused memorization.
Imagine that, at a moment, someone intends to fly from Sydney to Sofia and
he discovers that there are no direct flights between those cities. Then, he
recalls which are the biggest European airports and decides to check for
connecting flights. This is a targeted additional data search to fill in an
evident gap. Once a connection is found, the waiting time between flights is
assessed as non-risky short or not inconveniently long, and the ticket price
is judged as acceptable, the plan is recognized as ready to apply. The in-
form becomes available, and significantly, it belongs only to its author. AI
follows a similar path. Once the in-form is produced, the intelligent
procedure has two options:
First, the intelligent body decides to keep the in-form in privacy. The
human or the machine puts the plan in memory and never presents it to
others. If this option is chosen, the intelligent procedure ends here.
However, this happens more rarely than the second option – step 4.
Step 4 relates to making the in-form accessible to other intelligent bodies

by either performing it, or by conducting transformations.
The In-form, Per-form, Trans-form Triad

Making the in-form accessible to other intelligent bodies can occur by either
per-form or trans-form. Per-form means providing communication that
involves just information to data exchange. The in-form creator shares
ideas, concepts, assessments, opinions, impressions, desires, feelings,
attitudes, plans, as data patterns that undergo data collection by others. This
may happen through speech, writing, gesture, drawing, etc. The per-form
mode may be either direct or encrypted, and data identification and
acquisition may occur immediately or in the future.
At first glance, it seems illogical to think about the in-form reaching others
in advance of its author willingness to present it. Significantly, there is a
tendency to develop AI to read human’s thoughts, even when one has no
intention to publish them. In a philosophical discourse, this is a critical
aspect of the human-to-AI interface that may change human subjectivity.
However, I want to underscore two features that can appear and characterize
any in-form performance. Demonstrating the in-form by performing it never
brings considerable direct changes of state within the physical and social
environment. Those may only happen later if the intelligent procedure has
provoked some activity. Simultaneously, without exception, the per-form
requires another intelligent body’s involvement. Any type of in-form
sharing between two or more intelligent actors is what contemporary
philosophy accepts as virtual communication.
Trans-form, unlike the in-form performance, denotes actions focused on

causing a change of state within the external environment driven by the
intelligent body targets. Transforming is a specific expression of his or its
informational product. In such cases, another intelligent body in play is
possible, but not obligatory. In other words, conducting transformations
may serve communication but they are not mandatorily tied to it. For
example, I am at home alone and I decide to have a cup of coffee. I make
one and drink it. All that action was the result of my informational
production that focused my trans-forms to reach the goal. Someone else
might have watched me and read my in-form contents or not.
Importantly, in any event, providing transformations inevitably brings

changes of state to the environment and they can be examined by both the
information creator as well as by other intelligent counterparts. The totality
of all evident changes of state are what contemporary philosophy considers
as real. However, sharing abstract informational products like assessments,
judgments, desires, impressions, attitudes, feelings, etc. is mainly possible
through virtual performance.
142 Chapter VIII
Research on the in-form, per-form, and trans-form triad must also consider
the de-form phenomenon. Deformations occur once the in-form is
presented, no matter of whether the last appears as performance or
transformation. In cases of per-form, the de-form is expressed as some
misunderstanding between what the in-form author made accessible, and
what its counterpart has understood during the communication process.
There are many reasons that cause de-forms, but their analysis falls beyond
this paper’s scope. In cases of trans-form production, the de-form is the
difference between the info-product creator’s intended goal and what was
successfully put into practice. One can imagine various reasons that cause
such deviations, ranging from incorrect planning of activities to some
general impossibilities due to physical or social circumstances.
To illustrate de-form in the case of human-to-human in-form performance:

I have a Bulgarian friend who worked as an architect in Africa in the late
1960ies. One day, he gave a servant a few US dollars and asked him to buy
a pack of Marlboros from a neighboring shop. The servant asked how to
proceed if Marlboros were unavailable, and the instruction was to buy
something else. Most humans would assume that the order’s intent was to
purchase some other brand of cigarettes. However, the servant returned
bringing a hot-dog and described how excited and busy he was to decide
what to buy instead of the unavailable Marlboros. This type of
misunderstanding seems possible when communicating with intelligent
machines if they literally copy a request. Here, future human-to-AI interface
de-forms will be a challenging issue.
To summarize, the important fact to bear in mind is that deformation is

typical for any information to data exchange. Its level may vary from
insignificant to entirely spoiling the mission.
The Difference Between AI and Robots

Following the intelligent procedure analytic line that describes in-form
generation and decision-making about whether and how to publish, the
borderline between AI and robots is clear. AI is the information producer,
while the robot is a peripheral tool that provides the in-form visibility in a
manner corresponding to the targeted goal. Therefore, if the AI running
system is dedicated only to performance, it needs only a network, a display,

or even only load-speakers to serve it, so there are no robots needed. For
example, an artificial chess-player or music composer needs no mechanical
apparatus. Simultaneously, depending on their purpose, robots vary
considerably in construction: humanoid, flying drones, small, crawling, and
insect-like, etc. They may or may not have AI installed to guarantee their
balance and operation. However, in any event, it is possible to connect them
to an external AI system characterized by a high level of intelligence. Then,
robots become engaged to act as AI trans-form instrumentation.
Significantly, depending on the set of memory and data processing speed
capacity, a single AI-operating machine can conduct many parallel multi-
goals in-form generations and drive multiple robots.
What is Advanced AI (AAI)?

One can observe a variety of AI applications, designed for various activities
in our lives: machine language recognition and interpretation, face and other
image recognition, navigation, driverless vehicles, medical diagnostics,
gaming, etc. Many of these AI solutions perform successful non-conserved
big data streams identification and examination, applying deep algorithmic
learning via neural networks parallel to the classic mathematical logic-
grounded computer processing. Thus, they discover data patterns that are
related to a variety of running interactions, links, and co-relations. In this
perspective, contemporary AI systems’ analytic/synthetic capacities strongly
exceed human thinking and imagination in particular areas of research. Here
it is useful to define advanced AI (AAI) and clarify where it falls within the
framework of narrow (specialized) AI and artificial general intelligence
(AGI).
To better understand this issue, one should know the three essential
characteristics of big data:
First, it is an enormous volume of data – a volume that no human could

memorize and explore.
Second, these are unstructured, real-time, and rapidly changing data

flows.
144 Chapter VIII
Third, the diverse data streams to investigate have heterogeneous

origins.
Because being unemotional, AI operating machines can ask a single

question as many times as they need without becoming annoyed or
dysfunctional. Moreover, humans, being unable to deal with big data,
experience a blind witnessing as to what machines do during data input and
data processing. Also, we cannot foresee the computers’ activity based on
studying all the rules they follow in their processing because, as Duranton
outlines, AI constantly learns from its experience and thus, it continually
produces and applies new rules.11 In other words, humans have no choice
but to wait passively until AI intelligently generates results. Further, this
black box phenomenon drives the conclusion that we can never be sure
whether the AI has demonstrated the totality of its intelligent production or
just a part of it. To study this issue, laboratory experimentation with AI
systems practices a periodic data processing switch-off so that experts can
survey retrospectively the many lengthy lines of logged mathematical
activity. Although the success of this research is questionable, it seems that
AI systems memorize more in-form than they publish. It has also been
shown that AI deep learning retains some of its activity results for
application in the future. Therefore, we may never be aware of all processing
experiences, which it has accumulated.
With this as background, the philosophical concept on AI development,

which a few decades ago was seen as evolving from slow to similar to
human mental faculties, and in some far future to AGI, is now accepted as
obsolete. The contemporary argument states that the AI’s successful data
processing, which overpowers our human abilities, does not make it smarter
than we are. Therefore, the AI classification was oriented to narrow-
specialized AI, designed as a human-centered and entirely human-
dependent assisting instrument, and AGI as something that will emerge in
the future as a self-managing non-supervised actor. Pessimists predict its
occurrence in some hundred years ahead, while optimists insist that it will
11 Duranton, “Humans and AI.”

happen much sooner because of the exponential growth of the machines’

data memory and processing capacities.
I will not enter this discussion of timing, although it is impressive that the
focus on AGI emergence falls on when rather than whether. However, from
my perspective, the crucial question is also how it will arise. Most texts,
dedicated to AGI issues focus either on its differences with human
intelligence (often claiming that machines just simulate intelligence), or
draw a picture in which suddenly, a new revolutionary AGI performing
automation is invented. Perhaps, thinkers are inspired by publications about
the Deep Mind algorithms involved in demonstrations with the Japanese
board game Go.12 Arguably, this is the hardest human intellectual game.
Initially, the AlphaGo deep learning algorithm won the World Championship
in Go by studying the entire human history of the game. Subsequently,
AlphaZero achieved brighter success having been provided with just the
game rules and having played for six weeks against itself. It studied its own
experience, entirely ignoring the human practice history. Logically, as this
AI managed learning Go in a short period, it was expected later to run into
new spheres. Thus, radically new AGI expectations emerged. However,
there are no publications to confirm success in such a trend.
Although not positioning myself on the list of the AGI revolution path
followers, I do not exclude it. Simultaneously, I can also draw two distinct
scenarios for the narrow AI’s development:
First, any narrow AI Deep Learning algorithm progress could follow a

step-by-step intensifying specialization in new investigative fields, so
that the particular AI operating system adds new capabilities. Thus, it
will gradually expand its narrow competence in analytic and synthetic
data processing. In this way, it would follow human educational
practice. AI will become more and more applicable to various specific
explorations and activities, but it will never become completely general.
At the same time, a second creative option also seems possible. A new
type of narrow AI could be developed, which could coordinate the
12 For more detailed information, see
https://deepmind.com/research/case-studies/alphago-the-story-so-far .
146 Chapter VIII
activity of several or even all existing narrow intelligent systems so that

limited universality arises by conducting synchronized multi-
specialization in various spheres.
I recognize that these paths for narrow AI evolution are easier to design
compared to AGI and thus they could emerge more rapidly. I call this
eventual technology Advanced AI (AAI). In my opinion, it will be the
apparent narrow AI next generation and as it will bring considerable new
circumstances, due to having innovative intelligent bodies in play, our
contemporary society should get prepared to meet it in short terms.
Advanced AI Communication and Networking

The essential mission of any type of communication is sharing content and
discovering meanings (thoughts, viewpoints, attitudes, feelings) to achieve
mutual understanding, ergo, producing common successful predictions
about future events even if they refer to issues retrospectively. In this
fashion, mutual understanding among intelligent bodies means an
agreement on what is about to happen combined with a common standpoint
on how to react effectively to the estimated changes of state. Moreover,
communication operates within a framework defined by the range of the in-
form production that every intelligent actor decides to present.
All of us traditionally communicate among ourselves in a human-to-human

fashion, and over a long time we have experienced the resulting benefits and
disappointment. Everyone has their own approach, which includes
consideration of the goal of the communication and the identity of the
counterpart. However, the human-AI and AI-AI interfaces are different. Our
experience here is based on just two decades of practice and its theoretical
analysis. Further, human-to-AAI communication assessment is just
prognostic because AAI is not yet available.
Therefore, I will focus first on what seems common to us and them and what
immediately appears is the intelligence procedure, which both humans and
machines employ. Here, the issues to highlight are the following:
As evident from many specialized AI actions, humans appear generally

slower in data processing compared to intelligent machines, and this will
not change. Moreover, in any event, the intelligent data input to in-form
output process will suffer delays in relation to the examined changes of
state because of the necessary period to process the acquired data and
then determine follow-up moves on behalf of the intelligent actor. In
many cases, the shorter the intelligent processing period is, the fewer the
number of prediction errors. In this trend, it seems that machines will
overpower humans.
No matter the data volume, due to technological capacity when copying

mathematical code, in any event AI-to-AI network data flow exchange
always runs faster compared to human-to-human or human-to-AI
communication. From this perspective, AI exceeds the human mental
faculty, and in the future, this feature will be typical for AAI as well.
The de-form level in cases of AI-to-AI data exchange is much lower

compared to human-to-human, or human-to-AI communication. This is
due to the direct mathematical code copying that skips the conversion to
language, image, or another mode of the in-form signification,
connotation etc. Advanced AI versions will keep this characteristic.
Simultaneously, (at least at present) it will most often interpret a single

meaning as embodied in a particular code in a one-to-one fashion, rather
than denoting some multilayer approaches typical to humans. This issue
needs special attention and care to avoid various misunderstandings with
the machines.
In my opinion, AAI, which arises as an autonomous intelligent body that

can drive peripherals, will also add the following specific components that
we will have to consider:
First, we must expect AAI to focus on shaping its future together with
us. Therefore, we shall have to assess all ongoing physical and social
processes based on a prediction of what the innovative automation is
about to demonstrate. Simultaneously, AAI will do the same about us.
By investigating big data, AAI will apply deep learning comparative

analyses to real-time data streams with heterogeneous origins,
something that humans cannot do. Bearing in mind that any intelligent
148 Chapter VIII
procedure deals with only a partial data input of what is potentially

available, which in turn depends on the intelligent body’s sensors,
memory, and data processing capacity, in many cases, a prediction’s
success is related to what has been involved in the in-form’s creativity.
Therefore, compared with humans, AAI has a better chance for accurate
predictivity, and we will need to add this assessment in communicating
with it.
Moreover, we will run our AAI-to-human communication never

knowing if it shares with us all its informational production or just a part
of it. The same is true when evaluating the next generation of AI-to-AI
network informational exchange.
Additionally, any AAI informational product that is presented to us will

require an analysis of what kind and what amount of data shaped its in-
form generation. We should also consider the AAI type in play because
unlike AGI, AAI is about to be just a set of synchronous specialized AI
operations. Therefore, it is necessary to continually judge the level of its
universality because distinct specialized sets may produce different
outputs for similar inputs.
The above list of issues may not cover all the variations typical for human-
to-AI and AI-to-AI communications if compared to our human-to-human
practices. However, I see them as a comprehensive background that sets out
a framework for future research and analysis. I believe, many other issues
will emerge. However, regarding human-to-human communication, many
thinkers note that effective communication is less about talking and more
about listening to reach a common understanding. I will not discuss if this
is the best behavior among humans, but I think that it would work perfectly
in our communication with advanced and general intelligence computers. I
do not claim that AAI is smarter than we are—there are many arguments
against such a statement—but without doubt, AAI will be smart enough and
completely foreign to us. Thus, at least at the first stage, many of its
conclusions will seem surprising both as to content and analytic perspective.
However, many of them will be correct.
Conclusion
Today, our civilization is at the dawn of a new era. We are about to meet an
unknown innovative autonomous intelligent actor—AAI operating
machines that surpass us in some areas of our mental faculty. Moreover,
AAI memorizes its deep learning investigation results for future application.
Therefore, in the future, when analyzing a particular event or process, a
group of similar machines all running similar hardware and software
solutions, will generate diverse in-forms depending on what they have
explored in the past. Although this is not an expression of human
subjectivity, no doubt, this is a step toward AI individualization. Therefore,
in addition to our human-to-human face-to-face communication traditions,
we must develop a new human-to-AI and vice-versa interface that should
not follow a pure subjective-objective path but should recognize the
following new circumstances:
Advanced AI systems can act in our favor, but also as our competitors
and opponents.
Moreover, to avoid encounters with them, we will need to predict AAI’s

reaction in various situations while keeping in mind all its peculiarities,
including new ones that we discover in future research.
In cases of our synchronous interface with AAI solutions, we will need

to recognize their ability to run in a parallel, multi-task mode that can
result in the driving of multiple robots. Thus, we will need to be aware
of whether regarding diverse activities we are facing a single system, or
different intelligent machines.
Finally, as AAI is about to become a cloud-based technology, we will

need to develop the habit of communicating with nonsingular intelligent
bodies, which not only borrow from common data sources including the
deep learning results experiences, but also might share and exchange
parts of their expertise to serve parallel missions.
Therefore, contemporary progress in AI analytic and synthetic capacities,

which seem close to the human mental faculties, requires a philosophical
approach with a framework and range that we have never experienced.
150 Chapter VIII
Moreover, it needs to not only direct AI development, but also be prepared

to meet the new challenges that will arise as AAI emerges. In my opinion,
this task is so complex that philosophers of all schools and traditions can no
longer draw a strict borderline separating their wisdom from what science
and engineering demonstrate. First, they should focus on how to model the
technological progress to protect human nature from AI targeted
breakthroughs of our subjective firewalls and disclosure of in-forms that one
does not want to publish. At the same time, we must develop a philosophical
concept for our transition from human-to-human face-to-face communication
to human-to-AI and vice-versa interface and interaction that will avoid
mutual encounter and conflict.
References
AlphaGo. Accessed July 1, 2022. https://deepmind.com/research/case-
studies/alphago-the-story-so-far.
Duranton, Sylvain. How Humans and AI Can Work Together to Create
Better Businesses. Accessed May 7, 2020.
https://www.ted.com/talks/sylvain_duranton_how_humans_and_ai_ca
n_work_together_to_create_better_businesses?language=en.
Floridi, Luciano. “What the Near Future of Artificial Intelligence Could
Be.” Philosophy & Technology 32 (2019): 1–15.
https://link.springer.com/article/10.1007/s13347-019-00345-y.
Floridi, Luciano, and J. W. Sanders. “On the Morality of Artificial Agents.”
Minds and Machines 14, No. 3 (2004): 349–379.
Hanson, David. “Expanding the Design Domain of Humanoid Robot.”
Paper presented at the ICCS Cognitive Science Conference, Special
Session on Android Science, Vancouver, USA, 2016.
Leibniz, Gottfried. The Monadology. Translated by Robert Latta. Oxford:
Clarendon Press, 1898.
McCarthy, John, Marvin L. Minsky, Nathaniel Rochester, and Claude E.
Shannon. “A Proposal for the Dartmouth Summer Research Project on
Artificial Intelligence, August 31, 1955.” AI Magazine 27, No. 4 (2006):
12–14.
Pareira, Luis Moniz. “AI & Machine Ethics and Its Role in Society.”
Keynote speech at the International Conference of Artificial Intelligence
and Information, Porto, Portugal, Dec. 5–7 2017.
Russell, Stewart J., and Peter Norvig. 2016. Artificial Intelligence: A
Modern Approach, 4th edition. London: Pearson Education Ltd.
Turing, Alan. “Computing Machinery and Intelligence.” Mind: A Quarterly
Review of Psychology and Philosophy (October: 1950): 433–460.
Wissner-Gross, Alex. “A New Equation for Intelligence.” Accessed
September 7, 2016.
https://www.ted.com/talks/alex_wissner_gross_a_new_equation_for_i
ntelligence .
CHAPTER IX
THE STRUCTURE OF ARTIFICIAL RATIONALITY
BORIS GROZDANOFF
Abstract
Deep Reinforcement Learning (DRL) is at the forefront of modern artificial
general intelligence (AGI) research and formalizes AI tasks in terms of
agent, environment, state, action, policy and reward and also harnesses the
function approximation power of the other leading AI instrument, the
artificial neural networks (ANNs). One of the main challenges contemporary
efforts to build artificial general intelligence (AGI) systems face, in a broad
Turing-test framed setting, is the artificial emulation of key components of
human language-based reasoning, like meaning of singular terms and
expressions but also semantically enabled logical reasoning. Here I envision
a high-level AGI system, RAISON, that could be driven by a DRL
architecture. I suggest that we can use Frege’s influential distinction
between sense and reference in order to emulate linguistic meaning as found
in real world human practice. For that purpose, I propose that a semantic
graph space (SGS) can be trained on available NL datasets to deliver
artificial semantic ability. I also suggest that the formal abilities of syntax
manipulation and rules-based logical reasoning can harness the expressive
power of SGS and thus enable knowledge-based «educational» training of
an emerging general AI model.
Introduction
In May 2022, the lead research scientist at Google’s DeepMind, arguably
the world leading company in the development of Artificial General
Intelligence systems, Nando De Freitas, provocatively claimed that
The Structure of Artificial Rationality 153
“It’s all about scale now! The Game is Over! It’s about making these models
bigger, safer, compute efficient, faster at sampling, smarter memory, more
modalities, INNOVATIVE DATA, on/offline. Solving these scaling
challenges is what will deliver AGI. Research focused on these problems, eg
S4 for greater memory, is needed. Philosophy about symbols isn’t. Symbols
are tools in the world and big nets have no issue creating them and
manipulating them”. 1
This statement, although not officially sanctioned as a DeepMind's position,

created an intense discussion by some of the most influential AGI
researchers and developers in the field. 2In its crux, it contains several key
theses:
1. That there is a well-founded conception of an AGI (implicit) and

2. That AGI is achievable
3. The multimodal deep artificial neural networks (DANNs) and the
deep reinforcement learning (DRL) architectures of AI are by and
large sufficient for AGI
4. That «philosophy about symbols» cannot achieve AGI
De Freitas’ position is characteristic of the prevailing view of AGI, which

derives much of its influence from its success in narrow AI systems. This
does not come from a won dominant position in the heated ongoing
scientific debates in AI and related fields, like computer science,
mathematics, cognitive science, neurophysiology, logic and analytic
philosophy. In De Freitas’ statement we see very clearly the affirmative
thesis that the general computational structure of A(G)I is already available
and the (undisputed, sic!) gap between available (narrow, sic!) models and
accepted A(G)I is a mere matter of mathematical and engineering
improvements, that should focus on volumes of data, types of data, sizes of
models and related engineering improvements like data storage (better
memory) and computational processing. The negating thesis that
1https://twitter.com/NandoDF/status/1525397036325019649
2 For a brief summary of the discussion see Sparkes, Matthew article in “Is
DeepMind's Gato AI really a human-level intelligence breakthrough?” in New
Scientist, 19.05.2022. https://www.newscientist.com/article/2320823-is-deepminds-
gato-ai-really-a-human-level-intelligence-breakthrough/
154 Chapter IX
complements it is the one that while the above can deliver AGI, «philosophy
of symbols» cannot. I will discuss both theses in more depth, but two points
deserve mentioning before that: first, the term «more modalities» in De
Freitas' claim and second, the apparent derogatory formulation of
«philosophy of symbols». Under modalities De Freitas clearly has in mind
modalities of multi-modal systems like GATO,3 where each modality has a
distinct function and nature: playing a certain game, leading a conversation,
image recognition and the like.
On historical grounds, we can identify those modalities with different

human abilities and in particular, such that contribute to human intelligence.
Thus, the desired greater number of modalities effectively translates as «we
need AI systems that have greater number human-like abilities». The
«symbol philosophy» is easily decoded as pointing, mainly backwards,
toward the now largely considered as unsuccessful early AI systems,
founded on expert systems,4 which used symbolic manipulation and not
ANN or DRL foundation. The implicit attack, however, is much deeper and
is directed against the choice of formal syntax-based models of AI as
opposed to current and argued as generally successful ANN and DRL-based
systems. Unpacked, it translates as the following: «AI systems, built on any
syntactic structure, are AI inferior to ANNs and other non-syntactic
models».
We can summarize that in De Freitas' thesis, the computational ANN and

DRL architectures can deliver A(G)I whereas syntax architectures,
expressing logical and other formalisms, cannot. The obvious assumption
behind this is that the nature of human intelligence (HI) is best modeled via
ANNs and DRL models, and not via syntactic models. We see that digital
brain models are taken to (be able to) directly lead to human-like AI (HLAI).
Or, to formulate it more clearly: proper brain function exhausts human
intelligence. I will completely ignore the theoretical deficiencies of this
clearly physicalist assumption, which identifies rationality with physical
3 For the GATO system see DeepMind’s paper: Scott Reed et al. (2022) “A
Generalist Agent” in arXiv:2205.06175v2 [cs.AI].
4
Puppe, F. (1993) “Characterization and History of Expert Systems” In: Systematic
Introduction to Expert Systems. Springer, Berlin, Heidelberg.
https://doi.org/10.1007/978-3-642-77971-8_1
processes, since my purpose here is not to develop a philosophical argument

but instead to offer a practical suggestion, that, eventually, might even
happen to inform, backwards, the philosophical debates on the nature of
human rationality.
Marcus5 and LeCun6 were among the first to dispute De Freitas, on different
grounds and in different directions. LeCun disagrees about AGI on a
fundamental level and argues that AGI is not and could not be well
formulated, let alone achieved technically. The next best thing, according to
him, is HLAI or human-like AI and our actual progress is only to be
expected in the direction of HLAI, not AGI. The main challenges before
HLAI are the lack of generalized self-supervised learning, no learning
paradigm for machines to learn how “the world works”. I agree with him
that it is best first to attempt HLAI, but I also believe, unlike LeCun, that
HLAI can lead to AGI. The argument behind this is fairly trivial: if a true
HLAI is artificially emulated, it would be trained on virtually all human
knowledge and there is no reason whatsoever to think that it would not learn
superhuman abilities in knowledge but also human-like rationality. From
DRL systems like AlphaGo, we already know that AI can find better
“moves” than humans have managed to for millennia. There is no obvious
reason why the DRL model that manages to reach HLAI would not do the
same with rational human abilities.
Marcus, following his long defended positions, that step on the shoulders of
AI giants, in his own words, like MacCarthy, Minsky and Simon, among
others, directly argued against the thesis (3). Following upon his long-
standing view that understanding the mind even at a high level is a necessary
prerequisite for success in AGI and the impressive results of artificial neural
networks (including in deep reinforcement learning models with neural
networks for value and policy learning functions) could not alone achieve
this goal, but should also be complemented with what he calls, following a
5 Markus’ position is developed here: https://garymarcus.substack.com/p/the-new-
science-of-alt-intelligence?s=r
6 Matsuoa, LeCun et Al. (2022) “Deep learning, reinforcement learning, and world
models” in Neural Networks, Volume 152, August 2022, Pages 267-275.

156 Chapter IX
tradition from some of AI's founding fathers, «symbol manipulation».7 He

rejects the claim that what we take as artificial intelligence in its, arguably,
more modest human-like AI form, versus the disputed immodest version as
artificial general intelligence, is now solved and reduced to the effectively
technical issue of scaling and related technical quantitative problems:
«Alt Intelligence isn’t about building machines that solve problems in ways
that have to do with human intelligence. It’s about using massive amounts
of data – often derived from human behavior – as a substitute for
intelligence. Right now, the predominant strand of work within Alt
Intelligence is the idea of scaling. The notion that the bigger the system, the
closer we come to true intelligence, maybe even consciousness.» 8
Marcus argues that not just De Freitas provocative statement that AI in its
HLAI form is being reduced to the above alternative intelligence, but also
that our best and most successful systems, like the recent GATO and
Flamingo,9 express nothing else but alternatively intelligent capabilities and
not at all human-like intelligent capabilities, least of all, generally intelligent
capabilities. The core of his argument can be unpacked in the following
form:
1. Artificial (General) Intelligence should mirror human intelligence

(HI)
2. Key for HI is the (human rational) ability to solve problems
3. Current DANNs and DRL systems do not mirror human ability in
(2)
4. Therefore, they are not and could not be human-like A(G)I
The «alternative» intelligence systems, no matter how successful in

practical tasks, all share the notorious problem of unexplainability, where
any functional success is driven by a computational black-box. And not just
on the parameter level, for example, with concrete optimized values of
ANNs weights and biases, but on the general architectural level, which
7 Marcus, G. F. (2001). The Algebraic Mind: Integrating Connectionism and Cognitive

Science. Cambridge, MA: MIT Press.
8
In https://garymarcus.substack.com/p/the-new-science-of-alt-intelligence?s=r
9 Jean-Baptiste Alayrac, Jeff Donahue et al. (2002) “Tackling multiple tasks with a
single visual language model” in https://arxiv.org/abs/2204.14198

concerns the very structure of the AI model: the kind of networks chosen,
the type of their integration, the learning algorithm, the activation function,
etc. If a model like GATO or subsequent more successful and more
powerful multimodal «generalizing» agents pretend human-like general
intelligence they need to prove that the intelligent functionality of the model
mirrors intelligence abilities of humans. Short of that, I am afraid that even
Markus' alternative intelligence term would be too beneficial for such
systems, since our only scientific criteria for intelligence are our criteria for
human intelligence. Systems pretending general human-like intelligence
would not only have failed to prove that they are human-like intelligent,
they would have failed to prove that they are intelligent at all and thus they
would be at best, only considered «alternative».
Here I suggest that the leading ANN and DRL technologies are not to be
opposed with syntactic (but also semantic and reasoning AI models) but,
on the opposite, their power can be harnessed to drive such models in order
to learn human-like abilities of syntax proficiency, semantic proficiency,
reasoning proficiency and knowledge proficiency or all the kinds we find
in intelligent human practice based on natural language (NL).
“Solving Intelligence” Approaches

Intelligence, historically and scientifically, has always been related mainly
to the human ability to understand (as clear in the term from Latin verb
intelligere – to understand, to comprehend). While this ability certainly does
not exhaust the semantic load of the term we cannot and we should not
ignore it when attempting an A(G)I architecture. Another notion, implicit in
intelligence, but quite present in its semantic explication, is the notion of
reasoning, the ability to draw conclusions from knowledge, to use the
principles of logic to further it and thus to justify real life choices. Together,
arguably, they can deliver the content of the meaning of the term rationality,
as found and as sought for by AI in the expression human rationality. Thus,
for a proper human-like AI to be intelligence at all it would need to
demonstrate the ability to understand, both expressions formed in natural
language and real world situations (as opposed to simulated virtual
situations); that is, to identify and artificially comprehend the meaning of
expressions, texts and real life conversations. The already notorious
158 Chapter IX
LaMDA10 model ability to intelligently respond to such expressions and to

adequately participate in such conversations is not an identical ability.
The AI model should also be capable to reason on the basis of comprehended

meaning, to draw analogies and inferences, and to perform all reasoning
actions humans do, such as drawing logical inferences and following logical
laws and rules of introduction and discharge of logical connectives, as found
in predicate logic, a powerful system that can formalize with rigor
expressions in natural language. It should also be able to reason with respect
to making a decision and taking an action, that is, to found decision and
action on reasoning. Such human-like ability would justify the behavior of
the model as truly rational and intelligent as opposed to random, chaotic and
unjustified. The black-box behavior of available models exhibits no
intelligent reason to justify an action before all alternatives in a certain
available set of actions. What they have instead is unexplained
computational functionality that cannot be rationally understood, that is, in
the light of the intelligere term, it cannot be accepted as intelligent.
The AI model would also need to reason causally, among other things. It
would need to understand the usage of the human terms cause and effect and
to learn to use them in the same way. And at the end, it would need to unite
all those abilities in a coherent unity, such as we find in a human's conscious
rational mind. Only then would we have grounds to assess it as a human-
like artificial intelligence, if not in structure, at least in function. And that
would be considered a success by the interdisciplinary community that
slowly erects the complex theoretical foundation of human intelligence,
which could only serve as a basis for its artificial counterpart, unlike the
black-box model. The alternative would be to formulate virtually ex novo a
novel, non-human intelligence and to the best of my knowledge, science
knows much less about such intelligences than it does about the human one.
All of the above considerations historically and scientifically fall under De

Freitas', sadly, derogatory expression «philosophy of symbols». Here, I
believe, we see a mistake of a category type. To accept that computational
10Thoppilan, R. De Freitas et al. (2022) “LaMDA: Language Models for Dialog

Applications”, in https://arxiv.org/abs/2201.08239v3
models, run on classical or, increasingly, quantum computers, can emulate

the biophysical processes in the human brain that bring about what we
observe as intelligent abilities and intelligent behavior is one thing, being
limited physicalism as it is. But to identify the former with the latter would
be the same as to affirming that what we observe as intelligent abilities and
behavior are a chosen set of brain processes. This, naturally, would be
absurd, besides being plainly false, for it is a plain fact of life that humans
not merely do not observe such processes, let alone during communication, but
never take them as the intelligent ability or the intelligent behavior
themselves. What we observe is the intelligent behavior that runs on top and
in some, still unknown, relation to those processes. It is therefore a much
better justified model to attempt to model human intelligence not on the
driving electro-chemical, that is, physical processes, effectively emulated
by the function approximation of neural networks, but on the acting
functionality of human rational behavior, much of which is self-conscious
and not physical per se.
The oldest and, arguably, the richest scientific probe into the nature and
structure of human intelligence, as comprised by understanding, reasoning
and self-awareness, is the western civilization's endeavor of philosophy that
has developed immensely complex and critically evolved debates on the
problems of semantics, reasoning, rationality and consciousness. This paper
follows in the steps of this tradition, which, I believe, can be immensely
useful for the practical purposes of arriving at a functioning general artificial
intelligence. This approach, to structure an AGI model, based on
philosophical doctrines, is surely unusual and highly unpopular in today's
exclusively computer science based projects, which loosely take
mathematical models of human neurons (like the ANNs) and models of
human-environment interaction (like the reinforcement learning model) as
sufficient to emulate the yet «unsolved» riddle of human intelligence. At
the end of the day, however, all approaches would reach the same judge and
the same field of assessment: the one that works first or best would
undoubtedly be considered a valuable if not the valuable approach. Thus,
the philosophy, mostly in its analytic line, faces an opportunity that is rarely
if ever present, to prove success in a practically measurable setting, such as
the AI and AGI race.
160 Chapter IX
Symbolism Unpacked
The generally quoted «symbolism», however, both in pure mathematics and
in logic, but also in written natural language, always comes in two parts:
syntax and semantics.11 Computation, in its forms of, say, functional
analysis, linear algebra or tensor calculus, is impossible without syntax and
equally impossible without the semantics that interprets expressions in this
syntax as well formed, as true or false, but also as meaningful. If the last
seems like too hard a requirement for a mere purely mathematical
expression of the form f(x)=y let us compare it with an alternative ()fyx=,
which contains the exact same symbols and the same number of them.
Where the two «expressions» differ syntactically is only their order. The
«symbols», to use De Freitas preferred term, are the same. They are just not
ordered in the same way in the two symbol strings. And yet the difference
is highly non-trivial, for their meaning, and for their ability to be interpreted
in truth-value. For the first is a well-formed formula of a function and as
such is mathematical, whereas the latter is not; but is a meaningless and
uninterpretable string of the same symbols not even capable to be
approached with truth or falsity. And computations, such as those we use in
ANN and DRL models, can only use the former kind, whereas the latter is
certainly of no use not just for computer science but also for the whole of
mathematics; symbolic gibberish can only figure with some valuable
significance in science fiction or in a modernist poem.
What is of key importance here, is that the syntax and its interpretation are
inseparably intertwined, even if we decide to take a slight historical detour
on Hilbert's formalism school of thought. From Hilbert, we know that any
choice of syntax, of symbols and order assigned to them, could be, of course,
arbitrary.12 But once chosen, it ceases to be arbitrary and begins to follow
formal rules and principles and to require ultimate rigor in its writing in
order to be semantically interpretable, that is, to be formally functional and
thus significant and meaningful. The semantic interpretation of any
11 Van Fraassen, Bas C. (1971) Formal Semantics and Logic, The Macmillan
Company, New York.

12 David Hilbert (1984) “On the infinite” in Philosophy of Mathematics, Edited by
Paul Benacerraf and Hilary Putnam, pp. 183 – 201, Cambridge University Press.
syntactically introduced in a formal system symbol produces its

functionality; the functionality enables the emergence of significance,
meaning and truth. They enable the ability, familiar from logic of natural
language, to reason with semantically interpreted syntactic expressions.
Once humans have proper syntactic terms, like terms in English, they can
interpret them semantically, that is, they can understand their meaning and
they can judge their truth. On the basis of these interpretations they can
attempt logical reasoning with them, like drawing an inference from a set of
previously available semantically interpreted linguistic expressions. This
ability, characteristic of human rational practice, is a conditio sine qua non
in any artificial model that pretends human-like intelligence.
If we cannot interpret the numeral 9 as the integer number «nine» we have

no rational access to the object of the term, the number, if we assume «9» is
the only symbol in the system reserved for it. Thus, the dynamic unity of
properly interpreted good syntax is essential for all formal systems, the
whole of mathematics included. Mathematical formulae have value only in
virtue of their understanding. Mathematical proofs, that elevate conjectures to
theorems, require an utmost rigor of understanding the formulae, the
relations that hold among them as well as the logical rules, including the
rules of inference. Those rules allow the rigorous transition from one
formula to another in a manner that would furnish the totality of the steps
where at the end we reach a theorem, which emerges in virtue of the
mathematical proof and not in virtue of arbitrarily concatenated symbols,
let alone uninterpreted ones. The «symbol manipulation» covers both syntax
and semantics and is highly non-trivial for the most rigorous domains of
sciences, mathematics and logic, and the fact that neural network-based
computation simply vanishes without both does not help much to oppose
Markus' point.
In what follows, I will unpack my reading on what is needed behind the

«symbol manipulation» to reach a structure of neural network
differentiation, both at high, functional level but also on computationally –
architectural level. Symbols can only provide a material for order, which
needs to be semantically interpreted, in truth-value and in meaning, so that
it can be logically reasoned with in order to reach inferential power.
Arguments can be formally valid but, upon actual world data training, also
162 Chapter IX
materially sound. Therefore, much differentiation is needed beyond mere

symbol manipulation, for only a set of distinct syntax, semantics and
reasoning modules, functional after successful training, can begin mirroring
the rational practice of humans, as found in everyday life.
The two main characteristics of the environment of operation of human

rational agents, most generally defined and devoid of any content, are its
immense complexity and its notorious resistance to knowability. Combined,
they render the modern state of the art approaches in AGI research
defocused and ineffective from the stage of very beginning, which is the
formulation of the task for which AI algorithms need to be proposed, trained
and assessed. It is safe to note that even the most successful results of world
leading AGI teams, such as DeepMind and Open AI, focus initially on very
specific tasks, such as chess or Go superhuman abilities, with the expectation
that the algorithms would eventually generalize to much different and
broader tasks, eventually reaching an AGI scale generality. Perhaps the
most successful system along this path has been DeepMind’s recent MuZero
algorithm,13 which harnesses the learning capabilities of deep reinforcement
learning in an unprecedented way not just for specific task supremacy, but
with demonstrated potential for practical implementation and success
retention in general domains as industry and, eventually, AGI. The world
leading researchers and developing teams, however, do not share the same
understanding of what would be a successful structure for an AGI system
and what are the most adequate methods to approach it. Perhaps the best
illustration of this is the recent debate between De Freitas and Markus, who
defined very clearly the two leading stances.
Modern AI Technologies that Approach Human-Like

Intelligence
The nature of the tasks before the AI model determines the scope of
technologies available for implementation to successfully perform the task.
The modern field of Deep Learning (DL) is highly successful in acquiring
the ability to learn from enormous amounts of data, mimicking in a very
13 Schrittwieser, J., Antonoglou, I., Hubert, T. et al. (2020) “Mastering Atari, Go,
chess and shogi by planning with a learned model” in Nature 588, 604–609.
restrained way how the human brain functions. What is characteristic of the
DL technology is that it is able to be fed with large volumes of previously
unstructured data or largely unstructured data, and then process it via a
dynamically structured architecture of neural networks, with input layer,
which receives the data, hidden (or deep) layers, that process the data sent
from the input layer and an output layer that classifies or predicts the desired
output. DL is able to classify and suggest predictions with enormous
accuracy and precision.
The supervised learning (SL) technology essentially manages to arrive at a

complex function that maps input data to output data after a process of
training that is performed on categorized and previously labeled data. The
technology is enormously successful in classification tasks.
The technology of unsupervised learning (UL) is a different instrument that

can be used to discover patterns and ordered volumes of information within
large amounts of completely unstructured data. The UL is integrable with
both the SL and RL technologies as of particular interest is the integration
of UL with the environment layer in RL, where UL can be used to enhance
and update in virtually real time the environment dataset, which plays a key
role for the RL success. Thus, in a general RL system where the
Environment, observed by the agent, constantly develops and changes,
functionality that the AI architect would very much desire is the
environment to evolve novel features and agents, previously unfamiliar to
it. The novelty and unfamiliarity of such features can be introduced by the
simultaneous work of an UL system serving this purpose. The system could
be devised to observe actual environment and discover novel patterns in it,
to categorize them, as environment features or as novel agents, and then
introduce them into the RL environment layer, thus making them
susceptible to be observed by the RL agent, functioning in a Markov
Decision Process (MDP)14 setting.
14MDP analizes tasks in terms of agent, environment, observations and actions, for
an in-depth discussion see Olivier Sigaud and Olivier Buffet (Eds.) (2010) Markov
Decision Processes in Artificial Intelligence, Wiley.
164 Chapter IX
Finally, the technology of Reinforcement Learning (RL) uses the MDP

architecture in order to approach, structure and perform its tasks. The key
modules in MDP are environment, agent, states of environment, actions of
the agent and rewards. The RL technology is extremely successful in a
number of famous cases, by and large goal-oriented tasks, as illustrated by
the supremacy of RL systems in game-like settings and tasks. What is of
particular interest for the AI architect here is that most real life situations
that intelligent humans function in are very much akin to game-like
scenarios. Thus, the success of RL up to date is very informative and
suggestive of further success of RL in next stage complexity tasks,
approaching real life scenarios of humans in everyday human functioning
environments.
Deep Reinforcement Learning Model for Human-Like

Rationality (Raison)15
The enormously successful approach of DRL analyzes the task within the
MDP framework of an agent, operating via observing states of an
environment and taking actions with quantified success toward a final
predefined task; the model is driven by reward feedback. Eventually, the
agent develops an improved policy of actions, which can be generally
understood as the strategy to determine better next actions in order to reach
the final task. The policy is a mapping of observed environment states to an
agent’s actions, which provides the best “justified” option for a highest
reward stimulus. The integration of deep neural networks into reinforcement
learning proved immensely fruitful and particularly in scenarios where the
spaces of the agent’s actions and the states of the environment are too large
to be operated effectively within by a pure, non-neural reinforcement
learning system. In DRL, neural networks can approximate both value and
policy functions in RL and typically receive at input layer both value related
data and agent’s policy-related data.
15The RAISON name of the model is devised to accent on the English word reason,
as one of the key components of the suggested model of human-like intelligence, the
other being understanding, and contains the AI abbreviation for obvious purposes.
The RL and neural network modules have been successfully integrated with
two additional AI methods: self-play (SP) and tabula rasa (TR) training,
that is, training with no previously provided data sets. The potential of DRL
systems integrating SP and TR (DRL-SPTR) was first demonstrated by
DeepMind’s algorithms AlphaGo Zero, AlphaZero and most recently
MuZero and LaMDA. What is truly remarkable in MuZero is the novel
ability of the system to learn the rules of the game by itself and then to
develop the ability of superhuman performance in the recognized game.
The unprecedented success of DRL-SPTR systems is by itself a reason to

investigate their prospects for human-like intelligent tasks, such as natural
language comprehension and natural language based logical reasoning. But
it is the functionality framework of DRL-SPTR that reveals an AGI
potential given the characteristic nature of human-like AI tasks, like
machine emulation of NL-based comprehension, reasoning, communication
and problem solving, among others.
Here I suggest that a DRL-SPTR system has the specific potential to

formalize syntax tasks, semantic tasks and logical reasoning tasks. The
reasons to expect that a DRL-SPTR system would have an AGI potential
are much broader than the mere specific task success demonstrated by
DeepMind’s and OPEN AI’s algorithms. First, the general RL setting,
where an agent operates in an environment, is effectively isomorphic to the
real life setting where a rational human agent operates in a real world
environment. The agent-environment and the respective observation, states,
and actions reflect very adequately the rational process where a rational
human observes the world, attempts to comprehend it, attempts to reason
about it, on the basis of comprehended observed information about it,
formulates actions and decides to perform them on the basis of reasoning,
which provides justification for both the type of the action (drink water) and
the reasons to undertake it (dehydration needs to be eliminated out of self
preservation). The reward stimulus is exceptionally evolutionary in its
structural functionality. And the self-play method of learning, while
uncharacteristic of human rational agents, is extremely productive in
simulating human rational agent interaction with the environment.
166 Chapter IX
While the domain of artificial intelligence is highly interdisciplinary and

encompasses fields such as mathematics, computer science, neuroscience,
cognitive science and psychology, the history of “solving intelligence” is
much older and richer than the history of all of these scientific disciplines,
with the exception of pure mathematics. My stance in approaching the task,
today described as “solving intelligence”, comes from the tradition of
analytic philosophy, where the problems of natural language based
comprehension, logical reasoning and normative justification have an
intense, deep and very rich history. It is an assumption, both in analytic
philosophy and in this work, that human rational activity flows at their level
of functionality through the framework of human natural language, such as
English. Within this tradition, human intelligence is generally grasped as
the integrated set of necessary conditions of human rational abilities: NL
mastery, NL understanding, both in syntax and semantics (that operate
successfully in real life scenarios, including problem solving and NL
communication with other human agents), NL based reasoning, founded on
logical rules, such as rules of introduction and discharge of logical
operators, as well as logical laws such as modus ponens and modus tollens,
syllogistic reasoning and use of acquired knowledge in reasoning.
Abilities: Syntax, Semantics, Understanding, Reasoning

and Rationality
The AGI tasks, due to their distinctive nature, can be addressed by different,
ad hoc implementations of the DRL-SPTR systems. Each task would
emulate a distinct rational ability that is accepted as a component of human-
like intelligence and would have a dedicated DRL-SPTR system, which
learns it. This ability would be acquired by the agent in the DRL-SPTR
system after DRL training.
The syntax ability (SYA) should consist of syntax structures for recognition
and classification and it would need to be integrable with the semantic and
the reasoning systems. To illustrate, if we want the syntax ability to be
trained in 2nd order predicate logic, it would need to recognize 2nd order well
formed formulae and differentiate them from non-wwfs in the same logic.
This ability, albeit formal, is essential for human-like NL abilities and would
allow systems for semantic interpretation and logical reasoning to be

trained.
The semantic ability (SA) is essential for natural language comprehension

and communication. For our purposes, we would differentiate between
semantic interpretation in truth-value (SIT), which provides the ability to
interpret NL expressions with respect to truth and falsity, and semantic
interpretation in meaning (SIM), which provides the ability to interpret NL
expressions according to meaning. For the latter, I choose Gottlob Frege’s
influential distinction between sense (semantic interpretation in sense - SIS)
and reference (semantic interpretation in reference - SIR) as functionally
powerful enough to solve the task in real life scenarios.
The reasoning ability (RA) will represent human’s real life practice where
we reason, for our concrete purposes and along the main thesis of analytic
tradition, within the natural language.16 Generally, this is the ability to
operate with syntactically well-formed linguistic expressions, to interpret
them semantically in truth-value, in sense and in reference, and on the basis
of these to introduce and discharge connectives for well-formed formulae
(atomic and molecular) as well as to concatenate them into larger sets of
expressions. The RA system would learn to construct ordered chains of
expressions following the logical forms found in human practice: the logical
laws, such as modus ponens and modus tollens and many others. Most of
all, the system would acquire the ability to reason syllogistically, to draw
logically valid conclusions from available premises and to assess logical
arguments as valid, invalid, sound or not sound.
Human intelligence, when approached within the general framework of

indo-European natural languages, limited in RAISON to English,
necessarily contains the ability to comprehend (AC) the meaning and
references of well-formed linguistic expressions and the ability to reason
with them (RA). Comprehension and reasoning are distinct, albeit
dynamically intertwined abilities, that we cannot fail to find in rational
16Central thesis of analytic philosophy accepts that all reasoning occurs within NL
– for a discussion see Martinich, A. P. and Sosa, David (eds.). 2001a: Analytic
Philosophy: An Anthology, Blackwell Publishers, and Martinich, A. P. and Sosa,
David (eds.). 2001b: A Companion to Analytic Philosophy, Blackwell Publishers.
168 Chapter IX
human behavior, be it on the level of its description or on the levels of its

explanation and justification. Therefore, I accept that human-like intelligence
contains as necessary components NL understanding and NL reasoning.
The fourth necessary condition of human-like intelligence could best be

defined as rational ability (RAA), which endows SYA, SA and RA with
actual knowledge (AK) about the real world. Thus, in the hypothetical case
where an integrated system (SYA+SA+RA or SSR for short) functions
without training in actual knowledge 17about the different domains of the
actual world,18 it would not be able to introduce true premises into its
arguments. Thus, the arguments would at best be valid arguments, but not
sound ones19 and the comprehension and reasoning abilities of the system
would be exclusively limited to the form of the expression and not to any
practically useful semantic load that is necessary for real world
comprehension, real world reasoning, communication and problem solving,
as found in human practice.
Therefore, in order to obtain human-like rational ability (RAA) we must

train an SSR system onto actual world knowledge (@WK) and here I
strongly differentiate between mere data and knowledge, where data is raw
unstructured information with no or virtually no practically useful
connections of order between its elements. For knowledge, I see no better
theory of epistemology,20 both for expressing the nature of human
17 Using the symbol “@” for “actual” as in @W or “actual world”

18 Such as scientific facts from biology, physics, geography, history, etc. as well as
behavior facts about history and possible explanations of human behavior, provided
by psychology, cognitive science and most of all, non-rigorous common sense,
found much more frequently in real life scenarios than scientifically rigorous facts.
19
Sound arguments are valid arguments with true premisses and possess the highly
non-trivial value to transfer not mere logical validity from premises to the
conclusion, but to transfer truth of premises onto truth of conclusion, thus making
them immensely important for scientific purposes much transcending the formal
rigor of pure mathematics and logic. For an overview see Etchemendy, J. (1999) The
Concept of Logical Consequence, Stanford: CSLI Publications.
20 Epistemology is one of the oldest disciplines of philosophy and it deals with all
problems of knowledge: its nature, its kinds, criteria for, etc. It is my assumption
that analytic theories of knowledge are both the most promising ones and most
suitable for the practical purposes of building AGI models. For a high-level
knowledge and as a basis for practical AI success in emulating human-like

knowledge into artificially intelligent systems, than the leading analytic
theory of knowledge of knowledge as justified true belief + x (or JTB+). In
JTB+, the cognitive agent knows that P (the NL expression) if and only if
he is justified to hold that P (justification or J), he actually is in a state of
holding it (artificial belief or AB), and P is true as guaranteed by our SSR
system. The exceptions, denoted by the “+”, and proven by Gettier cases 21,
showed that the J, T and B conditions are necessary but not sufficient
conditions for knowledge. Non-Gettier cases of knowledge, statistically
dominating human practice, have very high survivability against
counterfactual attacks, but still need to be supplemented with additional
conditions where different positions in the debate offer different candidates.22
For our practical purposes here, I will ignore the “+” condition and I will
accept that modeling RAISON on the JTB set of necessary conditions for
knowledge is still a very significant success. A model that would emulate
the sufficient “+” condition, could be further developed, once analytic
debates in epistemology establish an accepted “winner” that solves the
Gettier cases.
In this way, I hope it is clear that only a @WK trained SSR system can
become artificially rational and truly artificially intelligent. The SSRR
system (syntax, semantics, reasoning and rationally trained system or SR2)
would be an integrated AI system that exhibits human-like abilities to
comprehend expressions, reasons using them and which knows, like
humans, via @WK “educational” training in AI training datasets, like the
WiKi Corpus, facts about the real world. But most of all, it would be able
discussion see Ernest Sosa (2018) Epistemology (Princeton Foundations of

Contemporary Philosophy, 14) Princeton University Press.
21
Gettier famously demonstrated that the JTB formula only provides the set of
necessary conditions of knowledge and not the complete set of conditions for
knowledge by constructing, statistically rare but epistemologically significant
Gettier cases where cognitive agents satisfy J, T and B conditions and yet fail to
possess knowledge, where their state is generally attributed to the so called
“epistemic luck”. The original argument is presented in Gettier, Edmund (1963). “Is
Justified True Belief Knowledge?”. Analysis. 23 (6): pp. 121–123.
22
For a discussion on Gettier solutions see Lycan, W. G. (2006). “On the Gettier
Problem Problem.” In Epistemology Futures, (ed.) S. Hetherington, Oxford University
Press.
170 Chapter IX
to form sound logical arguments and communicate via them. The AI ability
in SR2 would represent a successful operation of human-like intelligent
abilities in the actual world. Such ability is arguably the best candidate to be
a decisive criterion to judge the intelligence of humans - it could not be mere
communication between humans, as presented in the classical Turing test
setting, but it should be the evolutionary environment of successful
functioning, survival, adaptation and evolution of rational humans in the
real world. A fully functional artificial agent, like RAISON, acting in this
environment without being identified by humans as an artificial vs. human
agent would be a much stronger version of the Turing test than the original
one.
Only recently in history has humanity discovered (or developed, or both)

the domains of pure mathematics, logic, human natural science, art and
ethics. They are an essential part of the actual world humans operate in since
millennia and even if perhaps inaccessible to other living species, like plants
and animals, or other intelligent species, like animals and AI run robots,
they further and enrich the evolutionary environment of homo sapiens and
therefore they need to be included in the @WK training of the SR2 system.
The data for the DRL system training would need to allow its successful
training and this is a non-trivial task. It would need to be mathematically
encoded as n-dimensional tensors, as popular ML systems of language
encoding of the word2vec 23kind would lack expressive power and
sufficiently powerful integrability with semantic spaces like SGS. Without
these properties, however, traditional encodings would be unable to serve
their AGI purpose; the necessary mathematical expressive power can be
delivered only by general n-dimensional tensors and thus the word-level NL
encoding would be a word2tensor (or w2t) encoding. The necessary
encoding for the DRL-SPTR system would need, on the one hand, to
provide the specific encoding for each system (SYA, SA, RA and RAA):
syntax2tensor (SY2T), semantics2tensor (SE2T), logic2tensor (L2T) and
actualknowledge2tensor (@2T), but on the other, the dimensionality and the
23 Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean (2013) “Efficient
Estimation of Word Representations in Vector Space”, in
https://arxiv.org/pdf/1301.3781.pdf
shapes of each tensor should provide mathematical integrability between

each encoding.
The SY2T would encode the syntax well formed formulae of words and
expressions. The SE2T would encode the graph content of terms and
expressions that are the results of the three semantic interpretations, in sense,
in reference and in truth-value, that comprise the semantic interpretation in
meaning. The L2T would assemble semantically interpreted SY2T formulae
into argument premises according to the rules of introduction and rules of
discharge of logical connectives. The @2T would encode the interpretation
of expressions with respect to actual world knowledge.
The semantic space SGS that would enable the SA system but that would
also host the @WK training data would best be formalized as a highly
expressive graph space with operanda, syntactic and semantic, as nodes and
operations as edges, where the 2nd order logical functionality would be
introduced by allowing edge to edge connections that would represent
predication of predicates, but would also allow for the natural formal
expression of universal and existential quantifiers. Expressions of all kinds
in the SR2 system, linguistic, syntactic, and semantically interpreted, would
be graphs and graph-chains that would be attributed a number of indices,
which would express the different systems’ properties and roles. In SGS, we
introduce a syntactic index for well-formedness, possessed by graph g and
not, say, graph e, a semantic index that expresses sense and another semantic
index, that represents reference (pointing to the graphs that express the
reference and negating a number of graphs that oppose this reference
interpretations), logical index for a graph that is a valid premise in a valid
argument A and a logical index for a graph that is a premise in a sound
argument S. Sound arguments thus would be ordered sets of connected
graphs in SGS and sound reasoning would be an ordered set of graphs where
every graph represents either a true premise or a true conclusion.
In the suggested DRL HL-AGI model RAISON, all of the above abilities
would be abilities of the agent who would operate in an environment
segmented in syntactic layer, semantic layers, reasoning layers and actual
world data trained or educated layer. The model needs to learn a high-level
structure that emulates the function. The elements of RAISON are
172 Chapter IX
integrated in the general space of the DRL model, which consists of an

agent, environment, states of environment, actions of the agent and a reward
signal. We also highly desire the agent to be capable of learning from self-
play, just like the immensely successful DRL models of AlphaZero and
MuZero demonstrated.24
The DRL Structure of the Syntax Ability

In reinforcement learning the deep neural network (DNN) integration can
be extremely useful in cases when either the state space of the environment
is very large, or when the action space of all available to the agent actions
is very large; or both. For our concrete purposes here we need first to identify
how the syntax ability could be modeled in the DRL setting and, in
particular, how the deep neural network can serve in this task. In the general
case, the DNN can be learned to map environment states to values, in the
role of value function approximator; or it can learn to map state-action pairs
to quality or Q values.
In the case of the syntax ability we need first to identify what would be the
syntax environment of the agent, what would be the states of the
environment and what be the actions of the agent in it. The very agent would
be a syntax agent and his agency role in the general DRL model of the HL
rational agent would effectively be a one of a syntactic avatar which is
integrated with the general agent and mediates between it and the syntactic
environment. The environment would be the complete space of available
ordered syntactic elements, like well-formed formulae and non-well formed
formulae. However, since the syntactic function is inseparable in the
integrated functionality of the AGI-DRL agent, and since in real life
scenario humans never face a syntactic only environment, but face a
naturally linguistic environment of expressions, in order for the syntactic
agent to contribute to the DRL agent in a human-like way he would need to
24 See David Silver, Thomas Hubert et al. (2017) “Mastering Chess and Shogi by
Self-Play with a General Reinforcement Learning Algorithm” in
https://arxiv.org/pdf/1712.01815 and Schrittwieser, J., Antonoglou, I., Hubert, T. et
al. (2020) “Mastering Atari, Go, chess and shogi by planning with a learned model”
in Nature 588, 604–609.
face the same linguistic environment as the one the DRL faces. Because its
task is only syntactic, upon observation of a fully linguistic environment
(FLE – an environment of all available NL expressions that follow syntactic
rules, receive semantic interpretations and participate in reasoning
algorithms), that is, an environment of expressions such as the one humans
face in real-time dialogues, speeches, or readouts, the syntactic agent would
need to extract only syntactic features and structures from it.
The approach to construct a text environment within a deep reinforcement

learning model is certainly not novel. Ji He and al. used a similar approach
for constructing the novel deep reinforcement relevance network (DRRN)
for evaluating its performance in text games.25 They note the immensely
large size of the state space times action space table for “vanilla” Q-learning
recursion for any such pair of large spaces. He at al. use two DNNs, one for
“state text embedding” and the other for “action text embedding”. We see
that in He’s setting, both the states of the environment and the actions of the
DRL agent are represented as text expressions in a natural language. The
agent-environment interaction in their DRRN model is performed by the
sequence of observed text states and executing an action text-state that
changes the text-structured environment. The agent needs to learn the most
relevant responses, action-texts, which, set in a text-game model, are
generally analogous to a human-computer interaction, including the one we
see in a classical Turing test setting. I find this approach generally applicable
to our technical task of embedding in a syntactic DRL setting, where we
will embed logical formulae as states and actions.
If we aim to model an episode of syntactic agent interaction with the text

environment in a text-game-like scenario where the DRL agent listens to
and talks to another agent, human or artificial, the environment should
consist of all linguistic expressions rendered during this episode and the
“rules of the game” would be the totality of syntactic, semantic and
reasoning rules. For the syntactic goals of the agent, the syntax of the logical
25 Ji He, Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng and Mari
Ostendorf (2016) Deep Reinforcement Learning with a Natural Language Action

Space in: arXiv:1511.04636v5 [cs.AI] https://doi.org/10.48550/arXiv.1511.04636 ,
last accessed 17.06.2022.
174 Chapter IX
form of the expression-state would need to be extracted by formalization.

This task contains the trained ability to recognize the form, which could be
performed by the syntactic DNN. The input of the network would be the
observed state-expression of the environment and the output would be its
logical form. The logical form would thus be available to the general DRL
agent for integration with semantic interpretations and reasoning operations.
The ability of a syntactic agent would be as syntactically human-like as

practically possible to emulate. It would need to be able to recognize wffs
from non-wffs, it would need to recognize the type of a concrete wff, for
example JLJ, as found in the linguistic expression “John loves Jane”.
Besides this, the syntactic agent would need to deliver points of integration
of the syntactic output with the semantic environment and the reasoning
environment, with respect to the semantic agents and the reasoning agents,
to be defined below. Thus, the agent would observe and act in a syntactic
environment that orders all syntactic material, available for the game: a
communication session, Turing set dialogue, or a history of such dialogues,
analogous to the history of played games in the same way AlphaGo played
with itself, via self-play, an enormous number of games. Thus, the syntactic
material would form the state space where the agent would observe one state
per observation and would act upon it via his action, that would change the
syntactic environment and would provide the next syntactic state for
observation.
The syntax ability would be modeled by a deep neural network that would
take as an input naturally linguistic expressions and would learn to
recognize its logically formalized form solely on the basis of its syntax. In
the DRRN model of He and al. they use two separate DNNs, one for the
states of the text environment and one for the text actions, due to the
expected significant difference between the two inputs in terms of
complexity and length. In our syntactic model, we do not need to account
for such differences and we can therefore use one DNN that would take both
states and actions as inputs. Our initial choice for syntax would be 2nd order
predicate logic, which provides immense expressivity, sufficient for
formalizing scientific expressions but also, and more importantly, it mirrors
human linguistic practice more naturally than other logical formalisms
because it allows quantification and predication over predicates. Again, the
rigor of the system would be available to the DRL model, but on the surface,
in its mirroring human behavior, expressivity would need to dominate over
rigor.
The DRL System: Environment, States, Actions

The space of states of the general RL environment would actually be ordered
expressions. The order expresses the three kinds of relations that hold in NL
expressions: the syntactic, the semantic and the logical. Inferences would be
sub-states of selected by the reasoning engine premises that can or cannot
contribute to conclusions, where falling short of reaching a conclusion on
the basis of a given set of expressions, taken as premises candidates, would
be substituted by a different but yet following conclusion. The conclusion
would be of the qualificatory kind “given premises [p1, p2, … pn] I cannot
conclude that x” or “given premises [p1, p2, … pn] I can conclude that y”.
The environment in our setting would be the initially unordered set of all
available text expressions to the agent. The DRL system would use a deep
neural network with an input layer that would take as input data the state of
the environment, which the agent would “observe”. In our case, this would
be the equivalent to the “move” of the computer in this “game”: in a Turing
set dialogue, the computer would utter an expression and that would be the
move of the machine in a gamified setting. The move would be a novel and
last expression, which appears in the environment and which would change
its state. The agent would take the move within the general state of the
environment as an input and would employ the neural network to output a
tensor of “move” probabilities for his actions and a scalar value for the
expected “outcome” from the new state of the environment which would
include its action.
The success of the agent response would be the adequacy between the
agents’ response and the sequence of the observed state-response pairs. This
adequacy would be optimized for syntax adequacy, semantic adequacy and
logical adequacy, by the corresponding layers of the model.
The action of the agent is also an expression and is analogous to the

movement of the DRL agent in a typical game setting. The task of the agent
176 Chapter IX
would be to construct a response expression. The choice of the construction

would be from the available options, but the response would be integration
from the three layers: syntactic, semantic and reasoning. Thus, from the
syntactic layer, the agent would have, say, a thousand options of well
formed logical formulae. The best chosen one would be the one that receives
the highest reward for syntactic response success. The semantic choice in
sense would be a set of options derived from the semantic graph space with
a special accent on the sub-space defined by the response of the machine.
The reference choice would be from a set of possible options from the
semantic space that would fix the references of the singular terms and the
reference of the expression.26 The reasoning choice would be to construct
the agent’s response as the one logically best related to the response of the
machine response.
The Structure of Semantic Ability: Frege’s Model of Sense

and Reference
In the case of the semantic ability we need first to identify what would be
the semantic environment of the agent, what would be the states of the
environment and what be the actions of the agent in it. The agent here would
be the semantic avatar of the general DRL agent and his agency role in the
general DRL model would be semantic. Its output would be integrated with
the other layers’ outputs by the general agent and it would mediate between
the syntactic environment and the reasoning environment. The semantic
environment would be the complete space of available ordered semantic
elements, which would be loaded with the complex and highly responsible
obligation to deliver the elements of natural language semantics and to
execute the semantic interpretations of both text-states and text-actions in
the general DRL model.
An important qualification is due here. As it is evident across several

scientific domains let alone across historical traditions, in particular the
26 Frege’s reference of an expression can best be formalized in DRL as a (sub)state
of the environment. An English translation can be found in Gottlob Frege (1948)

“Sense and Reference” in The Philosophical Review, Vol. 57, No. 3 pp. 209-230 (22
pages), Duke University Press.
tradition of philosophy, which deals with key semantics as meaning,

significance and truth, there is no predominantly accepted theory of
meaning and truth. No theory that has been both accepted by the research
community in any involved discipline and successfully implemented into an
AI system that even loosely resembles a semantically capable machine
functionality. Instead, what we see everywhere, is an intense and heated
inter-disciplinary and, much more rarely, a cross-disciplinary debate on the
true theory of truth and meaning of expressions in natural language. Thus,
an AGI architect can either choose to devote his entire academic life to
participate actively in this debate, which would certainly prevent him from
building an AGI system whatsoever, or attempt a much more practical
approach and see what works best in implementation and use it as a
justificatory, albeit practical, foundation to rise the AI implemented theory
of truth and meaning in rank among the leading theoretical rivals, which,
still fall short of delivering a successful AI implementation. To sum it up: if
a certain theory of truth and meaning demonstrably performs on a human-
like level in a certain AGI system with Turing test potential, this could be
used as a side argument to claim primacy among rival theories until they do
not demonstrate equal or better results in a competitive system.
In my research field of analytic philosophy of language, I chose Frege’s

influential theory of sense and reference 27as having a high practically
implementable potential in an AGI system. Here I would avoid completely
any stance in the ongoing debates on it, which would have been a conditio
sine qua non if the purposes of this paper were theoretical. My purposes
here are, however, entirely practical: the semantic success of the actual
model, when developed in code and trained on data, would become
measurable. If the system proves successful, this could expectedly be used
as a novel argument in the modern philosophy of language debates on truth
and meaning.
27 For my stance on Frege’s theory of sense and reference I am very much indebted
to my doctoral supervisor and mentor, Nenad Miscevic, as well as to Katalin Farkas
and Howard Robinson from the CEU, and, later, to Gabriel Uzquiano and Ofra
Magidor from the University of Oxford. I am also indebted to the late Sir Michael
Dummett, whom I had the chance to listen to on a number of seminars at Oxford just
a year before his passing.
178 Chapter IX
The semantic agent would interact with a semantic environment, where the
states of the environment and the agent-actions are semantic. In order for
the semantic environment to be as human-like as practically possible, it
would need to provide recognition of semantic truth and semantic meaning.
Human linguistic practice is devoid of truth determinacy and meaning
determinacy because we almost never find in real life human practice truth-
rigorous expressions and meaning-rigorous expressions. In Frege’s theory
of sense and reference, meaning of linguistic expressions can be analyzed
in terms of sense and reference and this dynamic duality exhibits the
potential of expressing a significant volume of real life human language
interactions while having the potential for actual rigor in truth and meaning.
The semantics in this layer is present in the observed text-state of the

environment and to illustrate this let us choose a well-worn example from
Frege’s discussion on identity statements: “Hesperus is Phosphorus”28 (ST).
If we take this expression with a semantic focus, our semantic agent would
need to perform the following semantic tasks:
1. To recognize the semantics of the individual terms in ST (Hesperus,

is, Phosphorus)
2. To recognize the semantics of the statement in ST
3. To relate (1) and (2) to the semantics of the history of the
environment (the dialogue so far or the actual world data “educated”
part of the larger environment)
4. To relate (1) and (2) to the semantics of the history of the
environment (the relevant semantic contexts of the intended usage of
the terms and the semantic intention of the statement)
These tasks are key to human-like reasoning via language and the first acts
that build up the human understanding ability. In this case, the semantic
observation of the text-state by the semantic agent would need to understand
the meaning of the terms and then to understand the meaning of the intended
statement of ST. The understanding of the terms would consist in identifying
in our semantic graph-modeled space (SGS) the concepts of the terms and
28Gottlob Frege (1948) “Sense and Reference” in The Philosophical Review, Vol.
57, No. 3 pp. 209-230 (22 pages), Duke University Press.
the connections with other concepts in other expressions that convey best
the meaning they have in ST. Thus, the meaning of every term in SGS would
be the node of the term (the node for “Hesperus” or the edge of “is”, as
operandae are modeled on graph nodes and operatii are modeled on graph
edges) and the graphs, which employ it in other expressions. The totality of
actual expressions with the node-term would represent the totality of the
human practice of the term. The sub-space of the usage of the term in typical
contexts, which form a certain meaning of the term (any term can have
numerous meanings, some of the very distinct and remote from others), are
represented by the connected graphs of the expressions, where graphs are
expressions in the form aRb where a and b, as operandae are nodes and R,
as an operatio, is an edge.
The SGS graph space would represent the semantic source available to the
environment, in our DRL model, but also to the general RL agent. In order
for any semantic agent to be able to perform a semantic interpretation, it
would need access to the SGS.
To give a cursory illustration of the meaning of the term, the name

“Hesperus” in SGS, we can assume that SGS already has a trained structure
and provides us, or the agent, upon a query, with the following list:
1. “Hesperus” – node, operandum, name

2. Total connected edges for graphs – a natural number n
3. Definition graphs: star, planet, physical, radiating light, rises in
morning, sets in evening, etc.
4. Expressions: a natural number m
On the basis of these expressions, the agent can retrieve the actual meaning
of the term in human practice (we assume that SGS has been trained on
actual human practice). Graph selection can serve to form the sense of the
term and a ST or the sense of an expression. It can also serve to fix the
reference of the expressions. After the semantic interpretations in sense and
reference, the agent can perform the semantic interpretation in truth-value.
180 Chapter IX
Frege defines the sense as the content of the cognitive result upon
understanding the linguistic expression by the human agent.29 The sense can
be exchanged with a fellow conversant in the same language or translated
into a different language. The sense determines the reference (the object or
state of affairs) to which the linguistic expression applies in a desirably
unique way. To use Frege’s own illustration with the identity statements,
which have a simple yet immensely significant structure of two names or
descriptions being connected by a single linguistic identity operator like “is”
or “are”: if we consider the two different expressions “Venus is the morning
star” (Vm) and “Venus is Venus” (VV) we observe that they differ
significantly in their informative content even if they actually refer to the
same object, the planet Venus. This and many similar cases raise a puzzle
about meaning, which requires an explanation by a successful theory of
meaning. Frege’s explanation distinguishes between two elements of the
meaning of the expressions. The first one is the sense of the meaning, which,
in this puzzle, carries the load of the difference in informational content,
evident from the different syntactic structure of the expressions, which
differ in syntactic content and order. The second one is the reference of the
expression, which in the puzzle happens to be one and the same object. The
identity in the meanings of both expressions is explained by the identity in
reference, while the difference in the meaning of the expressions is
explained by their different senses.
The sense, if it manages to determine the reference successfully so that the

act of referring is performed, would satisfy a necessary condition for the
referring; but not a sufficient condition. There is nothing in the language
practice that requires the conversant to formulate the sense in a way that it
would require or guarantee the reference as an accomplished act. To
illustrate, we might speak meaningfully, that is, with a well-defined sense,
of quantum particles like the graviton, included in the standard model of
quantum mechanics, and yet we will fail to fix the reference and thus to
accomplish the act of referring, since science still does not know if gravitons
exist at all. The act of sense formulation only manages to form a beam of
semantic light that begins to search for its defined reference, in a process
29 Ibid.
that sometimes captures it, as in expressions like “John loves Jane” and
sometimes fails, as in “The car’s engine released a lot of phlogiston” where
there is no real object “phlogiston”, referred to by the term. In mathematical
cases like “5+7=12” or “5+7=10”, which mathematicians do find non-
trivially distinct, in terms of truth and meaning, the inquiry into the referents
of purely mathematical expressions continues the heated debates even
today, as philosophers of mathematics clash their preferred views within the
frames of Platonism, structuralism, formalism, fictionalism and other
influential stances.
We need to distinguish between an objectively occurring reference, that is,

a reference that has happened as an act and an intended reference, which is
the reference the conversant has in mind, when she formulates the
expression and encodes it, within her linguistic abilities, in the expression.
The two are evidently not identical. An actual act of reference might happen
despite the sense, due to an ambiguous encoding, for example. And the
intended in the sense reference might land on a different, unintended object.
For example, the conversant might say “The balloon is growing in size”
intending to refer to an actual latex inflatable balloon. But if in a written
context, for example, written next to a financial column in the newspaper, a
reader might mistakenly understand it as referring to an inflation balloon. In
these and similar cases the act of understanding of the reference by the
listener furnishes the act of the referring of the expression and the term. This
happens according to Frege’s context principle, according to which every
single term receives its sense only within an expression.30 The listener takes
the reference as an inflation balloon, whereas the speaker intends the term
in the expression to refer to a latex balloon. We have one expression, one
intention, and yet two references, one of intention and one of comprehension.
Both are essential for successful communication, but only the second is a
communicated accomplished referring act.
We can observe that the sense can be formalized as a recipe for an operation
and thus as the initial step of a semantic algorithm, where in a finite number
30 An illuminating analysis on Frege’s context principle and the closely related

principle of compositionality can be found in Linnebo, Øystein (2008)
“Compositionality and Frege’s Context Principle” in Philosophy.
182 Chapter IX
of distinct steps the semantic definition, interpretation and operation

execution are performed. We can generally define the semantic algorithm
as having the following form:
Semantic Algorithm
1. Intention of the speaker
2. NL encoding of intention following a syntactic selection of symbols
and an order that holds between them with a semantic load of the
terms
3. Formation of expression
4. Formation of intended reference
5. Utterance
6. Acquisition by a listener (or the computer)
7. Decoding of the symbolic structure
8. Semantic interpretation in meaning: decoding the acquired sense
9. Semantic interpretation in reference: attempting to comprehend the
intended reference
10. Act of referring after (9)
11. Semantic interpretation in truth value: the listener is able to assess
truth of the expression on the basis of (8) and (9)
12. End of communication cycle (1 - 11)
We see that in the SA, the steps contain both operandae and operatii. This
structure makes it quantifiable for the purposes of the DRL model.
From the perspective of implementation, it is significant to observe that the

Fregean sense of an expression can be expressed as a semantic graph in a
semantic space where nodes express concepts-operandae and edges express
concepts-operatii. Such formal expression can be particularly useful in the
task of training the semantic agent, but also, to provide the formal structure,
which formally expresses a non-trivial graph isomorphism between natural
language and mathematical expressions. This space can also be especially
useful for an agent's memory, as a training database and as a knowledge
base, where it can serve for the actual world data training of the DRL agent.
The semantic graph space can provide the expressibility, the necessary rigor
and the integrability with syntax, reasoning and rationality necessary for the
“education”.
Frege’s theory of sense and reference can systematically resolve ambiguity

cases through its power to identify the reference of a term, the reference of
an expression and an order that holds between the references of the primitive
terms in the expression and across expressions. The theory is also capable
of identifying well-formedness of syntax and semantics via sense
formulation and via its figuring in an expression. It is also capable of
identifying the act of referring as a separate stage process with distinct steps
with measurable states: definition of reference, intention of reference,
identifying the object of reference, interpretation of reference, incomplete
act, complete act, wrong reference and successful or unsuccessful
communication of the reference to a conversant. Each step and state of the
term or expression can be identified and assessed. These capabilities provide
a powerful semantic toolkit for artificially modeling the semantic behavior
of humans.
Atoms of Reasoning
The first and minimal syntactic act of reasoning is, formally and
syntactically, the predicate attribution to a bearer of a predicate, a subject,
or, in 2nd order predicate logic, a predicate as well. This predication is also
the first and minimal expression. The syntactic consideration of a single
syntactic element, of any kind (subject, predicate, relation or quantifier) is
not sufficient to be accepted as the first and minimal syntactic act of
reasoning because it receives a good syntacticity or a syntactic well-
formedness only at the general level of predicate or relation attribution to an
x that is not itself the predicate or the relation and yet can accept it and carry
it. The availability of a single element allows for the first act of
quantification, since we can define both existential and universal quantifiers
over it and we cannot do it without the availability of an element. In order
to be able to form the quantification “There is an x” we need the x to be
present, for otherwise the expression could only take the form “There is”,
denoted syntactically by the existential quantifier symbol and this is not a
well-formed expression in logical formalism, which also, for that very
reason, fails to be able to carry any meaningful semantic load. The
184 Chapter IX
introduction of quantifiers thus becomes first possible only after our syntax
provides us with an x the quantifiers can range over. After the quantification,
the element, however, is not a single element anymore: we have two
elements, a quantifier and an x, connected in a syntactically proper way
which allows their concatenation to be regarded as the formal structure of
an expression. The expression represents a formal act of quantification,
which, when considered as a schema, has the same formally general form
as the schema of predication of a predicate to a subject.
The first and minimal semantic act of reasoning is grasping (comprehending,

understanding) the meaning of a linguistic term. In our model, based on
Frege’s theory of meaning, that amounts to grasping the sense of the term
and grasping its reference, or, rather, attempting to grasp the sense and
attempting to grasp the reference, since nothing requires or guarantees that
either would be successful and finished. The second semantic act of
reasoning is grasping the meaning of an expression; and again in our model,
this amounts to attempting to grasp the sense and the reference of the
expression. But at the level of semantic interpretation of an expression, a
novel possibility emerges, unavailable so far: an interpretation of the
expression with respect to its truth-value. Again, this can only figure as an
attempt, for in a real life linguistic situation, a truth-value interpretation can
be any of true, false, undetermined, or unavailable as a procedure. The third
and last act of semantic reasoning is the semantic ordering of an expression,
along with its constituent terms and their senses and references, with respect
to other expressions and their terms, senses and references. This order, in a
text body of any kind, acquires its own sense and own reference, as well as
its own truth value. This furnishes the general semantic structure of
reasoning based on meaning, structured by sense and reference and
complemented by a semantic interpretation in truth-value.
The availability of two distinct semantic elements, carried by different

syntactic strings, allows all logical operations as firstly possible: negation,
conjunction, disjunction, implication, identity. The availability of elements
and the possibility to execute logical operations over them allows well-
formed linguistic expressions to be analyzed both syntactically and
semantically. This schema also allows for the elements of reason to be
identified, analyzed, produced, and exchanged in a conversation-like
setting. For example, the singular term “John” does not form a meaningful
expression and cannot be regarded as expressing something beyond itself.
By itself, it does not provide us with the possibility to quantify over it and
to predicate it: we have no quantifiers and no predicate terms available. So
no introductions of logical operators can be performed on “John” unless we,
before that, make available a second term, like “sings”. The availability of
two terms allows all of the above that can give the rise of a syntactically
well-formed expression “John sings” that expresses something beyond what
the singular terms express: it expresses an order that holds between them.
This order is encoded by Frege’s sense and fixes the references of the
singular terms and the expression.
This is highly non-trivial and represents the emergence of the semantic

meaning of an expression, as opposed to the semantic meanings of the two
singular terms, “John” and “sings”, the expression becomes meaningful
semantically. The term assembly allows, for the first time, the functioning
of the elements of reason. For example, we can affirm that “John sings” and
by the act of affirmation or mere utterance we make the expression available
for semantic interpretation with respect to the meaning of the expression.
We can examine its meaning in accordance with our preferred theory of
meaning and see that the singular terms denote a person, John, who acts, as
denoted by “sings”. This is minimally the sense of the expression, conveyed
to the conversant and which also defines the state of affairs, which would
represent the reference of the expression. Singular terms have singular
references, John refers to the person x such that x is a person, a human, male,
physical, living, etc., who is connected in the depicted by the sense real life
state of affairs to an act of singing, which refers to producing sounds, that
have a melody (presumably), with the intention of expressing an emotion,
etc.
The connection between the reference of the name John and the reference
of the act of singing in real life furnishes the state of affairs, which should
provide the reference of the expression. Our initial expression “John sings”
gives no information as to what other things John does or is: we have no
information in the sense that he is a man or a dog, etc., but in a context we
can infer those from the context and see that the utterer implicitly means
that John is a man. Once the sense portion of the meaning of the expression
186 Chapter IX
is sufficiently interpreted, further steps can be added or modified later for

the semantic interpretation is an ongoing process for the duration of the
conversation. New and relevant expressions can relate to “John sings” and
thus add to, take from and modify its sense and its reference.
We can now, armed with our sense, enter the domain of referential
interpretation, say the set of observational data that we have, and investigate
it as to the reference state of affairs of the expression. We should first look
for the existence of the referents of the singular terms, John and the singing.
If we find them in the data, we should investigate if an order holds between
them and if there is one if the order is the one described in the sense. Thus,
we can find in the data both John and singing, but we can also happen to
find Jane and, say, the only order that we can find, is an order that holds
between the singing and Jane and not John. In this case, we would have
found the referents of the singular terms, John and the singing, but not the
presence of the sense prescribed order to hold between them. Our semantic
interpretation of the expression would be that referentially John does not
sing and consequently, the expression, while well formed semantically, does
not express the true state of affairs that actually obtains. This would
represent the semantic interpretation of the expression in truth-value, where
we have approached and examined the expression with respect to its truth
or falsity. Note that being meaningful does not at all lead automatically to
truth, but allows for an interpretation in truth-value.
To take another example, the expression “5 + 7 = 12”. The sense of the

expression directs its meaning toward an interpretation, which defines the
expression as formed within a concrete domain of description, mathematics.
The domain comes with all its rules, elements and practices and therefore,
an adequate interpretation of the expression should take place within it. The
sense of the expression would be its examination with respect to the nature
of the elements [5, +, 7, =, 12] and the referential interpretation would need
to look for term and sense references within mathematics. What is also of
interest here is that the sense of the expression defines the conditions for the
reference examination: we should not look for the element “5” availability
in the real life physical world, say, on the street, as we did when we looked
for John. Instead, knowing that 5 is a (natural) number the adequate place
to look for it are the sets of numbers. Those sets come with their rules of
existence onboard, established by the historical practice of mathematics and

number theory in particular.
Other practices are also relevant and inform the domain of numbers, like
foundations of mathematics and mathematical epistemology. From them,
among others, we know that numbers, even if being all the time related to
the real life states of affairs, as in expressions like “John sang two songs”,
do not by themselves inhabit the physical world but only the abstract,
causally inert, outside physical space and time domain of abstract objects.
Abstract mathematical objects, like numbers and functions, can be applied
to non-abstract objects, via a non-trivial process of application;31 non-trivial
because it combines two distinct domains with very different rules of
meaning construction, meaning examination and most of all – reference
examination. The main epistemological method in the empirically
accessible domain, such as the physical world with (some, and far not all,
as quantum mechanics revealed to use) its states of affairs, is empirical
observation and measurement. This is the main epistemic tool of natural
sciences 32 and common sense knowledge acquisition, the empirical method.
Whereas in abstract domains such as the ones of pure, unapplied
mathematics, logic and metalogic, we have very different conditions of truth
and meaning, which are impenetrable for the empirical method 33. Thus, if
we want to be able to interpret purely mathematical and logical expression,
in meaning and in truth value, and especially, with respect to their delivering
31 Perhaps the best place to see a still very much up to date discussion of the
applicability of pure mathematics is the classical paper by Wigner, E. P. (1960) “The

unreasonable effectiveness of mathematics in the natural sciences. Richard Courant
lecture in mathematical sciences delivered at New York University, May 11, 1959”.
Communications on Pure and Applied Mathematics. 13: 1–14.
32 As I have argued elsewhere, not the only tool of scientific epistemology, see
Grozdanoff, B. [2014] A Priori revisability in Science, Cambridge Scholars

Publishing. Newcastle upon Tyne.
33 With the exception of theories like John Stuart Mill’s and Philip Kitcher’s, as well
as Penelope Maddy’s. For their views see: The Collected Works of John Stuart Mill,
edited by John M. Robson (Toronto: University of Toronto Press, London:
Routledge and Kegan Paul, 1963–91); Philip Kitcher (1980) “Arithmetic for the
millian” in Philosophical Studies 37 (3):215 - 236 (1980); Penelope Maddy (1992)
Realism in Mathematics, Clarendon Press; James Robert Brown (2008) Philosophy
of Mathematics: A Contemporary Introduction to the World of Proofs and Pictures,
Routledge.
188 Chapter IX
knowledge, we need a different tool than the one we use in the empirical
domain. We cannot observe numbers (perhaps only in a neo-Platonic
epistemology of mathematics or in a neo-formalist kind of epistemology of
mathematical knowledge as the one influentially proposed by Charles
Parsons34) the way we observe birds and cars, but we nevertheless know that
humanity has an enormous body of truths about them that are considered by
science firm truths that bring about knowledge to those who are lucky
enough to gain access to them, if not even firmer than the ones of physics.
The reference of a singular term, as it is considered by itself and as it figures

in a well-formed sentence, is not atomic. That is, it is not functionally and
structurally identical to any other reference of a term used in the same way.
This difference can be best captured as a difference in content but also as a
difference in function and as a difference in structure. We should
distinguish between distinct stages in the process of reference, which
furnishes an event of a semantic nature. Thus, roughly and not exhaustively,
we can distinguish between the following stages of meaning formation:
1. Formation of sense
2. Formation of reference
3. Execution of sense – via uttering, communicating or formalizing
4. Execution of reference – act of intended reference, act of fixing
reference (by conversant)
(4) obtains when the sense and the reference have been well-formulated and
an interpretation of both has been executed. To use a loose analogy from
quantum mechanics, we can approach the wff sense and reference as a
quantum object that remains in a certain state while unobserved, that is,
while not semantically interpreted. In QM, any observation of a quantum
object collapses its wave function and thus changes the parameters of its
physical state, and correspondingly, values that figure in its mathematical
description. In our “quantum sentence” case, we can assign a structure of
quantification that would have one set of values while the sense of the
34Charles Parsons (2007) Mathematical Thought and its Objects, Cambridge

University Press.
sentence is in an unobserved state, or in an uninterpreted state and another

after observation. The same can be done for the reference of the sentence.
Any interpretation of the sense changes the formal uninterpreted state. For
example, the semantic load of the interpreter is anything but trivial: if the
sense of the term Venus as Hesperus is the only one used by the formulator,
and Venus as Phosphorus is the only one available at the interpretor, we will
end up in a miscommunication situation, where same string of symbols (for
Venus) is intended as Hesperus by the formulator and interpreted as
Phosphorus by the interpretor. Thus, the identity of syntax would not be
sufficient in order to convey the intended sense to a user different from the
formulator and this would prevent the communication with a conversant.
And we need not even agree that the main function of language is the
communicative function.35 We should only agree that the communicative
function of language is non-trivial for its emergence, existence and usage.
Thus, we can distinguish between functioning structures on the layer of the

sense of formulator and on the level of the sense as interpreted. The same
distinct structures can be found immediately on the layers of the reference
as brought about semantically by the sense of the formulator and the
interpreted reference by the interpreter. In our simple case, we see that we
can arrive at a structural version of the Frege’s puzzle on the dynamic level
of communication: the interpreter fails to acquire the intended formulation
due to distinctly different senses and references even given the same syntax
of the wff of the formulator and the wff of the interpreter. Therefore, we
should formulate a set of measures that would eliminate or diminish the
probability of such miscommunications at the core of our AI rational engine.
The first and minimal logical act of reasoning is the first logical operation
over a logical operandum. This could be, on an intra-expression level, the
attribution of a predicate to a subject (“John is tall”), the quantification over
subjects (“there is an x”), or quantification over predicates (“for all x” where
x is a predicate). But on the inter-expression level, the logical acts of
reasoning emerge via the usage of logical operators between expressions
35 Halliday, M.A.K. (1973) Explorations in the Functions of Language. London:
Edward Arnold.
190 Chapter IX
(“P & Q” where P and Q are expressions). Those become available only
when we first have the expressions, syntactically well formed and
semantically interpreted. Once we have a number of available expressions,
like the text-actions in the DRL model, we can use the rules of introduction
and discharge of logical operators to establish order onto them. Thus, if we
have P, Q and R, we can introduce conjunctions to form “P&Q” or “P&R”
or “P&R”, etc. We can further develop logical complexity by using the
forms of logical laws (modus ponens, modus tollens, etc.) and attempt to
find if one of their forms holds for our available expressions, given their
syntactic well-formedness, and their semantic interpretation.
If we manage this, we would have reached a logical conclusion of some

form, where we can use selected expressions as premises of a logical
argument in order to deduce a conclusion. The logical conclusion is, since
the times of Aristotle, the best candidate for a reasoning activity of humans
with naturally linguistic abilities. If the premises are not interpreted with
respect to their truth-value but the conclusion follows from them with
necessity, we have a valid argument. If we interpret the premises with
respect to their truth-values, given the valid argument, we have arrived at an
immensely valuable for human reasoning, rationality and knowledge sound
argument, where the truth of the premises is preserved via the validity of
the argument into a truth of the conclusion, allowing reasoning with truths
and arriving at novel truths. This ability, following the tradition of western
philosophy, I take as the highest form of human rationality; the ability to
arrive at truths by reasoning with truths, which, if emulated by DRL, would
provide us with a very strong reason to accept the DRL model as true
human-like artificial intelligence.
The DRL Structure of the Reasoning Ability

The reasoning ability (RA) represents human’s real life practice where we
reason within natural language. Generally, this would be the ability to
operate with syntactically well-formed linguistic expressions, to interpret
them semantically, and on the basis of these to introduce and discharge well-
formed atomic formulae and to concatenate them into molecular wffs. All
this would allow the system to construct ordered chains of expressions
following the logical forms found in human practice: the logical laws, such
as modus ponens and modus tollens and others. Most of all, the system
would acquire the ability to reason syllogistically, to draw logically valid
conclusions and to assess logical arguments as valid, invalid, sound or
unsound.
The observed text-states from SGS, ST, can now, after a full and successful
semantic interpretation in meaning, be fed to the reasoning ability of our
DRL agent. Each term, formalizable syntactically can be retrieved from
SGS and presented to the reasoning engine for logical operations. Thus, if
we have as a ST the expression “Hesperus is Phosphorus” we can send it to
the reasoning engine, along with all relevant definitions for the terms and
connected expressions and we can attempt to reason about it. For example,
the agent already has the results of the semantic interpretations and knows
that the referent r of both terms is actually one and the same, the planet
Venus. But he also knows that the statement uses two distinct meanings,
which even if they happen to refer to the same r are not identical. The graph
structures for the 1st sense and the 2nd sense do not coincide in SGS. The
agent can compile tentative sets of expressions, connect them according to
logical rules and see whether a law can deliver a conclusion from some
assembled lists.
Methods
The RAISON model is a deep reinforcement learning model, which would
develop the suggested abilities for syntax recognition, semantic
interpretations, logical reasoning and rationality. These abilities are very
distinct, but in order RAISON to function they need to be seamlessly
integrated. In the DRL setting, the abilities will be modeled as possessed by
the RL agent, which develops them and perfects them in the course of its
interaction with an environment. For the AGI purposes the environment
would possess a complex structure and would have two layers: the deep
layer,36 which possesses the syntactic, the semantic and the reasoning
structures, modeled by the semantic graph space (SGS), and the surface
36 Not to be confused with the “deep” notion from the deep neural networks (DNNs).
192 Chapter IX
layer, which is the portion of the environment the agents observes and with
which it interacts.
The agent would learn a strategy to find the most adequate text response-
actions, modeled by a policy. The policy in DRL is best found through deep
neural networks and especially so in cases, like the present one, where the
space of environment states or the space of agents’ actions is immense and
practically unknowable. The general deep neural networks (NG) of the
RAISON model are functions ƒș(s), where the learnable parameters are
denoted by ș and the observed by the agent state of the environment is
denoted by s; ƒș(s) takes natural linguistic text data as input and outputs the
probability of a text-response given state s: pa = Pr (a|s) and a scalar value
v, which assesses the quality of the adequacy of the response. Each ability
in the suggested architecture would have a dedicated DRL agent whose
expertise would be integrated in NG.
The syntactic ability would have a DRL agent ASY which, which interacts
with the environment but whose DNN, NSY, learns to recognize syntactic
formulae.
The semantic ability would have a DRL agent ASE, which interacts with the
environment but whose DNN, NSE, learns the human-like semantic
interpretations via dedicated deep neural networks. The semantic
interpretations to be learned by NSE are the above formulated interpretations
in sense, reference and truth-value. Each semantic interpretation would have
its own dedicated neural network, NSS, for the sense interpretation, NSR for
the reference interpretation and NST for the truth-value interpretation. Once
each semantic neural network learns its ability to perform adequate
interpretations in sense, in reference and in truth-value, these outputs would
be fed to the RL agent in order to choose an action, given the state of the
environment. He would grow to learn a strategy that would endow it with
the ability to “beat” the machine, that is, to deliver the most adequate
semantic text-response to the observed state of the environment, or the
“move” of the environment, in game terms.
Thus, the reasoning ability would have a DRL agent AR, which interacts
with the environment but whose DNN, NR, learns to reason like humans.
The reasoning ability, acquired by NR, would take as input syntactically

recognized linguistic expressions that are already semantically interpreted
(in sense, reference and truth-value) and would operate over them with the
logical operations of rules of introduction and discharge of logical
connectives in order to recognize logical forms. NR would output a number
of logically adequate (meaningful, true and relevant) expressions, ranked by
scalar values and these would provide the policy expertise that would be fed
to the AR agent, in order to come up with its best reasoning “move”,
effectively his reasoning response to the machine-player.
Thus, the rational ability would have the same DRL agent ARA, which
interacts with the environment, with the key difference that his DNN, NRA,
has been trained on immense volumes of actual world data which training
would render him immensely “educated”. NRA would take as input the
training data and it would output the semantic graph space, SGS, that would
receive upon training artificial education of real world knowledge. This
process would be the artificial analog of an artificial education of the agents,
that would draw on the trained SGS for real world facts.
The distinction between a reasoning agent and a rational agent would be that
the rational one is educated and knows many things, whereas the reasoning
agent only needs a minimal amount of training, which would make it able
to reason like humans. The rational agent would be a functional cognitive
agent in a possession of human-like knowledge.
Self-Play, Training Datasets and Hardware

The training of the syntactic ability would be executed by supervised
learning since the rules of syntax in 2nd order predicate logic are explicit and
as rigorous as they could be. The training of the semantic abilities of sense
interpretation, reference interpretation and interpretation in truth-value are
a function of the SGS space and thus they require some minimal structuring
with actual semantic data. This structuring would “carve” graph-represented
data for subjects, predicates, quantifiers and relations and would also be
executed by supervised or semi-supervised learning. The reasoning ability
would be learned by training on actual logical rules of introduction and
discharge of logical connectives, as well as on syllogism forms and laws of
194 Chapter IX
human thought. Again, since they are explicit and rigorously defined the
training would be performed by supervised learning. The last task of
training would be actual world knowledge training, or artificial “education”.
Due to the actuality of knowledge, the technique of training would again be
supervised learning.
The tabula rasa, approach, familiar from Aristotle’s epistemology and used
to great success by DeepMind in their DRL agents, if at all available, would
be present only within the agent-environment interaction, where the agent
would learn in a semi-tabula rasa fashion the environment of the conversant,
which would consist of the ad hoc naturally linguistic communication free
of any previous load but the choice of the environment. Those can be
performed artificially, emulating, say, a human Turing kind jury, by using
quantum randomness37 to determine the choices for the topic and the
directions of the conversation with the agent.
The enormously successful technique of self-play, familiar by a growing

number of impressive AI systems, like DeepMind’s AlphaZero and
MuZero, shows a highly non-trivial potential in training AI models. For our
purposes, the RAISON model would benefit enormously by a self-play, as
the environment is driven by quantum random choices of topics, themes and
text-actions that would provide the conversant side of the RL agent-
environment interaction.
The RAISON model can be trained on available NLP datasets, like (as of
2022) UCI’s Spambase, Enron dataset, Recommender Systems dataset,
Dictionaries for Movies and Finance, Sentiment 140, Multi-Domain
Sentiment dataset, Legal Case Reports Dataset, but mostly on the Wiki QA
Corpus and the Jeopardy datasets. The syntax training would be supervised
and the syntax rules would be explicitly trained, as well as the reasoning
rules.
The main challenge is the semantic graph space model SGS, the main layer
of RAISON, which would directly benefit from the NLP datasets training.
37 Most suitable would be the quantum random number generator or QRNG

hardware, that provides objectively random numbers in a variety of hardware form-
factors, including PCIe cards.
The semantic structures, that will emerge upon training the SGS space on
available big volume NL datasets would establish a significant number of
subject-predicate structures, predicate-predicate structures, the subject-
relations structures, predicate-relation structures as well as linguistic forms,
found in real life human linguistic practice. Those structures are available
in the datasets and would be embedded in the SGS space upon the training
that would seek the functionality, devised for the semantic module of
RAISON.
The RAISON model is devised as an AGI candidate and, given the

generality of its purpose would naturally require an immense computational
resources in order to be trained, tested and improved as well as to acquire
the artificial abilities. The RAISON model would need CPU or TPU
computational resources in order to perform the discrete interactions
between the agent and the environment. The most important computational
resource here is the number of threads that would allow the parallelization
of the computation. The accurate estimate of the number of CPU cores and
threads needed at this stage is difficult to arrive at but a multi-core and multi-
thread CPU supercomputer would perhaps be the only practical way to
develop, preliminary assess the expected performance of RAISON and train
its agent in the discrete tasks of environment observations and agent
responses. The other type of required computational resource are GPU or
graphic processor units that would handle the tensor multiplications
required by the dedicated deep neural networks of RAISON. In order the
policies learning not to become a computational bottleneck in the agent-
environment interaction speed, we would need a GPU supercomputer
matching the computational power of the CPU (TPU) supercomputer. The
GPU computational load is a function of the immense volumes of training
data, syntactic, and reasoning but mostly semantic.
The suggested AGI model has the main justification for its proviso structure
in the rich and critically evolved tradition of analytic philosophy, which is
reflected in the syntax, semantics, reasoning and relational abilities to be
learned. Due to the immense development and training work, the model
would be truly in a position to be assessed both as an AI potential and as
prospects for improvement only when developed in code and after a certain
preliminary but still significant training on large volumes of data.
196 Chapter IX
Bibliography
Alayrac, J.B., Donahue, J. et al. (2002) “Tackling multiple tasks with a
single visual language model” in https://arxiv.org/abs/2204.14198
Brown, J.B. (2008) Philosophy of Mathematics: A Contemporary
Introduction to the World of Proofs and Pictures, Routledge.
Halliday, M.A.K. (1973) Explorations in the Functions of Language.
London: Edward Arnold.
Etchemendy, J. (1999): The Concept of Logical Consequence. Stanford:
CSLI Publications.
van Fraassen, Bas C. (1971) Formal Semantics and Logic, The Macmillan
Company, New York.
Frege, Gottlob (1948) “Sense and Reference” in The Philosophical Review
Vol. 57, No. 3 pp. 209-230 (22 pages), Duke University Press.
Gettier, E. (1963). “Is Justified True Belief Knowledge?”. Analysis. 23 (6):
pp. 121–123.
Grozdanoff, B. (2014) A Priori revisability in Science, Cambridge Scholars
Publishing. Newcastle upon Tyne.
He, J. Jianshu Chen, Xiaodong He, Jianfeng Gao, Lihong Li, Li Deng and
Mari Ostendorf (2016) Deep Reinforcement Learning with a Natural
Language Action Space in: arXiv:1511.04636v5 [cs.AI]
https://doi.org/10.48550/arXiv.1511.04636 , last accessed 17.06.2022.
Hilbert, David (1984) “On the infinite” in Philosophy of Mathematics,
Edited by Paul Benacerraf and Hilary Putnam, pp. 183 – 201, Cambridge
University Press
Kitcher, Ph. (1980) “Arithmetic for the millian” in Philosophical Studies 37
(3):215 – 236.
Linnebo, Øystein (2008) “Compositionality and Frege’s Context Principle”
in Philosophy.
Lycan, W. G. (2006). “On the Gettier Problem Problem.” In Epistemology
Futures, (ed.) S. Hetherington, Oxford University Press.
Maddy, P. (1992) Realism in Mathematics, Clarendon Press.
Marcus, G. F. (2001) The Algebraic Mind: Integrating Connectionism and
Cognitive Science, Cambridge, MA: MIT Press.
Martinich, A. and Sosa, D. (eds.). 2001a: Analytic Philosophy: An

Anthology, Blackwell Publishers, and Martinich, A. and Sosa, D. (eds.).
2001b: A Companion to Analytic Philosophy, Blackwell.
Matsuoa, LeCun et Al. (2022) “Deep learning, reinforcement learning, and
world models” in Neural Networks, Volume 152, August 2022, 267-275.
Mill, John Stuart (1963-91) "The Collected Works of John Stuart Mill",
edited by J. Robson (University of Toronto Press, London: Routledge
and Kegan Paul, 1963–91)
Olivier, S. and Olivier B. (Eds.) (2010) Markov Decision Processes in
Artificial Intelligence, Wiley.
Parsons, Charles (2007) Mathematical Thought and its Objects, Cambridge
University Press.
Puppe, F. (1993) “Characterization and History of Expert Systems” In:
Systematic Introduction to Expert Systems. Springer, Berlin, Heidelberg.
https://doi.org/10.1007/978-3-642-77971-8_1
Reed, S. et Al. (2022) “A Generalist Agent” in arXiv:2205.06175v2 [cs.AI].
Schrittwieser, J. et al. (2020) “Mastering Atari, Go, chess and shogi by
planning with a learned model” in Nature 588, 604–609.
Schrittwieser, Silver et al. (2020) «Mastering Atari, Go, Chess and Shogi
by Planning with a Learned Model» in
https://arxiv.org/abs/1911.08265
Silver, D., Hubert, T. et al. (2017) “Mastering Chess and Shogi by Self-Play
with a General Reinforcement Learning Algorithm” in
https://arxiv.org/pdf/1712.01815 .
Sosa, Ernest (2018) Epistemology (Princeton Foundations of Contemporary
Philosophy, 14) Princeton University Press.
Sparkes, M. (2022) “Is DeepMind's Gato AI really a human-level
intelligence breakthrough?” in New Scientist, 19.05.
Sutton, R. and Barto, A. (2018) Reinforcement Learning: An Introduction,
Second Edition, MIT Press, Cambridge, MA.
Thoppilan, R. De Freitas et al. (2022) “LaMDA: Language Models for
Dialog Applications”, in https://arxiv.org/abs/2201.08239v3
Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean (2013) “Efficient
Estimation of Word Representations in Vector Space”, in
https://arxiv.org/pdf/1301.3781.pdf
198 Chapter IX
Wigner, E. P. (1960) “The unreasonable effectiveness of mathematics in the

natural sciences”, Richard Courant lecture in mathematical sciences
delivered at New York University, May 11, 1959”. Communications on
Pure and Applied Mathematics. 13: 1–14.
CHAPTER X
DISCRIMINATOR OF WELL-FORMED
FORMULAE AS A COMPONENT
OF ARTIFICIAL HUMAN RATIONALITY
DIMITAR POPOV
The contemporaneous progress of modern Artificial Intelligence (AI)

methodologies has had an unprecedented impact on nowadays industrialized
society. Such applications are heavily employed to predict, discriminate,
rule-govern and generate relations upon learning knowledge state
representation between entities in a given data set. Deep Learning (DL) has
enabled such representations to be integrated with automated reasoning
mechanisms, pursuing the goal of Artificial Human Cognition (AHC).
Although deep learning architectures have had significant success in
resolving well-defined tasks, where optimization criteria rely on mathematical
approximation between data inputs and data outputs, such approximations
remain a volatile candidate for achieving AHC, since rationality needs to be
persistent outside the training domain, but ANNs fail to adapt greatly, to
domain changes, domain crossings or just domain expansions.1 That is why,
the underlying building blocks of DL, Artificial Neural Networks (ANN),
although having tremendous power of discovering correlations between
data and system states, are poor performers when it comes to transferring or
applying bare abstract model representation to new tasks in order to achieve
robustness. Even more, the generalization outside the training set seems at
1Michael McCloskey and Neal J. Cohen, “Catastrophic Interference in Connectionist

Networks: The Sequential Learning Problem. Psychology of Learning and
Motivation,” Vol. 24 (January: 1989): 109–165. Roger Ratcliff, “Connectionist
Models of Recognition Memory: Constraints Imposed by Learning and Forgetting
Functions,” Psychological Review, 97 No. 2 (April: 1990): 285–308.
200 Chapter X
best debatable and only when certain caveats are applied 2before training.
Neural-symbolic computing has been an active area of research for many
years seeking to bring together robust learning in artificial neural networks
with human-like reasoning capabilities and self-explainability via symbolic
computational frameworks. The central aspect of such symbolic frameworks
is that intelligence results from the manipulation of abstract compositional
representations whose elements stand for objects and relations.3 In this
paper, I argue the foundation of a state-of-the art computational framework,
which relies on logical formalization and operation for reasoning and neural
networks for knowledge embedding. By knowledge embedding, we mean
the elicitation of features of a particular real-world object or concept in a
given domain, those features are then encoded inside the connections of the
NN, representing real world justifiable true beliefs. The utilization of this
encoding is transformed into a set of relational rules that collectively
represent the captured knowledge. The end goal of achieving systems with
better robustness, as it is believed to be one of the basic trends of Human
Rationality (HR), constitutes what Gary Marcus defines as:
22 Yann LeCun, Yoshua Bengio and Geoffrey Hinton, “Deep Learning,” Nature,
521(7553) (May: 2015): 436-444. Irina Higgins Loïc Matthey, Acra Pal et al., “Beta-
VAE: Learning Basic Visual Concepts with a Constrained Variational Framework,”
Proceedings of ICLR (2017), accessed May 3, 2022,
https://www.semanticscholar.org/paper/beta-VAE%3A-Learning-Basic-Visual-
Concepts-with-a-Higgins-
Matthey/a90226c41b79f8b06007609f39f82757073641e2. Irina Higgins, Nicolas
Sonnerat, Loïc Matthey et al., “Scan: Learning Abstract Hierarchical Compositional
Visual Concepts,” Proceedings of ICLR (2018), accessed March 23, 2021, arXiv
preprint arXiv:1707.03389. Marta Garnelo, Kai Arulkumaran and Murray
Shanahan, “Towards Deep Symbolic Reinforcement Learning”, accessed May 25,
2022, arXiv preprint arXiv:1609.05518. Brendan M. Lake and Marco Baroni,
“Generalization without Systematicity: On the Compositional Skills of Sequence-
to-Sequence Recurrent Networks,” accessed June 14, 2022, arXiv preprint
arXiv:1711.00350. Gary Marcus, “Deep Learning: A Critical Appraisal,” accessed
June 29, 2022, arXiv preprint arXiv:1801.00631. Matthew Botvinick, David Barrett
and Peter Battaglia, “Building Machines that Learn and Think for Themselves:
Commentary on Lake et al.,” Behavioral and Brain Sciences (2017), accessed July
2, 2022, arXiv preprint arXiv:1711.08378.
3
Marta Garnelo and Murray Shanahan, “Reconciling Deep Learning with Symbolic
Artificial Intelligence: Representing Objects and Relations,” Current Opinion in
Behavioral Sciences, Vol. 29 (October: 2019): 17-23.
Discriminator of Well-Formed Formulae as a Component of Artificial 201
Human Rationality
“...systems that can routinely acquire, represent and manipulate abstract

knowledge, using that knowledge in the service of building, updating, and
reasoning over complex, internal models of the external world”. 4
Contemporary Uncertainty
Integration between learning and reasoning is one of the key challenges in
artificial intelligence and Machine Learning (ML) today. Where in humans
the ability to learn and reason, from and to the surrounding environment is
categorized as a central preordain feature of the natural intelligence,
engineering the same two interrelated concepts inside programmable
hardware has spurred a plethoric creation of novelty approaches in
Computer Science.
Naturally, those approaches derive a substantial amount of their underlying

operating principles from the approximation of chemical, biological, and
psychological mechanisms occurring inside the human brain, moderating
our individual behavior, whenever the need of response due to
environmental interaction arises. Categorically, we group those responses
as the result of two separate systems, automatic and conscious once,
performing intuitive and conscious reasoning, not independently from each
other or in hierarchical fashion, but operating in synergy, simultaneously.5
Intuitive reasoning is exerted predominantly upon innate mental activities,
such as a preparedness to perceive the world around us, recognizing objects,
orient attention. Conscious activities, such as when we do something that
does not come automatically and requires some sort of conscious mental
exertion, like rule. We could then surmise learning, as being the set of
processes of creating lasting change in behavior that is the result of
experience, and conscious reasoning, being the process of applying logical
rules to produce valid arguments in formal or informal environments.6
4 Gary Marcus, “The Next Decade in AI: Four Steps towards Robust Artificial
Intelligence,”
accessed February 12, 2022, arXiv:2002.06177.
5 Daniel Kahneman, Thinking, Fast and Slow (New York: Farrar, Straus and Giroux,
2011).
66Jaakko Hintikka, “Philosophy of Logic”, Encyclopædia Britannica, accessed
March 21, 2022, https://www.britannica.com/topic/philosophy-of-logic.

202 Chapter X
To emulate natural learning, the research field of machine learning is aiming

to create computer algorithms that also improve automatically through
experience. Applications range from data-mining programs that discover
general rules in large data sets, to information filtering systems that
automatically learn users’ interests.7 Deep Learning is a subset of Machine
Learning, which relies on the creation of complex interwoven acyclic
graphs. Any vertices of the graph are implemented by a computational unit
named Artificial Neuron (Figure1), and any edge that propagates from and
to any neuron is weighted. A particular method of stacking those neurons
together in layers and establishing the number of interconnections to each
neuron describes the network architecture. The first layer of the network is
called the input layer and its last is called the output layer, while any other
layer between the input and the output layer is named hidden layer. Data is
fetched into the network through the input layer, for example, if we have an
input layer of eight neurons, we could pass information in the form of eight
floating point numbers or eight vectors (floating point arrays) for each
neuron.
Figure 1. Artificial Neuron, image generated by TikZ open source code.
Each neuron at given moment t, could only exist in an active or inactive

state. Fetching data to the input layer will cause propagation through the
network in the form of a signal, denoted as x and xn denotes the signal
coming from the input neurons to the interior of the network. Propagating
7 Tom Mitchell and Hill McGraw, Machine Learning (New York: McGraw Hill,
1997).
Human Rationality
forward, the signal is first scaled by the weights of the edges, wi,j , where i
denotes the position of the neuron in the previous layer and j denotes the
position of the neuron in the layer the signal is been propagated to, for
example w3,5 is the weight between the third and the fifth neurons between
two consecutive layers. The signal is then fed to the first hidden layer. Inside
each neuron in the hidden layer, two operations take place, summations of
all weighted signals from a previous layer and applying an activation
function, if a provided threshold is exceeded, the neuron is activated and
propagation through this neuron is enabled, or we are saying that the neuron
fires. Then, at time tk to tk+1, the input of a neuron is calculated as:
Figure 2. Input of a neuron
Where bi is called a bias and is an arbitrary value predefined for each neuron,
it could also be omitted. The activation function could vary, depending on
the problem at hand, in our case we are using the Softmax activation
function:
Figure 3. Softmax activation function
Here, the exponent of a given neuron is divided by the sum of all exponents
of the rest of the neuron. The Softmax function returns the probability of a
given example to be in a class. After each cycle, an optimization technique
is applied to adjust the weights, if the output value of the Softmax functions
in the output neurons differs from those of the pre-labeled training
examples. This optimization is performed by calculating the Gradient
Descent of each neuron weight in relation to the difference between the
current output value and the desired value, for every single neuron in the
entire network. The difference between the sum of the outputs of the last
layer and the pre-labeled training examples is computed by a math function
called a loss function, which could take various mathematical forms. That
difference is then used to start an automatic differentiation algorithm called
204 Chapter X
backpropagation.8 For our particular work, we are using a neural network

architecture called a Recurrent Neural Network (RNN). The special thing
about this network is that it can process all inputs as a sequence and
remembers its intermediate state. This is done by connecting the input for
every step, back to the input layer of the function, combining it with the new
data. At each new input at time t + 1, the new intermediate state is calculated
by:
Figure 4. Recurrent network intermediate state
Where,
Ɣ is the input at time step

Ɣ is the output at time step
Ɣ is the intermediate state as vector at time
Ɣ are weights associated with inputs in
recurrent layer.
Ɣ are weights associated with hidden

units in recurrent layer.
Ɣ is the bias associated with the recurrent layer.
and the output of the network is:
Figure 5. Output of RNN
8 David Rumelhart, Geoffrey Hinton and Ronald Williams, “Learning Representations
by Backpropagating Errors,” Nature 323 (October: 1986): 533-536.

Human Rationality
Neuro-Symbolic Approach
Neuro-Symbolic Artificial Intelligence can be defined as a subfield of
Artificial Intelligence (AI) that combines neural and symbolic approaches.
By neural, we mean approaches based on artificial neural networks –
sometimes called connectionist or sub-symbolic approaches – and in
particular this includes deep learning, which has provided very significant
breakthrough results in the recent decade and is fueling the current general
interest in AI. By symbolic we mean approaches that rely on the explicit
representation of knowledge using formal languages – including formal
logic – and the manipulation of language items (“ symbols”) by algorithms
to achieve a goal. Mostly, neuro-symbolic AI utilizes formal logics, as
studied in the Knowledge Representation and Reasoning subfield of AI, but
the lines blur, and tasks such as general term rewriting or planning, that may
not be framed explicitly in formal logic, bear significant similarities, and
should reasonably be included.
The neuro-symbolic approach has gained significant attention, primarily

because the leading trend of trying to leverage statistical inference over
massively parametrized network models cannot be backtracked. To
understand sufficiently the prediction or even how the result was derived,
the inference process itself needs to be demystified. Although that behavior,
admittedly, cannot be considered a showstopper in commercial applications,
where statistical dependencies are sufficient in narrowing down a group of
patterns as best fitting solutions to a recommendation or classification
problem, the obscurity of the pattern generator, cannot be abstracted to any
intelligible or human-interpretable reference, so a due-process by carefully
following a rational argumentative sequence is not possible. If we follow
this train of thoughts, a certain philosophical suggestion could be furnished,
or namely that Artificial Intelligence based exclusively on statistical
models, takes the vague likeness of human hunch or gut feeling that cannot
be substantially rationalized by the person experiencing it. The latter,
however, possesses potentially destructive capability, when statistical
inferences are applied in decision-making systems, where the rationale
behind a certain decision must be traceable, in order to be justified,
defended, or appealed. As a whole, the pursuit of automated human-like
rationality, or the quality of being on or in accordance with reason and logic,
206 Chapter X
cannot be successfully achieved, without the successful adherence to the

rules of logic. The key contribution of this article is to propose a
foundational layout of the neuro-symbolic model, which has in its core
logical inference engine. That engine will gain as an input, syntactically
correct logical expressions derived from text or speech and by applying
rules of inference in a given formal system, should conclude if the provided
argument could be judged as a valid one. In order for the inputs to be
syntactically correct, we need to grand filtering or discriminating
capabilities of the engine. For example, a text-to-expression parser creates
symbolic representation of the arguments out of provisioning context,
following the rules of valid syntax the expression could be accepted and
incorporated into this argument, or returned for reparsing. This article will
use the rules of first-order logic to present a discriminator of well-formed
formulas. That approach is to train a sequence-to-sequence model based on
Recurrent Neural Networks (RNN), that will grant the capability of
distinguishing between any correct formal expression and such that we
could consider as noise. Refinement on the training approach, brings
forward even better discriminator performance, as the RNN model yields
good results while distinguishing between atomic and molecular well-
formed formulas.
Neuro-Symbolic Computational Framework

In this section, a blueprint layout of a state-of-the-art computational
framework based on the neural-symbolic paradigm will be discussed. We
argue that a pure connectionism system cannot account for a valid theory of
cognition. In such a system, combinatorial and semantic structures of mental
representation seem impossible to achieve. On the other end, purely
symbolic theories cannot be regarded as more than an approximation of what
happens in the brain, they cannot be accounted as a complete description of
a cognitive phenomenon.9 Our view is that between these two philosophies
the most stubble and yet the least unexplored path is the integration between
symbolic and sub-symbolic. We will now go forward and describe the
9Steven Pinker and Alan Prince, “On Language and Connectionism: Analysis of a
Parallel Distributed Processing Model of Language Acquisition,” Cognition, 28(1-
2) (March: 1988): 73–193.
Human Rationality
existing taxonomical classification of such integrations that researchers

engage while implementing such systems. Usually a construction of neuro-
symbolic architectures falls inside two strategic paths: unified and hybrid.
Unified strategies are research paths that aim at creating neural structures,
which have symbolic capabilities or translating symbolic computation
inside neural networks alone. They can be additionally subdivided into
neuronal symbol processing and connectionist symbol processing. The
neuronal sub-approach stands for close emulation with the properties of the
biological neuron, but it only applies general similarities to actual neuron.
Connectionist symbol processing is a strategy that relies on a formal
mathematical approximation of a neuron, that we have introduced in section
II. Stacking such neurons is used to create complex architectures that
perform symbolic computations. To create a model based on that design
pattern, first a general conception about the desired function has to be
introduced as possible mapping between data distribution points, that
implementation in the form of architecture is explored. For example, if we
want to develop an architecture that could perform logical conjunction
operation (AND, ‫ )ר‬symbolic computation between two variables
we need first to ground these variables to data points inside a given
data distribution and then learn syntactic and semantic mapping
transformation to it. Of course, what we hope to learn is the truth-value table
of logical conjugation. In this specific example, we are taking advantage of
the particular state, about the data distribution of the outputs, or precisely
that the outputs of the mapping function are linearly separable, and the
implementation could be achieved by applying only a single artificial
neuron (perception). In the case of nonlinear separability of the outputs, for
example, in learning how to compute logical expression with exclusive
disjunction (XOR, Չ), and the respective symbolic notation (x Չ y) more
complex architecture has to be created with minimum two stacked
perceptrons and as final the symbolic computation that will be learned by
the model could be the following,
. We are hypothesizing the exact symbolic
representation since accurate elicitation of symbolic expression is not
possible, once mapping is learned inside the neural nets. In this case, we
208 Chapter X
could also have a symbolic representation of the form,

. Both representations are logically equivalent and in both of the cases, two
additional neurons must be included in the architecture, one for the operator
and one for the operator. In the end, what matters is the learned mapping
function and the internal representations of the networks. Hybrid strategies
are strategies that rely on a synergistic combination of neural and symbolic
models. The goal in mind while utilizing this strategy is that only by
constructing a modular neuro-symbolic structure we could attain a full range
of cognitive and computational potentials. Hybrid modules also subdivide
themselves into translational or functional hybrids. Translational models are
composites that we could put between models based on unified strategies
and functional models, as well as in unified models in translational models,
the sole computational process relies on neural networks, but that model
does not confide itself on bottom-up approach from a single to multiple
stacked neurons when it comes to its core implementation. The translational
model, as the name suggests, translates the provided symbolic representations
from a formal symbolic system or structure to a neural network
computational core. These systems also, as in the provided example of a
unified system, try to elicit after neural network computation symbolic
representation back to its initial symbolic structure. Such systems are
assigned to symbolic model discovery from physical systems.10 Functional
hybrids in addition to neural networks have complete symbolic structures
and processes, like rule interpreters, parsers, reason and inference engines,
theorem provers. These systems rely on synergy between the different
functional modules. The idea is that great model robustness could be
achieved to a certain degree by adding compositionality to these models
with levels of integration between the different functional models. This
degree of integration is a quantitative benchmark, which reflects the
progression from loosely to tight coupling between the modules, for
example, as loosely coupled we could classify systems that only possess
simple juxtaposition between the symbolic and the neural models. In
strongly coupled, a continuous interaction between the neural and symbolic
10Miles Cranmer, Alvaro Sanchez-Gonzalez, Peter Battaglia et al., “Discovering

Symbolic Models from Deep Learning with Inductive Biases”, NeurlPS 2020,
accessed July 3, 2022, arXiv:2006.11287.
Human Rationality
submodules operates in internal structures, meaning that knowledge and

data are shared during the runtime of the system and a change in the
operational state of a certain neural module will trigger a unified response
across all symbolic modules, provided that more than one neural module is
present in the architecture, this unification will also constitute an adaptation
in the rest of the neural modules. Now after the introduction of the different
strategies, which are used in neuro-symbolic systems, we will discuss our
proposed design as a particular instance of these strategies. Considering that
any design pattern carries within its pros and cons and cons. Our choice in
this matter is to follow the hybrid strategy, as a design paradigm.
Considering that in the past 65 years since the dawn of the field of artificial
intelligence, there was little progress in decoding the essence of human
rationality and implementing common sense as a machine ability, providing
a schematic what might catalyze or induce as final all missing ingredients
to the creation of artificial human rationality, in a single paper would not be
possible, what we will describe here is our view of what a system might
need in order to at least being able to get modular abilities that would enable
more elaborative future research to be based upon this one.
We approach the problem by firstly defining what we think is the most

pressing lack of abilities that could curb an artificial system from obtaining
rationality levels next to that of a human. Deep learning has been
predominantly a huge success when it comes to perception tasks, like object
and pattern recognition in computer vision or natural language processing,
but for systems that perform automated reasoning over a set of descriptive
rules, such models have proven to be inadequate, for example, in closed
environment, competitive board games or Atari, a learning mechanism has
to be combined with symbolic computational mechanisms like Monte Carlo
Tree Search (MCTS). For example, in 2016 the AlphaGo11 system was able
to beat the current GO world champion by continuously inferring better
pieces’ positions and ergo building better strategies, the model, although
constricted to the geometry of board desk, was able to fill the gaps in
previously unseen pieces’ configurations, based on the existing knowledge
representation which was rooted in the time of past offline learning. The
11
David Silver, Aja Huang, Chris Maddison et al., “Mastering the Game of Go with
Deep Neural Networks and Tree Search,” Nature 529 (January: 2016): 484-489.
210 Chapter X
learning procedure was a special case of supervised learning named

Reinforcement Learning (RL). To beat the reigning champion, however,
only RL was not enough, what was learned had to be integrated with a
symbolic search technique to create the best reasoning patterns over the
board. Referring back to section II and the definition of reasoning, this
comes in a close approximation to what people do. Our schematic is a
strategy for creating artificial agents, capable of reasoning over unseen
environmental configurations, much like AlphaGo. Of course, however, we
are interested in the general case, searching for an approach or architecture
that will discover and apply strategies to solve a variety of problems. To be
able to create those strategies, there are three crucial aspects that must be
imbued as abilities into an agent, environment identification, problem
semantic representation, and decision making. Our test studies focus on
natural language processing, meaning that for now we are constraining the
environment to knowledge elicitation and transformation from text. In a
corpus of textual information, in order to make sense of the information
interlocked, an agent needs to discover the semantics or the meaning behind
each sentence, both as a single unit and as a text. A decision making, to a
specified problem, like query-answer entailment or sentiment analysis. Our
approach here is to convert a text to sets of logical expressions using a
technique called logical chunking, with that technique what we are aiming
at is a system for parsing and generating text using combinatory categorial
grammar for syntax and hybrid logic dependency semantics for, well, the
semantic representation. For example, the sentence Brian loves Jain would
be transformed to love(B, J) or even L(b, j), where L, b and j are respectively
the predicate and two constants bounded to real objects Bounding of
variables, constants and definitions is a particular hard problem when it
comes to natural language, our approach is to define them as relational
connections or intersections between sets In this, particular example, two
members of the intersection of the sets containing the names of all people,
the members of all leaving peoples, the set of people loving each other etc..
Historically, such relational categorization is painfully slow process,
provided that group of experts is responsible for defining, categorizing and
connecting all the relations, such approach has already been tried as part of
Human Rationality
the infamous CYC project,12 which results, although strikingly impressive,

when it comes to completing tasks as automatic story generation for specific
literary works it remains extremely limited when knowledge relations are
not encoded, and has to be discovered on a flight.13 For automation of this
part, an automatic semantic parser has to be employed. The architecture of
this parser should rely on the already discovered connections. Meaning we
could leverage the existing knowledge represented as graph- connection in
Google Knowledge Graphs or Wikipedia. Furthermore, convolution of this
graph’s spaces would discover deeper connections between the entities.
This discovered knowledge in a graph formalism, must exist on demand and
will be used as a semantic interpretation later on. The syntactical parser must
also have unified functionality to take those relations and combine them
with predefined syntax to create syntactically valid logical expressions, we
believe that the best formal system to be used here by which these logical
expressions should be constructed is the Second-Order Logic (SOL). The
syntax must also be learned via supervised learning, however, the rich
expressive power of SOL will require a more powerful technique through
RL to be applied here since a large enough dataset with enough possible
symbolic representations is not feasible. After that, the expressions, that
then will be passed to two components, namely the Semantic and the
Reasoning engines. The semantic engine will interpret the expression using,
interpretation by sense and reference, and truth. Interpretation by truth is
simply the logical value of the expressions, and by sense and reference we
are referring here to the idea proposed by Gottlob Frege in Begriffsschrift.14
The Reasoning engine will have encoded into itself, rules of introduction of
logical operators, rules of discharge and rules of logical inference (modus
ponens, modus tollens…). Following this blueprint, an agent will be
endowed with a powerful enough mechanism to enable common sense
reasoning or to rationalize over presented ideas and topics from text. What
we have described above is just the architectural approach: current
12 “CYC”, Encyclopedia Britannica, accessed June 6, 2022,

https://www.britannica.com/topic/CYC.
13 Doug Lenat, “What AI Can Learn from Romeo & Juliet,” Cognitive World,
Forbes, July 3, 2019.

14 Gottlob Frege, Begriffsschrift: Eine der arithmetischen nachgebildete formelsprache
des reinen denkens (Halle a/S: L. Nebert, 1879).

212 Chapter X
implementations as modules of functional hybrids are still under

investigation. However, as a part of creating valid logical expressions, the
hybrid system must have an integrated module for discriminating, between
valid and invalid ones. That discrimination or classification will be
conducted by a unit that, we have developed, and it will be integrated in the
RL pipeline.
Discriminator of a Well-formed Formula

In this section, a technical implementation of the previously discussed idea
of syntactical classifiers is implemented. The technical implementation is
performed using the Python programming language and the popular
machine learning framework PyTorch, which provides extensive capabilities
for creating sequential models for natural language processing in Python.
We chose a vanilla Recurrent Neural Network for the architecture of the
model. When a standard RNN network is exposed to long sequences or
phrases it tends to lose the information because it cannot store the long
sequences and as the methodology is concerned it focuses only on the latest
information available at the node. This problem is commonly referred to as
vanishing gradients. In our case, we adopt RNNs as the implementation
because of the nature of the training data set. Since our model will be trained
on artificially synthesized expressions, we are converting them to relatively
short strings. The aim of this model is to acquire the ability to classify
between noise and syntactically correct formulas which are generated by the
graph parser described in the previous section. The motivation behind
developing such model is that formal systems constitute logical formalism
as a reasoning tool. There are many different formal systems and their
referential syntactical rules. We hope that this model could be used as a
general study case when it comes to creating preprocessing pipelines for
reasoning engines based as part of functional hybrids.
We follow the syntactical rules for creating logical expressions for First-
Order Logic to restrict the formalism. Such formalism, although having its
limitations, for example, cannot define finiteness or countability, is
considered powerful enough to represent the natural language statements
sufficiently well. The allowed symbols are restricted over a signature, here
Human Rationality
we are considering all letters from the English alphabet as well as the
following operational symbols.
Ɣ Identity operator and iff :
Ɣ Implication operator :
Ɣ Entailment (semantic and syntactic):
Ɣ Binary operators for Boolean algebra:
In the last row, the last three symbols are making, respectively, the NOR,
NAND and XOR Boolean operators; these particular symbol’s annotations
are purely our own choice. We will refer to this list as the operational
symbol list. We are also using the following symbols for existential (exists,
does-not-exist) and universal quantifiers:
Every symbol described above is used to construct the following sets, when
we create a syntactically valid expression:
1. Set of predicate symbols P.

Contains all uppercase letters of the English language.
2. Set of constant and variables and functions symbols S

Every lowercase letter of English, which is not a member of the
function symbol set F. Provided that we are just interested in the
form of an expression and do not ground the variables and the
constants, we will not create two separate sets of symbols for
assignment, since we are not interested in the semantics of the
expression.
214 Chapter X
3. Set of function symbols F

Every lower-case letter of the English language, which is not a
member of the set S. The rule that we should follow is that once a
symbol is selected for a function notation it cannot be used for a
variable or a constant
4. Set of function strings Fs.

It contains every string of the form , where is
an unbound variable or a constant from S. A function has an arity of
a maximum of six symbols. These are artificial constraints that we
are introducing, theoretically it must be adjusted for every particular
formal system.
5. Set of terms T.
Is the union .
6. Set of atomic formulas strings A.

It contains all strings of the form , where
. The terms inside the brackets after the
predicate symbol Q are called arguments. Atomic formulas are also
restricted to a maximum of six terms as arguments to their predicate.
A could also be noted as the union , where
contains all the strings of the form and contains
all the strings of the form .
7. Set of molecular formulae strings M

Contains every string m such that,
( ), where is a quantifiable
atomic formula such that and ‘o’ is any operational symbol
from the operational symbol list.
Human Rationality
Training Data
The training data will be synthesized from a custom code that will first
create the mentioned collection of sets . Second, the
sets (A, M) will be written into two text files, one for the atomic formulas
and one for the molecular. In these files, each member of the set will be
written in a separate line as the string that we want to pass to the input layer
of our RNN. One additional text file will also be added, any line of this file
will be a string of jumbled random characters in ascii, this will be our noise.
As a whole, we have three labels for classification Noise, Atomic,
Molecular. The training set contains the following records described in the
table below.
Training set
Label Records
Noise 18
Atomic 32617
Molecular 300000
Table 1. Training set of the experiment
In our experiment, the atomic set is used to generate the molecular set, this
is a tricky part for the network to discover, since the information encoded
into the molecular set is also encoded in the atomic set. The noise represents
a particularly blunt difference between the form of a syntactically correct
expression and random strings, that is why the network, should need very
few of these examples to train itself.
Discriminator’s Architecture
As we pointed out, the discriminator would be a many-to-one recurrent
neural network. Our first layer will have a size of 100 input neurons, this is
exactly to cover all printable characters, digits, punctuation and brackets in
ascii encoding. The hidden layer is constructed out of 128 fully connected
neurons, which are connected to the three output neurons. In practice,
because of the specifics of PyTorch, what we end up is a combination of
216 Chapter X
two regular feedforward neural networks. One is connected to the output

layer of three neurons for each output label that we want to classify and the
second is reconnected again with the input coming from the training data,
in order to combine the current state of the network with the new state. After
combination, the new values are fed as inputs to both networks. To begin
training, the training data must be converted from a string to a tensor.
To put it simply, a tensor is just a multidimensional matrix. The notation

corresponds to the dimensionality of the tensor, for example 3-d is a 3-
dimensional tensor. We are taking advantage of the positional index that
each character has in Python “string.printable”; this command returns a
string containing all the numbers, letters and special printable characters in
Python “0..9A..Z..-./:;<=>?@[..”. Each character has an indexed position
in it, for example the character ‘?’ is in position 82 and it is at 81 characters
away from the beginning of the string. We denote this index as r. Each line
will be encoded to 3-d tensor where, with r, 2-d tensors inside and each two-
dimensional tensor will have one 1-d tensor with m = 100 elements. For
example, let is the shape of the final three-dimensional tensor,
if we encode the line as:
Figure 6. An example expression from the learning set.
We will get a tensor with dimensions (‘’ consists of two printable

characters ‘-’ and ‘>’), From here, the generated 3-d tensor will contain
twelve 2-d tensor, each having one row of one hundred zeros, except for one
1.0, which corresponds to the same position indexed from the
“string.printable”. If a character happens not to be contained in
“string.printable”, the last element of the respectable 2-d tensor is filled
with “1.0”. From here, each 3-d tensor is passed to the network, and the
collected 2-d tensors are passed sequentially to the network. The output is a
1-d tensor with 1 element equal to the positional index of a list with the
labeled outputs (Noise, Atomic, Molecular).
Human Rationality
Training and Learning

The learning is conducted on GPU NVIDIA 2060 RTX with 6 GB RAM.
The required loss function for minimization in the test study is the negative
log-likehood function:
Figure 7. Log-likehood loss function 15
The training is performed along the course of 100 000 cycles, at each cycle
a random training sample is selected from any of (N, A), and is fed to the
100 input neurons, for each character in the line. The selected learning rate
is . We are selecting such a miniscule learning parameter
because we want the model to learn the subtle difference which members of
A are used for in the creation of the members of M.
From here, the cycle of learning is the following:
Ɣ Create the input output tensor.

Ɣ Create initial hidden state
Ɣ Fed the input tensor corresponding to a single line from N, A, M
Ɣ Compare with the desired output
Ɣ Perform backpropagation
Ɣ Repeat
The result from the training can be seen in Figure 8. This is the so-called
confusion matrix, which represents the average normalized prediction of the
model for each 1000 cycles. We could see that the model has achieved great
performance, during the training phase, as the yellow color in the diagonal
corresponds to the normalized degree of prediction in this case 1.0, meaning
extremely close to 100%.
15 Lester J. Miranda, “Understanding Softmax and the Negative Log-likelihood,”
accessed July 3, 2022,

https://ljvmiranda921.github.io/notebook/2017/08/13/softmax-and-the-negative-
log-likelihood/.
218 Chapter X
Figure 8. Confusion matrix
Testing
In order to test our model, we have generated a new logical expression using
anew construction of the sets (S, F, Fs, T, A, M). The table below shows the
record distribution of the test set and the test results.
Test set
Label Records
Noise 3 274 584
Atomic 2 976 230
Molecular 3 000 000
Table 2. Test set with records from the categories.

Human Rationality
Results from testing

Random test records 3 000 000
Correct predictions 2 972 853
Incorrect predictions 27147
Success rate 99.095%
Elapsed time 2:29:39h
Table 3. Test Results
Conclusion
This work has introduced a theoretical compositional framework for neural-
symbolic integration that utilizes functional hybrid as a leading design
paradigm. The main contribution of this paper is the elaborated schema of
compositional design for an artificial agent with human-like level of
rationality. One module of this framework, a sub-symbolic discriminator for
valid logical syntax, is constructed in this article. The results from training
and test cycles have achieved over 99.13% accuracy. In future work, a better
discriminator of valid logical arguments, semantic parser from natural
language to logical expression reasoning and semantic engines will be built
to complete the neuro-symbolic framework proposed in this article.
References
Botvinick, Matthew, David Barrett, Peter Battaglia, Nando de Freitas,
Darshan Kumaran, Joel Leibo, , Timothy Lillicrap et al.. “Building
Machines to Learn and Think for Themselves: Commentary on Lake et
al.” Behavioral and Brain Sciences (2017). arXiv preprint
arXiv:1711.08378.
Cranmer, Miles, Alvaro Sanchez-Gonzalez, Peter Battaglia, Riu Xu, Kyle
Cranmer, David Spergel, and Shirley Ho. “Discovering Symbolic
Models from Deep Learning with Inductive Biases”. NeurlPS 2020.
Accessed July 3, 2022. arXiv:2006.11287.
Encyclopedia Britannica. “CYC”. Accessed June 6, 2022.
https://www.britannica.com/topic/CYC.
220 Chapter X
Frege, Gottlob. 1879. Begriffsschrift: Eine der arithmetischen nachgebildete

formelsprache des reinen denkens. Halle a/S: L. Nebert.
Garnelo, Marta, Kai Arulkumaran, and Murray Shanahan. “Towards Deep
Symbolic Reinforcement Learning”. Accessed May 25, 2022. arXiv
preprint arXiv:1609.05518.
Garnelo, Marta, and Murray Shanahan. “Reconciling Deep Learning with
Symbolic Artificial Intelligence: Representing Objects and Relations.”
Current Opinion in Behavioral Sciences, Vol. 29 (2019): 17-23.
https://doi.org/10.1016/j.cobeha.2018.12.010.
Higgins, Irina, Loïc Matthey, Arka Pal, Christopher Burges, Xavier Glorot,
Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. “Beta-
VAE: Learning Basic Visual Concepts with a Constrained Variational
Framework.” Proceedings of ICLR (2017). Accessed May 3, 2022.
https://www.semanticscholar.org/paper/beta-VAE%3A-Learning-
Basic-Visual-Concepts-with-a-Higgins-
Matthey/a90226c41b79f8b06007609f39f82757073641e2.
Higgins, Irina, Nicolas Sonnerat, Loïc Matthey, and Arka Pal. “Scan:
Learning Abstract Hierarchical Compositional Visual Concepts.”
Proceedings of ICLR (2018). Accessed March 23, 2021. arXiv preprint
arXiv:1707.03389.
Hintikka, Jaakko. “Philosophy of Logic”. Encyclopædia Britannica.
Accessed March 21, 2022.
https://www.britannica.com/topic/philosophy-of-logic.
Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar,
Straus and Giroux.
Lake, Brendan M., and Marco Baroni. “Generalization without
Systematicity: On the Compositional Skills of Sequence-to-Sequence
Recurrent Networks”. Accessed June 14, 2022. arXiv preprint
arXiv:1711.00350.
LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. “Deep Learning.”
Nature, 521(7553) (2015): 436-444.
Lenat, Doug. “What AI Can Learn from Romeo & Juliet.” Cognitive World,
Forbes. July 3, 2019.
https://www.forbes.com/sites/cognitiveworld/2019/07/03/what-ai-can-
learn-from-romeo--juliet/.
Human Rationality
Marcus, Gary. “The Next Decade in AI: Four Steps toward Robust Artificial
Intelligence”. Accessed February 12, 2021.
arXiv:2002.06177.
Marcus, Gary. “Deep Learning: A Critical Appraisal”. Accessed June 29,
2022. arXiv preprint arXiv:1801.00631.
McCloskey, Michael, and Neal J. Cohen. “Catastrophic Interference in
Connectionist Networks: The Sequential Learning Problem.”
Psychology of Learning and Motivation, Vol. 24 (1989): 109–165.
https://doi.org/10.1016/S0079- 7421(08)60536-8.
Miranda, Lester J. “Understanding Softmax and the Negative Log-
likelihood”. Accessed July 3, 2022.
https://ljvmiranda921.github.io/notebook/2017/08/13/softmax-and-the-
negative-log-likelihood/.
Mitchell, Tom, and Hill McGraw. 1997. Machine Learning. New York:
McGraw Hill.
Pinker, Steven, and Alan Prince. “On Language and Connectionism:
Analysis of a Parallel Distributed Processing Model of Language
Acquisition.” Cognition, 28(1-2) (1988): 73–193.
Ratcliff, Roger. “Connectionist Models of Recognition Memory: Constraints
Imposed by Learning and Forgetting Functions.” Psychological Review,
97 No. 2 (1990): 285–308.
https://doi.org/10.1037/0033-295x.97.2.285.
Rumelhart, David, Geoffrey Hinton, and Ronald Williams. “Learning
Representations by Backpropagating Errors.” Nature 323 (1986): 533-
536. https://doi.org/10.1038/323533a0.
Silver, David, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre,
George van den Driessche, Julian Schrittwiesser et al. “Mastering the
Game of Go with Deep Neural Networks and Tree Search.” Nature 529
(2016): 484-489. https://www.nature.com/articles/nature16961.

Rationality and Ethics in Artificial Intelligence

Uploaded by

Copyright:

Available Formats

Rationality and Ethics in Artificial Intelligence

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Rationality and Ethics in Artificial Intelligence

Uploaded by

Copyright:

Available Formats

Rationality and Ethics

Boris D. Grozdanoff, Zdravko Popov

Edited by Boris D. Grozdanoff, Zdravko Popov and Silviya Serafimova

This book first published 2023

Cambridge Scholars Publishing

Lady Stephenson Library, Newcastle upon Tyne, NE6 2PA, UK

British Library Cataloguing in Publication Data

Copyright © 2023 by Boris D. Grozdanoff, Zdravko Popov,

ISBN (10): 1-5275-9441-6

Acknowledgements .................................................................................. vii

Chapter III ................................................................................................ 35

Chapter VII ............................................................................................. 116

Chapter VIII ........................................................................................... 131

Chapter IX .............................................................................................. 152

Chapter X ............................................................................................... 199

This volume includes a selection of papers, written on the basis of talks

A BRIEF HISTORY OF COMPUTER ETHICS

1 Bynum,”A Very Short History of Computer Ethics”, accessed July 8, 2022,

York: Doubleday Anchor Book, 1950).

author warns us not to accept computers as available entities, but to always

Norbert Wiener made valuable recommendations nearly 70 years ago,

3Walter Maner, Starter Kit on Teaching Computer Ethics (Self-published in 1978.

Deborah Johnson also contributed to this issue. In her article, “Computer

Vol.16, No. 4 (October 1985): 319-322.

Back in 1995, Krystyna Górniak-Kocikowska predicted in her article6 that

6Górniak-Kocikowska, Krystyna. “The Computer Revolution and the Problem of

consequences of interacting with artificial intelligence will not be a regional

However, when addressing general or superartificial intelligence, it will be

Will the code of ethics for artificial intelligence be objective if the

Mara Hvistendahl, a scientific correspondent for the Guardian, in her article

7 Mara Hvistendahl, “Can We Stop Outsmarting Humanity?”, The Guardian, Match

Stuart Armstrong of the Future of humanity institute at the Oxford Institute

Boris Grozdanoff in his article “Prospects for a Computational Approach to

8Boris D. Grozdanoff, “Prospects for a Computational Approach to Savulescu’s

There is a call from the European Commission to companies, when creating

Undoubtedly, it is crucial that there is a global strategic factor in the face of

UNESCO is also developing a solid and intergovernmental document - an

No. 3 (2020): 107-120. https://jesbg.com/eticheski-izsledvania-br-5-

ETHICAL CLASHES IN THE PROSPECTS

In turn, the trolley dilemma is modified within so-called moral design

Oxford Review, No. 5 (1967): 5-15.

in Road Traffic Scenarios: Applicability of Value-of-Life-Based Models and

context, moral programming can be examined as displaying adaptations of

Automated vehicle technologies are “the computer systems that assist

Current projects for AVs aim at building partially autonomous vehicles,

Influences of Time Pressure,” Front. Behav. Neurosci, No. 11 (December 2017):

“Human Decision-Making Biases in the Moral Dilemmas of Autonomous

Similar to the methodological pitfalls in objectifying the moral outcomes of

The aforementioned specifications show that if one wants to find objective

9 Frank, Chrysochou, Mitkidis and Ariely, “Human Decision-Making Biases in the

Moral Dilemmas of Autonomous Vehicles,” 1.

Moral Dilemmas of Autonomous Vehicles,” 2.

Moral Dilemmas of Autonomous Vehicles,” 17.

contractualist project and some of Frank et al.’s thought experiments which

The Section “A contractualist prospect for AVs” is devoted to the role of

In Section “Building “utilitarian” AVs”, I analyze some Frank et al.’s thought

13 The choice of projects demonstrates how both theoretical ethical approaches

A Contractualist Prospect for AVs

Leben’s “contractualist” answer to the moral design problem is grounded

For the purposes of maintaining the objective foundations of justice, Rawls