Chapter Two

Intelligent Agent
 Agent and Environment

• An agent is anything that can be viewed as

perceiving its environment through sensors and
acting upon that environment through actuators.
• Example: -
– A robotic agent might have cameras and infrared range
finders for sensors and various motors for actuators.
– A software agent receives keystrokes, file contents, and
network packets as sensory inputs and acts on the
environment by displaying on the screen, writing files,
and sending network packets.
 Sensors Percept
E
Agent N
V
I

? R
O
N
M
E
Action
Actuators N
T

Agents interact with environments through sensors and actuators.

 Some terminologies
• Percept: - refer to the agent’s perceptual
inputs at any given instant.
• Percept sequence: - is the complete history of
everything the agent has ever perceived.
• Agent function: - maps any given percept
sequence to an action.
 Count…
• Internally, the agent function for an artificial
agent will be implemented by an agent
program.
• The agent function is an abstract
mathematical description;
• the agent program is a concrete
implementation, running within some physical
system.
 Example-The vacuum-cleaner world
• This particular world has just two locations: squares A
and B.
• The vacuum agent perceives which square it is in and
whether there is dirt in the square.
• It can choose to move left, move right, suck up the dirt,
or do nothing.
• One very simple agent function is the following:
– if the current square is dirty, then suck; otherwise, move to
the other square.
• A partial tabulation of this agent function is shown
below.
 Good Behavior: The Concept of Rationality
• A rational agent is one that does the right thing.
• What does it mean to do the right thing?
– When an agent is plunked down in an environment, it
generates a sequence of actions according to the percepts
it receives.
– This sequence of actions causes the environment to go
through a sequence of states.
– If the sequence is desirable, then the agent has performed
well.
• This notion of desirability is captured by a performance measure
that evaluates any given sequence of environment states.
• As a general rule, it is better to design performance
measures according to what one actually wants in the
environment, rather than according to how one thinks
the agent should behave.
 Rationality
• What is rational at any given time depends on four
things:
– The performance measure that defines the criterion of
success.
– The agent’s prior knowledge of the environment.
– The actions that the agent can perform.
– The agent’s percept sequence to date.
• This leads to a definition of a rational agent:
– For each possible percept sequence, a rational agent
should select an action that is expected to maximize its
performance measure, given the evidence provided by the
percept sequence and whatever built in knowledge the
agent has.
• Doing actions in order to modify future
percepts—sometimes called information
gathering—is an important part of rationality.
– Information gathering is provided by the
exploration that must be undertaken by an agent
in an initially unknown environment.
• A rational agent not only needs to gather
information but also to learn as much as
possible from what it perceives.
– The agent’s initial configuration could reflect some
prior knowledge of the environment, but as the
agent gains experience this may be modified and
augmented.
 Task Environment

• Before thinking about designing a rational agent

there is the task environments that we should think
about.
• Task environments are problems to which rational
agents are the “solutions”.
• The task environments consist of PEAS
(Performance, Environment, Actuators, and
Sensors).
• In designing an agent, the first step must always be
to specify the task environment as fully as possible.
 Example
• The task of designing an automated taxi
– Performance: - Measure safety, destination, profits, legality,
comfort …
– Environment: - the streets, traffics, pedestrians, weather …
– Actuators: - steering, accelerator, break, horn,
speaker/display…
– Sensor: - video, accelerometers, GPS, keyboard…
• Internet shopping agent
– Performance: - price, quality, appropriateness, efficiency…
– Environment: - current and future WWW sites, vendors,
shippers …
– Actuators: - display to user, follow URL, fill in form…
– Sensor: - HTML pages (text, graphics, and scripts)…
 Types of environment
• There are different types of environment:
– Fully observable vs. partially observable:
– Single agent vs. multiagent
– Deterministic vs. stochastic
– Episodic vs. sequential
– Static vs. dynamic:
– Discrete vs. continuous
– Known vs. unknown
Fully observable vs. partially observable

• If an agent’s sensors give it access to the

complete state of the environment at each
point in time, then we say that the
environment is fully observable.
• An environment is effectively fully observable
if the sensors detect all aspects that are
relevant to the choice of action
– Relevance depends on the performance measure.
 Count…
• Fully observable environments are convenient
because the agent need not maintain any
internal state to keep track of the world.
• An environment might be partially observable
because of:
– noisy and inaccurate sensors
– parts of the state are simply missing from the
sensor data
 Count…
• For example:
– a vacuum agent with only a local dirt sensor
cannot tell whether there is dirt in other squares
– an automated taxi cannot see what other drivers
are thinking.
 Single agent vs. multiagent
• How do we distinguish whether an environment is
single agent or multiagent, given two agents A
and B?
– It depends on which entities must be viewed as
agents. Does an agent A (the taxi driver for example)
have to treat an object B (another vehicle) as an agent,
or can it be treated merely as an object?
• The key is whether B’s behavior is best described
as maximizing a performance measure whose
value depends on agent A’s behavior.
 Count…
• Example
– Solving Puzzle – single agent environment
– Playing chess – two agent environment
 Deterministic vs. stochastic
• If the next state of the environment is
completely determined by the current state and
the action executed by the agent, then we say
the environment is deterministic; otherwise, it
is stochastic.
– Example: - Taxi driving is clearly stochastic in this
sense, because one can never predict the behavior
of traffic exactly;
– The vacuum world as we described it is
deterministic.
 Episodic vs. sequential
• In an episodic task environment, the agent’s
experience is divided into atomic episodes.
• In each episode the agent receives a percept
and then performs a single action.
• Crucially, the next episode does not depend
on the actions taken in previous episodes.
 Count...
• For example:-
– an agent that has to spot defective parts on an
assembly line bases each decision on the current part,
regardless of previous decisions; moreover, the
current decision doesn’t affect whether the next part
is defective. Therefore it is episodic
– Chess and taxi driving are sequential: in both cases,
short-term actions can have long term consequences.
Episodic environments are much simpler than
sequential environments because the agent does not
need to think ahead.
 Static vs. dynamic:
• If the environment can change while an agent
is deliberating, then we say the environment is
dynamic for that agent; otherwise, it is static.
• Static environments are easy to deal with
because the agent need not keep looking at
the world while it is deciding on an action, nor
need it worry about the passage of time.
 Count…
• Dynamic environments, on the other hand,
are continuously asking the agent what it
wants to do;
– if it hasn’t decided yet, that counts as deciding to
do nothing
• If the environment itself does not change
with the passage of time but the agent’s
performance score does, then we say the
environment is semi-dynamic.
 Count…
• Example
– Taxi driving is clearly dynamic: the other cars and
the taxi itself keep moving while the driving
algorithm dithers about what to do next.
– Chess, when played with a clock, is semi-dynamic.
– Crossword puzzles are static.
 Discrete vs. continuous
• The discrete/continuous distinction applies to
– The state of the environment,
– The way time is handled and
– The percepts and actions of the agent.
• For example: - the chess environment has a
finite number of distinct states. Chess also has
a discrete set of percepts and actions. Taxi
driving is a continuous-state and continuous
time problem
 Known vs. unknown
• In a known environment, the outcomes (or
outcome probabilities if the environment is
stochastic) for all actions are given.
• If the environment is unknown, the agent will
have to learn how it works in order to make
good decisions.
 The Structure of Agents

• The job of AI is to design an agent program

that implements the agent function that maps
percepts to actions which run on some sort of
computing device with physical sensors and
actuators—we call this the architecture:
agent =architecture + program
 Count…
• Example: - If the program is going to recommend
actions like Walk, the architecture had better have
legs.
• The architecture might be just an ordinary PC, or it
might be a robotic car with several onboard
computers, cameras, and other sensors.
• In general, the architecture makes the percepts from
the sensors available to the program, runs the
program, and feeds the program’s action choices to
the actuators as they are generated.
 Types Intelligent Agents
• We have four basic kinds of agent programs
that embody the principles underlying almost
all intelligent systems:
– Simple reflex agents
– Model- based reflex agents
– Goal- based agents and
– Utility- based agents
 Simple Reflex Agents
• The simplest kind of agent is the simple reflex
agent.
• These agents select actions on the basis of the
current percept, ignoring the rest of the
percept history.
– For example: - the vacuum agent is a simple reflex
agent, because its decision is based only on the
current location and on whether that location
contains dirt.
 Count…
• Simple reflex behaviors occur even in more complex
environments.
– Imagine yourself as the driver of the automated taxi. If
the car in front brakes and its brake lights come on, then
you should notice this and initiate braking.
– In other words, some processing is done on the visual
input to establish the condition we call “The car in front is
braking.” Then, this triggers some established connection
in the agent program to the action “initiate braking.” We
call such a connection a condition–action rule, written as
• if car-in-front-is-braking then initiate-braking.
 Count…
• Simple reflex agents have limited intelligence.
• It works only if the correct decision can be
made on the basis of only the current percept
that is if the environment is fully observable.
 Model-based reflex agents
• The most effective way to handle partial
observability is for the agent to keep track of
the part of the world it can’t see now.
• It requires two kinds of knowledge to be
encoded in the agents program: -
 Count…
1. We need some information about how the
world evolves independently of the agent
2. We need some information about how the
agent’s own actions affect the world
– For example: - When the agent turns the steering
wheel clockwise, the car turns to the right, or that
after driving for five minutes northbound on the
freeway, one is usually about five miles north of
where one was five minutes ago.
 Count…
• This knowledge about “how the world
works”—whether implemented in simple
Boolean circuits or in complete scientific
theories—is called a model of the world.
• An agent that uses such a model is called a
model-based agent.
 Goal-based agents
• Knowing something about the current state of the
environment is not always enough to decide what to
do.
– For example: - at a road junction, the taxi can turn left,
turn right, or go straight on. The correct decision depends
on where the taxi is trying to get to.
• In other words, as well as a current state description,
the agent needs some sort of goal information that
describes situations that are desirable—for example,
being at the passenger’s destination.
• Notice that decision making of this kind is
fundamentally different from the condition action
rules described earlier, in that it involves
consideration of the future—both “What will happen
if I do such-and-such?” and “Will that make me
happy?”
– In the reflex agent designs, this information is not explicitly
represented, because the built-in rules map directly from
percepts to actions.
– The reflex agent brakes when it sees brake lights.
– A goal-based agent, in principle, could reason that if the
car in front has its brake lights on, it will slowdown.
– Given the way the world usually evolves, the only action
that will achieve the goal of not hitting other cars is to
brake.
 Utility-based agents
• Goals alone are not enough to generate high-quality behavior
in most environments.
• For example: - many action sequences will get the taxi to its
destination (thereby achieving the goal) but some are quicker,
safer, more reliable, or cheaper than others.
• Goals just provide a crude binary distinction between “happy”
and “unhappy” states.
• A more general performance measure should allow a
comparison of different world states according to exactly how
happy they would make the agent. Because “happy” does not
sound very scientific, economists and computer scientists use
the term utility instead.
• An agent’s utility function is essentially
internalization of the performance measure.
• If the internal utility function and the external
performance measure are in agreement, then
an agent that chooses actions to maximize its
utility will be rational according to the external
performance measure.
 Count…
• The two cases where utility- based agents are
better than goal- based agents: -
– When there are conflicting goals, only some of which
can be achieved (for example, speed and safety), the
utility function specifies the appropriate tradeoff.
– When there are several goals that the agent can aim
for, none of which can be achieved with certainty,
utility provides a way in which the likelihood of
success can be weighed against the importance of
the goals.