Agent-based systems for supply chain management

Fernando D. Mele, Gonzalo Guillén-Gosálbez, Antonio Espuna and

Luis Puigjaner
Department of Chemical Engineering
Polytechnic University of Catalonia, Barcelona, Spain

EWO Seminar, 11 December 2007

1
 Outline

• Introduction

• Simulation models for SCM

• Multi-agent system for SCM

• Simulation-based optimization (Sim-Opt)

• Multi-objective Sim-Opt

• Mathematical programming vs Sim-Opt

• Conclusions

2
 Supply chain management
Manufacturing
plant

Retailer
Customer
Raw material
extraction
Customer
Warehouse
Manufacturing
plant

Manufacturing
plant

Customer
Retailer
Supply Chain
A SC can be defined as a network of business entities who work together in an effort to acquire
raw materials, transform these raw materials into intermediate and finished products, and
distribute these final products to retailers (Simchi-Levi et al., 2000)

Introduction 3
 SCM: Trends and challenges
Dynamic considerations:
• Dynamic simulation, simulation/optimization approaches

Integrated approaches
• Horizontal integration (manufacturing, distribution, etc.)
• Vertical integration (different decision-making levels: strategic, tactical and operational))

Multi-objective optimization
• Different conflictive criteria (economic, environmental issues, etc.)
• Rigorous approaches (weighted-sum method, ε-constraint method, goal-programming
method)
• Heuristic approaches (GA-based, SA-based, etc.)

Introduction 4
 Approaches for SCM

SCM models and tools

Transactional

Enterprise
Enterpriseresource
resourceplanning
planningsystems
systems(ERP)
(ERP)


Material
Materialresource
resourceplanning
planningsystems
systems(MRP)
(MRP)


Customer
Customerrelationship
relationshipmanagement
managementsystems
systems(CRM)
(CRM)


. .. .. .

Normative or optimization
approaches
Deterministic math. prog.
Analytical
Deterministic math. prog.


Stochastic
Stochasticmath.
math.prog.
prog.
(LP,
(LP,MILP,
MILP,NLP,
NLP,MINLP)
MINLP)

Descriptive approaches


Forecasting
Forecastingmodels
models


Simulation
Simulationmodels
models


. .. .. .

Introduction 5
 Analytical approaches for SCM
Planning-based approaches

Operations research
optimization models
LP,
LP, MILP,
MILP, NLP,
NLP, MINLP
MINLP
Deterministic
Deterministic // stochastic
stochastic
centralized
centralized approaches
approaches math.
math. programming
programming
with global information
with global information
sharing
sharing Demand-driven approaches

Discrete
Discrete event-
event-
based
based simulation
simulation
Deterministic
Deterministic // stochastic
stochastic
simulation centralized
centralized or
or
simulation decentralized approaches,
decentralized approaches,
variable
variable degree
degree of
of
information
information sharing
sharing

Introduction 6
 Deterministic simulation

Performance indexes Equations

Equationsororother
otherrelationships
relationshipsdescribing
describinghow
howthe
thestate
state
variables change over time as a function of the decisions
variables change over time as a function of the decisions
and
andexternal
externalevents
events

Index
Index Parameters
calculator
calculator

Simulation
Simulation Decision variables
Simulation outputs model
model

Simulation models for SCM 7

 Stochastic simulation

Probabilistic
Probabilistic
Probabilistic performance indexes
index
index calculator
calculator
Simulation
Simulationruns
runsare
are
repeated
repeatedseveral
several
times,
times, eachone
each onewith
with
Sampling
Sampling different
different values ofthe
values of the
tool
tool uncertain parameters
uncertain parameters
Performance indexes

Index
Index Uncertain parameters
calculator
calculator

Simulation
Simulation Decision variables
Simulation outputs model
model

Simulation models for SCM 8

 Discrete event-based simulation
Modeling principles: generic modeling unit for SCM

ORDER input (OR_in)

Generic modeling unit

MATERIAL output
Order
Delivering
reception (MAT_out)

ORDER output Material

Ordering
(OR_out) reception

MATERIAL input
(MAT_in)

Simulation models for SCM 9

 Discrete event-based simulation
Demand model: orders placed by customers ORDER input (OR_in)

Uniform quantities and times Variable quantities and uniform times

Quantities:
Quantities:
Normal
Normal distribution
distribution
Times:
Times:
Poisson
Poisson distribution
distribution

Uniform quantities and variable times Variable quantities and times

Simulation models for SCM 10

 Discrete event-based simulation
The state order reception States
Transition nodes
Sates and transitions
Logic conditions

ORDER input (OR_in)

Turning
Turningback
backto
tothe
the Idle
Idlestate
stateafter
afterthe
the
necessary time for updating the inventories
necessary time for updating the inventories

[O>5]/O-=a Inv1/
entry:
[O<=5]
[t>=Inv1_c+5]/a=b
Inv1_c=t;
exit:
IF-=0.6*a;
IG-=0.4*a;
[(IF<10)|(IG<10)]/O+=a H+=a;
Idle/
on ORin:
[(IF>=10)&&(IG>=10)] O+=b;

ORin{demand}/a=b Updating
Updating
inventories
inventories
Checking
Checkingavailability
availability of
ofFFand
andG G
of
of productsFFand
products andG G
Event
Event that
thatwakes
wakes
up the system
up the system

Simulation models for SCM 11

 Discrete event-based simulation
Inventory model (state ordering) ORDER output
Inventory control policies (OR_out)

Continuous

(s, k) System

(s, Q) System

(R, S) System

Periodic
s

(R, S) System
R

(R, s, S) System
Check
Check

(R, s, k) System
inventories
inventories Safety stock (s)
Net stock level
On-hand inventory

Simulation models for SCM 12

 Discrete event-based simulation
Performance measures: Key Performance Indicators (KPIs)

Economic: Total SC profit

decision scenario
scenario
decision variables
variables

expected
expected value
value

Social: Customer satisfaction

Environmental: Greenhouse
effect impact

Simulation models for SCM 13

 Software agents

An
Anagent
agent isis an
an encapsulated
encapsulatedcomputer
computer system
system that
that isis situated
situatedin
in
some
some environment,
environment, and
and that
thatisis capable
capable of
of flexible,
flexible, autonomous
autonomous
action
action in that environment in order to meet its design objectives.
in that environment in order to meet its design objectives.
Wooldridge
Wooldridge and
andJennings,
Jennings, 1995
1995

Autonomy: it should have some control over its actions and should work without
human intervention.

Social ability: it should be able to communicate with other agents and/or with human
operators.

Reactivity: it should be able to react to changes in its environment.

Pro-activeness: it should also be able to take initiative based on pre-specified goals.

Multi-agent system for SCM 14

 Agent-based simulation model
Manufacturing
plant
Suppliers
Retailer (external) Customers
Customer (external)
Raw material EMULATION Sales
extraction AGENTS Inventory
Customer
Factories Production
Warehouse Other agents
Manufacturing
plant Purchasing
Warehouses
Transportation

Distribution
Manufacturing centres
plant MULTI-AGENT
Customer
SYSTEM Retailers
Retailer

Material flows

Forecasting Information
module MESSAGES flows

Cash flows
Negotiation
module
Economic
indicators Financial
KPI indicators
MODULES
evaluators
Environmental
impact indicators
Optimisation
...
module
...

Multi-agent system for SCM 15

 Agent-based simulation model
Implementation of a multi-agent system

Real supply chain

Simulation model
(multi-agent system)

Central agent
• Exchanges messages
• Provides access to modules

Multi-agent system for SCM 16

 Sub-agents
Implementation of a multi-agent system

Tasks
• Storage of raw materials
• Production
• Storage of final products

Sub-agents
• Sales
• Inventory
• Purchases
• Transport

Multi-agent system for SCM 17

 Storage agent
Plants (storage), warehouses, distribution centers and retailers
Storage agent
Subagents: Sales, Inventory, Purchasing and Transport States and transitions
[O>5]/O-=a
Messages [O<=5] [t>=Inv1_c+5]/a=b
Inv1/
entry:
Inv1_c=t;
exit:

external customer IF-=0.6*a;

IG-=0.4*a;
customers [(IF<10)|(IG<10)]/O+=a H+=a;

manufacturing plant Idle/
[(IF>=10)&&(IG>=10)]
on ORin:
O+=b;

ORD_in

distribution center
ORin{demand}/a=b
MAT_out RORD_in
...
Confirm MAT_in s
u
p
p
RFQ RRFQ RFM MAT_in ORD_out ORD_out l
RORD_out RORD_out i
e
RFQ Notice_purch r
Inventory Purchasing
Sales subagent RRFQ
subagent subagent
Confirm Confirm s
RFM Review

RRFM
Notice_deliv
external supplier

manufacturing plant
Transportation
subagent

distribution center

...
MAT_out

Multi-agent system for SCM 18

 Stochastic simulation
Probabilistic
Probabilistic
Probabilistic performance indexes
index
index calculator
calculator

Sampling
Sampling
tool
tool
Performance indexes

Index
Index
Environmental
Environmental Uncertain parameters
Financial
calculator
calculator
Financial
(modules)
(modules)
Simulation
Simulation Decision variables
Simulation outputs model
model
Multi-agent
Multi-agentsystem
system

Multi-agent system for SCM 19

 Software implementation
Retailers, warehouses and plants
• C#, .NET of Microsoft ®
• SC model: XML files

Multi-agent system for SCM 20

 Software implementation
Simulation outcome

Multi-agent system for SCM 21

 Agent-based simulator

Framework for dynamic simulation of SCs

• Very few approaches in the literature for the CPI industry (Goodwin y col., 1997,
Fox y col., 2000, Swaminathan y col., 1998, Julka y col., 2002
• Centralized or decentralized decision-making
• Different production, storage and transportation policies
• Connection with different modules:
• Demand forecast
Multi-agent
Multi-agentsystem
system
• Planning, scheduling, etc.
• Flexibility

Lack of optimization capabilities!!

Multi-agent system for SCM 22

 Simulation-based optimization
Decision-making under uncertainty
orders

Order Order Order

Inventory fulfilled fulfilled
level in i
Inventory
decisions

Production
decisions

Final
sub-period products

period

Inventory
Inventory
control scheduling
controlpolicies
policies scheduling

Sim-Opt architecture 23
 Sim-Opt architecture
Meta-heuristic algorithms
Probabilistic (SA, GAs…)
Probabilistic
Probabilistic performance indexes Initial values
index
index calculator
calculator
Optimization
Optimization tools
tools
Sampling
Sampling
tool
tool
Performance indexes
Improved
Uncertain
Index values
Environmental
Environmental
Index parameters
Financial…
calculator
calculator
Financial…

Simulation
Simulation Decision variables
Simulation outputs model
model
Multi-agent
Multi-agentsystem
system

Sim-Opt architecture 24
 Case study Sim-Opt
SC design problem
Control policies Optimization design
outcome
GA
GA
Objective function

Probabilistic
Probabilistic index
index
Capacities evaluator
evaluator

Sampling
Sampling tool
tool
KPI
(profit)
Uncertain
parameters
Index
Index (demand)
evaluator
evaluator

Strategic-Tactical levels (capacities and inventory
control policies)
Multi-agent
Multi-agent

Uncertain demand (time and amount) Simulation system
system Decision
outcomes

System (R, s, S) for distribution centers variables
(inventory

System (s, S) for retailers control)

Maximize expected profit

Sim-Opt architecture 25
 Case study Sim-Opt
Results
R1

R2
F1 D1
R3

F2 D2 R4

Design I R5

R1

R2
F1 D1 R1
R3 R2
F1 D1
F2 D2 R4
R3
R5 F2 D2 R4
Design II
R5
Design III
R1 R1

D1 R2 D1 R2
F1 F1
R3 R3

F2 D2 R4 F2 D2 R4

Design IV R5 Design V R5

Sim-Opt architecture 26
 Case study Sim-Opt
Results

Tuning parameters

3,25—104

Large CPU times!!

Sim-Opt architecture 27
 Sim-Opt architecture
Meta-heuristic algorithms
(SA, GAs…) Improved
Probabilistic values
Probabilistic
Probabilistic performance indexes Initial values
index
index calculator
calculator
Optimization
Optimization tools
tools
Performance indexes

Sampling
Sampling Neural
tool
tool networks
Uncertain filter
Index
Index parameters
calculator
calculator Metamodel
Metamodel

Simulation
Simulation
Simulation outputs model
model
Multi-agent Filtering
Filtering rule
rule
Multi-agentsystem
system
Filtered decision
variables

Sim-Opt architecture 28
 Sim-Opt architecture

Region Minimization
Objective function

Simulator outcome
Metamodel outcome

Region
Best solution found so far

IF , , THEN DISCARD

Sim-Opt architecture 29
 Sim-Opt architecture
Tuning parameters
Genetic algorithm parameters
Maximum number of generations
Population size
Number of Monte Carlo samples
Probability of cross-over
Probability of mutation

Metamodel

Maximum tolerance
Frequency of metamodel training

Maximum number of simulations

Sim-Opt architecture 30
 Case study
SC with embedded multi-product plants Outer
optimization
Initial
Simulation values
outcome
GA
GA
Objective function
Control policies
Probabilistic
Probabilistic index
index
calculator
calculator Filter
Filter

Index Sampling
Sampling

Uncertain
Loonkar and Index
parameters
Index
calculator
calculator
Robinson, 1970

Integration of tactical-operational levels
Multi-agent
Multi-agent

Uncertain demand (quantity and amount) Simulation system
system Decision
outputs variables

System (R, s, S) for distribution centers and (s, S) for
retailers Scheduler
Scheduler Inner
(MILP)
(MILP) optimization

Maximize expected profit

Sim-Opt architecture 31
 Case study

Outer loop optimization

100 generations
50 samples (scenarios)
10 s/run (total time 28 hours)
2·104 simulation runs
Inner optimization
0,1-1 seconds
GAMS-CPLEX

Sim-Opt architecture 32
 Multi-objective Sim-Opt
Optimization
Optimizationoutcomes
outcomes Initial solution


Approximation
Approximationtotothe
thePareto
Pareto set
set


Improved
Improved values
valuesof
ofthe
thedecision
decision variables
variables Initial
values
Optimizer
Optimizer

Objective function
and constraints Probabilistic
Probabilistic
index
index
Monte Carlo

Sampling
Sampling tool
tool
Financial
Financialmodule
module KPI
Evaluate financial
Evaluate financial Uncertain
impact
impact parameters
Environmental
Environmentalmodule
module Index Meta-heuristic
Meta-heuristic
Index
Evaluate
Evaluateimpact
impactbased
based calculation algorithms
algorithms
on LCA principles
calculation
on LCA principles NSGA,
NSGA,NSGAII,
NSGAII, PSA,
PSA,
etc.
etc.
Simulation
Simulation
Simulation model
outcomes model
Decision
variables

Multi-objective Sim-Opt 33
 Environmental concerns
Environmental issues in SCM

TOTAL
TOTAL

TOTAL
cradle
cradle 2 1 Waste
Waste
2 3 1 2 3
1 2 3 Management Total impact
3
Environment
Environment 1 Management

Local impact (each echelon) grave (entire SC)

grave

Multi-objective Sim-Opt 34
 Life Cycle Assessment (LCA)
LCA
LCA Objective
Objectivestrategy
strategy to
to evaluate
evaluatethe
theenvironmental
environmental
loads
loads associated
associated with
with aaproduct,
product, process
process or
or activity
activity
(ISO
(ISO 14040
14040series
series on
on LCA)
LCA)

by to
toevaluate
evaluateopportunities
opportunities to
todo
do
byquantifying
quantifying energy
energy and
and materials
materials used
used
and improvements
improvements
and waste
wastereleased
released

ItItincludes
includes the
theENTIRE
ENTIRE LIFE
LIFE CYCLE
CYCLE of
of the
theproduct
product

Main drawback: a systematic procedure to search

alternatives for environmental improvements is lacking

Combine LCA with optimization tools

(Azapagic et al., 1999, Hugo and
Pistikopoulos, 2005)

Multi-objective Sim-Opt 35
 Case study
Multi-objective tactical-operational SCM problem

Uncertain demand (quantity and amount)

System (R, s, S) for distribution centers and (s, S) for retailers

Maximize expected profit

Minimize expected contribution to global warming

Multi-objective Sim-Opt 36
 Case study

Approximation to the Pareto set

100 generations
90 samples
Inventory profiles of
10 s/simulation
product B in distribution
1·105 simulations center D1
Total time 100 hours (NSGAII)

Multi-objective Sim-Opt 37
 Case study

GA parameters

Graphical user interface of the Life

Cycle Assessment module (LCA)

Graphical results
Multi-objective Sim-Opt 38
 Formal comparison
Mathematical Programming vs Sim-Opt approach

Based on algebraic equations
Based on logic rules (states and transitions).

Steady state models
Captures the dynamics of the system

Centralized system
Centralized/decentralized system

Simplifications
Realistic SC representation

Well established theory
Need to develop efficient optimization strategies

Available software (GAMS, AIMMS, etc.)
Customized solution (generic programming
languages: C++, C#, Java, Visual Basic, etc.)

Optimal solutions

Optimality is not guaranteed

Math. Prog. vs Sim-Opt 39

 Practical guidelines
Mathematical Programming vs Sim-Opt approach

Push systems (make-to-stock)
Pull systems (make-to-order)

Centralized system (global information
Decentralized system (partial sharing of
sharing) information)

Strategic/Tactical/Operational
Strategic/Tactical

Solve well defined design/planning/scheduling
Provides insights: capture dynamics and delays in
problems information flows

Optimization skills (math. prog.)
Software development skills (C++, Java, etc.)

Available decomposition strategies
Cannot avoid large CPU times

Optimal or near-optimal solutions
Optimality is not guaranteed

Math. Prog. vs Sim-Opt 40

 Conclusions

Multi-agent system for SCM

Decentralized SCs

Dynamics

Realistic industrial practices

Sim-Opt framework to improve the SCM

Use of meta-models to decrease CPU time

Multi-objective optimization

Conclusions 41