Chapter 01 Simulation Modeling With SIMIO A Workbook
Chapter 01 Simulation Modeling With SIMIO A Workbook
Chapter 01 Simulation Modeling With SIMIO A Workbook
The word “simulation” has a variety of meanings and uses. Probably its most popular usage is in games. A
video game “simulates” a particular environment, maybe a formula one race, a battle, or a space encounter.
The game allows the user to experience something similar to what it is like to drive in a race, maybe with other
people participating. In the military, commanders create a battlefield simulation where soldiers act according
to their training perhaps defending a base or assaulting an enemy position. In aeronautical engineering, an
engineer may take a model airplane to a wind tunnel and test its aerodynamics. Most simulations have
elements of reality with the intention that the participant will learn something about the environment through
the simulation or perhaps only learning to play the game better.
We employ simulation to study and improve “systems” of people, equipment, materials, and procedures. We
use simulation to mimic or imitate the behavior of systems like factories, warehouses, hospitals, banks,
supermarkets, theme parks – just about anywhere a service is provided or an item is being produced. Our
simulations are different from gaming and training simulations in that we want the simulation to model the
real system, so we can investigate various changes before making recommendations. In that sense, simulation
is a performance improvement tool. The simulation acts as an experimental laboratory, except that our
laboratory is not physical but instead a computer model. We can then perform experiments on our computer
model.
Modeling
Many performance improvement tools (i.e., Lean, Six Sigma, etc.) rely on models. For example value stream
maps, spaghetti diagrams, process flow charts, waste walks, etc. are useful conceptual/descriptive models for
performance improvement based on direct observations of the system. These models provide a wide-range of
vehicles for describing and analyzing various systems. More formal models employ mathematical and
statistical methods. For example, linear regression is a popular modeling technique in statistics. Queuing
models offer a means for describing an important group of stable stochastic processes. Linear programming is
a formal optimization method of finding the values of variables that minimize or maximize a linear objective
function subject to linear constraints. Nevertheless, all these methods require a variety of assumptions about
the system being modeled. For example, the variables are constants (i.e., real or integer valued) and often
related linearly. If the variables have statistical variation, they are assumed to be normally or exponentially
(i.e., Markovian) distributed.
Simulation is a model-based improvement tool. However, few assumptions need to be made to build the
model. The model can be non-linear, described by arbitrary random variables, have a complex relationship,
and change with time (i.e., dynamic). In fact, the simulation model is limited only by your imagination and
the nature of the system being considered. You determine the nature of the model, based on what you think is
important about the system being studied.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
performance measures. Based on what we learn about the system, we modify the computer simulation models
to study alternative systems (i.e., experiment). By comparing these alternative systems statistically, we are
able to offer performance improvement recommendations.
Of course, we could just experiment directly with the system. If we thought an additional person on the
assembly line would improve its production, we could try that. If we thought a new configuration of the
hospital emergency room would provide more efficient care, we could create that configuration. If we thought
a new inventory policy would reduce inventory, we could implement it. But now the value of a computer
simulation model becomes apparent. A change in a computer simulation model is clearly cheaper and less
risky as compared to changing the real system. It is also faster to use a computer simulation to determine if
the changes are beneficial. Also, it might be safer to try a change in a simulation model than to try it in real
life. In general, it is much easier to try changes in a computer model than in a real operating system,
especially since changes disrupt people and facilities.
The word “verification” refers to the model and its behavior. We most often develop our simulation models
with a simulation language. This language translates our modeling “intent” into a computation structure that
produces output statistics. Most simulation languages can be used to describe a complex operating system and
the language provides a framework of components for viewing that system. As our models become more
complex, we employ more complex simulation language constructs to model the behavior. This relationship is
critically dependent on our understanding of the simulation language intricacies. We may or may not fully
understand computational code and thus we confront the key question in verification: Does the simulation
model behave the way we expect? Is it possible that we may have made a mistake in employing the simulation
language or perhaps the simulation language creates an unexpected behavior in the computational code?
Therefore, even after we have created our simulation model, it needs to be tested to see if it behaves as
expected. If we make the processing time longer, does it result in longer times in the system? If we reduce the
arrival rate to a queue, does the waiting time decrease? The answers to any of these questions are specific to
the system and your model of it. But, above all else, we need to be sure that our model is behaving the way
we expect without error.
The word “validation” refers to the relationship between the model and the real system. Does the simulation
produce performance measures consistent with the real system). Now, we are assuming our model has been
verified but can we validly infer to the real system? This question strikes at the heart of our modeling effort
because without a valid model we cannot legitimately say much about performance improvement. Many
people new to simulation may want the model to be a substitute for the “real system” and that is a limitless
task. After all, the only “model” that is perfectly representative of the real system is the system itself!
We must always remember in simulation that our model is only an approximation of the real system. So the
most relevant way to validate our model is to concern ourselves with its approximation. How do we decide
what to approximate? It depends on our performance measures. If our performance measure is time in the
system, then we concern ourselves with those factors that impact time in system. If our performance measure
is production, then we focus on those factors that impact production. Usually we are interested in several
2 Simulation Modeling with SIMIO: A Workbook
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
performance measures, but those measures will be the focus of our concern and will limit our modeling
activity. Otherwise without a clear set of performance objectives, we are left with a search for reality in our
model, which is a never-ending task.
Generally, people who have only a general familiarity with simulation, think simulation models can mimic
anything and often drive the development of a model that is needlessly complex. One of the difficult
responsibilities of anyone engaged in simulation modeling is the education of the stakeholders on the benefits
and limitations of a simulation model as well as the simulation modeling activity.
We have chosen to use the simulation language SIMIO which is a relatively new language, having been
developed over the past several years. The developers of the language had previously developed the Arena
simulation system. SIMIO benefits from more recent developments in object-oriented design and agent
modeling. SIMO is a “multi-modeling” language having agents as well as discrete event and continuous
language components. SIMIO was developed to provide visual appeal through its 3-D animation and
graphical representations. SIMIO provides a wide range of extensions, from direct modification of executing
processes to user developed objects. SIMIO also provides interoperation with various spreadsheets and
databases. Finally, SIMIO has become widely adopted in industry as well as academic institutions. Learning
SIMIO provides you with one broadly-based simulation modeling tool which can help you learn to use others
if the need arises.
A fundamental understanding of simulation will be beneficial throughout your study of simulation. It’s easy
to get caught up in the creation of a computer model using a simulation language and miss important basic
principles of simulation modeling. Too many people associate simulation with a simulation language and for
them, simulation is simply learning a simulation language. Learning a simulation language is necessary to
using simulation, but it does not substitute for understanding at a fundamental level. If you understand the
fundamentals of simulation, then you establish a basis for understanding any simulation language and any
simulation model. In fact, this understanding will be a key to learning almost everything else in this text.
Question 1: If you owned or managed the ice cream store, what might be your operational concerns?
________________________________________________________________________________
Likely one of your most prominent concerns would be this store’s operation and how you can improve it. For
example, could you buy a new cone making machine to make cones faster or should you hire someone to help
service customers? Should you resize the waiting space? These are performance improvement concerns.
Introduction to Simulation: The Ice Cream Store 3
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Figure 1.1: The Ice Cream Store
Question 2: If you made one of these changes to the ice cream store, how would you decide if it improved
the store’s operation?
________________________________________________________________________________
In other words, what are the performance measures? Here are some possible measures: number of customers
served per day, time customers spent in the store, the waiting time of customers, the number of customer
waiting, and the utilization of servers. We will discover these are common performance measures and will
often be your first performance measurement choices.
Question 3: How can you expect to improve performance if you don’t know what is going on inside the
ice cream store?
________________________________________________________________________________
You need to employ all the descriptive tools you know to understand what is going on inside the system. At
this stage, one might create a value stream map, a flow chart, a relationship chart, a spaghetti diagram, etc.
These techniques will greatly improve your understanding of what happens to customers and the attendant
during the sale of an ice cream cone. Perhaps you develop the conceptual model (i.e., flow chart) presented in
Figure 1.2.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Table 1.1: Direct Observation Time Study (Event View) of Ice Cream Store
Time Customer Process
0 Store Opens – Server Idle
Arrives –
0 1
Start service on Customer 1 – Server Busy
8.36 1 Departs service – Server Idle
Arrives –
9.01 2
Start service on Customer 2 – Server Busy
9.98 3 Arrives – Customer 3 waits
13.5 4 Arrives – Customer 4 waits
2 Departs Service – Customer 2 leaves:
19.36
3 Start service on Customer 3 – Server Busy
23.07 5 Arrives – Customer 5 waits
3 Departs service – Customer 3 leaves:
27.22
4 Start service on Customer 4 – Server Busy
4 Departs service – Customer 4 leaves:
33.82
5 Start service on Customer 5 – Server Busy
38.18 6 Arrives – Customer 6 waits
40 End observations
In our time study, we simply record all the “events” and the event time that occurs in the ice cream store. An
event occurs when something operationally happens in the store, like an arrival of a customer or the departure
of a customer from service. Also we recorded when the attendant (i.e., server) becomes busy or idle. Here we
only observed the first 40 minutes of the store’s operation. We recognize this isn’t really enough time to gain
a full understanding, but we are not intending to solve a problem at this time – only to demonstrate a method.
The time study is sufficient for us to compute some performance measures, but first it will be helpful to re-
organize our time study data. We are not going to add (or subtract) any information. We are simply going to
re-organize our data from an event view to an entity (i.e., customer) view. The event view recorded events but
the entity view allows us to follow our customers. The re-organized data is shown in Table 1.2 which shows
when the customer arrives to the store, enters service, and leaves the store. Notice, customers five and six
service and exiting store happen after the 40 minutes.
Table 1.2: Re-organized Event Data in Customer View From
Customer Arrives to Store Enters Service Leaves Store
1 0.00 0.00 8.36
2 9.01 9.01 19.36
3 9.98 19.36 27.22
4 13.50 27.22 33.82
5 23.07 33.82 ??
6 38.18 ?? ??
From the entity view of the time study data, we can easily compute a number of performance measures.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
The time in system is calculated by averaging the difference when the customer entered the system
from when they exited the system. Again, only four customers exited the system and contributed to the
average time in system in our simple ice cream system as seen in Table 1.3.
Table 1.3: Calculating Average Time in System for the Ice Cream Store
Customer 1 Customer 2 Customer 3 Customer 4 Average Time in System
8.36-0.0 19.36-9.01 27.22-9.98 33.82-13.5 56.27/4 = 14.07 minutes
Figure 1.3: Graph of the Number in Queue in the Ice Cream Store
Consider, computing the average number in queue. Suppose we observed the number in queue to be
two one time and ten another time. Would you say that on average number we would expect to see six
in line? Of course “not”! You need to know how long the value of two was observed and how long
the value of ten was observed. Suppose we observe two in queue for ten minutes of the time and ten
in the queue for only one minute. So the queue was observed a total of 11 minutes. As a result, the
value of two was observed 10/11ths of the total time while ten was observed 1/11th of the total time.
Our average then is 2*(10/11) + 10 *(1/11) or = 2.73 customers. Or another way of computing is to
realize that the total waiting time2 observed was 20 + 10 over a total of 11 minutes which also yields
2.73 people. Now, looking at the data from the ice cream store, we need the percentage of time (40
minutes) that there was zero waiting, one waiting , two waiting , three and so forth3. In our case, a
maximum of two was observed and the time waiting would be computed as seen in Table 1.5.
2
Total waiting time is also the “area” under the curve in Figure 1.3.
3 We don’t include 0 since it makes no contribution to the total waiting time.
6 Simulation Modeling with SIMIO: A Workbook
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Table 1.5: Calculating Average Number in Queue
Number In Queue Time Period Time Spent (min)
0 0.00 – 9.98 0*9.98 = 0.000
1 9.98 – 13.5 1*3.52 = 3.520
2 13.50 – 19.36 2*5.86 = 11.72
1 19.36 – 23.07 1*3.71 = 3.710
2 23.07 – 27.22 2*4.15 = 8.300
1 27.22 – 33.82 1*6.60 = 6.600
0 33.82 – 38.18 0*4.36 = 0.000
1 38.18 – 40.00 1*1.82 = 1.820
Total Waiting Time 35.67 minutes
Average Number in Queue 35.67/40 = 0.89 Customers
· Utilization:
This performance is the percentage time the server is busy servicing customers which is calculated by
dividing the time spent servicing customers divided by the time available. For our example, the
attendant is only idle during the time period between finish servicing customer one and the arrival of
customer two (i.e., 9.01 – 8.36 = .65 minutes). Therefore, the attendant’s utilization will be 39.35/40
or 98.4%:
· Other possible performance measures are maximum values, standard deviations, time between
departures, etc.
Table 1.6: Types of Performance Simulation Measures/Metrics
Type Description/Examples
Counts The number of parts that exited, entered, etc. (e.g., production).
These measures typically deal with time like waiting time, time in system
Observation-based Statistics
etc. SIMIO calls observation-based statistics “Tally” statistics.
Time-Persistence/Time- These measures deal with numbers in system, queue, etc. when the values
Average Statistics can be classified into different states. SIMIO calls these “State” statistics.
Question 7: Is the amount of time that a job is late a time-persistent or observation-based statistic?
________________________________________________________________________________
Figure 1.4 shows some elements of a “common” queueing system with the normal terminology. In general we
will refer to the arriving objects as “model entities” and the counter/attendant as a “server”. You can see the
members of the queue and the customer in service. The input processes for this system are the arrival process
and service processes. We will provide details on these inputs later.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Ice Cream Store
4 Counter
Arriving 2 Departing
46 35
Customers Serviced Customers
(Model Entity)
Queue (FIFO) Customer in Service
We want to be able to reproduce the time-study data collection exercise that we used for the ice cream store.
But we want to synthesize it numerically as opposed to observing it – in essence simulate it! If we can re-
create the time study data, we can compute the performance measures. Notice that the time study records are
centered on “events”. Recall these events are points in time when the system changes it state (i.e., status). A
quick review of that data reveals the following three events occur.
Let’s add some specifics to our simulation problem (i.e., the ice cream store). We will use “minutes” as our
base measure of time. The input data that we currently have is shown in Table 1.8.
To synthesize the system as presented in the time study, we need a systematic means of moving through time
by removing and inserting events on the event calendar. Consider the simulation algorithm shown in Figure
1.5.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Algorithm Step 1: Setup the simulation (i.e., initialize the system)
Algorithm Step 2: Remove the next event from the event calendar
Algorithm Step 3: Update simulation time (i.e., TimeNow) to the time of that event
Algorithm Step 4: Execute the processes associated with the event (i.e., adding additional events
as needed and collecting statistics)
Arrival Event Process (see Figure 1.6)
Departure Event Process (see Figure 1.7)
End Event
Algorithm Step 5: Repeat Algorithm Steps 2-4 until complete
Figure 1.5: Simple Simulation Algorithm
Remember that the event calendar is ordered according to the next most recent event. So we can move
through time by removing the “next” event from the event calendar, updating time to the time of that event,
and executing whatever processes are associated with that event. This simple method is then repeated until we
reach some terminating time or condition.4 In our case, the “events” are the arrival of an entity, the service
(departure) of the entity, and the end of the simulation period. The arrival and departure of the entities are the
most important. Consider now how the arrival (see Figure 1.6) and the departure (see Figure 1.7) are
processed within the context of our simple single queue, single server system. The word “schedule” means to
insert this event into the event calendar.
Yes
Schedule Next Arrival Entity Goes Into
Arrival of Entity Is Server Busy Arrival Event Ends
(Create a New Event) Queue
No
Start Service
Resource
Schedule End of
Becomes Busy
Service Event
No
Other Resource End of Service Event
End of Service Entity Exits System
Entities Waiting Becomes Idle Ends
Yes
First Entity in
Schedule End of
Queue Starts
Service Event
Service
The arrival of an entity creates a new event because we know the next entity’s arrival time since we have the
interarrival times of entities. The newly arriving entity either must wait because the server (i.e., resource) is
busy or that entity can engage the server and start service. If the entity can start service, we now know a new
future event, namely the service departure because we know the processing time. So if the removed event is
an arrival, we insert a next arrival into the event calendar and may insert the service departure event in the
event calendar provided the entity can start service immediately. If the event removed from the event calendar
is a service departure, then the entity that has finished service will exit the system. If the waiting queue is
empty, then the resource (server) becomes idle. On the other hand, if there is at least one customer in queue,
4
The practice of removing the “next” event has caused some to refer to our simulation as a “next or discrete event simulation”.
Introduction to Simulation: The Ice Cream Store 9
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
then the first customer is brought into service and the resource remains busy and a new future event, namely
the service departure is inserted into the event calendar because we have the processing time. Note that a
service departure causes the entity to depart the system regardless of what happens to the server.
Question 8: What is the maximum number of new events that are added to the event calendar when an
entity arrival event occurs? What are they?
________________________________________________________________________________
Question 9: What is the minimum number of new events that are added to the event calendar when an
entity arrival event occurs?
________________________________________________________________________________
Question 10: What is the maximum number of new events that are added to the event calendar when an
entity departure event occurs? What are they?
________________________________________________________________________________
Question 11: What is the minimum number of new events that are added to the event calendar when an
entity departure event occurs?
________________________________________________________________________________
Finally, we note that the very first step of the simulation algorithm calls for the system to be “initialized.” In
other words, how will we start operation relative to the number of entities in line and the state of the server? It
will be convenient to use the “empty and idle” configuration. By “empty and idle” we mean that the server is
idle and the system is empty of all entities.
To further understand how a discrete event simulation operates, a simple data structure will be employed to
execute a manual simulation as seen in Figure 1.8. Our manual simulation will consist of a “system animation”
graphical representation of what is going on in the system with the current customer being serviced shown
inside the square and other customers waiting outside the square. The current simulation time (i.e., called
“Time Now”) will be shown. The current event will be identified, followed by a description of the process.
Finally, the “Event Calendar” will be maintained, consisting of the event time and typ and the entity ID
number.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Figure 1.9: System Initialization
Step 2: Executing the simulation algorithm, we remove the next event (Algorithm Step 2) from the event
calendar, update simulation time to the time of that event (Algorithm Step 3), and execute the appropriate
processes (Algorithm Step 4). The next event is the “Arrival of Entity #1” at time 0.0. The arrival of this
entity allows us to schedule into the Event Calendar the “Arrival of Entity #2” to occur since the
interarrival time between Entity #1 and Entity #2 is 9.01 minutes and thus the Arrival event for Entity #2
is at time 9.01 (i.e., 0.0 + 9.01). Because Entity #1 arrives when the server is idle, that entity enters
service and we can now schedule its Departure event because we know the processing time for Entity #1
as 8.36 minutes thus the event time is 8.36 (i.e., 0.0 + 8.36). The result is that our data structure now
appears as Figure 1.10.
Question 13: Why do we compute the new event times as 0.0 +? What does the 0.0 mean?
________________________________________________________________________________
Step 3: We are done with the processes associated with time 0.0. Now, the next event in the event calendar
is the “Departure of Entity #1” from the system at time 8.36. Since there are no entities in the queue, the
server is allowed to become idle and no new events are added to the event calendar as seen in Figure 1.11.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Figure 1.12: At Time 9.01
Step 5: The next event occurs at time 9.98 which is the arrival of Entity #3. Its arrival allows us to
schedule the next arrival (i.e., Entity #4) at time 13.50. But now the arriving entity must wait which is
indicated in the “System Animation” section of the data structure in Figure 1.13.
Step 6: The next event is the arrival of Entity #4, which will schedule the arrival of Entity #5, but will have
no other actions since the server remains busy. The new simulation time is 13.50 and the updated status is
shown in Figure 1.14
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Question 17: What new events are added as a result of this event? (Give the event time, the entity, and the
event type?
________________________________________________________________________________
Step 8: From Figure 1.15, we see the next event is the arrival of Enity #5 at time 23.07 which triggers the
addition of the next arrival (Entity #6).
Question 19: What new events are added as a result of this event? (Give the event time, the entity, and the
event type)?
________________________________________________________________________________
Step 10: The result of this next event is shown in Figure 1.18.
Question 21: What new events are added as a result of this event? (Give the event time, the entity, and the
event type?
________________________________________________________________________________
Step 11: We are getting closer to the time to quit observing the system; however we have one more event
and one more time update. The result of this next event is shown in Figure 1.18.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Figure 1.19: At Time 38.18
Step 12: Finally, the next event calls for the “End” of the simulation at time 40.0. Only time is updated as
the event calendar is unchanged. The final state is given in Figure 1.20.
Since we now have simulated the ice cream store time study, we can re-organize this information from an
entity viewpoint, and compute the same performance statistics that we computed previously!
Question 22: Do we have enough information to draw conclusions about the present system?
________________________________________________________________________________
If we had a total of 45 interarrival times and 43 processing times, we could simulate a total of 480 minutes of
time. For example, we may obtain the information given in Table 1.9. We have included the minimum,
maximum, and average values over the 480 minutes. Currently we simulated “only” one day of 480 minutes.
Question 24: Now do you have enough information for a “present systems analysis?
________________________________________________________________________________
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Table 1.9: Final Performance Measures for 480 Minutes
Performance Measure Value
Total Production 43
Average waiting time in queue 9.59
Maximum waiting time in queue 35.65
Average total time in system 17.87
Maximum total time in system 42.65
Minimum total time in system 7.46
Time-average number of parts in queue 0.88
Maximum number of parts in queue 4
Ice Cream Attendant utilization 78%
One day doesn’t give us much information about the day so more “days” of information is needed. But that
means more interarrival times and more processing times. If we did a simulation of ten days, we would need
approximately 450 interarrival times and 430 processing times. If that information came from an electronic
record, then getting more data may not be a problem. 5 But if our only alternative is time studies, then we will
need to observe 10 days and that may be a greater intrusion into the actual operation than we can expect.
An alternative to more data collection is for us to find a “model” of this input data (i.e., interarrival and
processing times). Perhaps we can use a statistical representation since processing times and interarrival times
are most certainly random variables. We can often match the processing times and interarrival times to
standard statistical distributions. First we make a histogram of the data we observed. Second, we try to pick
out a statistical distribution, like a Gamma, Lognormal, Weibull, or Pert, to match the data. In fact, there is a
wide range of software that can help with this undertaking. An example of this activity is shown in Figure
1.21.
5
An implication of this need for more data is that performance improvement should be one of the bases on which an information
system is designed. The information should not be limited to accounting and reporting, but also performance improvement.
Introduction to Simulation: The Ice Cream Store 15
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Once we characterize an input with a statistical model, there is simulation technology6 that allows us to
“sample” from that statistical model. Repeated sampling will statistically reproduce the input model, but more
importantly give us an “infinite” supply of input data. With that extended data, we can simulate many days of
operation. Having many days of simulation provides us with the opportunity to gain an estimate of the
precision of the average performance measures we compute. For example suppose we simulate our system for
five days. Using simulation terminology, we performed five “replications” or “runs” of our simulation model.
Each replication obtains new “samples” for its interarrival times and processing times. To compute the
average and the standard deviation, we use the averages from each day (one day yields one average). The
results are shown in Table 1.10.
Question 27: Why only the averages instead of all the queue waiting times within a given replication?
________________________________________________________________________________
We know that these statistics, like the average and the standard deviation, are themselves random variables. If
we looked at another five days (either simulated or real) we wouldn’t get the exact same results because of the
underlying variability in the model. We need to have some idea about how precise these summary statistics
are. In order to judge how precise a given statistic is, we often use a confidence interval. For example, we
computed the average waiting time as 13.47, but does this estimate have a lot of variability associated with it
or are we pretty confident about that value?
Question 28: Confidence intervals in statistics are based on what famous distribution as well as famous
theorem?
________________________________________________________________________________
Any observations for which we want to create a confidence interval must satisfy the assumptions that the
observations are “independent and identically” distributed. As a consequence the Central Limit Theorem
(CLT) can be employed to compute a confidence interval.
Question 29: Are the observations, say of the waiting times, during a simulated day independent and
identically distributed?
________________________________________________________________________________
Question 30: For example, would the waiting time for entity #1, entity #2, and entity #3 be independent of
each other? Would you expect the distributions of these waiting times to be identical?
________________________________________________________________________________
Question 31: Would the number waiting in the queue at 9am be independent of the number waiting at
8:45am? Would the distribution of the number waiting be identical?
________________________________________________________________________________
6
Random number and random variate generation methods.
16 Simulation Modeling with SIMIO: A Workbook
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Almost no statistic (performance measure)7 computed during a single simulation replication will be
independent and identically distributed (e.g., the waiting time of entity #4 maybe dependent on entity #3
waiting and processing times). So instead, we use, for example, the average values computed over the day as
an observation (i.e., we will determine the average of the averages).
Question 32: Are daily averages independent and identically distributed? Why?
________________________________________________________________________________
Since these daily averages (or maximums or others) are independent and identically distributed, we can
compute confidence intervals for our output statistics (based on the Central Limit Theorem).
where is the sample mean, is the sample standard deviation of the data, is the upper critical
point from the Student’s t distribution with degrees of freedom, and is the number of observations
(i.e., replications). Note, is often referred as the standard error or the standard deviation of the mean. As the
number of replications is increased, the standard error decreases. So a 95% confidence interval of the expected
time in the queue using the data from Table 1.10 would be computed as the following calculation.
Question 34: Would you bet your job that the “true” average waiting time is 13.47 minutes?
________________________________________________________________________________
So another way to express the confidence interval is given by the Mean ± the Half-width, as in
which SIMIO will use. The Central Limit Theorem allows the calculation of the confidence interval by
asserting that the standard error (or the Standard Deviation of the Mean) can be computed by dividing the
standard deviation of observations by the square root of the number of observations. If the observations are
not independent and identically distributed then that relationship doesn’t hold. As n increases sufficiently large
the CLT and the inferential statistics on the mean of the population become valid.
Question 35: Does the confidence interval we computed earlier (i.e., ) mean that 95% of the
waiting times fall in this interval?8
________________________________________________________________________________
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
Question 36: Does it mean that if we simulated 100 days that 95% average daily waiting times would fall in
this interval?
________________________________________________________________________________
Question 37: Or looked at another way. Can we say that there is a 95% chance that the “true, unknown”
overall mean daily waiting time would fall in this interval?
________________________________________________________________________________
So the confidence interval measures our “confidence” about the computed performance measures. A
confidence interval is a statement about the mean (i.e., the mean waiting time), not about observations (i.e.,
individual waiting times). Confidence intervals can be “wider” than one would like. However, in simulation,
we control how many days (i.e., replications) we perform in our analysis thus affecting the confidence on the
performance measures.
Question 38: Using simulation, how can we improve the precision of our estimates (reduce the confidence
interval width?
________________________________________________________________________________
So we can run more replications in our simulation if we want “tighter” confidence intervals.9
In simulation, we usually refer to a single model as a simulation “scenario” or a simulation “experiment”. Our
model of the present ice cream store is a single scenario. However using simulation as a performance
improvement tool, we are interested in simulating alternative models of, for example, our ice cream store. We
would refer to each model as a scenario. So, for example, if we added a new ice cream making machine, this
change would constitute a different simulation scenario. If we started an ad campaign and expected an
increase in business in the ice cream store, we would have yet another scenario. In general, we would
typically explore a whole bunch of scenarios, expecting to find improvements in the operations of the ice
cream store. The simulation is our experimental lab.
So let’s reconsider a different scenario for our ice cream store. What would happen if the arrival rate was
increased by 10% (i.e., more customers arrive per hour owing to an ad campaign)? We could reduce the
interarrival time by 10% and make five additional replications of the 480 hour day. The results of the original
and this alternative scenario are shown in Table 1.11 along with the confidence intervals.
9
Note that we can also create a smaller confidence level by using is a larger significance level or α value)
18 Simulation Modeling with SIMIO: A Workbook
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
When comparing different sets of statistics like this, we would resort to the Student’s t-test. A way to conduct
the t-test is to compare confidence intervals for each of the two scenarios. If the confidence intervals overlap
then we will fail to reject the null hypothesis that the mean time in system for the original system equals the
new system.
The 95% confidence interval for the original five days was [6.94, 20.00] while for the increased arrival rate,
the confidence interval is [12.48, 29.76]. Now comparing the confidence intervals, we see that they “overlap”,
meaning we cannot say there is a statistical difference in the average waiting time. Without a statistical
difference, any statement about the practical difference10 is without statistical foundation.
Question 39: What can we do to increase our chance of obtain a statistical difference?
________________________________________________________________________________
Although it is probably unnecessary to have a statistical foundation for every recommendation, we strive to do
so to avoid the embarrassing situation of making a claim that is later shown to be erroneous – especially since
our job may be at risk. By striving to have a statistical foundation for our recommendations, we take
advantage of our entire toolbox in decision-making and promote our professionalism.
The entire simulation study is composed of a number of elements, which we present here. Although these are
presented in a sequential manner, rarely is a simulation study done without stopping to return to an earlier
issue whose understanding has been enhanced. In many instances we work on several of the elements at the
same time. But regardless of the order, we usually try to complete all the elements.
· Understand the system: Getting to know and understand the system is perhaps the most intense step and
one that you will return to often as you develop a better understanding of what needs to be done.
· Be clear about the goals: Try to avoid “feature or scope creep”. There is a tendency to continue to
expand the goals well beyond what is reasonable. Without clear goals, the simulation effort wanders.
During this phase identify the performance measures of interest that will be used to evaluate the
simulation.
· Formulate the model representation: Here you are clearly formulating the structure and input for your
model. Don’t spend a lot of time doing data collection at this point because, as you develop the model, its
data needs will become clearer. Also be sure to involve the stakeholders in your formulation, to avoid
missing important concerns.
· Translate your conceptual representation into modeling software, which in our case is SIMIO. A lot of
time is spent learning SIMIO so you have a wide range of simulation modeling tools with which to build
this and other models.
· Determine the necessary input modeling: At some point data will need to be collected on the inputs
identified during the formulation and translation of the system into a computer model. The initial
simulation model can be used to determine which inputs are the most sensitive with regard to the output
which need to be collected. Fitting distributions is generally better but expert opinion can be used to get
the model up and running.
· Verify the simulation: Be sure the simulation is working as it is expected without errors. Do some “stress
tests” to see if it behaves properly when resources are removed or when demand is increased. Explain any
10
A practical difference means that the difference is important within the context of the problem. When we are concerned with a
practical difference of unimportant or cheap items, then a statistical difference is not necessary.
Introduction to Simulation: The Ice Cream Store 19
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]
“zeros” that appear in the output. Don’t assume you are getting counter-intuitive results when they may
just be wrong.
· Validate the model: How does the model fit the real world? Is the simulation giving sufficient behavior
that you have confidence in its output. Can you validly use the model for performance improvement?
· Design scenarios: Determine which alternatives you think will improve the performance of the present
system and create the alternative simulation models and associate the models with scenarios.
· Make runs: do the simulation experiments for the scenarios. Be sure your simulation output generates the
appropriate performance measures. Make multiple runs for each scenario.
· Analyze results and get insight: Examine carefully the output from the scenarios and begin to develop
judgments about how to increase the performance of the system. Be sure the statistical analysis supports
your conclusions.
· Make Recommendation and Document: Be sure to discuss the results with all the stakeholders and
decision-makers. Make sure you have addressed the important problems and developed feasible
recommendations. Document your work so you or someone else can return one year later and understand
what you have done.
If you have worked carefully through this chapter, you will have a fundamental understanding of simulation
that is completely independent of the simulation software or any particular application. Some of the key
points have been.
Licensed to: Patricio Matus Carrillo - Avenida Los Carrera 2114 Concepción 4030000 - [email protected]