Lecture Notes in Output Analysis in

Simulation

Adeeba Alam Ansari1

November 16, 2024

1 Department of Industrial and Systems Engineering,

Indian Institute of Technology Kharagpur, West Bengal 721 302, India,
E-mail: [email protected]
Output Analysis
Output analysis in simulation refers to the process of interpreting the re-
sults generated by a simulation model to draw meaningful conclusions about
the system being modeled. Simulations typically generate outputs based on
random numbers drawn from sample generators, which represent probabilis-
tic processes that reflect the inherent variability and uncertainty of the real
system. These outputs help us understand how the real system might be-
have under different conditions, providing probabilistic estimates of system
performance rather than deterministic predictions. This allows us to eval-
uate potential outcomes and risks, guiding decision-making in the face of
uncertainty.

0.1 Example of Single Server Queuing Sys-

tem

Customers
x Server
x x x x

Figure 1: A Single Server Queuing System

In this figure, customers are served by a single server, and while one
customer is being served, others wait in a queue. If we obtain the waiting
times for each of these customers, denoted as W1 , W2 , ..., Wn , can we infer
something about the original system? How can we validate the model and
determine whether it truly replicates the behavior of the actual system?
We get the waiting time of customers and obtain the mean waiting time as
n
1X
W̄ = Wi , i ∈ {1, 2, . . . , n}
n i=1

We run the model for multiple replications and obtain the value of W̄ . This
is because we want W̄ to be the unbiased estimator of waiting times. If
we obtain the mean from just the first replication, W̄ won’t be an unbiased
estimator, since the Wi would not be independent and identically distributed
(I.I.D.), and Wi s would be correlated.

1
 n1
1 X
W̄1 = W1j , j ∈ {1, 2, . . . , n1 }
n1 j=1
n2
1 X
W̄2 = W2j , j ∈ {1, 2, . . . , n2 }
n2 j=1
..
.
nm
¯ 1 X
Wm = Wmj , j ∈ {1, 2, . . . , nm }
nm j=1

To determine the average waiting time we take the mean of average replica-
tion times across replications.
m
1 X
W̄ = W̄i
m i=1

[W̄1 , W̄2 , W̄3 , . . . , W¯m ] provides confidence interval to check if the original
waiting time of system lies within this. Before analysing the output from
this single server queue we must verify and validate our model.

0.1.1 Verification
Verification is the process of ensuring that the model has been implemented
correctly. This involves walking someone through the code, adding comments
for clarity, and testing with different inputs to confirm that the model behaves
as expected and is free from errors.It also includes checking that the logic
and algorithms align with the model’s specifications.

0.1.2 Validation
Validation is the process of assessing how closely the model resembles the
true system. It involves comparing the model’s output to real system to
evaluate its accuracy and consistency. The goal is to ensure the model reliably
represents the system.

2
0.2 Types of Simulation
The type of simulation depends on the nature of the system being modeled.
It could be a non-terminating simulation, which models systems that oper-
ate continuously over time, such as an airport. Alternatively, it could be a
terminating simulation, which models systems with a defined start and end
time, such as the operation of a bank.

0.3 Warm Up Period

In simulation modeling, the warm-up period refers to the initial phase of the
simulation where the system is not yet in a steady state.The steady state is
the point at which the system’s behavior becomes consistent over time, and
the performance measures (such as waiting times, queue lengths, etc.) are
stable and reflective of the long-term behavior of the system.

Statistic

t0 Time
Do all systems have a queue length ? In terms of queue length, the
system is in a steady state when the average queue length remains constant
over time. This means that the rate at which customers or tasks arrive
and are processed is balanced in the long run, and the queue does not grow
indefinitely. A system is in steady state when ρ = µλ , is greater than 1, i.e.,
the service rate µ is smaller than the arrival rate λ. In unsteady state the
queue keeps on increasing always and servers are always busy. If we observer
this pattern in our model ,it implies that it is transient.

0.4 Measures of Performance

Performance measures in simulation are metrics used to evaluate the effec-
tiveness and efficiency of a system.We discuss two of them here -: Steady

3
State Parameter and Steady State Cycle Parameter.

0.4.1 Steady-State Parameters

They describe the system’s behavior once it has reached a steady state, where
it behaves consistently over time. Example - Utilization of server.

0.4.2 Steady-State Cycle Parameters

These parameters are particularly useful in systems where entities follow
periodic or regular processes such as situations with multiple shifts. During
day and night shift the parameters are different.

0.5 Construction of Confidence Intervals/Prediction

Intervals
Let X1 , X2 , . . . , Xn be independent and identically distributed random vari-
ables with mean µ and variance σ 2 . Then, by the Central Limit Theorem,
as n → ∞, the sample mean X̄ follows

σ2
 
X̄ ∼ N µ, .
n
X
Xi ∼ N (nµ, nσ 2 )
2
Further if X1 , X2 , . . . , Xn belong to N (µ, σ ), then the sample mean X̄
2
belongs to N µ, σn for any value of n.

(n − 1)S 2
∼ χ2n−1
σ2
Here, the sample variance follows a chi-squared distribution with n−1 degrees
of freedom.
For T Statistic
X̄ − µ
√ ∼ tn−1 for X1 , X2 , . . . , Xn ∼ N (µ, σ 2 )
Sn−1 / n

4
 The confidence interval for the population mean µ is given by:
Sn−1 Sn−1
X̄ − t α2 ,n−1 × √ ≤ µ ≤ X̄ + t α2 ,n−1 × √
n n

where X1 , X2 , . . . , Xn ∼ N (µ, σ 2 )

0.5.1 Example Question

Consider the output that briefs about the time spent in system .

i Observation 1 Observation 2 Observation 3

1 20 35 25
2 10 15 15
3 20 10 15
4 15 35 30

To construct the confidence intervals, we first determine Ȳ¯ , which is the

overall mean of the means from each replication. It is calculated as:
4
1X
Ȳ¯ = Ȳi
4 i=1

The mean for each replication Ȳi is calculated as:

3
1X
Ȳi = Yij
3 j=1

where Yij represents the observations in replication i for the j-th obser-
vation.
The values for each replication mean are:

Ȳ1 = 26.67, Ȳ2 = 13.33, Ȳ3 = 15, Ȳ4 = 26.67

The overall mean Ȳ¯ is then:

Ȳ¯ = 20.4
2
The sample variance Sn−1 is calculated as:

5
 2 (26.67 − 20.4)2 + (13.33 − 20.4)2 + (15 − 20.4)2 + (26.67 − 20.4)2
Sn−1 = = 52.6
3
q √
2
Sn−1 = Sn−1 = 52.6 = 7.3
The confidence interval for µ is calculated as:
 
7.3
µ ∈ 20.4 ± 3.18 × √
4
This gives:

µ ∈ [8.89, 31.93]

0.5.2 Calculation of prediction interval

A prediction interval is similar to a confidence interval, but instead of esti-
mating a population parameter, it provides a range within which a future
individual observation is likely to fall, accounting for both model uncertainty
and the variability in the data. The prediction interval for Xn+k is given by:
" r #
n−1 1 1
Xn+k ∈ X̄ ± K · tα/2,n−1 · Sn−1 · +
n m
where:
- m is the number of statistics collected per replication.
-k is the number of consecutive periods for which we want to predict
intervals
From above question if we want to predict the intervals for 5th replication
from given 4 replications - :
" r #
1
Y5 ∈ 20.4 ± 3.18 × 7.3 × +1
4

Y5 ∈ [5.55, 46.35]
if negative or infeasible limits are obtained then they should be truncated
depending upon the random variable .