Symmetry 13 02402 v2
Symmetry 13 02402 v2
Symmetry 13 02402 v2
Article
Application of Three Metaheuristic Algorithms to
Time-Cost-Quality Trade-Off Project Scheduling Problem for
Construction Projects Considering Time Value of Money
Omid Kebriyaii 1 , Ali Heidari 1 , Mohammad Khalilzadeh 2, * , Jurgita Antucheviciene 3
and Miroslavas Pavlovskis 3
1 Department of Industrial Engineering, Iran University of Science and Technology, Tehran 16846-13114, Iran;
[email protected] (O.K.); [email protected] (A.H.)
2 CENTRUM Católica Graduate Business School, Pontificia Universidad Católica del Perú, Lima 15023, Peru
3 Department of Construction Management and Real Estate, Vilnius Gediminas Technical University,
Sau-letekio al. 11, LT-10223 Vilnius, Lithuania; [email protected] (J.A.);
[email protected] (M.P.)
* Correspondence: [email protected]
Abstract: Time, cost, and quality have been known as the project iron triangles and substantial factors
in construction projects. Several studies have been conducted on time-cost-quality trade-off problems
so far, however, none of them has considered the time value of money. In this paper, a multi-objective
mathematical programming model is developed for time-cost-quality trade-off scheduling problems
in construction projects considering the time value of money, since the time value of money, which is
decreased during a long period of time, is a very important matter. Three objective functions of time,
Citation: Kebriyaii, O.; Heidari, A.;
cost, and quality are taken into consideration. The cost objective function includes holding cost and
Khalilzadeh, M.; Antucheviciene, J.;
negative cash flows. In this model, the net present value (NPV) of negative cash flow is calculated
Pavlovskis, M. Application of Three
considering the costs of non-renewable (consumable) and renewable resources in each time period
Metaheuristic Algorithms to
Time-Cost-Quality Trade-Off Project
of executing activities, which can be mentioned as the other contribution of this study. Then, three
Scheduling Problem for Construction metaheuristic algorithms including multi-objective grey wolf optimizer (MOGWO), non-dominated
Projects Considering Time Value of sorting genetic algorithm (NSGA-II), and multi-objective particle swarm optimization (MOPSO) are
Money. Symmetry 2021, 13, 2402. applied, and their performance is evaluated using six metrics introduced in the literature. Finally, a
https://doi.org/10.3390/ bridge construction project is considered as a real case study. The findings show that considering the
sym13122402 time value of money can prevent cost overrun in projects. Additionally, the results indicate that the
MOGWO algorithm outperforms the NSGA-II and MOPSO algorithms.
Academic Editor: Deming Lei
Keywords: project scheduling; time-cost-quality trade-off; discounted cash flow; MOGWO; NSGA-II;
Received: 17 November 2021
MOPSO; metaheuristic algorithm
Accepted: 4 December 2021
Published: 12 December 2021
while the quality is maximized [4]. Traditional project scheduling methods are not ca-
pable of meeting today’s needs of the construction industry. Presenting an appropriate
mathematical programming model which simultaneously optimizes different conflicting
objectives, as well as efficient multi-objective optimization techniques, help to achieve the
defined goals [5].
If the required resources are not sufficient for completing activities, they will cause
delays in the implementation of projects. Hence, this study aims to properly assign
renewable resources to project activities in each time period of activity execution. On the
other hand, considering the indirect costs and time value of money is very important in
project planning and can affect long-term investments. In this study, the discounted cash
flow is considered by calculating the costs of resources in each period of time, while in
other studies the costs of resources are calculated after completing each activity.
The research questions are as follows:
1. What is the significance of considering time value money in time-cost-quality (TCQ)
project scheduling problems?
2. Which methods can be used to optimize the proposed multi-objective mathematical
programming model for scheduling construction projects?
3. Is it possible to implement the proposed model in a real case construction project?
4. Which method can find the best solution among the Pareto frontier?
In this paper, a mathematical programming model is developed for Time-Cost-Quality
(TCQ) trade-off multi-mode resource-constrained project scheduling problem (MRCPSP).
The first objective function minimizes the total costs of the project including negative
cash flows and holding cost of finished activities. The second objective function seeks to
minimize the project make-span, and the third objective function maximizes the quality of
activities. The cost objective function consists of two parts. The first part is the holding cost
of activities and the second part is the negative cash flows of the project. For calculating
the holding cost of activities, the total cost of the project is obtained cumulatively after
completing each activity and a percent of this cost is considered as holding cost. The
second part is obtained as a percentage of non-renewable and renewable resource costs in
each time of executing activities. A decision variable (uimt ) is applied for considering the
execution of activity i in each time and the renewable resources are assigned to activities by
using this variable. In this model, two types of resources (renewable and non-renewable
resources) exist, and we have limiting for using both resources. The quality level of
activities must not be lower than a threshold. The objective functions (time, cost, and
quality) are in conflict and increasing the total cost of project leads to the better values of
the other objective functions.
The outstanding points can be concisely counted as follows; considering negative cash
flow per unit of time period, using three well-known and highly recognized multi-objective
optimization algorithms named NSGA-II, MOPSO, and MOGWO for solving a number
of problem instances with different sizes, and applying the proposed model to a real case
of a bridge construction project, finding the best solution using the Shannon’s entropy
technique and WASPAS method. The main contribution of this study is considering the
time value of money. In addition, other relevant research works have considered the
negative cash flow at the completion of each project activity, however, in this study, the
negative cash flow is calculated in each time period of executing activities in order to
prevent cost overrun. Additionally, in this paper, a bridge construction project including
88 activities and 17 renewable resources is taken into consideration as a real case study.
This paper is structured as follows. In Section 2, the literature review is briefly
presented. Section 3 describes the mathematical programming model and the methodology
of solving model. The model validation, the experimental results and the case study are
presented in Section 4, and finally the paper concludes in the Section 5.
Symmetry 2021, 13, 2402 3 of 25
2. Literature Review
Time, cost, and quality are considered as important objectives in any project scheduling
problem [6,7]. Most of the studies on time-cost-quality (TCQ) trade-off problems considered
only one non-renewable resource. Afruzi et al. (2014) proposed a simple TCQ trade-
off model and defined the duration of activities in two execution modes (crashed and
normal durations). They employed the multi-objective imperialist competitive algorithm
(MOICA) to tackle this problem and compared the algorithm performance with other
frequently used algorithms [8]. Tavana et al. (2014) introduced a resource-constrained
TCQ trade-off problem considering the preemption of activities. They also applied the
ε-constraint method to solve the model [9]. Zareei and Hassan-Pour (2015) proposed a
resource-constrained project scheduling problem (RCPSP) to maximize the net present
value (NPV) [10]. Saif et al. (2015) used a meta-heuristic algorithm known as problem data-
based optimization (PDBO) to solve this problem [11]. Fu and Zhang (2016) proposed a non-
linear programming model with the objective of minimizing total quality costs for the multi-
mode resource-constrained project scheduling problem (MRCPSP). It should be noted that
RCPSP deals with finding the start times of project activities taking the activity precedence
relationships as well as the availability of resources into account. Additionally, they
developed a shuffled frog-leaping algorithm (SFLA) to solve the problem [12]. Hosseini
et al. (2017) proposed a multi-objective TCQ trade-off mathematical programming model
for an underground project and a sample project with 10 activities was solved by the
NSGA-II algorithm. These multi-objective methods obtain the Pareto optimal frontiers
with two significant challenges: The Pareto set may not be properly distributed and may
be very large which raises the question of how to select a specific solution [13].
Orm and Jeunet (2018) reviewed the literature on TCQ trade-off problems and found
out that the number of activities in the sample problems is quite small compared to the
ones in real-world projects [14]. Geshniani et al. (2020) presented a model for maxi-
mizing contractor net present value and solved it with metaheuristic algorithms [15].
Mehrdad (2020) developed a model for the multi-objective RCPSP in which the NPV and
the quality of executing activities are maximized, and the project completion time is mini-
mized [16]. Mahmoudi and Javed (2020) proposed a mathematical programming model
for the time-cost trade-off project scheduling problem [17]. Sharma and Trivedi (2020) pre-
sented a TCQ and safety trade-off model and used the NSGA-III algorithm for solving
it [18]. Moghadam et al. (2020) compared the performance of two metaheuristic algo-
rithms in solving the TCQ trade-off problem [19]. Jeunet and Bou Orm (2020) developed
an optimization model for the TCQ trade-off problem [20]. Keshavarz and Shoul (2020)
used a single-objective optimization approach to solve the TCQ trade-off problem [21].
Banihashemi and Khalilzadeh (2021) considered the multi-mode project scheduling prob-
lem and evaluated the efficiency of each mode in terms of time, cost, quality, and environ-
mental impacts [22]. Mao et al. (2020) Presented a mathematical model in shipbuilding
project scheduling problem. There are several projects (multi-projects) in order to execute
in their model. Decreasing the cost and increasing the quality was the objective func-
tions of shipbuilding project problem. The findings showed the distributed scheduling
process can be effectively connected by parameterizing the local optimization and the
interorganizational coordination process [23]. Panwar and Neeraj Jha (2021) used the
NSGA-II algorithm to deal with the TCQ trade-off problem considering safety issues [24].
Hosseinzadeh et al. (2021) presented a heuristic approach to solve the TCQ trade-off prob-
lem in an uncertain environment [25]. Luong et al. (2021) exploited Opposition-based
Multiple Objective Differential Evolution (OMODE) to tackle TCQ trade-off problem con-
sidering a high-way construction project [26]. Hamta et al. (2021) developed a goal
programming model for the TCQ trade-off problem and implemented the model on
a case study [27]. Nguyen et al. (2021) used the multiple objective whale optimiza-
tion (MOWO) approach to tackle the TCQ trade-off problem in reiterative construction
projects [28]. Huynh et al. (2021) compared the performance of four metaheuristic algo-
rithms in solving the TCQ trade-off problem considering carbon dioxide emission [29].
Symmetry 2021, 13, 2402 4 of 25
Banihashemi et al. (2021) attempted to simultaneously minimize the project cost, time, and
environmental effects [30]. Liu et al. (2021) developed a model to deal with RCPSP taking
consumable resources into account, and solved the proposed model by using the constraint
programming (CP) method [31].
It should be noted that most of the papers on RCPSP used case studies with small and
medium sizes [32–34].
The holding cost of completed activity and value of money are very important in
large-scale projects. Dodin and Elimam (2001) paid attention to this subject and used
CPLEX, which is one of the solvers of GAMS optimization software, for many projects [35].
Seifi and Tavakkoli-Moghaddam (2008) presented a multi-mode RCPSP with the goal of
maximizing the NPV and minimizing the holding cost of completed activity by the projects’
completion time [36]. Khoshjahan et al. (2013) developed a model for RCPSP with the
objective of minimizing the NPV of the earliness–tardiness penalty costs [37]. A summary
of the relevant studies is presented in Table 1.
Authors Year Multi-Objective MCDM Time Value of Money Holding Cost Case Study
Mahmoudi and Javed [17] 2020 √
Sharma and Trivedi [18] 2020 √ √
Moghadam et al. [19] 2020 √ √
Jeunet and Bou Orm [20] 2020 √ √
Keshavarz and Shoul [21] 2020 √
Banihashemi and
2021 √ √
Khalilzadeh [22]
Mao et al. [23] 2020 √ √
Panwar and Neeraj Jha [24] 2021 √
Hosseinzadeh et al. [25] 2021 √ √
Luong et al. [26] 2021 √ √
Hamta et al. [27] 2021 √ √
Nguyen et al. [28] 2021 √ √
Huynh et al. [29] 2021 √ √
Banihashemi et al. [30] 2021 √ √
Liu et al. [31] 2021 √ √
This paper 2021 √ √ √ √ √
In the related studies on project scheduling in which the NPV is considered as the
objective function, cash flows are converted to the NPV after completion of activities [36],
while cash flows should be converted to the NPV in each unit of time due to the long
implementation time of the large projects (mega projects). Therefore, in this paper, the
cash flows of the activities are calculated in each time unit. On the other hand, timely and
appropriate allocation of renewable resources to activities is extremely important in project
scheduling. In this study, a variable is defined which considers the execution of each activity
in each unit of time and appropriately allocated the renewable resources to the activities. In
addition, the holding cost of activities which is one of the main costs of the project is taken
into account in this paper, while it has received less attention in other studies. Additionally,
there are few studies that implemented mathematical programming model on a real case
study, solved it with three different multi-objective optimization algorithms, and found the
best Pareto solution using the multi-criteria decision-making methods.
The highlights of current research are enumerated as follows:
• Considering time value of money in the TCQ trade-off problem for construction projects;
• Applying three metaheuristic algorithms of MOGWO, NSGA-II, and MOPSO for
solving the proposed problem;
• Implementing the proposed model on a real case of bridge construction project with
88 activities;
• Using Shannon’s entropy technique and WASPAS method for finding the best
Pareto solution.
Symmetry 2021, 13, 2402 5 of 25
Sets
i: Set of activities where i = {0, 1, . . . , N, N + 1}
mi : Set of modes (executive mode for activity i where mi = {1, 2, . . . , Mi })
r: Set of renewable resources where r = {1, 2, . . . , R}
l: Set of nonrenewable resources where l = {1, 2, . . . , L}
t: The period of time
Parameters
esi : Earliest start time of activity i
lsi : Latest start time of activity i
E f sij : Precedence relationship between activity i and activity j.
f sij : Minimum delay for activities i and j with precedence relationship finish to start
Duimi : Duration of activity i in execution mode mi
Drimi r : Amount of renewable resource r for executing activity i in mode mi in each time.
limi l : Amount of nonrenewable resource l for executing activity i in mode mi .
Nl : The availability of the nonrenewable resource l.
Ar : The availability of the nonrenewable resource r.
qimi : The quality level of activity i in mode mi .
Cr : The cost of one unit of renewable resource r in each time.
Cl : The cost of one unit of non-renewable resource l.
Symmetry 2021, 13, 2402 6 of 25
Variables
Ltnsi : Lateness of activity i
Wt : Non-renewable and renewable resource costs in each time of executing activities
WPt : Project worth obtained by completing each activity
Binary variables
ximi t : 1 if activity i starts in mode mi at time t, 0 otherwise
uimi t : 1 if activity i execute in mode mi at time t, 0 otherwise
Fimi t : 1 if activity i finishes in mode mi at time t, 0 otherwise
H H
Min Z1 = ∑ S.WPt + ∑ SSt .Wt (1)
t =1 t =1
M
Min Z2 = ∑t=1 ∑m=N1+1 t.x N +1,mN+1 ,t (2)
N Mi T
Max Z3 = ∑ ∑ ∑ qimi ximi t Vi (3)
i =1 m =1 t =1
Equation (1) represents the cost objective function which contains the holding cost and
NPV of cash flows as nonrenewable and renewable resource costs. Equation (2) represents
the time objective function which is calculated using the critical path method. Equation (3)
represents the quality objective function which is calculated through the multiplication of
activity quality and its weight [9].
s.t
Mi li (4)
∑ ∑ ximi t = 1 ∀i ∈ N
m i =1 t = ei
Mi li Mj lj
t−1+ Duim
∑ uimi k ≥ Duimi ximi t ∀i ∈ N, mi ∈ Mi (6)
k=t
N Mi li
∑ ∑ ∑ lil ximi t ≤ Nl ∀l ∈ L (9)
i =1 m i =1 t = ei
Mi li
∑ ∑ qimi ximi t ≥ Qmini ∀i ∈ N (10)
m i =1 t = ei
Symmetry 2021, 13, 2402 7 of 25
N Mi
Wt ≥ ∑ ∑ wimi uimi t ∀t ∈ T (11)
i =1 m i =1
N Mi
WPt ≥ WPt−1 + ∑ ∑ wimi .Fimi t .Duimi ∀t ∈ T (12)
i =1 m i =1
limi l
wimi = Cr × Drimi r + Cl × (13)
Duimi
Ltnsi ≥ 0, Wt ≥ 0, WPt ≥ 0, ximi t , uimi t , Fimi t ∈ {0, 1} (14)
Equation (4) mandates that each activity must start in one mode only between its
earliest start time and latest start time. Equation (5) satisfies the precedence relationship.
Equation (6) represents the binary variable uimi t is equal to 1 in executing time period.
Equation (7) indicates the binary variable Fimi t is equal to 1 by finishing each activity.
Equations (8) and (9) dictate the total resource usage per unit of time cannot exceed
the total resource availability. Equation (10) shows that the quality of activities must be
upper than a predefined minimum level. Equation (11) is the costs of non-renewable and
renewable resources in each activity executing time. Equation (12) is the calculated project
worth by completing each activity. The parameter wimi is calculated through Equation (13).
Equation (14) guarantees that the decision variables are binary and positive.
This permutation shows where the activity is located. For example, activity 2, whose
number was the lowest in the solution, is in the first place.
4 1 8 7 9 2 6 3 5 10
4 1 8 7 9 2 6 3 5 10
Figure 1. Solution representation.
Symmetry 2021, 13, 2402 Figure 1. Solution representation. 8 of 25
This permutation shows where the activity is located. For example, activity 2, whose
This
number was permutation
the lowestshows in the where
solution, theisactivity is located.
in the first place. For example, activity 2, whose
number was
According the lowest in the solution, is in the first place.
Accordingtotothe theprecedence
precedencerelationships,
relationships,this thisorder
orderisisrepaired.
repaired.If Ifnonoactivity
activityisis
According
planned, it will to shifted
be the precedence
to the relationships,
following activity. this order is repaired. If no activity is
planned, it will be shifted to the following activity.
planned,
The it willpart
be shifted solution
to the followingthe activity. determination. Each activity number
Thethird
third partofofthe the solutionshows shows the mode mode determination. Each activity number is
The third
ismapped
mapped tothe part
the of the mode
selected solution shows
using the mode determination.
thefollowing
following equation:⎣ Each activity number
to selected mode using the equation:
is mapped to the selected mode using the following equation:⎣
⌊𝑋 × 𝑚𝑜𝑜𝑑𝐿𝑖𝑠𝑡⌋ + 1
⌊𝑋 × 𝑚𝑜𝑜𝑑𝐿𝑖𝑠𝑡⌋ + 1
where 𝑋 represents the solution of the activity, and 𝑚𝑜𝑜𝑑𝐿𝑖𝑠𝑡 is the number of execution
mods. 𝑋 represents the solution of the activity, and 𝑚𝑜𝑜𝑑𝐿𝑖𝑠𝑡 is the number of execution
where
where X represents the solution of the activity, and moodList is the number of execution mods.
mods. The third part of the solution is to determine the start time. It determines the default
The thirdpart partofofthe
thesolution
solutionisistotodetermine
determinethe thestart
starttime.
time.ItItdetermines
determinesthe thedefault
default
start time.third
The Due to the resource constraints, the timing of activity should be delayed.
start
start Thetime. Due
time.following
Due to the to the resource
resource constraints, the timing of activity should be delayed.
equation is constraints,
used to determine the timing of activity
the default timeshould be delayed.
of performing an activ-
Thefollowing
The followingequation
equationisisused usedto todetermine
determine the the default
default time
time ofof performing
performing an
anactivity:
activ-
ity:
ity:
(𝑙𝑠𝑖 − 𝑒𝑖) × 𝑋 + 𝑒𝑖
(𝑙𝑠𝑖 − 𝑒𝑖) × 𝑋 + 𝑒𝑖
where 𝑙𝑠𝑖 is the latest time to perform an activity, and 𝑒𝑖 is the fastest time to perform
it.where𝑙𝑠𝑖
where lsi is
is the
the latest time to to perform
perform an anactivity,
activity,and andei 𝑒𝑖
is the fastest
is the time
fastest to perform
time to performit.
it.
3.3.3. The NSGA-II Metaheuristic Algorithm
3.3.3. The NSGA-II Metaheuristic Algorithm
3.3.3. The
As aNSGA-II Metaheuristic
population-based Algorithm
approach, NSGA-II is one of the well-known and most widely
As a population-based approach, NSGA-II is one of the well-known and most widely
used metaheuristic algorithms solving multi-objective optimization problems. After the
usedAs a population-based
metaheuristic algorithms approach,
solvingNSGA-II is one ofoptimization
multi-objective the well-known and most
problems. widely
After the
introduction
used metaheuristicof the first version
algorithms of this algorithm in 1995, the second version was released
introduction of the first version solving multi-objective
of this algorithm in 1995,optimization problems.
the second version wasAfter the
released
in 2002 under
introduction of the first
the acronym
version NSGA-II
of this [45,46].
algorithm in 1995, the second version was released
in 2002 under the acronym NSGA-II [45,46].
in 2002 under the acronym NSGA-II [45,46].
CrossoverOperator
Crossover Operator
Crossover Operator
AccordingtotoFigure
According Figure2, 2, first, parentsx1
first, two parents 𝑥1and andx2𝑥2 areareselected for for
selected reproduction.
reproduction. Each
According
parent is a string to Figure
with a 2,
lengthfirst,
of two
N. parents
Then a 𝑥1
string and
with 𝑥2
a are
length
Each parent is a string with a length of N. Then a string with a length of N is randomly selected
of N is for reproduction.
randomly generated
Each
withparent
generatednumbers is abetween
with string with
numbers zero a and
betweenlength
one
zeroofcalled
N.
andThen
oneTwo
α. acalled
string α.with
children Twoy1a and
length of 𝑦1
y2 are
children Ngenerated
isand
randomly
𝑦2 from
are
generated
the parents,
generated with
from numbers
which between
have a share
the parents, whichofhavezero and
eachaparent’sone called α.
share ofcharacteristics Two children 𝑦1 and 𝑦2 gene
according to α.according
each parent’s characteristics Each are
generated
toofα.each fromisof
Eachchild
gene the parents,
created
each which
as following
child have
is created asa following
shareusing
equation of each parent’s
parental
equation characteristics
genes.
using parental genes.according
to α. Each gene of each child is created as following equation using parental genes.
𝑦1y1==𝛼α ∗∗ x1 𝑥1 ++(1(1− − α) 𝛼)
∗ ∗x2𝑥2 (15)
(15)
𝑦1 = 𝛼 ∗ 𝑥1 + (1 − 𝛼) ∗ 𝑥2 (15)
𝑦2 == 𝛼α ∗∗ x2 𝑥2 + (1 − ––α𝛼)𝛼)∗∗ x1𝑥1 (16)
y2=
𝑦2 𝛼 ∗ 𝑥2++(1(1 ) ∗ 𝑥1 (16)
(16)
Mutation Operator
The mutation gives a chance to the algorithm to escape from convergence to local
optimality. As shown in Figure 3, after selecting a member for mutation, a number of the
genes are selected and randomly changed based on the mutation steps.
It should be noted that a solution representation method is used so that the solutions
are continuously displayed between zero and one and are converted to continuous numbers
through decoding. Hence, the algorithm only receives and processes numbers between
Symmetry 2021, 13, 2402 9 of 25
zero and one so that the infeasible solutions are not generated. In addition, infeasible
solutions are avoided by using the penalty function.
The first group, alpha (α), includes leaders who make decisions. The second group,
beta (β), assists the leaders as advisors. The third group, delta (δ), does the duties and
tasks such as hunting, scouting and guarding. The fourth group, omega (ω), takes the
commands of the higher groups.
The most appropriate sets of solutions to a single-objective model are alpha (α),
beta (β), delta (δ), and omega (ω), respectively.
α
β
δ
ω or any other hunters
Estimated position of
the prey
In which, a linearly decreases from 2 to 0 during each iteration, and r1 and r2 randomly
change between 0 and 1 [51].
unlike algorithms would not easy [52]. Nevertheless, the quality of the Pareto sets should
be quantitatively assessed. Therefore, six metrics are applied in this study as follows.
∑in=1 ci
MID = (19)
n
q
where n is the number of non-dominated set and ci = f 1i2 + f 2i2 . The lower value of MID,
shows the better of performance for an algorithm.
The higher values of SNS, show the better solution quality, in other words, more
diversity of solutions.
Utility function includes three quantitative criteria and two qualitative criteria. The
weight of 2 is allocated to the quantitative criteria, and the weight of 1 is assigned to the
qualitative criteria, shown in the following equation [60]:
q
(Spacing)1 + ( MID )2 + ( RAS)2 + ( DM)1 + (SNS)1 + (SNS)1
2
(24)
Symmetry 2021, 13, 2402 13 of 25
f ij denotes the jth objective function of the jth solution. Pij represents the linear
normalization of the jth objective for the jth solution, which is employed to determine the
value of the Shannon’s entropy (Ej ) of the jth objective. The number of objectives and
solutions are displayed by m and n, respectively. Eventually, the corresponding relative
normalized weight for jth solution indicated by ω j , is then obtained using Equation (27).
Figure 7.
Figure 7. Activity
Activity on
on node
node network.
network.
1 2 3
Cost of using one unit of renewable resource r
400 500 200
Holding cost factor (S) 0.1
This sample was solved with GAMS 24.1.2 software by augmented ε-constraint
method (AEC). The thirtieth point of this frontier (cost = 126,116.5, makespan of project = 16,
quality = 0.826) was selected, shown in Table 7, variables ximt and uimt take values properly
according to duration and due date of each activity. The chart of resource allocation is
shown in Figure 8.
Symmetry 2021, 13, 2402 15 of 25
x1.3.2 = 1 du1.1 = 4
u1.3.2 = 1 u1.3.3 = 1 u1.3.4 = 1 u1.3.5 = 1
x2.2.1 = 1 du2.2 = 3
u2.2.1 = 1 u2.2.2 = 1 u2.2.3 = 1
x3.2.7 = 1 du3.2 = 4
u3.2.7 = 1 u3.2.8 = 1 u3.2.9 = 1 u3.2.10 = 1
x4.3.5 = 1 du4.3 = 4
u4.3.5 = 1 u4.3.6 = 1 u4.3.7 = 1 u4.3.8 = 1
x5.1.10 = 1 du5.1 = 2
u5.1.10 = 1 u5.1.11 = 1
x6.1.13 = 1 du6.1 = 2
u6.4.13 = 1 u6.4.14 = 1
x7.2.15 = 1 du7.2 = 2
u7.2.15 = 1 u7.2.16 = 1
Furthermore, this sample was solved by three meta-heuristic algorithms and the
Pareto optimal frontiers are shown in Figure 9.
Symmetry 2021, 13, 2402 16 of 25
Figure 9. Pareto frontiers of time-cost-quality by AEC and three metaheuristic solution methods.
As seen in Figure 10, the quality increases with cost. Additionally, project make-span
increases with cost decrease. It is clear that higher project quality and shorter make-span
will incur higher project costs.
Symmetry 2021, 13, 2402 17 of 25
Number of Activities
Number of Activities
Number of Modes
Number of Modes
Number of Modes
Example Number
Example Number
Example Number
1 8 2 2 11 30 3 3 21 55 4 4
2 10 2 2 12 32 3 3 22 60 4 4
3 12 2 2 13 34 3 3 23 65 4 4
4 14 2 2 14 36 3 3 24 70 4 4
5 16 2 2 15 38 3 3 25 75 4 4
6 18 2 2 16 40 3 3 26 80 4 4
7 20 2 2 17 42 3 3 27 85 4 4
8 22 2 2 18 44 3 3 28 90 4 4
9 24 2 2 19 46 3 3 29 95 4 4
10 26 2 2 20 48 3 3 30 100 4 4
Symmetry 2021, 13, 2402 18 of 25
Table 11. The average of six metrics for three proposed algorithms in small, medium, and large sizes.
The statistical analysis of the model was implemented in the form of analysis of
variance (ANOVA) for all metrics. Figure 11 shows the means plot of the different metrics
in numerical example (at the 95% confidence level).
Comparison of MOGWO and NSGA-II show that there are significant differences
between two algorithms based on Diversity, RAS, SNS, and Spacing, MOGWO outperforms
based on Diversity, SNS and RAS, and NSGA-II has a better performance based on Spacing.
There are no significant differences in term of MID and CPU time.
Comparison of MOGWO and MOPSO show that MOGWO outperforms based on
Diversity, RAS, and SNS. There are no significant differences in term of other indices.
The results also indicate that NSGA-II has better performance in term of Spacing.
However, MOPSO outperforms based on Diversity. There are no significant differences in
term of other indices.
MOPSO is better than MOGWO and NSGA-II in term of the CPU time.
Figure 11. The plot of all metrics for all the algorithms.
Figure 12. Pareto sets of solutions for the three-criteria and two-criteria multi-objective
Figure 12. Pareto sets optimization problem.
of solutions for the three-criteria and two-criteria multi-objective optimization problem.
Comparison
Comparison with
with the
the case
case study
study where
where the
the NPV
NPV isis calculated
calculated at
at the
the completion
completion of
of
the activity.
the activity.
As shown in Figure 13, the cost objective function has a lower value when NPV is
As shown in Figure 13, the cost objective function has a lower value when NPV is
calculated at the end of each activity. Therefore, calculating NPV at the end of each activity
calculated at the end of each activity. Therefore, calculating NPV at the end of each activity
leads to not considering some amounts of costs, and the cost overrun is occurred, which is
leads to not considering some amounts of costs, and the cost overrun is occurred, which
typical in most projects.
is typical in most projects.
In addition, the Gantt chart related to the best solution is presented in Figure 14.
Figure 14. The corresponding Gantt chart with the best solution of Pareto frontier.
Symmetry 2021, 13, 2402 22 of 25
4.4. Discussion
Considering the small-sized problems, the results of the three metaheuristic algorithms
of NSGA-II, MOGWO, MOPSO, and the results obtained from GAMS software were
compared in order to validate these metaheuristic algorithms. Due to the NP-Hardness of
the proposed model, the exact methods and GAMS optimization software are not able to
find the optimal solutions for the large-sized problems. Hence, these three metaheuristic
algorithms were employed using the MATLAB software. Thirty small, medium and large
size test problems were designed, and six indices of time, DM, RAS, MID, Spacing, and
SNS were considered for performance evaluation. The results showed that the MOGWO
algorithm has better performance than the other two algorithms.
Additionally, the impact of considering the time value of money was evaluated. As
shown in Figure 13, the cost objective function is greater when NPV is calculated by
executing activities in each period. Therefore, calculating NPV at the end of each activity
leads to not considering some amounts of costs and the cost overrun which is typical in
most projects. In this case study, if we do not consider the time value of money there is
about 22 percent cost overrun in project. Therefore, it is very important to consider the
time value of money and also the other indirect cost of the project in order to prevent
cost overrun.
Figure 12 shows that cost objective function and time and quality objective functions
conflict with each other so that by increasing project costs, the duration of the project
decreases, and the total quality of the project increases. As can be seen in Figure 13, in the
interval (149 weeks,154 weeks) period of time that project is completed, by increasing 0.4
to 1 percent in the total cost of the project can reduce 1 period of time of project make-span.
Additionally, it can reduce 19 periods of time by increasing 16 percent in project total cost.
Regarding the quality objective function, it can be said that with 8 percent increasing in
project total cost, 1.3 units can be added to the overall quality level of the project.
After ranking the Pareto Set of solutions using the MCDM method, the best solution is
selected regarding the project decision-makers’ policies and preference. The corresponding
Gantt chart of this solution is represented in Figure 14. According to the Gantt chart, the
project was started in July 2015 and was completed in July 2018, with a total duration of
154 weeks. However, the total project duration of 153 weeks was obtained based on the
best Pareto solution that indicates the efficient performance of the proposed methodology.
5. Conclusions
In this paper, a multi-objective mathematical programming model was presented for
resource constraint project scheduling problem (RCPSP) with the aim of time, cost, and
quality trade-off considering the time value of money. Additionally, the proposed model
was implemented on a real case study of a bridge construction project with 88 activities
and solved by three different metaheuristic algorithms named NSGA-II, MOPSO, and
MOGWO. The best Pareto solution was selected using the MCDM techniques including
the Shannon’s entropy technique and WASPAS method.
It should be noted that the value of the cost objective function is greater when the net
present value (NPV) is calculated based on each unit of time period. However, calculating
NPV based on the finish times of activities leads to not considering the real amount of costs
and results in cost overrun. In addition, the findings of this study showed that the objective
functions of the TCQ trade-off problem are conflicting. In other words, improving the value
of one objective function leads to deteriorating the values of the other objective functions.
Moreover, the results demonstrated the efficiency of the metaheuristic algorithms in solving
the proposed multi-objective optimization model. However, the MOGWO algorithm
outperforms the others in terms of the six well-known metrics including time, MID, SNS,
RAS, Spacing, and DM.
The limitation of research that can be concisely counted, are as follows: Gathering
input data of the real construction project with 88 activities was time-consuming and
challenging. In addition, due to the NP-Hardness of the problem in hand, the CPU
Symmetry 2021, 13, 2402 23 of 25
run time for the problem instances (especially large size problems) was quite long. As
suggestions for future research, it is recommended to implement the proposed model
on large-sized projects and analyze the results. Additionally, the environmental impacts
of project activities and social aspects of the project can be considered as other objective
functions. Moreover, the fuzzy set theory may be applied to deal with the uncertain
parameters of projects. Furthermore, other MCDM methods may be applied to weight the
objective functions and choose the best solution.
Author Contributions: Conceptualization, O.K.; methodology, O.K. and A.H.; software, A.H.; vali-
dation, M.K.; formal analysis, O.K.; investigation, O.K.; resources, M.K.; data curation, O.K.; writing—
original draft preparation, O.K., A.H. and M.K.; writing—review and editing, M.K. and J.A.; funding
acquisition, J.A. and M.P.; final revision and layout, J.A. and M.P. All authors have read and agreed
to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Data is available upon request.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Nagyová, A.; Pačaiová, H.; Markulik, Š.; Turisová, R.; Kozel, R.; Džugan, J. Design of a Model for Risk Reduction in Project
Management in Small and Medium-Sized Enterprises. Symmetry 2021, 13, 763. [CrossRef]
2. Kaveh, A.; Vazirinia, Y. Chaotic Vibrating Particles System for Resource-Constrained Project Scheduling Problem. Sci. Iran. 2020,
27, 1826–1842. [CrossRef]
3. Banihashemi, S.A.; Khalilzadeh, M.; Antucheviciene, J.; Šaparauskas, J. Trading off Time–Cost–Quality in Construction Project
Scheduling Problems with Fuzzy SWARA–TOPSIS Approach. Buildings 2021, 11, 387. [CrossRef]
4. Sonmez, R.; Bettemir, Ö.H. A hybrid genetic algorithm for the discrete time–cost trade-off problem. Expert Syst. Appl. 2012, 39,
11428–11434. [CrossRef]
5. Monghasemi, S.; Nikoo, M.R.; Fasaee, M.A.K.; Adamowski, J. A novel multi criteria decision making model for optimizing
time–cost–quality trade-off problems in construction projects. Expert Syst. Appl. 2015, 42, 3089–3104. [CrossRef]
6. Afshar, A.; Ziaraty, A.K.; Kaveh, A.; Sharifi, F. Nondominated archiving multicolony ant algorithm in time–cost trade-off
optimization. J. Constr. Eng. Manag. 2009, 135, 668–674. [CrossRef]
7. Mungle, S.; Benyoucef, L.; Son, Y.J.; Tiwari, M.K. A fuzzy clustering-based genetic algorithm approach for time–cost–quality
trade-off problems: A case study of highway construction project. Eng. Appl. Artif. Intell. 2013, 26, 1953–1966. [CrossRef]
8. Afruzi, E.N.; Najafi, A.A.; Roghanian, E.; Mazinani, M. A multi-objective imperialist competitive algorithm for solving discrete
time, cost and quality trade-off problems with mode-identity and resource-constrained situations. Comput. Oper. Res. 2014, 50,
80–96. [CrossRef]
9. Tavana, M.; Abtahi, A.R.; Khalili-Damghani, K. A new multi-objective multi-mode model for solving preemptive time–cost–
quality trade-off project scheduling problems. Expert Syst. Appl. 2014, 41, 1830–1846. [CrossRef]
10. Zareei, M.; Hassan-Pour, H.A. A multi-objective resource-constrained optimization of time-cost trade-off problems in scheduling
project. Iran. J. Manag. Stud. 2015, 8, 653–685.
11. Saif, A.; Abbas, S.; Fayed, Z. The PDBO algorithm for discrete time, cost and quality trade-off in software projects with expressing
quality by defects. Procedia Comput. Sci. 2015, 65, 930–939. [CrossRef]
12. Fu, F.; Zhang, T. A new model for solving time-cost-quality trade-off problems in construction. PLoS ONE 2016, 11, e0167142.
[CrossRef] [PubMed]
13. Hosseini, S.A.; Akbarpour, A.; Ahmadi, H.; Aminnejad, B. Balance of cost, time, and quality related to construction projects
regarding the reinforced concrete of underground structures using a meta-heuristic algorithm. Arch. Civ. Eng. 2017, 63, 103–121.
[CrossRef]
14. Orm, M.B.; Jeunet, J. Time Cost Quality Trade-off Problems: A survey exploring the assessment of quality. Comput. Ind. Eng. 2018,
118, 319–328. [CrossRef]
15. Geshniani, Y.V.; Kamranrad, R.; Emam, I. A multi-objective scheduling payment pattern for project cash flow by considering
resource constraints (a case study in power transfer system). Int. J. Ind. Syst. Eng. 2020, 35, 299. [CrossRef]
16. Mehrdad, P. A new method for scheduling resource constrained projects using a heuristic. Int. J. Adv. Heuristic Meta-Heuristic
Algorithms 2020, 1, 77–104.
17. Mahmoudi, A.; Javed, S.A. Project scheduling by incorporating potential quality loss cost in time-cost tradeoff problems: The
revised KKH model. J. Model. Manag. 2020, 15, 1187–1204. [CrossRef]
Symmetry 2021, 13, 2402 24 of 25
18. Sharma, K.; Trivedi, M.K. Latin hypercube sampling-based NSGA-III optimization model for multimode resource constrained
time–cost–quality–safety trade-off in construction projects. Int. J. Constr. Manag. 2020, 1–11. [CrossRef]
19. Moghadam, E.K.; Sharifi, M.; Rafiee, S.; Chang, Y.K. Time–Cost–Quality Trade-Off in a Broiler Production Project Using Meta-
Heuristic Algorithms: A Case Study. Agriculture 2020, 10, 3. [CrossRef]
20. Jeunet, J.; Orm, M.B. Optimizing temporary work and overtime in the Time Cost Quality Trade-off Problem. Eur. J. Oper. Res.
2020, 284, 743–761. [CrossRef]
21. Keshavarz, E.; Shoul, A. Project Time-Cost-Quality Trade-off Problem: A Novel Approach Based on Fuzzy Decision Making. Int.
J. Uncertain. Fuzziness Knowl.-Based Syst. 2020, 28, 545–567. [CrossRef]
22. Banihashemi, S.A.; Khalilzadeh, M. Time-cost-quality-environmental impact trade-off resource-constrained project scheduling
problem with DEA approach. Eng. Constr. Archit. Manag. 2021, 28, 1979–2004. [CrossRef]
23. Mao, X.; Li, J.; Guo, H.; Wu, X. Research on Collaborative Planning and Symmetric Scheduling for Parallel Shipbuilding Projects
in the Open Distributed Manufacturing Environment. Symmetry 2020, 12, 161. [CrossRef]
24. Panwar, A.; Jha, K.N. Integrating quality and safety in construction scheduling time-cost trade-off model. J. Constr. Eng. Manag.
2021, 147, 04020160. [CrossRef]
25. Hosseinzadeh, F.; Paryzad, B.; Pour, N.S.; Najafi, E. Fuzzy combinatorial optimization in four-dimensional tradeoff problem of
cost-time-quality-risk in one dimension and in the second dimension of risk context in ambiguous mode. Eng. Comput. 2020, 37,
1967–1991. [CrossRef]
26. Luong, D.L.; Tran, D.H.; Nguyen, P.T. Optimizing multi-mode time-cost-quality trade-off of construction project using opposition
multiple objective difference evolution. Int. J. Constr. Manag. 2021, 21, 271–283. [CrossRef]
27. Hamta, N.; Ehsanifar, M.; Sarikhani, J. Presenting a goal programming model in the time-cost-quality trade-off. Int. J. Constr.
Manag. 2021, 21, 1–11. [CrossRef]
28. Nguyen, D.T.; Doan, D.T.V.; Tran, N.N.C.; Tran, D.H. A novel multiple objective whale optimization for time-cost-quality tradeoff
in non-unit repetitive projects. Int. J. Constr. Manag. 2021, 1–12. [CrossRef]
29. Huynh, V.H.; Nguyen, T.H.; Pham, H.C.; Huynh, T.M.D.; Nguyen, T.C.; Tran, D.H. Multiple Objective Social Group Optimization
for Time–Cost–Quality–Carbon Dioxide in Generalized Construction Projects. Int. J. Civ. Eng. 2021, 19, 805–822. [CrossRef]
30. Banihashemi, S.A.; Khalilzadeh, M.; Zavadskas, E.K.; Antucheviciene, J. Investigating the Environmental Impacts of Construction
Projects in Time-Cost Trade-Off Project Scheduling Problems with CoCoSo Multi-Criteria Decision-Making Method. Sustainability
2021, 13, 10922. [CrossRef]
31. Liu, S.S.; Budiwirawan, A.; Arifin, M.F.A.; Chen, W.T.; Huang, Y.H. Optimization Model for the Pavement Pothole Repair Problem
Considering Consumable Resources. Symmetry 2021, 13, 364. [CrossRef]
32. Tiwari, V.; Patterson, J.H.; Mabert, V.A. Scheduling projects with heterogeneous resources to meet time and quality objectives.
Eur. J. Oper. Res. 2009, 193, 780–790. [CrossRef]
33. Kim, J.; Kang, C.; Hwang, I. A practical approach to project scheduling: Considering the potential quality loss cost in the time–cost
tradeoff problem. Int. J. Proj. Manag. 2012, 30, 264–272. [CrossRef]
34. Wood, D.A. Gas and oil project time-cost-quality tradeoff: Integrated stochastic and fuzzy multi-objective optimization applying
a memetic, nondominated, sorting algorithm. J. Nat. Gas Sci. Eng. 2017, 45, 143–164. [CrossRef]
35. Dodin, B.; Elimam, A.A. Integrated project scheduling and material planning with variable activity duration and rewards. Iie
Trans. 2001, 33, 1005–1018. [CrossRef]
36. Seyfi, M.; Tavakoli, M.R. A new bi-objective model for a multi-mode resource-constrained project scheduling problem with
discounted cash flows and four payment models. Int. J. Eng. 2008, 21, 347–360.
37. Khoshjahan, Y.; Najafi, A.A.; Afshar-Nadjafi, B. Resource constrained project scheduling problem with discounted earliness–
tardiness penalties: Mathematical modeling and solving procedure. Comput. Ind. Eng. 2013, 66, 293–300. [CrossRef]
38. Balali, A.; Valipour, A.; Antucheviciene, J.; Šaparauskas, J. Improving the results of the earned value management technique
using artificial neural networks in construction projects. Symmetry 2020, 12, 1745. [CrossRef]
39. Deb, K. Multi-Objective Optimization Using Evolutionary Algorithms; John Wiley & Sons: Hoboken, NJ, USA, 2001.
40. Punnathanam, V.; Sivadurgaprasad, C.; Kotecha, P. Multi-objective Optimal Integration of Biorefineries using NSGA-II and
MOGWO. In Proceedings of the 2016 International Conference on Electrical, Electronics, and Optimization Techniques (ICEEOT),
Chennai, India, 3–5 March 2016; pp. 3970–3975.
41. Agrebi, I.; Jemmali, M.; Alquhayz, H.; Ladhari, T. Metaheuristic algorithms for the two-machine flowshop scheduling problem
with release dates and blocking constraint. J. Chin. Inst. Eng. Trans. Chin. Inst. Eng. Ser. A 2021, 44, 573–582. [CrossRef]
42. Goodarzian, F.; Kumar, V.; Ghasemi, P. A set of efficient heuristics and meta-heuristics to solve a multi-objective pharmaceutical
supply chain network. Comput. Ind. Eng. 2021, 158, 107389. [CrossRef]
43. Mavrotas, G. Effective implementation of the ε-constraint method in multi-objective mathematical programming problems. Appl.
Math. Comput. 2009, 213, 455–465. [CrossRef]
44. Amirian, H.; Sahraeian, R. Augmented ε-constraint method in multi-objective flowshop problem with past sequence set-up times
and a modified learning effect. Int. J. Prod. Res. 2015, 53, 5962–5976. [CrossRef]
45. Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T.A.M.T. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans.
Evol. Comput. 2002, 6, 182–197. [CrossRef]
Symmetry 2021, 13, 2402 25 of 25
46. Li, K.; Pan, L.; Xue, W.; Jiang, H.; Mao, H. Multi-objective optimization for energy performance improvement of residential
buildings: A comparative study. Energies 2017, 10, 245. [CrossRef]
47. Eberhart, R.; Kennedy, J. A new optimizer using particle swarm theory. In MHS’95, Proceedings of the Sixth International Symposium
on Micro Machine and Human Science, IEEE, Nagoya, Japan, 4–6 October 1995; IEEE: New York, NY, USA, 1995; pp. 39–43.
48. Zhang, H.; Xing, F. Fuzzy-multi-objective particle swarm optimization for time–cost–quality tradeoff in construction. Autom.
Constr. 2010, 19, 1067–1075. [CrossRef]
49. Coello, C.A.C.; Pulido, G.T.; Lechuga, M.S. Handling multiple objectives with particle swarm optimization. IEEE Trans. Evol.
Comput. 2010, 8, 256–279. [CrossRef]
50. Mirjalili, S.; Mirjalili, S.M.; Lewis, A. Grey wolf optimizer. Adv. Eng. Softw. 2014, 69, 46–61. [CrossRef]
51. Mirjalili, S.; Saremi, S.; Mirjalili, S.M.; Coelho, L.D.S. Multi-objective grey wolf optimizer: A novel algorithm for multi-criterion
optimization. Expert Syst. Appl. 2016, 47, 106–119. [CrossRef]
52. Behnamian, J.; Ghomi, S.F.; Zandieh, M. A multi-phase covering Pareto-optimal front method to multi-objective scheduling in a
realistic hybrid flowshop using a hybrid metaheuristic. Expert Syst. Appl. 2009, 36, 11057–11069. [CrossRef]
53. Jolai, F.; Asefi, H.; Rabiee, M.; Ramezani, P. Bi-objective simulated annealing approaches for no-wait two-stage flexible flow shop
scheduling problem. Sci. Iran. 2013, 20, 861–872.
54. Schott, J.R. Fault Tolerant Design Using Single and Multicriteria Genetic Algorithm Optimization (No. AFIT/CI/CIA-95-039); Air Force
Institute of Tech: Dayton, OH, USA, 1995.
55. Kaveh, A.; Mahdavi, V.R. Multi-objective colliding bodies optimization algorithm for design of trusses. J. Comput. Des. Eng. 2019,
6, 49–59. [CrossRef]
56. Rabiee, M.; Zandieh, M.; Ramezani, P. Bi-objective partial flexible job shop scheduling problem: NSGA-II, NRGA, MOGA and
PAES approaches. Int. J. Prod. Res. 2012, 50, 7327–7342. [CrossRef]
57. Tabrizi, B.H.; Ghaderi, S.F. A robust bi-objective model for concurrent planning of project scheduling and material procurement.
Comput. Ind. Eng. 2016, 98, 11–29. [CrossRef]
58. Habibi, F.; Barzinpour, F.; Sadjadi, S.J. A mathematical model for project scheduling and material ordering problem with
sustainability considerations: A case study in Iran. Comput. Ind. Eng. 2019, 128, 690–710. [CrossRef]
59. Jafarian, A.; Rabiee, M.; Tavana, M. A novel multi-objective co-evolutionary approach for supply chain gap analysis with
consideration of uncertainties. Int. J. Prod. Econ. 2020, 228, 107852. [CrossRef]
60. Asefi, H.; Jolai, F.; Rabiee, M.; Araghi, M.T. A hybrid NSGA-II and VNS for solving a bi-objective no-wait flexible flowshop
scheduling problem. Int. J. Adv. Manuf. Technol. 2014, 75, 1017–1033. [CrossRef]
61. Khorasani, G.; Mirmohammadi, F.; Motamed, H.; Fereidoon, M.; Tatari, A.; Verki, M.R.M.; Khorasani, M.; Fazelpour, S.
Application of multi criteria decision making tools in road safety performance indicators and determine appropriate method
with average concept. Int. J. Innov. Technol. Explor. Eng. 2013, 3, 173–177.
62. Zavadskas, E.K.; Turskis, Z.; Antucheviciene, J.; Zakarevicius, A. Optimization of Weighted Aggregated Sum Product Assessment.
Elektron. Elektrotechnika 2012, 122, 3–6. [CrossRef]
63. Turskis, Z.; Zavadskas, E.K.; Antucheviciene, J.; Kosareva, N. A hybrid model based on fuzzy AHP and fuzzy WASPAS for
construction site selection. Int. J. Comput. Commun. Control. 2015, 10, 873–888. [CrossRef]
64. Paryzad, B.; Pour, N.S. Time-cost-quality-risk trade-off in GIGA projects using specific techniques of hunting dolphins. Int. J. Ind.
Syst. Eng. 2016, 22, 484–499. [CrossRef]