Sustainability 10 00507

sustainability
Article
A Methodology for the Selection of Multi-Criteria
Decision Analysis Methods in Real Estate and Land
Management Processes
Maria Rosaria Guarini * ID
, Fabrizio Battisti and Anthea Chiovitti
Department of Architecture and Design, Sapienza University of Rome, Via Flaminia 359, 00196 Rome, Italy;
[email protected] (F.B.); [email protected] (A.C.)
* Correspondence: [email protected]; Tel.: +39-06-49919293
Received: 15 December 2017; Accepted: 10 February 2018; Published: 13 February 2018
Abstract: Real estate and land management are characterised by a complex, elaborate combination of
technical, regulatory and governmental factors. In Europe, Public Administrators must address the
complex decision-making problems that need to be resolved, while also acting in consideration of the
expectations of the different stakeholders involved in settlement transformation. In complex situations
(e.g., with different aspects to be considered and multilevel actors involved), decision-making
processes are often used to solve multidisciplinary and multidimensional analyses, which support
the choices of those who are making the decision. Multi-Criteria Decision Analysis (MCDA) methods
are included among the examination and evaluation techniques considered useful by the European
Community. Such analyses and techniques are performed using methods, which aim to reach a
synthesis of the various forms of input data needed to define decision-making problems of a similar
complexity. Thus, one or more of the conclusions reached allow for informed, well thought-out,
strategic decisions. According to the technical literature on MCDA, numerous methods are applicable
in different decision-making situations, however, advice for selecting the most appropriate for the
specific field of application and problem have not been thoroughly investigated. In land and real
estate management, numerous queries regarding evaluations often arise. In brief, the objective of this
paper is to outline a procedure with which to select the method best suited to the specific queries of
evaluation, which commonly arise while addressing decision-making problems. In particular issues
of land and real estate management, representing the so-called “settlement sector”. The procedure
will follow a theoretical-methodological approach by formulating a taxonomy of the endogenous and
exogenous variables of the multi-criteria analysis methods.
Keywords: Multi-Criteria Decision Analysis (MCDA); Decision Making (DM); life cycle management;
decision-making
1. Introduction and Aims of the Work

Real estate and land management (including technical, regulatory and governmental aspects) are
typically highly complex sectors. Decision-making problems of this type are defined by multidimensional
objective profiles and by the multidisciplinary or multi-criteria nature of the factors needing to be
evaluated [1]. During the different phases of the construction process, while managing these kinds of
decision-making problems, evaluation methodologies have to be designed to interpret the objectives
of the land transformation and to translate the premise of the intervention into real actions.
Strategic planning of the decisions executed through decision support systems, or DSS [2],
have proven highly effective in the context of settlement transformations. DSS can also be utilised with
computerised systems that allow for an interactive use of data and models to support those making
decisions while problem solving [1]. In each of the iterative and interactive phases of the construction
Sustainability 2018, 10, 507; doi:10.3390/su10020507 www.mdpi.com/journal/sustainability

Sustainability 2018, 10, 507 2 of 28
process, the strategic planning of decisions is geared towards making informed choices based upon
methods of evaluation. These methods need to be well suited to the purpose and to the agenda of
needs and demands posed by the programs and projects being carried out in the geographical area.
Within DSS, the use of methods of Multi-criteria Decision Analysis (MCDA) can provide support
for the multidisciplinary management of the factors, which need to be optimized in fulfillment of the
objective being evaluated [3]. The literature regarding MCDA considers different schools of thought [4–7]
and proposes an extensive number of methods with which to resolve decision-making problems in
fields of application such as mathematics, management, information technology, psychology, the
social sciences and economics. More specifically in the last decade, we find examples in real estate
and land management, which demonstrate an increase of interest in the use of formalised analytical
decision methods employing structured and comprehensive databases [8]. Several authors act as a
reference source for describing the main MCDA methods, which have proven useful in different types
of decision-making problems and the approaches taken to resolve them [5,9–17]. A helpful synthesis is
provided by the contributions of, among others [18–21], Guitoni, Martel et Vincke (1999) [15], as well
as of Roy et Bouyssou (1993) [16], all of which already demonstrate attempts at taking a systematic
approach to the methods of MCDA. At the present moment (2018), about 100 different methods are
in circulation [22]. Consulting Guitoni, Martel et Vincke [15] and Ishizaka and Nemery [9], it can be
noted that the most frequently used and implemented are:
1. ELimination Et Choix Traduisant la REalitè (ELECTRE) [23];

2. Multi-attribute utility theory (MAUT) [24];
3. Analytic Network Process (ANP) [25];
4. Measuring Attractiveness by a Categorical Based Evaluation (MACBETH) [26];
5. Analytic Hierarchy Process (AHP) [27];
6. Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) [28];
7. Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE) [29].
A summary of the various contributions cited above [9,15,16] should include these points:
- No method can be considered perfect or applied to every type of problem [9].

- The range of available procedures offers many different operating opportunities, but also poses
the risk of using methods that are not suited to the decision-making problem at hand [16].
- A conclusive analysis of decision-making procedures has yet to be carried out [9].
Even if the use of MCDA techniques is by now an established practice, there are no specific
texts, either from Italy or from the rest of Europe, dealing with the exact procedures to be followed
when selecting the MCDA method best suited to the contexts of real estate and land management.
In particular when generally dealing with the settlement transformation processes and more specifically
the design and planning of public and private projects.
As a rule, the choice of the MCDA method best suited to the objectives of the decision-making
problem can significantly affect the efficiency of the procedure. Furthermore, it effects the proper
formulation of the decisions that need to be made. For example, the expression of the elements that
make up a multi-criteria decision-making problem reflects on the effectiveness of the different methods
and on the strength of the results. It follows that the choices in question play a key role in arriving
at a solution that provides an informed, suitable response to the needs and demands identified in
the project.
In light of these considerations, the present work, that represents an in-depth study of a
previous work [30], sets out to define a procedure which, taking into account the relevant factors
when addressing decision-making problems, makes it possible to select the MCDA methods best
suited to problems of real estate and land management, but which can also be employed in other
decision-making contexts. In terms of settlement transformations, these methods can expand the
horizons of fact-finding and decision-making situations from a simple consideration of the financial
requirements to the full range of relevant criteria (e.g., socio-economic, environmental etc.).
The proposed procedure serves as a useful tool, when added to the regulatory measures of the
European Union governing public tenders. The most recent is Directive 2014/24/EU, which has been
transposed into Italian laws regarding the Public Contracts Code (Codice dei Contratti Pubblici) via
Legislative Decree 50/2016 (plus its subsequent modifications and additions—s.m.a.) as well as its
regulations of implementation. These measures contemplate the use of MCDA in public tenders,
particularly for the selection of the most economically advantageous bid. MCDA has been considered
useful also in the context of planning and design choices for urban regeneration and or transformation
as early as 2006. To this end, the European Commission has drawn up a manual with recommendations
on systematic approaches to the use of MCDA in different fields of application, including real estate
and land management [31].
After having introduced, in the current Section 1, the subject of this study, hereinafter: firstly, after
the identification of the framework in which evaluation problems grow up in settlement transformation,
it will be proposed a theoretical-methodological approach with a taxonomic catalogue of variables
(‘endogenous’ and ‘exogenous’) that characterize MCDA and that must be considered during the
selection of the most appropriate method among those most commonly carried out in literature
(Section 2); afterwards, a procedure for selecting MCDA methods is built on the basis of proposed
taxonomic catalogue (Section 3); finally, the proposed procedure, is applied to a case study in its widest
form, taking into account a different points of view of stakeholders, and the results of this application
are discussed (Section 4). The conclusions of this study are argued at the end of this work (Section 5).
2. MCDA: Structure, Endogenous Variables, Exogenous Variables
2.1. Framework
Any MCDA is generally structured in two macro-phases. The first one involves the construction
and compilation, referring to the evaluation problem in question, of the evaluation matrix, which
consists of the different alternatives and their performance, based on the various criteria and
sub-criteria (and their weightings), plus their indicators of assessment. The second regards the
processing of the data in the evaluation matrix used to evaluate the alternatives, on the basis of the
objectives to be reached [30,32]. This operation is similar for all MCDA methods. The second phase
involves processing (or aggregating) data via a variety of different procedures, depending on which
method is being used, considering that each method comes with its own procedures of application
(referred to as endogenous variables later on in this paper).
When selecting the method best suited to meet the objectives posed by the evaluation, it is
necessary to take into account the context of the evaluation and that can give rise to many different
decision-making problems attached to the phases of the settlement process. As presented by the
literature on Life Cycle Management [33], the issues to be resolved during the different phases of the
Life Cycle of a settlement process correspond to a variety of objectives [34].
In Italy, the Unique Construction Text (“Testo Unico dell’Edilizia”) Presidential Decree 380/2001
(s.m.a.) and Public Contracts Code (“Codice dei Contratti Pubblici”) Legislative Decree 50/2016 (s.m.a.)
are the main laws that control the settlement transformation sector and its respective phases in both
private and public works. Using these laws, we can establish categories related to the decision-making
problems typical to different types of initiatives. The potential assessment questions and queries can
be classified according to the phases of the settlement process and for which of these phases solutions
are expected, based on the objectives of evaluation (Table 1).
Once the decision-making problem has been identified from among the possible assessment
queries that present themselves during the settlement transformation process, it is necessary to address
the evaluation question promptly, thus resolving the problem.
Table 1. Decision making problems in Life Cycle Management and action to solve them.
Action to Be Taken to Solve

Phases of the Building Process Valuable Question Action Fields of Decision-Making Problems
the Valuable Question
D.L. 351/2001 s.m.a.;
- Presidential Decree 380/2001
Normative D.L. 112/2008 s.m.a., Presidential Decree
s.m.a.; Leg. 50/2016 s.m.a. -
References (Italy) D.lgs. 42/2004 s.m.a., 380/2001 s.m.a.
- D.lgs. 50/2016 s.m.a.
D.L. 85/2010 s.m.a.
Identify between a set of items,
the most important ones based
Preliminary needs studies Priority of needs identification
on a limited amount of
information
Identify decision-makers, their
Programming Identification of subjects to be included in Life
Designers and advisors selection respective importance and their
Cycle Management
influence in decisions
Design solution that identifies the best relationship between Identify the best solution
Economic technical feasibility cost and benefit for the community, in relation to the among different proposals
specific needs to be met and performance to be provided - Settlement development;
project - Redevelopment, recovery, reuse, based on an average number of
(Legislative Decree 50/2016, Article 23, paragraph 5) information
urban regeneration;
Best design solution in accordance with the requirements, - Development of discarded
criteria, constraints, addresses and indications set by the areas/buildings; - Restoration and
Definitive project
contracting authority and, where applicable, the feasibility - Decision support in project conservation interventions; Identify the best solution
Design project (Legislative Decree 50/2016, Article 23, paragraph 7) management; [Article 3 par. 1(c)]; among different proposals
Best design solution in terms of form, type, quality, size and - Valuation of public buildings - Renovation of buildings; based on a large amount of
price and in relation to the solution proposed in the (Legislative Decree 351/2001, [Article 3 par. 1(d)]; information
Executive project Article 3-bis of Legislative Decree - New construction works;
maintenance plan of the work and its parts in relation to the
life cycle (Legislative Decree 50/2016, Article 23 par. 8) 112/2008, Article 58 of the Italian [(Article 3, par. 1 (e1–e7)];
Civil Code); - Urban planning
- Valorization of Cultural Heritage interventions; Identify the best offer among
(D.L. 85/2010, Articles 5–7 s.m.a.); [(Article. 3 par. 1(f)] different offers (of different
Finding the best deal (based on the most economically - Valorization of numbers depending on the
Work execution Relocation of work
advantageous bid criterion) landscape-environmental assets type of competition) on the
(D.L. 85/2010, Articles 5–7 s.m.a.) basis of an average number
of information
Identify the most advantageous management solutions Building a set of possible
Service delivery and/or the most suitable operator in accordance with solutions excluding hypotheses
the objectives that can not be prosecuted
Management during Ordinary and extraordinary
exercise Definition of the ordinary and extraordinary maintenance
maintenance (Presidential Identify all possible solutions in
solution in relation to the modalities and times for
Decree 380/2001 s.m.a., Article 3, relation to specific factors
the interventions
paragraph 1, letter a, b)
Sustainability
Sustainability2018,
2018,10,
10,x 507
FOR PEER REVIEW 6 5ofof2928
2.2. The Correlation between the Action to Be Taken and the Variables (Exogenous and Endogenous) of
2.2. TheMethods
MCDA Correlation between the Action to Be Taken and the Variables (Exogenous and Endogenous) of
MCDA Methods
Each action can be related to exogenous variables (strictly related to decision problems and the
context Each
from action
which can be related
they arise) and to exogenous
endogenous variables (strictlyproperties
ones (possible related to of
decision
MCDAproblems
methods). and the
This
is described in more detail respectively in Sections 2.3 and 2.4, where they are defined according tois
context from which they arise) and endogenous ones (possible properties of MCDA methods). This
described
their taxonomic in more detail respectively
classification (Figure 1).in Sections 2.3 and 2.4, where they are defined according to their
taxonomic classification
The Exogenous (Figure
variables have1). been defined taking into consideration the ‘external’ context of
The Exogenous variables
the MCDA. They vary depending on have beenthedefined takingand
regulations intoother
consideration
measuresthe in‘external’
place in context of the
the relevant
MCDA.inThey
country which vary depending
evaluation on theoccurs.
process regulations and other measures in place in the relevant country
in which evaluation process occurs.
The endogenous variables have been defined following an analysis of the specific literature
The MCDA.
regarding endogenous variables have
Understandably, therebeen defined
is not following
one single an analysis
definition, which canof the specific
explain literature
MCDA and
regarding MCDA. Understandably, there is not one single definition,
many ‘endogenous variables’ can thus be created. Concisely, Roy [10] has suggested that decision- which can explain MCDA
and many
making ‘endogenous
situations variables’ can
can be categorized thusbasis
on the be created.
of decision Concisely, Roy [10]
problematics [11]. has suggested
Different kindsthat
of
compensation logic are examined by Vincke [12] and shared by Colson and De Bruyn [13] andDifferent
decision-making situations can be categorized on the basis of decision problematics [11]. further
kinds of
studies compensation
[14,35], for example logicGuitony
are examined by Vincke
et al. [15] and Huang [12] and
et al.shared by investigated
[8] who Colson and De theBruyn [13]
required
and further studies [14,35], for example Guitony et al. [15] and Huang et al.
input information. In this paper, a set of endogenous variables is defined as that which best represents [8] who investigated the
required
the input occurring
most useful information. In this
in the paper,
related a set ofand
literature endogenous
accordingvariables is defined
to the survey as that
performed bywhich best
Ishizaka
represents
et Nemery P. [9].the most useful occurring in the related literature and according to the survey performed by
Ishizaka et Nemery P. [9].
Each variable (exogenous or endogenous) represents the qualifications and properties that
Eachthe
exemplify variable
various (exogenous
forms that or theendogenous)
variable can represents
take. Each the method qualifications and properties
retains specific properties,that
in
exemplify the various forms that the variable can take. Each
terms of how successfully it can be implemented. These properties are tied to the qualities method retains specific properties,
of the
in terms of how successfully it can be implemented. These properties are tied to the qualities of
variables.
the In
variables.
this study, the ensemble of exogenous and endogenous variables has been taken as a starting
point Inon this
which study, the ensemble
to structure of exogenous
the selection and endogenous
procedure for the proposedvariables
MCDA has method
been taken(seeasSection
a starting
3)
point onfor
intended whichuse into the
structure the selection
settlement procedure
transformation for the
process proposed MCDA method (see Section 3)
sector.
intended for use in the settlement transformation process sector.
EXOGENOUS ACTIONS TO BE TAKEN TO SOLVE ENDOGENOUS

VARIABLES THE VALUABLE QUESTION VARIABLES
Number of evaluation Identify between a set of items, the most important ones Type of decision-making
elements based on a limited amount of information problem
Typology of Identify decision-makers, their respective importance

and their influence in decisions Solution Approach
indicators
Expected solution Identify the best solution among different proposals Input Level
based on an average number of information
Technical support of a Identify the best solution among different proposals Implementation
Decision Aid Specialist based on a large amount of information Procedure
Identify the best offer among different offers (of

Stakeholders to be included Output Typology or
different numbers depending on the type of competition)
in the decision process ordering of alternatives
on the basis of an average number of information
Building a set of possible solutions excluding Decision Problem

Tool
hypotheses that can not be prosecuted Solution
Identify all possible solutions in relation to specific

factors
Actiontotobebetaken
Figure1.1.Action
Figure takenwith
withexogenous
exogenousand
andendogenous
endogenousvariables.
variables.
2.3.Exogenous
2.3. ExogenousVariables
Variables
InInthis
thissection,
section,exogenous
exogenousvariables
variableswill
willbe beexamined:
examined:(i)(i)By
Bythe
thenumber
numberofofevaluation
evaluationelements;
elements;
(ii)ByBythethe
(ii) typology
typology of indicators;
of the the indicators; (iii)stakeholders
(iii) The The stakeholders
involved involved in the process;
in the decision decision(iv)
process;
The
typology of the expected solution; (v) The presence of technical support from a Decision Aid
(iv) The typology of the expected solution; (v) The presence of technical support from a Decision
Aid Specialist during the implementation of the procedure. In the settlement transformation process,
the number of evaluation elements (criteria, sub-criteria and alternatives) depends on the nature of the
different decision-making problems [8,30,36–40].
The numeric configuration of evaluation elements (Table 2) is generally decided by the person
responsible for the evaluation process before the selecting the most suitable MCDA method with
reference to the specific problem in hand. In the case of settlement transformation processes, which
are generally rather complex, the number of criteria and sub-criteria are defined by the process
manager (Responsabile del Procedimento), role regulated by the Legislative Decree 50/2016 Article 31.
This is the individual who “provides the authority offering the contract with the data and information
regarding the main stages of the intervention, necessary for the coordination, guidance and checks of
the correct implementation, and oversees the economic management of the intervention” (Legislative
Decree 50/2016 Article 31, par. 4, point f). This kind of activity has been present since the definition of
administrative acts relating to settlement process initiation. The person responsible for the evaluation
process must attempt to construct the set of evaluation elements without allowing the potential
repercussions to overly influence the selection of an MCDA method. Although the creation of the set of
variables must of course represent the maximum number of requirements with the minimum number
of possible elements [35,41,42]. The number of alternatives can be deduced by taking into account
what has happened in the past in similar settlement proceedings in the same or in other administrative
contexts [43,44]. The typology of the indicators (Table 3) varies in accordance with the decision-making
problem to be resolved and the context in which it arises.
Table 2. Numeric configuration of evaluation elements.
Numeric Configuration of Evaluation Elements

Limited number of criteria and sub-criteria and a small number of alternatives
Limited number of criteria and sub-criteria and a large number of alternatives
Large number of criteria and sub-criteria and a small number of alternatives
Large number of criteria and sub-criteria and a large number of alternatives
Table 3. Typology of the indicators.
Typology of the Indicators Features of Indicators

Quantitative Measurable in specific units
Not measurable but subject to judgments of merit that may also employ
Qualitative
specially designed scales of measurement (ordinal, cardinal or mixed)
Mixed Both quantitative and qualitative
The number of stakeholders involved in the decision-making process may vary depending on
the different assessment questions that the MCDA needs to answer and from the decision maker’s
management related to the activation (or not) of Participatory Processes (P.P.) also deciding the number
of stakeholders to be considered (Table 4).
Table 4. Number of stakeholders.
Participatory Process Number of Stakeholders

Not activated Zero
Actived with a limited and specific number of stakeholders Narrow
Activated with a high number of stakeholders, if possible organised into categories Large
The involvement of stakeholders in the decision-making process aims to keep the decision-maker
informed as to the opinions of those who may be interested or influenced by the effects of the
decision. The types of stakeholders to be included in the participatory process [45,46] can be
classified into the following: (i) standard stakeholders “who have the legitimate responsibility to
participate in the process” [47], including all those who could be impacted by the results of the
program (e.g., beneficiaries or those who have suffered damage), design engineers and public
administration officials, etc. [31]; (ii) interest groups, stakeholders selected from local or professional
representatives, leaders of non-governmental organizations (such as stakeholder or environmental
protection, consumers and women’s rights), public sector bodies the representatives of financial
donors, etc. [31]. In short, interest groups are often political parties, civic organizations, or residents of
the impact area [47]. Each interest group, as well as the standard stakeholder, has their own motives
when evaluating the potential alternatives and often has different relational systems of preference.
The process manager, influenced by his own knowledge of the context from which the decision-making
arises, identifies the stakeholders to be included in the process. After attributing indices of importance
to each stakeholder, the process manager can select the solution (MCDA method) that is generally best
for them, for example the wishes of a simple majority (a solution that is preferred by stakeholders
whose added indices of importance is greater than 50%) [30].
The type of solution proposed for the decision-making problem in question depends on the
selection criterion sought during the evaluation process (Table 5).
Table 5. Type of solution.
Type of Solution Selection Criterion

Valid alternatives Based on the aims of the objective
Best alternative Based on the objective
Coherent alternative Closest to the objective itself
The technical support of a Decision Aid Specialist during the execution of the process is one
of the variables that must be considered. As has been noted by the European Commission [31],
the management of the multi-criteria evaluation processes can prove anything but simple. Therefore,
another factor influencing the selection of the most suitable method is the availability or non-availability
of a Decision Aid Specialist who can put into action the MCDA procedure (Table 6).
Table 6. The presence of technical support from a Decision Aid Specialist.
The Presence of Technical Support

Role of the Decision Aid Specialist
from a Decision Aid Specialist
Only technical manager of the MCDA method used
Yes
to respond to the problem under evaluation
Only facilitator for understanding the
Yes
decision-making phase(s) of the process
Both technical and facilitator roles Yes
No role for the Decision Aid Specialist No
The Decision Aid Specialist deals with the technical implementation of the MCDA by making
use of the various software available. In other studies, a wide variety of MCDA software has
been described, some of which is commercially available, discussing different kinds of packages
that have been developed to facilitate MCDA methods [9,48,49]. They involve several processes
to give structure to decision-making problems including the exploration of the situation and a
formulation and breaking down of the solution [48] by using mathematical algorithms. In reference
to the MCDA methods mentioned above (Section 1), a review of the literature draws attention to
some of the software available, which is easy to use and free to access or trial. Examples include
MakeItRational [49,50] or ExpertChoice [51] for AHP, SuperDecision [52] for ANP, RightChoice [53]
for MAUT, M-MACBETH [54] for MACBETH, SmartPickerPro [55] for PROMETHEE, Electre III–IV
software [56] for ELECTRE and Topsis [57] for TOPSIS. The Decision Aid Specialist in his position as
“technical” manager needs to have specific knowledge of programming and query languages in order
to perform MCDA using this kind of software. Acting as a “facilitator”, the Decision Aid Specialist as
a has the task of making the stages of the MCDA clear to non-specialist stakeholders; their presence
also increases the level of transparency in the evaluation process allowing for informed decisions to
be taken.
Table 7 summarizes the features of the exogenous variables.
Table 7. Features of exogenous variables.
Technical support
Number of Evaluation Typology of Stakeholders to Be Included
Expected Solution of a Decision Aid Tool
Elements Indicators in the Decision Process
Specialist
Limited number of criteria Definition of n
and sub-criteria and a small alternatives valid in ELECTRE
- Participatory process not
number of alternatives relation to objectives
activated;
Limited number of criteria - Participatory process
and sub-criteria and a large activated with a limited and MAUT
number of alternatives - Quantitative; specialized number of
A better overall - Yes; stakeholder;
Large number of criteria and - Qualitative;
alternative definition for - No - Participatory process
sub-criteria and a small - Mixed AHP; ANP
the purpose; The ideal activated with a significant
number of alternatives alternative definition number of stakeholder
Large number of criteria and closest to the lens preferably organized in MACBETH;
sub-criteria and a large categories PROMETHEE;
number of alternatives TOPSIS
2.4. Endogenous Variables

Each endogenous variable is described in the following: (i) the type of decision-making problem;
(ii) the solution approach; (iii) the input level; (iv) the implementation procedure; (v) the output
typology or order of alternatives and (vi) the solution to the decision problem [30].
The types of decision-making problem according to Roy’s viewpoint [16] and shared by Ishizaka
et Nemery P. [9] and Guitoni and Martel [11], among others, can be grouped into three categories that
express the qualification of the variable (Table 8).
Table 8. Types of decision-making problems.
Categories Decision-Making Problem

Description problem Identify the main distinctive features for a group of alternatives
Sorting problem The definition of homogeneous groups of alternatives by characteristics
Ranking and Choice problem The ranking of alternatives, from best to worst
Three clusters distinguish the approach to solving decision-making problems. They are identified
from the different qualifications that represent the resolution of the evaluation problem (Table 9).
Table 9. Approaches to resolving problems.
Method of Approach Qualification *

“A score is evaluated for each criterion and these are then synthesized into a
Full Aggregation Approach global score. This approach assumes compensable scores, i.e., a bad score for one
criterion is compensated for by a good score on another”.
“A bad score may not be compensated for by a better score. The order of the
option may be partial because the notion of incomparability is allowed.
Outranking Approach
Two options may have the same score, but their behavior may be different
and therefore incomparable”.
Goal, aspiration or reference “A goal for each criterion is defined, and then the closest options to the
level approach ideal goal or reference level are identified”.
* The definitions of qualifications for the solving approach are taken from Ishizaka et Nemery P. [9].
In the case of a full aggregation approach, the scores allow each alternative to be comparable
with another. They are expressed while taking into consideration the performance of the alternatives
according to the criteria and sub-criteria selected for the analysis.
With regard to the outranking approach the incomparability is defined by observing alternative
performance sets that are equally valid but qualified differently because they are based on different
sets of criteria. The allocation of a full or partial score to the alternative involves a consideration of the
performance set, based on the criteria and sub-criteria selected for the execution of the analysis.
In case of the “goal, aspiration or reference level approach”, the options (alternatives) are evaluated
using the aggregate collection (vector sum) of the performance in relation to the different criteria that
allow one to define how far (vector) the alternatives fall from the final objective.
The input level describes the “modelling effort” [9] needed to achieve the desired results and
is connected to the measurement (Indicators) of the data and the parameters to be drawn and
considered in order to solve the decision-making problem (Table 10). It is necessary to emphasize that
Strategic Planning Techniques [58] and Participation Techniques are employed in the evaluation and
collection of data used in the MCDA. Indeed, several stakeholders are often considered directly or
indirectly involved in the decision-making problem, including: (i) institutions (national, regional, local);
(ii) contracting stations; (iii) entrepreneurs, economic operators; (iv) property owners; (v) workers and
(vi) the population.
Table 10. Input levels.
Modelling Effort Parametres Indicators

Data and parameters to be traced and inserted into the evaluation model High, medium, low
Requested time to collect and process data Long, medium, short
Skills needed to manage and process data High, medium, low
Use of additional evaluation techniques for the collection of data used in the MCDA Necessary, advised, unnecessary
A score is assigned to each parameter by using the indicators. The input level is a synthetic indicator
of these scores, which expresses the level: High (H), Medium (M) or Low (L). The methodology of the
calculation is described in [30] Appendix A (Table A1).
The different implementation procedures are defined using logical mathematical operations in
order to process the data implemented in the evaluation matrix and to get a summary of results for the
classification of the alternatives. Different methods of data processing and aggregation are necessary
for the different implementation procedures. The procedures include: (i) preference thresholds,
indifference thresholds and veto thresholds [59]; (ii) utility function [9,17]; (iii) pairwise comparisons
on a ratio scale [27,60]; (iv) pairwise comparisons on a ratio scale with interdependencies [25];
(v) pairwise comparisons on an interval scale [26,61]; (vi) ideal option and anti-ideal option [9,28,62–64]
(Table 11) [30].
Table 11. Implementation procedures.
Implementation Procedures Data Processing and Aggregation

Pairwise preference degree comparing the performance of n alternatives. To find the preference level,
Preference thresholds, the evaluation must consider the preference and indifference thresholds. On the basis of these thresholds,
indifference thresholds, positive, negative and unicriterion net and global flows are created taking into account the weights attributed to
veto thresholds each criterion. If an action performs negatively according to a single criterion, it may also be included in a veto
threshold that definitively excludes that option from the final ranking.
The expression of the measure of desirability or preference of each alternative with respect to the others.
Different criteria are considered in the function. For each criteria, the marginal utility is determined as
representing the partial contribution that each criteria brings to the overall utility assessment. The Global utility
Utility function
is expressed by Global Utility Scores (generally expressed in values between 0 and 1) which are commonly
calculated by the additive method or with a weighted sum, based on the weighted importance (weight) for each
criterion, or by a simple addition.
The construction of evaluation matrices. The comparison of the elements included in the evaluation matrices,
Pairwise comparisons on a structured according to a hierarchical system of criteria, sub-criteria and alternatives. It is performed by
ratio scale simultaneously comparing two elements at a time with respect to the hierarchically superior element on the basis
of a rational numerical scale (Saaty Fundamental Scale).
Table 11. Cont.
Implementation Procedures Data Processing and Aggregation

The construction of evaluation matrices called Supermatrix. The Comparison of the elements included in the
Supermatrix, which are organised into clusters of criteria, sub-criteria and alternatives, is performed by
simultaneously comparing two elements at a time taking into account any interdependencies between them,
Pairwise comparisons on a
for example: (i) inner dependencies in cluster criteria; (ii) inner dependencies in the alternative cluster; (iii) outer
ratio scale with
dependencies (correlation between two different clusters). Based on the influences (also called nodes) between
interdependencies
elements or clusters, the Supermatrix is completed considering the influence of each node on the others and
expressed on a rational scale (Saaty Fundamental Scale). In the case of no interdependence between the elements
being compared, a value of zero is inserted into the Supermatrix.
The construction of evaluation matrices also called matrices of judgements. The comparison between the
evaluation elements (alternatives and criteria) is implemented by a pairwise comparison based on a semantic
Pairwise comparisons on an
qualitative scale (traditionally translated into quantitative values from 1 to 7). Values are generally included in
interval scale
the matrix of judgments where the relative attractiveness of the criteria and alternatives is also expressed by the
consideration of the weight attributed to each criterion.
The expression for each alternative, of the shortest distance to the ideal (virtual) solution and the longest distance
Ideal option and from the anti-ideal solution, taking into account the performance of alternatives referred to each criterion and to
anti-ideal option the weight of each criterion. The distance is expressed by calculating a distributive normalization and an ideal
normalization of the recorded performances.
It is possible to obtain output modalities by putting in order alternatives with different qualities.
The “granularity order” [9,15] varies according to the type of endogenous variables considered.
The output typologies are obtained as a result of the evaluation implementation referring to the
number (n) of alternatives evaluated (Table 12) [30]. This depends on different calculation methods,
which represent the comparability or incomparability between the alternatives and of the distance
(or the type of measurement or procedure by which the alternatives are ordered) of the alternatives
from achieving the defined objective.
Table 12. Output typologies.
Output Typologies Calculation Method

A simultaneous consideration of the positive and negative global performance flows
Partial and complete order obtained by
evaluated for each alternative or simply by considering the net flows that make it
expressing pairwise preference degrees
possible to understand whether the alternatives being deliberated obtain a higher rank,
and scores
a minor rank or if two or more alternatives are incomparable or equally valid.
Degrees of preference can lead to a partial rank (if two or more alternatives are
Partial and complete order obtained by incomparable) or a total rank (if the incomparability hypothesis is not allowed) of
expressing pairwise outranking degrees alternatives traditionally through the expression of degrees of concordance and
discordance according to the criteria considered.
Full order obtained by considering the
scores assigned to the alternatives in various By complex and general scores (a hypotheses of incomparability between two
ways (pairwise comparisons with or alternatives is not admitted) and a general approval of the ordering of alternatives
without interdependencies, utility functions, from the best to the worst.
pairwise comparisons on an interval scale)
Full order with a score closest to the The calculation of the proximity coefficient for each alternative traditionally expressed
desired objective in values between 0 and 1 where value 1 expresses the closest proximity to the aim.
The types of solution used to resolve the decision-making problem derive from the order (output)
of the alternatives and depend on whether the incomparability between two alternatives is admitted
or not. The solution is therefore based on different foundations (Table 13) [30].
Table 13. Solutions used for the decision-making problem.
Solutions Incomparability Solution foundation

n categories of alternatives of equal
Admitted The consideration of several valid alternatives at the same time
score but different behaviors
Alternative with the higher global score Not admitted The choice of alternative that gets the highest score
Choosing the alternative that gets a score, which is closest to
Alternative with the closest score to
Not admitted the ideal normalization of the recorded performances for the
the ideal solution
alternatives considered.
Table 14 summaries the features of the endogenous variables mentioned in this section.
Table 14. Features of the endogenous variables.
Type of
Solution Implementation Input Decision Problem
Decision-Making Output Typology Tool
Approach Procedure Level Solution
Problems
n categories of
Preference thresholds, Partial ordering obtained
Sorting/ Outranking alternatives of equal
indifference thresholds, Medium by expressing pairwise ELECTRE
Description approach score but different
veto thresholds preferences degrees
behaviour
Full ordering obtained by Alternative with the
Utility function High MAUT
considering the scores higher global score
Pairwise comparison on
Full ordering obtained by Alternative with the
rational scale and High ANP
considering the scores higher global score
Full aggregation interdependencies
approach Pairwise comparison on Full ordering obtained by Alternative with the
High MACBETH
interval scale considering the scores higher global score
Pairwise comparison on Full ordering obtained by Alternative with the
Low AHP
rational scale considering the scores higher global score
Goal, aspiration Full ordering with score Alternative with the
Ideal option and
or reference level Low closest to the aim closest score to the ideal TOPSIS
anti-ideal option
approach assumed solution
Ranking/Choice n categories of
Partial ordering obtained
alternatives of equal
by expressing pairwise
score but different
Preference thresholds, preferences degrees
behaviour
indifference thresholds, Medium ELECTRE
veto thresholds Total ordering obtained
Alternative with the
higher global score
Outranking preferences degrees
approach n categories of
Partial ordering obtained
alternatives of equal
score but different
preferences degrees
Preference thresholds, behaviour
Medium PROMETHEE
indifference thresholds Partial ordering obtained
Alternative with the
higher global score
preferences degrees
2.5. Properties of MCDA Methods Transposed into a Binary Mathematical System

The analyses carried out in Sections 2.3 and 2.4 above make it possible to construct a matrix
that summarizes the properties of the MCDA methods considered [30]. This matrix consists of
7 columns displaying the MCDA methods taken into consideration (Tn) and 38 rows representing
the qualifications (Qn) to be deliberated as a set of indicators for each variable (Vn). At the point
where each row and column meet, a score, Px(Tn;Vn;Qn), is assigned through a binary mathematical
system used to illustrate whether each of the relevant properties of qualification is present (1) or
absent (0) (Table 15) [30]. This transposition serves, in the following phase, for the implementation
of the procedure used to select the MCDA method best suited to the decision-making problem being
addressed. The transposition of the properties of MCDA methods into a binary mathematical system
was performed by examining the most popular MCDA [30].
Table 15. Properties of Multi-Criteria Decision Analysis (MCDA) methods transposed in a binary mathematical system.
Properties of MCDA Tool in Binary System

Type of Variables Variables Qualification of Variables
ELECTRE MAUT ANP MACBETH AHP TOPSIS PROMETHEE
Limited number of criteria and sub-criteria
1 0 0 0 0 0 0
and a small number of alternatives
Limited number of criteria and sub-criteria
0 1 0 0 0 0 0
and a large number of alternatives
Number of evaluation
elements Large number of criteria and sub-criteria and
0 0 1 0 1 0 0
a small number of alternatives
Large number of criteria and sub-criteria and
0 0 0 1 0 1 1
a large number of alternatives
Quantitative 1 1 1 1 1 1 1
Typology of indicators Qualitative 1 0 1 1 1 1 1
Mixed 1 0 1 1 1 1 1
Participatory process not activated 1 1 1 1 1 1 1
Participatory process activated with a limited
Stakeholders to be 1 1 1 1 1 1 1
Exogenous and specialized number of stakeholder
included in the decision
process Participatory process activated with a
significant number of stakeholder preferably 1 1 1 1 1 1 1
organized in categories
A better overall alternative definition for
1 0 0 0 0 0 0
the purpose
The ideal alternative definition closest to
0 0 0 0 0 1 0
the lens
Expected solution A better overall alternative definition for
0 1 1 1 1 0 1
the purpose
The ideal alternative definition closest to
0 0 0 0 0 1 0
the lens
Technical support of a Yes (advisable) 1 1 1 1 0 0 0
Decision Aid Specialist
No (not necessary) 0 0 0 0 1 1 1
Table 15. Cont.
Properties of MCDA Tool in Binary System

Type of Variables Variables Qualification of Variables
ELECTRE MAUT ANP MACBETH AHP TOPSIS PROMETHEE
Sorting 1 0 0 0 0 0 0
Type of decision-making
Description 1 0 0 0 0 0 0
problems
Ranking/Choice 1 1 1 1 1 1 1
Outranking approach 1 0 0 0 0 0 1
Solution approach Full aggregation approach 0 1 1 1 1 0 0
Goal, aspiration or reference level approach 0 0 0 0 0 1 0
Preference thresholds, indifference thresholds,
1 0 0 0 0 0 0
veto thresholds
Preference thresholds, indifference thresholds 0 0 0 0 0 0 1
Utility function 0 1 0 0 0 0 0
Implementation Pairwise comparison on rational scale and
procedure 0 0 1 0 0 0 0
interdependencies
Pairwise comparison on interval scale 0 0 0 1 0 0 0
Pairwise comparison on rational scale 0 0 0 0 1 0 0
Endogenous Ideal option and anti-ideal option 0 0 0 0 0 1 0
High 0 1 1 1 1 0 0
Input level Medium 1 0 0 0 0 0 1
Low 0 0 0 0 0 1 0
Partial ordering obtained by expressing
1 0 0 0 0 0 1
pairwise preferences degrees
Total ordering obtained by expressing
1 0 0 0 0 0 1
pairwise preferences degrees
Output typology Full ordering obtained by considering the
0 1 1 1 1 0 0
scores
Full ordering with score closest to the aim
0 0 0 0 0 1 0
assumed
n categories of alternatives of equal score but
1 0 0 0 0 0 1
different behaviour
Decision problem Alternative with the higher global score 0 1 1 1 1 0 0
solution
Alternative with the closest score to the ideal
0 0 0 0 0 1 0
solution
3. Selecting Methods of Multi-Criteria Decision Analysis: The Proposed Procedure

The selection of the most suitable MCDA method is carried out by comparing the framework of
properties that characterize each MCDA method (Table 15) with the qualifications that the method
should possess (the expected properties), depending on the decision-making problem to be addressed
and taking into consideration both the exogenous and endogenous variables. As already seen in
Section 1, the procedure is designed to satisfy the settlement transformation sector as follows:
1. The weighting of variables (optional action): A set of variables (that represents the criteria) and their
potential qualifications (Table 15) has been defined (see also Section 3). The variables can be
considered of equal importance or weight (equal weight method) or of different importance
and weight [65–67]. Should it be necessary to consider the varying importance of the variables,
a weight can therefore be assigned to each of them [68]. Different weights will directly influence
the results of MCDA procedure. Consequently, it is essential to define the rationality and veracity
of the criteria weights. Several methods of achieving this are discussed in the reference literature.
For example: (i) subjective weighting methods such as direct assignment, Simple Multi-Attribute
Rating Technique (SMART), SWING, SIMOS, pairwise comparison, AHP; (ii) Objective weighting
methods such as entropy method, TOPSIS and combination weighting methods [66,67,69–71].
The most appropriate weighting method can be chosen by taking into consideration: (i) the
variance in the degrees of criteria; (ii) the independency of criteria; (iii) the subjective preferences
of the decision-makers and stakeholders when communicating their weights [68]. The exact
number of criteria (and sub-criteria) may also have some relevance [35,59]. Direct assignment,
SMART and SWING are the most used methods for addressing decision-making problems related
to the settlement transformation process. The advantages of these being: the fast implementation
times and the possibility to collect the views of stakeholders through questionnaires. However,
the various weakness must also be considered including the difficulties connected to quantifying
the uncertainty of the human input [66] and the subsequent conflict between the thoughts and
priorities of the stakeholders and the expression of ranking and values. Appendix B describes
how stakeholders may express the index of importance for each variable and their aggregation
modalities [30].
2. Determining the framework of expected properties: This involves the identification (presence
or absence) of the qualifications needed by the different variables in order to address the
decision-making problem in question. Those responsible for the process of settlement
transformation must determine the needs and demands involved in the decision-making problem
being examined. The choice must be based on the set of exogenous and endogenous variables
and composed of both the required and expected properties, EP(Vn;Qn), of the method selected
for the decision-making problem. The framework of the expected properties for each exogenous
and endogenous variables (for the chosen method) is determined according to the formulas and
Table A2 [30] attached in Appendix C.
3. Calculation of the overall index of suitability: This is based on a comparison of the properties of
the various MCDA methods (Table 15) with their expected properties. A general index can be
obtained for the suitability of each potential method for resolving the evaluation problem. Before
an overall index of suitability can be calculated, the suitability, SR(Vn;Qn), must be determined
for each qualification of the variables listed on the new table. The suitability is determined
by comparing the data of the properties of the MCDA methods, for each qualification of the
variable (Table 15) with the data included on the table to be filled identifying the expected
properties for each exogenous and endogenous variables (see Appendix C Table A2) [30]. Refer to
Appendix D.1, for the possible configurations deriving from the calculation of the overall index
of suitability [30]. The suitability results, SR(Vn;Qn), for each variable are then combined for each
MCDA method in order to produce an aggregate index of suitability IS(Tn). In order to weigh
the variables, the suitability results must be multiplied by the index of importance for the factors
expressed by the stakeholders (see Appendix D.2 for the mathematical formula [30]). Should the
suitability of 2 or more qualifications have been determined for a single variable, then it holds
that if the binary system produces a number of results that are equal to 1, the overall result will
be 1 when calculating the overall suitability. In the case of it not being necessary to weigh the
variables, the aggregate index of overall suitability or IS(Tn) for each MCDA method is obtained
as displayed [30] in Appendix D.3.
4. The Identification of the method best suited to resolving the decision-making problem: Obtaining a ranking
of the MCDA methods with respect to the overall suitability indicators acquired. The ranking,
POS(Sn), of the overall indexes of suitability for each MCDA method is reached by listing the
indexes of aggregate suitability, IS(Tn) or ISW(Tn), in descending order. The most suitable method
is the one with the highest index of overall suitability.
As already mentioned in previous sections, the procedure being proposed has been designed by
considering the 6 endogenous variables that describe the different methods of MCDA [9]. In addition,
we have the 4 exogenous variables derived from the Italian regulatory framework integral to settlement
transformation processes. The selection procedure of the most appropriate MCDA method as
proposed below may be employed in other territorial contexts, however, the endogenous variables
remain unchanged in these possible applications as they are related to the implementation of MCDA
“techniques”. Instead, the exogenous variables can be reconsidered by using reduction, integration or
substitution depending on the evaluation problem to be answered.
4. Application of the Proposed Procedure to a Case Study
4.1. A Procedural Application: The Evaluation of Design Proposals Responding to the Call for Tenders for a
New Office Building at the Chamber of Deputies in Rome
The procedure proposed by this paper was applied to the selection of an MCDA method to
be used in a hypothetical international call for tenders. The structure in question is a new office
building at the Chamber of Deputies in Rome, for which the design ideas must be evaluated. The new
building is to be constructed in an urban void adjacent to Palazzo Montecitorio in the historic center
of Rome (Italy). The hypothetical request aims at identifying the best design idea for the solution
of a situation unresolved since the first call for tenders for this potential building at the Chamber of
Deputies in 1967 [1,72].
The purpose of the request is to choose from amongst multiple design proposals, the best
one considering a set of criteria, sub-criteria, indicators and weights for the variables specifically
designed by a team of experts (Table 16) [30,72] formed of technical officials from Rome’s Public
Administration (4) and professors (6) and researchers (8) involved in studying a new project for the
Chamber of Deputies.
4.2. Weighting of the Variables

In this case study, a direct assignment has been proposed; without any mathematical
normalization, so that the weighting operation can be performed easily and quickly. Direct assignment,
SMART and SWING, appear to be the most appropriate methods when considering the features of the
proposed procedure [35,59,68]. This is because the variables considered are different and independent
from each other and limited in number, and the different stakeholders (both standard and interest
groups) have the possibility to express their point of view. Direct assignment has been implemented,
by attributing to every stakeholder for each variable an index of importance, W(Vn), considering:
0 no importance; 0.25 low importance, 0.5 medium importance, 0.75 high importance and 1 very
high importance. This weighting operation has been performed by taking into account the opinions
gathered (through interviews) from a representative sample of stakeholders (technical officials from
Rome’s Public Administration; professors and researchers involved in studying a new project for the
Chamber of Deputies). The results of each category of stakeholders considered have been aggregated
(Table 17 column weight) according to the modalities shown in Appendix B considering a simple index
of importance between stakeholders.
Table 16. Evaluation elements to be considered in the call for tenders for the office building at the
Chamber of Deputies.
General Typology of
Goal Criteria Sub-Criteria Indicators
Objectives Indicators
Alignment of the new building to the - Total;
urban fabrics before demolition (Rilievo Qualitative - Partial;
Urban fabric filling in
IGM 1873) - Absent
relationship with the
historical development Presence of inner courts (covered or
- Present;
process uncovered) following the tradition of Qualitative
- Absent
the historical urban fabric
- Very high;
Connection between design spaces, - High;
Architectural urban spaces and parliamentary Qualitative - Medium;
and Urban functions close to the design area - Low;
quality - Very low
Organic relationship
between buildings and Mixed use providing by concentration
- Total;
urban spaces of commercial functions on Matrix
Qualitative - Partial;
route in order to restore its functional
- Absent
and morphological continuity
Easy access to non parliamentary - Total;
functions on matrix route (Via di Qualitative - Partial;
The urban void Campo Marzio) - Absent
solution by the
inclusion of - Very high;
new functions Minimizing of unmovable structures to - High;
Flexibility and reduce the impact on the dynamic and Qualitative - Medium;
Technical and integrability of inner and alternative use of spaces - Low;
functional outer spaces from - Very low
quality functional and distributive
Minimizing of technical and structural - Total;
point of view
elements to reduce the impact on the Qualitative - Partial;
dynamic and alternative use of spaces - Absent
% on base
amount
Cost reduction Quantitative
established for
call for tenders
Spending Control
Economic and Cost sustainability connected with
Quantitative €/year
financial energy saving
aspects Maintenance costs por year Quantitative €/year
Environmental costs Quantitative €
Economic Convenience Net Present
Costs Benefits ratio Quantitative
Value (€)
4.3. Determination of the Framework of Expected Properties

The expected properties of the MCDA method used to evaluate the design proposals have been
defined when considering the objectives and the details of the request as well as summarized in Table 17
(columns expected properties to the decision-making problems), according to the modalities [30] shown
in Appendix C.
Table 17. Expected properties for the MCDA method in the call for tenders for the office building at
the Chamber of Deputies in Rome.
Expected Properties to
Type of Weight
Variables Qualification of Variables Decision-Making Problem
Variables
Value Motivation
Limited number of criteria and
sub-criteria and a small number of 0 -
alternatives
Limited number of criteria and
sub-criteria and a large number of 0 -
Number of alternatives
evaluation 0.5 Large number of criteria and
elements sub-criteria and a small number of 0 -
alternatives
Related to Criteria, Sub-Criteria
Large number of criteria and
and Indicators of Evaluation;
sub-criteria and a large number of 1
Considering a significant
alternatives
participation in the call
Quantitative 0 -
Typology of Qualitative 0 -
0.75
indicators Related to Criteria, Sub-Criteria
Mixed 1
and Indicators of Evaluation
Exogenous Participatory.Process not activated 0
Participatory.Process activated with a
Stakeholders to limited and specialized number of 0
be included in stakeholder
the decision 1
Participatory.Process activated with a Need to activate a participatory
process significant number of stakeholder 1 process with a significant number
preferably organized in categories of categories of stakeholders
Definition of n alternatives valid in
0 -
relation to objectives
Expected A better overall alternative definition Need to select the best design
1 1
solution for the purpose proposal
The ideal alternative definition closest
0 -
to the lens
Technical support Need to speed up decision

Yes (advisable) 1
of a Decision Aid making
0.25
Specialist No (not necessary) 0 -
Sorting 0 -
Type of
Description 0 -
decision-making 0.5
problems Need to form a ranking among
Ranking/Choice 1
the design proposals
Outranking approach 0 -
Solution Full aggregation approach 1 Necessity of project proposals in
approach 1
Goal, aspiration or reference level relation to all achievements
1
approach
Preference thresholds, indifference
0 -
thresholds, veto thresholds
Endogenous Need to check the performance of

Preference thresholds, indifference
1 project proposals in relation to
thresholds
thresholds
Utility function 0 -
Implementation
procedure 1 Pairwise comparison on rational scale
0 -
and interdependencies
Pairwise comparison on interval scale 0 -
Pairwise comparison on rational scale 0 -
Need to check the performance of
Ideal option and anti-ideal option 1 project proposals in relation to
thresholds
Table 17. Cont.
Expected Properties to
Type of Weight Decision-Making Problem
Variables Qualification of Variables
Variables
Value Motivation
- Amount of data and parameters:
high (calculation for weighing the
modelling effort level in relation
to the input level parameters as
indicated in Table A1)
High 1 - Times for the definition: medium;
-Skills and degree of knowledge of
Input level 0.75
the decision-making
problem: high;
- Use of integrated techniques:
not necessary
Medium 0 -
Low 0 -
Partial ordering obtained by
expressing pairwise 0 -
Exogenous preferences degrees
Total ordering obtained by expressing
0 -
Output typology 1 pairwise preferences degrees
Full ordering obtained by considering
1
the scores
Need to measure the performance
Full ordering with score closest to the of project proposals
1
aim assumed
n categories of alternatives of equal
0 -
score but different behaviour
Decision problem Alternative with the higher Need to identify the project
1 1
solution global score proposal with the best
Alternative with the closest score to performance in relation to the
1
the ideal solution goals
4.4. Calculation of the Overall Index of Suitability

Following the procedure, the comparison between the property framework of MCDA methods
(Table 15) and the data included on the table to be filled identifying the expected properties for each
exogenous and endogenous variable as indicated [30] in Table A2 in Appendix C, provides the results
of consistency for each qualification of the variables (Table 18). We arrive at the synthetic global
coherence indicator, the overall suitability index IS(Tn), relative to each of the most commonly used
MCDA methods calculated according to the modalities [30] shown in Appendix D.
Table 18. Results of potential MCDA methods suggested for the selection of the best design proposal in the call for tenders for the office building at the Chamber
of Deputies.
Type of Consistency in Relation to the MCDA Tools in Relation to the Expected Qualification
Variables Weight Qualification of Variables
Variables ELECTRE MAUT ANP MACBETH AHP TOPSIS PROMETHEE
Limited number of criteria and sub-criteria and a small
0.00 0.00 0.00 0.00 0.00 0.00 0.00
number of alternatives
Limited number of criteria and sub-criteria and a large
0.00 0.00 0.00 0.00 0.00 0.00 0.00
Number of evaluation number of alternatives
0.5
elements Large number of criteria and sub-criteria and a small
0.00 0.00 0.00 0.00 0.00 0.00 0.00
Large number of criteria and sub-criteria and a large
0.00 0.00 0.00 0.50 0.00 0.50 0.50
Quantitative 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Typology of indicators 0.75 Qualitative 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Exogenous Mixed 0.75 0.00 0.75 0.75 0.75 0.75 0.75
ParticipatoryProcess not activated 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Stakeholders to be Participatory.Process activated with a limited and
0.00 0.00 0.00 0.00 0.00 0.00 0.00
included in the decision 1 specialized number of stakeholder
process Participatory.Process activated with a significant number
1.00 1.00 1.00 1.00 1.00 1.00 1.00
of stakeholder preferably organized in categories
A better overall alternative definition for the purpose;
0.00 0.00 0.00 0.00 0.00 0.00 0.00
The ideal alternative definition closest to the lens
Expected solution 1 A better overall alternative definition for the purpose 0.00 1.00 1.00 1.00 1.00 0.00 1.00
The ideal alternative definition closest to the lens 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Technical support of a Yes (advisable) 0.25 0.25 0.25 0.25 0.00 0.00 0.00
0.25
Decision Aid Specialist No (not necessary) 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Table 18. Cont.
Type of Consistency in Relation to the MCDA Tools in Relation to the Expected Qualification
Variables Weight Qualification of Variables
Variables ELECTRE MAUT ANP MACBETH AHP TOPSIS PROMETHEE
Sorting 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Type of
decision-making 0.5 Description 0.00 0.00 0.00 0.00 0.00 0.00 0.00
problem.s Ranking/Choice 0.50 0.50 0.50 0.50 0.50 0.50 0.50
Outranking approach 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Solution approach 1 Full aggregation approach 0.00 1.00 1.00 1.00 1.00 0.00 0.00
Goal, aspiration or reference level approach 0.00 0.00 0.00 0.00 0.00 1.00 0.00
Preference thresholds, indifference thresholds,
0.00 0.00 0.00 0.00 0.00 0.00 0.00
veto thresholds
Preference thresholds, indifference thresholds 0.00 0.00 0.00 0.00 0.00 0.00 1.00
Utility function 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Implementation Pairwise comparison on rational scale and
1 0.00 0.00 0.00 0.00 0.00 0.00 0.00
procedure interdependencies
Pairwise comparison on interval scale 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Pairwise comparison on rational scale 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Endogenous Ideal option and anti-ideal option 0.00 0.00 0.00 0.00 0.00 1.00 0.00
High 0.00 0.05 0.05 0.00 0.05 0.00 0.00
Input level 0.75 Medium 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Low 0.00 0.00 0.00 0.00 0.00 0.00 0.00
Partial ordering obtained by expressing pairwise
0.00 0.00 0.00 0.00 0.00 0.00 0.00
preferences degrees
Total ordering obtained by expressing pairwise
0.00 0.00 0.00 0.00 0.00 0.00 0.00
Output typology 1 preferences degrees
Full ordering obtained by considering the scores 0.00 1.00 1.00 1.00 1.00 0.00 0.00
Full ordering with score closest to the aim assumed 0.00 0.00 0.00 0.00 0.00 1.00 0.00
n categories of alternatives of equal score but
0.00 0.00 0.00 0.00 0.00 0.00 0.00
different behaviour
Decision problem
1 Alternative with the higher global score 0.00 1.00 1.00 1.00 1.00 0.00 0.00
solution
Alternative with the closest score to the ideal solution 0.00 0.00 0.00 0.00 0.00 1.00 0.00
Overall suitability index (IS) 0.23 0.53 0.60 0.64 0.57 0.61 0.43
4.5. Results: Identification of the Method Best Suited to Resolving the Decision-Making Problem
The decreasing order of the global coherence indicators obtained for each MCDA method
considered identifies MACBETH as the best performing method according to the objectives of the
request. Using this method could facilitate the evaluation problem of choosing the best design proposal,
by giving a suitable answer to the established objectives (Table 19).
Table 19. The order of potential MCDA methods to select the best design proposal for the new services
building at the Chamber of Deputies in Rome.
MCDA Tool Overall Suitability Index (IS) Ranking

MACBETH 0.64 1
TOPSIS 0.61 2
ANP 0.60 3
AHP 0.57 4
MAUT 0.53 5
PROMETHEE 0.43 6
ELECTRE 0.23 7
MACBETH sits at the top of the order with a consistency index of 0.64; TOPSIS (0.61) and ANP
(0.60) also obtain a high consistency; AHP (0.57) and MAUT (0.53) reach a medium consistency and
could potentially be used for solving the decision-making problem specific to the case study.
According to the objectives of the international call for tenders mentioned above, the consistency
of PROMETHEE (0.43) is low and ELECTRE (0.23) is not recommended at all.
5. Discussion and Conclusions

Applying the proposed procedure to a case study allowed us to identify the TOPSIS method as
that which was best adapted to the evaluation of design proposals in the call for tenders to construct a
new office building at the Chamber of Deputies in Rome.
During the case study, it was observed that the proposed procedure works better when used in
contexts where there is a proper knowledge of MCDA methods. Furthermore, a basic knowledge of
how to implement MCDA methods is required by the process manager. The process manager must
understand fully the relationship between the most suitable MCDA methods when compared with
the suitability and strength of the results for a specific decision-making problem. The implementation
of the proposed procedure may prove difficult at times because of a low level of MCDA knowledge,
especially in the context of settlement transformation processes. In order to promote a wider use of the
procedure proposed in this paper, some upgrades could be useful. These could focus particularly on
contexts where there is a low knowledge of MCDA.
Further developments to the proposed procedure could involve: (i) How to identify the relevant
stakeholders in relation to the decision-making problem to be solved. (ii) How to attribute indices
of importance to them, which truly represent their role in the decision to be taken. (iii) How the
process manager can arrive at a final decision (choice of MCDA method) in contexts characterized by
multiple stakeholders. (iv) An in-depth analysis of the selection and use of criteria weighing methods.
(v) The formulation of guidelines to facilitate creating the framework of expected properties, even by
people not particularly qualified in the field of MCDA [30].
As mentioned above, the examination of the decision-making problem in question and the
evaluation query (plus the related evaluation objective), represent the main critical aspects when
implementing the MCDA. It should also be observed that, during MCDA implementation, other
critically important stages could be incorporated. These may include: (i) a definition of the specific
objectives, the criteria, the sub-criteria and the indicators; (ii) the ability to obtain the input data
needed to structure the problem; (iii) the capacity to implement and control the analysis and to study
it in greater depth, in addition to defining the timeframe within which to resolve the evaluation
query [73,74].
Author Contributions: The paper must be attributed in equal parts to the authors.
Conflicts of Interest: The authors declare no conflict of interest.
Appendix A. Input Level Calculation

“To define the input level, a score is assigned to each of the 4 parameters considered in relation
to the required modeling effort (Table A1) and by creating an average of the scores attributed to the
parameters. If the total score is less than 0.33, the input level is low. If the total score is greater than
0.33 and not higher than 0.66, the input level is considered average. If the result is higher than 0.66,
the input level is high” [30].
Table A1. Weighing the modelling effort level in relation to the input level parameters.
Parameters of the Input Level Definition and Calculation

Score to Be Data and Skills and Level of Use of Other
Assigned Definition
Parameter Knowledge of the Integrated
Time (2)
Quantity (1) Decision Problem (3) Techniques (4)
1 High Long High Necessary
0.5 Medium Medium Medium Advised
0 Low Short Low Unnecessary
Appendix B. Weighting of the Variables by the Options Considered Preferable to

the Stakeholders
If the person responsible for the evaluation process opts for a weighting of the variables by the
stakeholders grouped into categories, the index of importance (of each variable) for each category
must be determined aggregating through mathematical media to define a synthesis result used for the
implementation procedure [30].
The different indices of importance for each variable can be aggregated by averaging them,
as follows:
- Simple, if all stakeholders are considered of equal importance;

- Weighted, if the stakeholders are considered of varying importance [74].
When selecting the most suitable MCDA method, the process manager can select the solution
that is preferred by the stakeholders whose added indices of importance are greater than a specified
threshold (related to a simple/relative/qualified majority and unanimity). The process manager can
set this threshold in relation to the composition of the stakeholders being considered (an increasing
threshold as the stakeholders’ points of view rise) [30].
Appendix C. Determination Modality of the Framework of Expected Properties

To define the set of expected properties for each exogenous and endogenous variables must be
filled (Table A2) [30] by deciding whether, for a given variable, each qualification is actually required
or not ((A1a) and (A1b)):
if EP(Vn;Qn) = request → EP(Vn;Qn) = yes (1) (A1a)
if EP(Vn;Qn) = no request → EP(Vn;Qn) = no (0) (A1b)

Table A2. Table to define the expected properties.
Expected Properties
Type of in Relation to
Weight Variables Qualification of Variables
Variables Decision-Making
Problem
Limited number of criteria and sub-criteria and a Request = 1;
small number of alternatives Not request = 0
Limited number of criteria and sub-criteria and a Request = 1;
Number of large number of alternatives Not request = 0
0≤W≤1 evaluation
elements Large number of criteria and sub-criteria and a small Request = 1;
number of alternatives Not request = 0
Large number of criteria and sub-criteria and a large Request = 1;
number of alternatives Not request = 0
Request = 1;
Quantitative
Not request = 0
Typology of Request = 1;
0≤W≤1 Qualitative
indicators Not request = 0
Request = 1;
Mixed
Not request = 0
Request = 1;
Exogenous Participatory process not activated
Not request = 0
Stakeholders to be Participatory process activated with a limited and Request = 1;
0≤W≤1 included in the specialized number of stakeholder Not request = 0
decision process Participatory process activated with a significant
Request = 1;
number of stakeholder preferably organized
Not request = 0
in categories
Definition of n alternatives valid in relation Request = 1;
to objectives Not request = 0
Request = 1;
0≤W≤1 Expected solution A better overall alternative definition for the purpose
Not request = 0
Request = 1;
The ideal alternative definition closest to the lens
Not request = 0
Request = 1;
Yes (advisable)
Technical support Not request = 0
0≤W≤1 of a Decision Aid
Request = 1;
Specialist No (not necessary)
Not request = 0
Request = 1;
Sorting
Not request = 0
Type of
Request = 1;
0≤W≤1 decision-making Description
Not request = 0
problems
Request = 1;
Ranking/Choice
Not request = 0
Request = 1;
Outranking approach
Not request = 0
Request = 1;
0≤W≤1 Solution approach Full aggregation approach
Not request = 0
Request = 1;
Goal, aspiration or reference level approach
Not request = 0
Preference thresholds, indifference thresholds, veto Request = 1;
thresholds Not request = 0
Endogenous
Request = 1;
Preference thresholds, indifference thresholds
Not request = 0
Request = 1;
Utility function
Not request = 0
Implementation
0≤W≤1 Pairwise comparison on rational scale and Request = 1;
procedure
interdependencies Not request = 0
Request = 1;
Pairwise comparison on interval scale
Not request = 0
Request = 1;
Pairwise comparison on rational scale
Not request = 0
Request = 1;
Ideal option and anti-ideal option
Not request = 0
Table A2. Cont.
Expected Properties
Type of in Relation to
Weight Variables Qualification of Variables
Variables Decision-Making
Problem
Request = 1;
High
Not request = 0
Request = 1;
0≤W≤1 Input level Medium
Not request = 0
Request = 1;
Low
Not request = 0
Partial ordering obtained by expressing pairwise Request = 1;
preferences degrees Not request = 0
Total ordering obtained by expressing pairwise Request = 1;
preferences degrees Not request = 0
Endogenous 0≤W≤1 Output typology
Request = 1;
Full ordering obtained by considering the scores
Not request = 0
Request = 1;
Full ordering with score closest to the aim assumed
Not request = 0
n categories of alternatives of equal score but Request = 1;
different behaviour Not request = 0
Decision problem Request = 1;
0≤W≤1 Alternative with the higher global score
solution Not request = 0
Alternative with the closest score to the Request = 1;
ideal solution Not request = 0
Appendix D. The Calculation of the Overall Index of Suitability
Appendix D.1. Possible Configurations

The comparison between properties of each method with expected properties can generate
4 possible configurations ((A2a)–(A2d)) [30]:
if P(Tn;Vn;Qn) = 1; EP(Vn;Qn) = 1 → SR(Vn;Qn) = 1 (A2a)
if P(Tn;Vn;Qn) = 1; EP(Vn;Qn) = 0 → SR(Vn;Qn) = 0 (A2b)
if P(Tn;Vn;Qn) = 0; EP(Vn;Qn) = 1 → SR(Vn;Qn) = 0 (A2c)
if P(Tn;Vn;Qn) = 0; EP(Vn;Qn) = 0 → SR(Vn;Qn) = 0 (A2d)
Appendix D.2. Equation to Obtain Weighted Suitability Results (Partial Coherence Results)
To obtain weighted suitability result must be considered the suitability results and the weight of
the variables [30]:
SRW = SR(Vn; Qn) ∗ W(Vn)
where
SRW(Vn;Qn): weighted suitability results (partial coherence results);

SR(Vn;Qn): suitability results (partial coherence results);
W(Vn): weighting judgement expressed in Vn variable (between 0 and 1).
Appendix D.3. Equation to Obtain Index of Overall Suitability (Overall Coherence Index)
To obtain weighted index of overall suitability must be aggregated the partial coherence results.
If the variables are not to be weighted, the aggregate index of overall weighted suitability ISW(Tn)
for each MCDA index is obtained via the equation [30]:
∑nk=0 SR(Vn; Qn)

IS(Tn) = (A3a)
NVn
where
IS(Tn): index of overall suitability (overall coherence index);

SR(Vn;Qn): suitability results (partial coherence results);
NVn: number of variables considered.
If the variables are to be weighted, the aggregate index of overall weighted suitability ISW(Tn)
for each MCDA index is obtained via the equation [30]:
∑nk=0 SRW(Vn; Qn)

ISW(Tn) = (A3b)
NVn
where
ISW(Tn): index of overall weighted suitability (overall coherence index);

SRW(Vn;Qn): weighted suitability results (partial coherence results);
NVn: number of variables considered.
References
1. Guarini, M.R.; Chiovitti, A.; Battisti, F.; Morano, P. An Integrated Approach for the Assessment of
Urban Transformation Proposals in Historic and Consolidated Tissues. In Computational Science and Its
Applications—ICCSA 2017; Gervasi, O., Murgante, B., Misra, S., Borruso, G., Torre, C.M., Rocha, A.M.A.C.,
Taniar, D., Apduhan, B.O., Stankova, E., Cuzzocrea, A., Eds.; Springer: Cham, Switzerland, 2017; pp. 562–574.
2. Marakas, G.M. Decision Support Systems in the 21st Century; Prentice Hall: Upper Saddle River, NJ, USA, 2003;
Volume 134.
3. Klapka, J.; Piňos, P. Decision support system for multicriterial R&D and information systems projects
selection. Eur. J. Oper. Res. 2002, 140, 434–446.
4. Belton, V.; Stewart, T. Multiple Criteria Decision Analysis—An Integrated Approach; Kluwer Academic Press:
Boston, MA, USA, 2002.
5. Figueira, J.; Greco, S.; Ehrgott, M. Multiple Criteria Decision Analysis—State of the Art Survey; Springer:
New York, NY, USA, 2005.
6. Nijkamp, P.; Beinat, E. Multi-Criteria Analysis for Land Use Management; Kluwer Academic Publishers:
Dordrecht, The Netherlands, 1998.
7. Hartog, J.A.; Hinloopen, E.; Nijkamp, P. A sensitivity analysis of multi-criteria choice-methods:
An application on the basis of the optimal site selection for a nuclear power plant. Energy Econ. 1989,
11, 293–300. [CrossRef]
8. Huang Ivy, B.; Keisler, J.; Linkov, I. Multi-criteria decision analysis in environmental sciences: Ten years of
application and trends. Sci. Total Environ. 2011, 409, 3578–3594. [CrossRef] [PubMed]
9. Ishizaka, A.; Nemery, P. Multi-Criteria Decision Analysis, Methods and Software; Wiley and Sons Ltd.: Chichester,
UK, 2013.
10. Roy, B. Méthodologie Multicritére d’Aide à la Décision; Economica: Paris, France, 1985.
11. Guitoni, A.; Martel, J.M. Tentative guidelines to help choosing an appropriate MCDA method. Eur. J.
Oper. Res. 1998, 109, 501–521. [CrossRef]
12. Vincke, P. L’aide Multicritère à la Décision, Édition de l’Université de Bruxelles; Bruxelles: Brussels, Belgium, 1989.
13. Colson, G.; De Bruyn, C. Models and Methods in Multiple Objectives Decision Making, Models and Methods in
Multiple Criteria Decision Making; Pergamon Press: Oxford, UK, 1989.
14. Fishburn, P.C. A survey of multiattribute/multicriterion evaluation theories. In Multiple Criterion Problem
Solving; Zionts, S., Ed.; Springer: Heidelberg, Germany, 1978; pp. 181–224.
15. Guitouni, A.; Martel, J.M.; Vincke, P.; North, P.B. A Framework to Choose a Discrete Multicriterion Aggregation
Procedure; Defence research establishment valcatier (DREV): Ottawa, ON, Canada, 1998. Available
online: https://pdfs.semanticscholar.org/27d5/9c846657268bc840c4df8df98e85de66c562.pdf (accessed on
28 July 2017).
16. Roy, B.; Bouyssou, D. Aide Multicritère à la Décision: Methodes et Cas; Economica: Paris, France, 1993.
17. Keeney, R.L.; Raiffa, H. Decisions with Multiple Objectives: Preferences and Value Trade-Offs; Cambridge
University Press: Cambridge, UK, 1993.
18. Cinelli, M.; Stuart, R.; Coles, K.K. Analysis of the potentials of multi criteria decision analysis methods to
conduct sustainability assessment. Ecol. Indic. 2014, 46, 138–148. [CrossRef]
19. Al-Shemmeri, T.; Bashar, A.; Pearman, A. Model choice in multi-criteria decision aid. Eur. J. Oper. Res. 1997,
97, 550–560. [CrossRef]
20. Celik, M.; Deha, I.E. Fuzzy axiomatic design extension for managing model selection paradigm in decision
science. Expert Syst. Appl. 2009, 36, 6477–6484. [CrossRef]
21. Kurka, T.; Blackwood, D. Selection of MCA methods to support decision making for renewable energy
developments. Renew. Sustain. Energy Rev. 2013, 27, 225–233. [CrossRef]
22. Saaty, T.L. The modern science of multicriteria decision making and its practical applications: The AHP/ANP
approach. Oper. Res. 2013, 61, 1101–1118. [CrossRef]
23. Roy, B. Classement et choix en presence de points de vue multiples: La méthode ELECTRE. Rev. Fr. Inform.
Rech. Opér. 1968, 8, 57–75. [CrossRef]
24. Dyer, J.S. MAUT—Multiattribute utility theory. In Multiple Criteria Decision Analysis: State of the Art Surveys;
Springer: New York, NY, USA, 2005; pp. 265–292.
25. Saaty, T.L. Analytic network process. In Encyclopedia of Operations Research and Management Science; Springer:
New York, NY, USA, 2001; pp. 28–35.
26. Bana e Costa, C.; Vansnick, J. MACBETH: An interactive path towards the construction of cardinal value
functions. Int. Trans. Oper. Res. 1994, 1, 387–500. [CrossRef]
27. Saaty, T. A scaling Method for priorities in hierarchical structures. J. Math. Psychol. 1977, 15, 234–281.
[CrossRef]
28. Hwang, C.L.; Yoon, K. Multiple Attribute Decision Making: Methods and Applications; Springer: Heidelberg,
Germany, 1981.
29. Brans, J.P.; Vincke, P. Note—A Preference Ranking Organisation Method: The PROMETHEE Method for
Multiple Criteria Decision-Making. Manag. Sci. 1985, 31, 647–656. [CrossRef]
30. Guarini, M.R.; Battisti, F.; Chiovitti, A. Public Initiatives of Settlement Transformation: A Theoretical-
Methodological Approach to Selecting Tools of Multi-Criteria Decision Analysis. Buildings 2018, 8, 1.
[CrossRef]
31. European Commission. Evaluation Methods for the European Union’s External Assistance; Evaluation Tool;
European Commission: Brussels, Belgium, 2006; Volume 4. Available online: http://ec.europa.eu/
europeaid/sites/devco/files/evaluation-methods-guidance-vol4_en.pdf (accessed on 28 July 2017).
32. Seixedo, C.; Tereso, A. A Multi-criteria Decision Aid Software Application for selecting MCDA Software
Using AHP, Conference Paper In 2nd International Conference on Engineering Optimization, Lisbon,
Portugal. 2010. Available online: http://hdl.handle.net/1822/19355 (accessed on 5 October 2017).
33. Campbell, J.D.; Jardine, A.K.; McGlynn, J. Asset Management Excellence: Optimizing Equipment Life-Cycle
Decisions; CRC Press: Boca Raton, FL, USA, 2016.
34. Guarini, M.R.; Battisti, F. Evaluation and Management of Land-Development Processes Based on the
Public-Private. Adv. Mater. Res. 2014, 869–870, 154–161. [CrossRef]
35. Bouyssou, D. Some remarks on the notion of compensation in MCDA. Eur. J. Oper. Res. 1986, 26, 150–160.
[CrossRef]
36. Chung, E.S.; Lee, K.S. Prioritization of water management for sustainability using hydrologic simulation
model and multi-criteria decision making techniques. J. Environ. Manag. 2009, 90, 1502–1511. [CrossRef]
[PubMed]
37. Liu, D.F.; Stewarr, T. Object-oriented decision support system modelling for multi-criteria decision making
in natural resource management. Comput. Oper. Res. 2004, 31, 985–999. [CrossRef]
38. Qin, X.S.; Huang, G.H.; Chakma, A.; Nie, X.H.; Lin, Q.G. A MCDM-based expert system for climate change
impact assessment and adaption planning—A case study for the Georgia Basin, Canada. Expert Syst. Appl.
2008, 34, 2164–2179. [CrossRef]
39. Guarini, M.R.; Locurcio, M.; Battisti, F. GIS-Based Multi-Criteria Decision Analysis for the “Highway in the Sky”,
ICCSA 2015; Springer: Cham, Switzerland, 2015; Volume 9157, pp. 146–161.
40. Haimes Yacov, Y. On the Universality and contributions of Multiple Criteria Decision Making:
A systems-based Approach. J. Mult. Criteria Decis. Anal. 2011, 18, 91–99. [CrossRef]
41. Bouyssou, D. Building criteria: A prerequisite for MCDA. In Readings Multiple Criteria Decision Aid; Springer:
Berlin/Heidelberg, Germany, 1990; pp. 58–80.
42. Roy, B.; Vanderpooten, D. The European school of MCDA: Emergence, basic features and current works.
J. Mult. Criteria Decis. Anal. 1996, 5, 22–38. [CrossRef]
43. Del Giudice, V.; de Paola, P.; Torrieri, F. An Integrated Choice Model for the Evaluation of Urban Sustainable
Renewal Scenarios. Adv. Mater. Res. 2014, 1030–1032, 2399–2406. [CrossRef]
44. Torrieri, F.; Batà, A. Spatial Multi-Criteria Decision Support System and Strategic Environmental Assessment:
A Case Study. Buildings 2017, 7, 96. [CrossRef]
45. Belton, V.; Pictet, J. A framework for group decision using a MCDA model: Sharing, aggregating or
comparing individual information? J. Decis. Syst. 1997, 6, 283–303. [CrossRef]
46. Ukeni, A.O.; Anthony, A.; Michael, S.; Sonia, G. Balancing stakeholder views for decision-making in steel
structural fire design. In Proceedings of the International Conference on Performance-based and Life-cycle
Structural Engineering, Brisbane, Australia, 9–11 December 2015; School of Civil Engineering, The University
of Queensland: Brisbane, Australia, 2015; pp. 983–992.
47. Lahdelma, R.; Salminen, P.; Hokkanen, J. Using Multi-criteria Methods in Environmental Planning and
Management. Environ. Manag. 2000, 26, 595–605. [CrossRef] [PubMed]
48. Li, Y.; Thomas, M.A. A Multiple Criteria Decision Analysis (MCDA) Software selection Framework.
In Proceedings of the 47th Hawaii International Conference on System Science (HICSS), Waikoloa, HI,
USA, 6–9 January 2014; pp. 1084–1094.
49. Kaspar, R.; Ossadnik, W. Evaluation of AHP software from a management accounting perspective. J. Model.
Manag. 2013, 8, 305–319.
50. MakeItRational AHP Software. Available online: http://makeitrational.com/analytic-hierarchy-process/
ahp-software (accessed on 5 October 2017).
51. Expertchoice. Available online: http://www.expertchoice.com (accessed on 5 October 2017).
52. Super Decisions CDS. Available online: https://superdecisions.com (accessed on 5 October 2017).
53. RightChoice. Ventana Systems UK. Available online: http://www.ventanasystems.co.uk/services/software/
rightchoice/ (accessed on 5 October 2017).
54. M-MACBETH Software. Available online: http://www.m-macbeth.com (accessed on 5 October 2017).
55. Smart Picker Pro: The Desktop Application. Available online: http://www.smart-picker.com/products.
(accessed on 5 October 2017).
56. Electre III-IV Software. Available online: http://www.lamsade.dauphine.fr/spip.php?rubrique64&lang=fr
(accessed on 5 October 2017).
57. Triptych: TOPSIS. Available online: http://www.stat-design.com/Software/TOPSIS.html (accessed on
5 October 2017).
58. Salet, W.G.; Thornley, A.; Kreukels, A. Metropolitan Governance and Spatial Planning: Comparative Case Studies
of European City-Regions; Taylor & Francis: Oxford, UK, 2003.
59. Bouyssou, D.; Perny, P. Ranking methods for valued preference relations: A characterization of a method
based on leaving and entering flows. Eur. J. Oper. Res. 1992, 61, 186–194. [CrossRef]
60. Saaty, T. The Analytic Hierarchy Process; Mcgraw Hill: New York, NY, USA, 1980.
61. Bana e Costa, C.; Vansnick, J. On the Mathematical Foundations of MACBETH. In Multiple Criteria Decision
Analysis: State of the Art Surveys; Springer: New York, NY, USA, 2005; pp. 409–442.
62. Lai, Y.J.; Hwang, C.L. Fuzzy multiple objective decision making. In Fuzzy Multiple Objective Decision Making;
Springer-Verlag: Berlin/Heidelberg, Germany, 1994; Volume 44, pp. 139–262.
63. Hwang, C.L.; Paidy, S.R.; Yoon, K.; Masud, A.S.M. Mathematical programming with multiple objectives:
A tutorial. Comput. Oper. Res. 1980, 7, 5–31. [CrossRef]
64. Behzadian, M.; Otaghsara, S.K.; Yazdani, M.; Ignatius, J. A state-of the-art survey of TOPSIS applications.
Expert Syst. Appl. 2012, 39, 13051–13069. [CrossRef]
65. Baudry, G.; Macharis, C.; Vallée, T. Range-based Multi-Actor Multi-Criteria Analysis: A combined method of
Multi-Actor Multi-Criteria Analysis and Monte Carlo simulation to support participatory decision making
under uncertainty. Eur. J. Oper. Res. 2018, 264, 257–269. [CrossRef]
66. Ascough, J.C., II; Maier, H.R.; Ravalico, J.K.; Strudley, M.W. Future research challenges for incorporation of
uncertainty in environmental and ecological decision-making. Ecol. Model. 2008, 219, 383–399. [CrossRef]
67. Salo, A.; Hämäläinen, R.P. Multi-criteria Decision Analysis in Group Decision Processes. In Handbook of
Group Decision and Negotiation, Advances in Group Decision and Negotiation; Kilgour, D.M., Eden, C., Eds.;
Springer: Dordrecht, The Netherlands, 2010; pp. 269–283.
68. Wang, J.-J.; Jing, Y.-Y.; Zhang, C.-F.; Zhao, J.-H. Review on multi-criteria decision analysis aid in sustainable
energy decision-making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [CrossRef]
69. Saaty, T.L.; De Paola, P. Rethinking Design and Urban Planning for the Cities of the Future. Buildings 2017,
7, 76. [CrossRef]
70. Ribeiro, F.; Ferreira, P.; Araújo, M. Evaluating future scenarios for the power generation sector using a
Multi-Criteria Decision Analysis (MCDA) tool: The Portuguese case. Energy 2013, 52, 126–136. [CrossRef]
71. Dulmin, R.; Mininno, V. Supplier selection using a multi-criteria decision aid method. J. Purch. Supply Manag.
2003, 9, 177–187. [CrossRef]
72. Guarini, M.R.; D’Addabbo, N.; Morano, P.; Tajani, F. Multi-Criteria Analysis in Compound Decision Processes:
The AHP and the Architectural Competition for the Chamber of Deputies in Rome (Italy). Buildings 2017, 7,
38. [CrossRef]
73. Nesticò, A.; Sica, F. The sustainability of urban renewal projects: A model for economic multi-criteria analysis.
J. Property Invest. Financ. 2017, 35, 397–409. [CrossRef]
74. Guarini, M.R.; Battisti, F. Benchmarking Multi-criteria Evaluation: A Proposed Method for the Definition of
Benchmarks in Negotiation Public-Private Partnerships. In Computational Science and Its Applications–ICCSA
2014; Springer: Cham, Switzerland, 2014; Volume 8581, pp. 208–223.
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Sustainability 10 00507

Uploaded by

Copyright:

Available Formats

Sustainability 10 00507

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Sustainability 10 00507

Uploaded by

Copyright:

Available Formats

sustainability

Received: 15 December 2017; Accepted: 10 February 2018; Published: 13 February 2018

1. Introduction and Aims of the Work

Sustainability 2018, 10, 507; doi:10.3390/su10020507 www.mdpi.com/journal/sustainability

1. ELimination Et Choix Traduisant la REalitè (ELECTRE) [23];

- No method can be considered perfect or applied to every type of problem [9].

2. MCDA: Structure, Endogenous Variables, Exogenous Variables

Action to Be Taken to Solve

EXOGENOUS ACTIONS TO BE TAKEN TO SOLVE ENDOGENOUS

Typology of Identify decision-makers, their respective importance

Identify the best offer among different offers (of

Building a set of possible solutions excluding Decision Problem

Identify all possible solutions in relation to specific

Table 2. Numeric configuration of evaluation elements.

Numeric Configuration of Evaluation Elements

Table 3. Typology of the indicators.

Typology of the Indicators Features of Indicators

Table 4. Number of stakeholders.

Participatory Process Number of Stakeholders

Table 5. Type of solution.

Type of Solution Selection Criterion

Table 6. The presence of technical support from a Decision Aid Specialist.

The Presence of Technical Support

Table 7. Features of exogenous variables.

2.4. Endogenous Variables

Table 8. Types of decision-making problems.

Categories Decision-Making Problem

Table 9. Approaches to resolving problems.

Method of Approach Qualification *

Table 10. Input levels.

Modelling Effort Parametres Indicators

Table 11. Implementation procedures.

Implementation Procedures Data Processing and Aggregation

Table 11. Cont.

Implementation Procedures Data Processing and Aggregation

Table 12. Output typologies.

Output Typologies Calculation Method

Table 13. Solutions used for the decision-making problem.

Solutions Incomparability Solution foundation

Table 14. Features of the endogenous variables.

2.5. Properties of MCDA Methods Transposed into a Binary Mathematical System

Properties of MCDA Tool in Binary System

Table 15. Cont.

Properties of MCDA Tool in Binary System

3. Selecting Methods of Multi-Criteria Decision Analysis: The Proposed Procedure

4. Application of the Proposed Procedure to a Case Study

4.2. Weighting of the Variables

4.3. Determination of the Framework of Expected Properties

Technical support Need to speed up decision

Endogenous Need to check the performance of

Table 17. Cont.

4.4. Calculation of the Overall Index of Suitability

Table 18. Cont.

MCDA Tool Overall Suitability Index (IS) Ranking

5. Discussion and Conclusions

Appendix A. Input Level Calculation

Parameters of the Input Level Definition and Calculation

Appendix B. Weighting of the Variables by the Options Considered Preferable to

- Simple, if all stakeholders are considered of equal importance;

Appendix C. Determination Modality of the Framework of Expected Properties