SS symmetry

Article
Determination of Objective Weights Using a New Method
Based on the Removal Effects of Criteria (MEREC)
Mehdi Keshavarz-Ghorabaee 1 , Maghsoud Amiri 2 , Edmundas Kazimieras Zavadskas 3, *, Zenonas Turskis 3
and Jurgita Antucheviciene 4

1 Department of Management, Faculty of Humanities (Azadshahr Branch), Gonbad Kavous University,

Gonbad Kavous 49717-99151, Iran; [email protected]
2 Department of Industrial Management, Faculty of Management and Accounting, Allameh Tabataba’i
University, Tehran 14348-63111, Iran; [email protected]
3 Institute of Sustainable Construction, Civil Engineering Faculty, Vilnius Gediminas Technical University,
Sauletekio al. 11, LT-10223 Vilnius, Lithuania; [email protected]
4 Department of Construction Management and Real Estate, Vilnius Gediminas Technical University,
Sauletekio al. 11, LT-10223 Vilnius, Lithuania; [email protected]
* Correspondence: [email protected]; Tel.: +370-5-274-4910

Abstract: The weights of criteria in multi-criteria decision-making (MCDM) problems are essential
elements that can significantly affect the results. Accordingly, researchers developed and presented
several methods to determine criteria weights. Weighting methods could be objective, subjective,
and integrated. This study introduces a new method, called MEREC (MEthod based on the Removal
Effects of Criteria), to determine criteria’ objective weights. This method uses a novel idea for





 weighting criteria. After systematically introducing the method, we present some computational
analyses to confirm the efficiency of the MEREC. Firstly, an illustrative example demonstrates the
Citation: Keshavarz-Ghorabaee, M.;
procedure of the MEREC for calculation of the weights of criteria. Secondly, a comparative analysis
Amiri, M.; Zavadskas, E.K.; Turskis,
Z.; Antucheviciene, J. Determination
is presented through an example for validation of the introduced method’s results. Additionally, we
of Objective Weights Using a New perform a simulation-based analysis to verify the reliability of MEREC and the stability of its results.
Method Based on the Removal Effects The data of the MCDM problems generated for making this analysis follow a prevalent symmetric
of Criteria (MEREC). Symmetry 2021, distribution (normal distribution). We compare the results of the MEREC with some other objective
13, 525. https://doi.org/10.3390/ weighting methods in this analysis, and the analysis of means (ANOM) for variances shows the
sym13040525 stability of its results. The conducted analyses demonstrate that the MEREC is efficient to determine
objective weights of criteria.
Academic Editor: José Carlos
R. Alcantud Keywords: decision-making; criteria weights; objective weights; weighting; MCDM; MADM

Received: 2 March 2021

Accepted: 22 March 2021
Published: 24 March 2021
1. Introduction
Publisher’s Note: MDPI stays neutral
Multi-criteria decision-making (MCDM) is an essential and interdisciplinary field in
with regard to jurisdictional claims in
operations research that has been taken into consideration in the past years. This field can
published maps and institutional affil- be categorized into two different branches: multi-objective decision-making (MODM) and
iations. multi-attribute decision-making (MADM) [1]. In MODM, we usually address the problems
of determining an optimal or near-optimal solution in a feasible solution space based on
multiple objectives and several variables, parameters, and constraints. Solving linear and
ono-linear programming models is a common way to deal with MODM problems. On
Copyright: © 2021 by the authors.
the other hand, MADM is a branch of MCDM that deals with issues defined by discrete
Licensee MDPI, Basel, Switzerland.
decision variables, a limited number of alternatives and attributes [2–5]. The focus of this
This article is an open access article
study is on this branch of MCDM. Many researchers have usually used the two terms:
distributed under the terms and MADM and MCDM. We also use the term MCDM to refer to the current research problems.
conditions of the Creative Commons In practical situations, we can be confronted with some problems in which a finite set
Attribution (CC BY) license (https:// of alternatives need to be evaluated in terms of multiple criteria [6–9]. MCDM methods
creativecommons.org/licenses/by/ provide us with a process that results in rational, explainable, and justifiable decisions. Sev-
4.0/). eral MCDM methods and techniques have been proposed in the literature. WSM (Weighted

Symmetry 2021, 13, 525. https://doi.org/10.3390/sym13040525 https://www.mdpi.com/journal/symmetry

Symmetry 2021, 13, 525 2 of 20

Sum Model), WASPAS (Weighted Aggregated Sum Product ASsessment), TOPSIS (Tech-
nique for Order of Preference by Similarity to Ideal Solution), VIKOR (Vise Kriterijumska
Optimizacija I Kompromisno Resenje), PROMETHEE (Preference Ranking Organization
METHod for Enrichment of Evaluations), ELECTRE (ELimination Et Choix Traduisant
la REalité), COPRAS (COmplex PRoportional ASsessment), EDAS (Evaluation based on
Distance from Average Solution), AHP (Analytic Hierarchy Process), and BWM (Best-
Worst Method) are some of the popular MCDM methods which have been utilized by
many researchers in different fields of study [10–15]. Generally, there are four steps in
the evaluation process using MCDM approaches: (i) defining the alternatives and criteria
related to the problem, (ii) determining weights of each measure, (iii) assigning individual
performance to each option on each measure, and (iv) evaluate the alternatives based on
the aggregate performance of them on all criteria [16,17]. The second step is the focus of
this study. We can say that determination of criteria weights is one of the most critical and
complicated processes in dealing with MCDM problems.
The weights of criteria show the importance of them. The easiest way, which has been
utilized in many studies, is to assign equal weights to the criteria [18]. However, the final
evaluation results are highly dependent on the weights of criteria, which is not appropri-
ate [19]. Several methods have been introduced to obtain criteria weights. These methods
fall into three categories—subjective weighting methods, objective weighting methods, and
hybrid weighting methods. In subjective methods, the determination of criteria weights is
dependent on the preferences of decision-makers. Direct ranking, point allocation, pairwise
comparisons, and SMART (Simple Multi-attribute Ranking Technique) are examples of
subjective methods [20]. The main disadvantage of these methods is that they are not effi-
cient enough when the number of criteria increases. In other words, expressing preferences
is a mental task for decision-makers, and the accuracy of their preferences decreases by
increasing the number of criteria [17]. In objective weighting methods, the preferences of
decision-makers have no role in determining criteria weights [20]. The objective weighting
methods yield criteria weights using a specific computational process based on the initial
data or decision-matrix (each alternative’s performance on each criterion).
In these methods, we do not use direct data on the significance of criteria expressed
by decision-makers [21]. Entropy method, Standard Deviation method, CRITIC (CRiteria
Importance Through Inter-criteria Correlation) and SECA (Simultaneous Evaluation of
Criteria and Alternatives) are in the category of objective weighting methods [9,20,22].
Hybrid methods use a combination of different subjective and objective weighting methods.
These methods have no distinctive characteristics, and they borrow the features of other
methods. Since the hybrid methods can utilize the decision-makers’ preferences and the
data of decision-matrix, they could give more realistic weights [23–28]. Each of these
methods has its advantages and disadvantages and can be efficient in different situations.
Here, we do not aim to study these advantages and disadvantages.
This study proposes a new objective weighting method, called MEREC (MEthod
based on the Removal Effects of Criteria), for determining criteria weights. As mentioned
previously, the objective weighting methods use the initial data or the decision-matrix
to determine criteria weights. These methods usually exploit the variations in different
alternatives’ performances concerning each criterion to determine the weights. The criteria
with more variations have greater weights [20]. Unlike the other methods, the proposed
method uses removal effects of each criterion on the aggregate performance of alternatives
for calculating criteria weights. In the proposed method, a criterion has a greater weight
when its removal leads to more effects on alternatives’ aggregate performances. Besides
weighting each criterion, this perspective may help decision-makers to exclude some
criteria from the decision-making process. Considering the variations, the performance of
an alternative based on removing criteria is a new perspective on determining the weights
of criteria. In other words, the concept of causality is the basis of the proposed method.
A logarithmic function is used in this study to measure the aggregate performance of
alternatives; however, as the main advantage of the proposed method, it is flexible so that
Symmetry 2021, 13, 525 3 of 20

decision-makers can use different functions to calculate the performances. Introducing

new MCDM methods based on novel perspectives could help experts and decision-makers
increase the robustness of results based on integration and combination of approaches.
In this paper, after presenting the proposed method, it is illustrated through a simple
MCDM example. Another numerical example is used to make a computational study
on the method and compare it with some other objective weighting methods. Besides, a
simulation-based analysis is conducted to make a more comprehensive comparison and
verify the stability of the results of MEREC by testing the homogeneity of variances of the
criteria weights. Symmetrical data are generated for the simulation-based analysis using
the normal distribution. The analysis of means (ANOM) for variances is applied to show
the homogeneity of variances in the simulation results. The comparative study results
show that the weights obtained by MEREC are consistent with those of other objective
weighting methods. Moreover, the simulation-based analysis demonstrates that MEREC
yields homogeneous variances and can give stable criteria weights.
The remainder of this paper is organized as follows. Section 2 reviews some recent
studies on the objective weighting methods and their applications. Section 3 systematically
presents MEREC. Section 4 examines the introduced method based on computational anal-
yses. This section includes three sub-sections. The first sub-section illustrates the procedure
of using MEREC for the determination of criteria weights. The second sub-section uses an
MCDM problem to present the results of a comparison between MEREC and three objective
weighting methods (CRITIC, Entropy, and Standard Deviation). The third sub-section
provides simulation-based analysis results to verify the credibility, reliability and stability
of the results determined using MEREC. Finally, Section 5 discusses conclusions.

2. Recent Studies on Objective Weighting Methods

In this section, some of the most recent studies on the applications and developments
of objective weighting methods are briefly surveyed. We focus on the three common
methods (Entropy, Standard Deviation, and CRITIC) because they are used in this study’s
computational analysis.
In a recent study, Nguyen et al. [29] studied the financial performance of retailer
companies traded in the Vietnam Stock Exchange Market. They examined the financial
performance concerning several ratios, including valuation, profitability, growth rate,
liquidity, efficiency, and leverage. Twelve companies were considered in different quarters
of 2019 and 2020. They used the Standard Deviation method to compute the weights
of eighteen financial ratios. Then, the Grey relational analysis was applied to rank each
company in each quarter (time). Their study showed that the leverage ratio could highly
impact the retailer companies’ financial performance and long-term investment planning.
Sałabun et al. [30] addressed the problem of choosing appropriate methods for MCDM
problems. Their study focused on benchmarking some of the prevalent methods in this
field based on a simulation-based analysis. TOPSIS, VIKOR, COPRAS, and PROMETHEE II
were selected to perform the analysis. Besides, they used different normalization techniques
and different weighting methods, including the Equal Weights, Entropy, and Standard
Deviation methods. It was concluded that the Equal Weights method was not the right
choice for weighting criteria compared with the Entropy and Standard Deviation methods.
Moreover, they found that using the Entropy weighting method led to smaller changes in
the correlation between rankings.
Another study was performed on the dimensions of cultural environment and de-
velopment. To evaluate the Czech theatres’ economy, Vavrek and Bečica [31] selected 11
criteria related to financial and technical efficiency. Three objective weighting methods,
including Coefficient of Variation, Standard Deviation, and Mean Weight (Equal Weights)
methods, were applied to determine the selected criteria’ importance and the variation
coefficient indicated that financial criteria are more significant than technical criteria, but
standard deviation gave importance to technical criteria. The TOPSIS method is then
utilized to make the overall evaluation concerning these two categories of criteria.
Symmetry 2021, 13, 525 4 of 20

Narayanamoorthy et al. [32] used the standard deviation as an objective weighting

method and hesitant fuzzy sets to develop three multi-criteria decision-making approaches:
hesitant fuzzy standard deviation with multi-objective optimization method by ratio analy-
sis (HFSDV-MOORA), hesitant fuzzy standard deviation with VIKOR (HFSDV-VIKOR),
and hesitant fuzzy standard deviation with TOPSIS (HFSDV-TOPSIS). They applied the
proposed approaches to evaluate water distribution systems and improve water resources.
Their problem consisted of four alternatives and five criteria. To validate the results of the
proposed approaches, they made a comparative analysis based on the studies of Liao and
Xu [33], Xu and Zhang [34], and Li [35].
In a recent research, Anitha and Das [36] tackled a problem to optimize electro-
discharge machines’ process parameters. They considered four input parameters of the
problem, including pulse-on time, current, voltage, and duty cycle and their effects on
surface roughness (Ra), and material removal rate (MRR). The Standard Deviation method
was utilized to obtain the criteria weights (MRR and Ra). Based on experimental results,
they applied the full multiplicative form of MOORA (MULTIMOORA) and multi-objective
optimization based on simple-ratio analysis (MOOSRA) to determine the best scenario of
parameters in the considered process. In this field, Anitha and Das [37] also made a similar
study and utilized the MOOSRA and Standard Deviation methods.
Maheshwari et al. [38] performed a study on design parameters of the solid ventilated
brake disc. They defined seven criteria, including inboard plate thickness, outboard plate
thickness, vane height, effective offset, central radius, fatigue life, and axial deflection. The
authors constructed several scenarios through a full factorial design (design of experiments),
and used the Standard Deviation method for weighting the criteria. They applied the
EDAS, COPRAS, TOPSIS, and ARAS (Additive Ratio ASsessment) methods to evaluate the
designed scenarios. The authors also compared results of the considered MCDM methods
to verify the optimum design parameters.
Ramasamy et al. [39] applied the TOPSIS method in research for choosing the opti-
mized set of experimental conditions to synthesize Bismuth Iron Tri Oxide (BiFeO3 –BFO)
nano-powder using the sol-gel route based on changing time duration and slotted calcina-
tion temperature. Their study’s main goal was to identify the optimum time duration and
slotted calcination temperature required to get single-phase powders with minimum strain
and crystallite size. The input weights of TOPSIS were assigned based on two approaches:
Equal Weights and Standard Deviation methods. Due to the low number of criteria, there
was not much difference in the results of these two approaches.
Şahin [40] addressed the material selection problem in the process of designing a
product. The Standard Deviation and CRITIC methods were used to determine the criteria’
importance, and the GRA (Grey Relational Analysis), TOPSIS, and ORESTE (Organiza-
tion, Rangement Et Synthese De Donnes Relationnelles) methods were applied to eval-
uate the alternatives. Six approaches were proposed based on these methods: Standard
Deviation-GRA, Standard Deviation-TOPSIS, Standard Deviation-ORESTE, CRITIC-GRA,
CRITIC-TOPSIS, and CRITIC-ORESTE. The author analyzed and compared the proposed
approaches and used the Copeland method to aggregate the results.
Yazdani et al. [41] made a study on the evaluation of renewable energy resources. They
developed a multi-criteria decision-making approach to handle this evaluation process with
conflicting objectives. Five renewable resources, including solar thermal, solar photovoltaic,
wind power, biomass, and geothermal, were chosen for evaluation. The set of evaluation
criteria consisted of economic, social, environmental and technical dimensions. Their
developed methodology was based on the EDAS and Entropy methods. A case-based
analysis (a case in Saudi Arabia) was performed to show the developed methodology’s
application, and wind power was selected as the most suitable alternative. They also made
a sensitivity analysis to verify the stability of the results.
In another study, Afshar et al. [42] tackled the subcontractor evaluation problem.
They used type-2 fuzzy sets to deal with the uncertainty of information in the qualification
assessment of subcontractors. Their study’s methodology was based on the TOPSIS method,
Symmetry 2021, 13, 525 5 of 20

and they utilized the Entropy method to determine criteria weights. More than ten criteria
and their corresponding sub-criteria were defined for the evaluation process. They used the
centroid of the interval type-2 fuzzy values in the decision matrix for determining criteria
weights by Entropy. To validate their methodology, they compared it with an approach
based on type-1 fuzzy sets.
Arya and Kumar [43] designed a methodology using the picture fuzzy set theory.
They study a new picture fuzzy entropy approach based on the probability theory to
determine criteria weights. The standard Entropy method is the basis of their proposed
picture fuzzy Entropy approach. They utilized the VIKOR and TODIM (a Portuguese
acronym for Interactive Multi-Criteria Decision Making) to evaluate alternatives in MCDM
problems. Mathematical properties of the proposed Entropy were investigated. Besides,
they carried out computational and comparative analyses to confirm the validity and
stability of the results.
Li et al. [44] addressed the machine tool selection problem in the manufacturing in-
dustry. They employed an integrated approach based on the fuzzy DEMATEL (DEcision
MAking Trial and Evaluation Laboratory) and Entropy methods to calculate the criteria
weights. Then a methodology based on the VIKOR method was applied to select an appro-
priate machining tool. A case-based analysis and a sensitivity analysis were performed to
assess the efficiency of the proposed approach. The analyses showed that their method is
efficient, and the results are valid.
A study carried out by Görçün [45] on the problem of selecting an appropriate con-
tainer seaport in the Black Sea region. The study presented two hybrid multi-criteria
decision-making approaches. The first approach used the Entropy and OCRA (Opera-
tional Competitiveness RAting) methods, and the second was based on the Entropy and
EATWIOS (Efficiency Analysis Technique WIth Output Satisfying). The Borda method was
also used to merge the results of the approaches. Although the proposed approaches were
applied to a significant problem, there were not enough analyses to verify and compare
the results.
Zha et al. [46] developed a hybrid multi-criteria decision-making approach. The
problem of facility layout evaluation was addressed in their study. An integration of
Delphi, fuzzy ANP (Analytic Network Process), and Entropy were employed to obtain the
criteria weights. Then a fuzzy extension of PROMETHEE was applied to evaluate facility
layout alternatives. They compared the results of their proposed approach with several
MCDM approaches in different levels of uncertainty. The comparative analysis showed
that the approach could efficiently deal with this type of problem.
Alao et al. [47] presented a study on evaluating technologies used in the waste-to-
energy process. They defined this problem as a multi-criteria decision-making problem.
The Entropy method was applied to compute the weights of criteria. Electricity generation
potential, technology maturity, investment cost, cost of energy, operation and maintenance
cost, and carbon dioxide emission were the criteria for the evaluation. Moreover, the
TOPSIS method was applied to make the evaluation. They used a case in Lagos, Nigeria,
to show the applicability of their methodology.
A study in the field of crisis management was also used as an objective weighting
method. Salehi et al. [48] chose five petrochemical plants and evaluated their crisis man-
agement systems in three criteria: technical, human, and organizational. They calculated
the weights of these criteria using the Entropy method. According to their results, the
human factor was the most critical criterion in their study. Then, the TOPSIS method and
the obtained weights were utilized to evaluate the petrochemical plants. A shortcoming
of their study was the lack of comparative analysis and sensitivity analyses to verify and
validate the results.
In a study of groundwater vulnerability assessment, Torkashvand et al. [49] used the
Entropy, SWARA (Stepwise Weight Assessment Ratio Analysis), and GA (Genetic Algo-
rithm) methods to improve the DRASTIC (Depth to the water table, net Recharge, Aquifer
media, Soil media, Topography, Impact of the vadose zone, and hydraulic Conductivity)
Symmetry 2021, 13, 525 6 of 20

framework. They used the data of nitrate concentration gathered from 50 wells in the study
region. Their study shows the importance of using objective and subjective weighting
methods in dealing with vital environmental problems and improving the decisions related
to human lives.
Wang et al. [50] focused on a particular supplier selection problem. They studied
battery-swapping stations as essential infrastructures in promoting electric vehicles (EVs)
and their inevitable concerns about selecting appropriate battery suppliers. An MCDM
framework was proposed in their article to evaluate the suppliers under uncertainty. The
uncertainty of data was defined using triangular fuzzy numbers. The problem of their
study consisted of four suppliers and fifteen criteria. They utilized the Entropy method
to determine the weights of criteria. Then, the MULTIMOORA method was employed in
evaluating the suppliers.
In addition to the Entropy and Standard Deviation methods, the CRITIC method is an
efficient objective weighting method that incorporates the standard deviation of data re-
lated to each criterion and the correlation between the criteria. Like the Standard Deviation
and Entropy methods, this method has also been developed in many research types and ap-
plied to different MCDM problems. Shipboard crane selection [51], healthcare management
evaluation [52], configuration optimization in energy systems [53], supply chain risk man-
agement [54], financial risk evaluation [52], quality assessment of wireless networks [55],
urban transportation [56], location planning [57], third-party logistics providers [58], air
quality assessment [59], waste disposal [60], and outsourcing evaluation [61] are some of
the recent and essential real-life problems in which the CRITIC method has been applied.
Table 1 presents a summary of the above-mentioned studies.

Table 1. A summary of the recent studies.

No. Author(s) Year Weighting Method Application/Development

1 Nguyen et al. [29] 2020 Standard Deviation Financial performance of retailer companies
Equal Weights, Entropy
2 Sałabun et al. [30] 2020 Benchmarking of the MCDM methods
and Standard Deviation
Coefficient of Variation,
3 Vavrek and Bečica [31] 2020 Standard Deviation Cultural environment and development
and Equal Weights
4 Narayanamoorthy et al. [32] 2020 Standard Deviation Development of hesitant fuzzy MCDM approaches
Optimizing the process parameters in
5 Anitha and Das [36,37] 2020 Standard Deviation
electro-discharge machines
Optimizing experimental conditions to synthesize
6 Ramasamy et al. [39] 2020 Standard Deviation
Bismuth Iron Tri Oxide Nano powder
Standard Deviation
7 Şahin [40] 2020 Material selection problem
and CRITIC
8 Yazdani et al. [41] 2020 Entropy Evaluation of renewable energy resources
Development of a methodology using the picture
9 Arya and Kumar [43] 2020 Entropy
fuzzy sets
Entropy and Machine tool selection problem in the
10 Li et al. [44] 2020
DEMATEL manufacturing industry
Evaluation of container seaports in the Black
11 Görçün [45] 2020 Entropy
sea region
Entropy, Delphi and
12 Zha et al. [46] 2020 Facility layout evaluation
fuzzy ANP
Evaluation of technologies used in the
13 Alao et al. [47] 2020 Entropy
waste-to-energy process
14 Salehi et al. [48] 2020 Entropy Evaluation of crisis management systems
Symmetry 2021, 13, 525 7 of 20

Table 1. Cont.

No. Author(s) Year Weighting Method Application/Development

15 Torkashvand et al. [49] 2020 Entropy and SWARA Groundwater vulnerability assessment
Shipboard crane selection with interval type-2
16 Mohamadghasemi et al. [51] 2020 CRITIC
fuzzy sets
17 Xu et al. [53] 2020 CRITIC Configuration optimization in energy systems
Abdel-Basset and
18 2020 CRITIC Supply chain risk management
Mohamed [54]
Evaluating the service quality of wireless
19 Li and Wang [55] 2020 CRITIC
sensor networks
Location planning of electric vehicle
20 Wei et al. [57] 2020 CRITIC
charging Stations
21 Piasecki and Kostyrko [59] 2020 Entropy and CRITIC Prioritization of indoor air pollutants
22 Narayanamoorthy et al. [60] 2020 CRITIC Assessment of bio-medical waste disposal methods
Evaluation and classification of outsourcing
23 Liaw et al. [61] 2020 CRITIC and DEMATEL
providers in manufacturing
Evaluation design parameters of solid ventilated
24 Maheshwari et al. [38] 2021 Standard Deviation
brake disc
Subcontractor evaluation problem using type-2
25 Afshar et al. [42] 2021 Entropy
fuzzy sets
26 Wang et al. [50] 2021 Entropy Supplier selection for battery swapping stations
27 Peng et al. [52] 2021 CRITIC Healthcare management evaluation
Evaluation of metro and tram vehicle for
28 Görçün [56] 2021 CRITIC
urban transportation
Selection of sustainable third-party reverse
29 Mishra et al. [58] 2021 CRITIC
logistics providers
30 Peng et al. [52] 2021 CRITIC Financial risk evaluation

The basis of the objective weighting methods used in the MCDM field is commonly
the input or the decision matrix. Unlike the former methods, this study aims to introduce a
novel objective weighting method, called MEREC, based on the relation between input or
decision-matrix and its effect on the output or the performance of alternatives. In other
words, we can say that causality is the foundation of MEREC.

3. Proposed Method
In this section, a new method based on the removal effects of criteria (MEREC) is pro-
posed to determine the criteria’ weights in a multi-criteria decision-making problem. This
method is in the category of objective weighting methods for obtaining criteria weights. As
previously mentioned, the MEREC uses each criterion’s removal effect on the performance
of alternatives to determine criteria weights. Greater weights are assigned to the criteria
that have higher effects on the performances. In this method, we should define a measure
for the performances of alternatives first. In this study, a simple logarithmic measure is
used with equal weights to calculate alternatives’ performances. To identify the effects of
removing each criterion, we use the absolute deviation measure. This measure reflects the
difference between the overall alternative’s performance and its performance in removing
a criterion. The following steps are used to calculate objective weights by MEREC.
Step 1: Construct the decision matrix. A decision matrix is constructed in this step,
which shows each alternative’s ratings or values concerning each criterion. The elements
of this matrix are denoted by xij , and these elements should be greater than zero (xij > 0).
If we have negative values in the decision matrix, they should be transformed into positive
values using an appropriate technique. Suppose that there are n alternatives and m criteria,
and the form of the decision-matrix is as follows:
 Symmetry 2021, 13, 525 8 of 20

 
x11 x12 ··· x1j ··· x1m
x
 21 x22 ··· x2j ··· x2m 
 . .. .. .. .. .. 
 . . .
 . . . . 

X= (1)
 xi1 xi2 ··· xij ··· xim 

 .. .. .. .. .. .. 
 
 . . . . . . 
xn1 xn2 ··· xnj ··· xnm
Step 2: Normalize the decision matrix (N). In this step, a simple linear normalization
is used to scale the elements of the decision-matrix. The elements of the normalized matrix
are denoted by nijx . If B shows the set of beneficial criteria, and H represents the set of
non-beneficial criteria, we can utilize the following equation for normalization:

minx
 k kj i f j ∈ B

x xij
nij = xij (2)
 maxxkj i f j ∈ H

k

• It should be noted that the normalization process is similar but different from the
process used in methods like WASPAS. The difference is in switching between the
formulas of beneficial and non-beneficial criteria. Unlike many other studies, we
transform all the criteria into the minimization type criteria.
Step 3: Calculate the overall performance of the alternatives (Si ). A logarithmic
measure with equal criteria weights is applied to obtain alternatives’ overall performances
in this step. This measure is based on a non-linear function depicted in Figure 1. According
to the normalized values obtained from the previous step, we can ensure that smaller
values of nijx yield greater values of performances (Si ). The following equation is used for
this calculation: !!
Symmetry 2021, 13, x FOR PEER REVIEW  1   
m∑
x 
Si = ln 1 + ln nij  (3)

 
j

 
Figure 1. The weights of the comparative analysis. 
Figure 1. The weights of the comparative analysis.

Step 4: Calculate the performance of the alternatives by removing each criterion.

In this step, we use the logarithmic measure in a similar way to the previous step. The
1
difference between this step and Step 3 is that the alternatives’ performances are calculated
𝑆 𝑙𝑛 1 |𝑙𝑛 𝑛 |  
𝑚
,

Step 5: Compute the summation of absolute deviations. In this step, we calcu
Symmetry 2021, 13, 525 9 of 20

based on removing each criterion separately. Therefore, we have m sets of performances

associated with m criteria. Let us denote by Sij0 the overall performance of ith alternative
concerning the removal of jth criterion. The following equation is used for the calculations
of this step: !!
1
0
Sij = ln 1 +
m k,k∑ |ln(nik )|
x
(4)
6= j

Step 5: Compute the summation of absolute deviations. In this step, we calculate the
removal effect of the jth criterion based on the values obtained from Step 3 and Step 4. Let
Ej denote the effect of removing jth criterion. We can calculate the values of Ej using the
following formula:  
Ej = ∑Sij0 − Si  (5)
 
i

Step 6: Determine the final weights of the criteria. In this step, each criterion’s objective
weight is calculated using the removal effects (Ej ) of Step 5. In what follows, w j stands for
the weight of the jth criterion. The following equation is used for the calculation of w j :

Ej
wj = (6)
∑k Ek

4. Computational Analyses
In this section, we present three sub-sections. The first sub-section using a simple
example systematically illustrates the way of using the MEREC. The second sub-section
performs a comparative analysis to show that the results of MEREC are valid and con-
gruent with those of existing objective criteria weight determination methods. The third
subsection presents a simulation-based analysis to test the variability of the results obtained
by MEREC.

4.1. Illustrative Example

To illustrate the procedure of utilizing MEREC for criteria weights determination, we
use a simple decision matrix in this sub-section.
Step 1: Table 2 shows the elements of this decision matrix. As shown in Table 2, we
have five alternatives, two beneficial criteria, and two non-beneficial criteria.

Table 2. The decision matrix of the illustrative example.

C1 ∈B C2 ∈B C3 ∈H C4 ∈H
A1 450 8000 54 145
A2 10 9100 2 160
A3 100 8200 31 153
A4 220 9300 1 162
A5 5 8400 23 158

Step 2: Decision-makers use Equation (2) and obtain the normalized decision matrix.
Table 3 represents this matrix.

Table 3. The normalized decision matrix of the illustrative example.

C1 C2 C3 C4
A1 0.011 1 1 0.895
A2 0.500 0.879 0.037 0.988
A3 0.050 0.976 0.574 0.944
A4 0.023 0.860 0.019 1
A5 1 0.952 0.426 0.975
Symmetry 2021, 13, 525 10 of 20

Step 3: In this step, decision-makers should obtain the overall performances of the
alternatives. They calculate these values based on Equation (3):
  
1
S1 = ln 1 + (|ln(0.011)| + |ln(1)| + |ln(1)| + |ln(0.895)|) = 0.77
4
  
1
S2 = ln 1 + (|ln(0.5)| + |ln(0.879)| + |ln(0.037)| + |ln(0.988)|) = 0.71
4
  
1
S3 = ln 1 + (|ln(0.05)| + |ln(0.976)| + |ln(0.574)| + |ln(0.944)|) = 0.65
4
  
1
S4 = ln 1 + (|ln(0.023)| + |ln(0.86)| + |ln(0.019)| + |ln(1)|) = 1.09
4
  
1
S5 = ln 1 + (|ln(1)| + |ln(0.952)| + |ln(0.426)| + |ln(0.975)|) = 0.21
4
Step 4: Based on Equation (4), decision-makers calculate the alternatives’ overall
performances by removing each criterion (Sij0 ) in this step. Table 4 shows these values. To
clarify the process of calculation, in the following, we present two examples. S11 0 is the
0
overall performance of A1 related to the removal of C1 , and S34 is the overall performance
of A3 connected with the removal of C4 .
  
0 1
S11 = ln 1 + (|ln(1)| + |ln(1)| + |ln(0.895)|) = 0.03
4
  
0 1
S34 = ln 1 + (|ln(0.05)| + |ln(0.976)| + |ln(0.574)|) = 0.64
4

Table 4. The values of Sij0 .

C1 C2 C3 C4
A1 0.03 0.77 0.77 0.75
A2 0.62 0.69 0.19 0.71
A3 0.15 0.64 0.57 0.64
A4 0.71 1.08 0.68 1.09
A5 0.21 0.20 0.02 0.20

Step 5: Decision-makers calculate the removal effect of each criterion on the overall
performance of the alternatives based on the deviation-based formula of Equation (5), the
results of Step 3, and the values of Table 4. These values are determined as follows:

E1 = |0.03 − 0.77| + |0.62 − 0.71| + |0.15 − 0.65| + |0.71 − 1.09| + |0.21 − 0.21| = 1.71

E2 = |0.77 − 0.77| + |0.69 − 0.71| + |0.64 − 0.65| + |1.08 − 1.09| + |0.20 − 0.21| = 0.04
E3 = |0.77 − 0.77| + |0.19 − 0.71| + |0.57 − 0.65| + |0.68 − 1.09| + |0.02 − 0.21| = 1.19
E4 = |0.75 − 0.77| + |0.71 − 0.71| + |0.64 − 0.65| + |1.09 − 1.09| + |0.20 − 0.21| = 0.03
Step 6: Calculation of each criterion’s weight is performed based on the effect of
their removal on the performance of the alternatives. Using Equation (6) and the values
computed in the previous section, we calculate the weights as follows:

∑ Ek = 1.71 + 0.04 + 1.19 + 0.03 = 2.97

k

1.71
w1 = = 0.5752
2.97
Symmetry 2021, 13, 525 11 of 20

0.04
w2 = = 0.0141
2.97
1.19
w3 = = 0.4016
2.97
0.03
w4 = = 0.0091
2.97

4.2. Comparative Example

The authors borrowed an MCDM example from [62] and used it for comparative
analysis. Table 5 represents the data of this example, including ten alternatives and
seven criteria. The authors chose three objective weighting methods to determine the
criteria weights and to perform the analysis. The research compares criteria weights
determined using CRITIC, Entropy, and Standard Deviation with the weights resulted
from the proposed method in this sub-section.

Table 5. The example utilized for making comparative analysis [62].

C1 ∈B C2 ∈B C3 ∈B C4 ∈H C5 ∈H C6 ∈H C7 ∈H
A1 23 264 2.37 0.05 167 8900 8.71
A2 20 220 2.2 0.04 171 9100 8.23
A3 17 231 1.98 0.15 192 10,800 9.91
A4 12 210 1.73 0.2 195 12,300 10.21
A5 15 243 2 0.14 187 12,600 9.34
A6 14 222 1.89 0.13 180 13,200 9.22
A7 21 262 2.43 0.06 160 10,300 8.93
A8 20 256 2.6 0.07 163 11,400 8.44
A9 19 266 2.1 0.06 157 11,200 9.04
A10 8 218 1.94 0.11 190 13,400 10.11

Table 6 shows the criteria weights determined by each method and the related Pearson
correlation coefficients (r). Figure 2 also represents these weights. The values of r in the
last row of Table 6 reflect the relationship between the results of MEREC and those of
the other considered methods in a significance level of α = 0.05 (confidence level of 95%).
The authors used a normalized decision matrix based on the linear normalization used in
WASPAS [63] to calculate the criteria weights by CRITIC, Entropy, and Standard Deviation,
and the CRITIC method skipped the normalization step.

Table 6. The weights and correlation coefficients of the comparative analysis.

Standard
CRITIC Entropy MEREC
Deviation
w1 0.171 0.199 0.221 0.324
w2 0.121 0.020 0.089 0.055
w3 0.110 0.040 0.115 0.086
w4 0.289 0.660 0.293 0.368
w5 0.075 0.016 0.080 0.044
w6 0.165 0.052 0.130 0.077
w7 0.069 0.013 0.073 0.045
0.852 0.873 0.975
r
(p-value < 0.05) (p-value < 0.05) (p-value < 0.05)
 𝑤  0.289  0.660  0.293  0.368 
𝑤   0.075  0.016  0.080  0.044 
𝑤   0.165  0.052  0.130  0.077 
𝑤  
Symmetry 2021, 13, 525 0.069  0.013  0.073  0.045  12 of 20

𝒓  0.852 (p‐value < 0.05)  0.873 (p‐value < 0.05)  0.975 (p‐value < 0.05)   

 
Figure 2. The weights of the comparative analysis. 
Figure 2. The weights of the comparative analysis.

Suppose the value of the correlation coefficient between two variables is more signifi-
Suppose the value of the correlation coefficient between two variables is more sig‐
cant than 0.4. In that case, we can say that there is a moderate relationship between them,
nificant than 0.4. In that case, we can say that there is a moderate relationship between 
and if the value is more significant than 0.6, the relationship is substantial [64]. According
them,  and  if  the tovalue  is  more 
the values significant 
of r shown in Table than  0.6, see
6, we can the  relationship 
that is  substantial 
the criteria weights determined[64]. 
using
According to the values of  𝑟  shown in Table 6, we can see that the criteria weights de‐
MEREC have a strong relationship with weights from CRITIC, Entropy, and Standard Devi-
ation. Figure 2 shows that the trend in varying criteria weights in MEREC is similar to the
termined using MEREC have a strong relationship with weights from CRITIC, Entropy, 
other methods considered in the comparative analysis. Therefore, the MEREC yields results
and  Standard  Deviation.  Figure  2  credible
that can be considered shows andthat  the  trend 
reliable weightsin forvarying  criteria 
the criteria in MCDMweights  in 
problems.
MEREC is similar to the other methods considered in the comparative analysis. There‐
4.3. Simulation-Based Analysis
fore, the MEREC yields results that can be considered credible and reliable weights for 
This sub-section examines the stability of the proposed method using a simulation-
the criteria in MCDM problems. 
based study. The authors generated several decision matrices for this purpose and com-
pared the results of the MEREC with those of the CRITIC, Entropy, and Standard Deviation
methods. Besides, the authors examined the homogeneity of criteria weights variances
determined by MEREC. Generated decision matrices fall into three categories: (i) 5 alter-
natives and 5 criteria, (ii) 10 alternatives and 10 criteria, and (iii) 20 alternatives and 20
criteria. The authors generated a decision matrix for each category ten times (ten sets in
each category) for simulation, and the values of the matrices follow the symmetric distribu-
tion. The authors used the normal distribution with a mean of 10 and a standard deviation
of 1. The authors used these generated decision matrices to obtain criteria weights using
the MEREC, and they can be found (in a format compatible with MATLAB) in Ref. [65].
ANOM for variances is an efficient test to examine the homogeneity of variances in the
obtained results [66]. ANOM for variances with Levene provides us with a test for com-
paring the group means of the absolute deviations from the median (ADM) to the overall
mean ADM [67]. The authors used the JMP-SAS software to perform all the statistical tests.
Tables 7–9, show the criteria weights determined by MEREC in each set of the generated
decision matrices of category (i), category (ii), and category (iii), respectively.
Symmetry 2021, 13, 525 13 of 20

Table 7. The criteria weights related to category (i).

Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 Set 7 Set 8 Set 9 Set 10
w1 0.387 0.217 0.258 0.291 0.384 0.226 0.113 0.210 0.213 0.161
w2 0.114 0.087 0.130 0.326 0.203 0.070 0.196 0.074 0.386 0.262
w3 0.223 0.256 0.245 0.129 0.141 0.264 0.327 0.204 0.170 0.286
w4 0.153 0.199 0.237 0.104 0.158 0.268 0.211 0.266 0.065 0.117
w5 0.123 0.241 0.129 0.150 0.114 0.172 0.152 0.246 0.166 0.173

Table 8. The criteria weights related to category (ii).

Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 Set 7 Set 8 Set 9 Set 10
w1 0.081 0.105 0.082 0.105 0.151 0.128 0.052 0.131 0.100 0.100
w2 0.085 0.104 0.070 0.129 0.044 0.071 0.052 0.177 0.107 0.126
w3 0.072 0.093 0.100 0.045 0.106 0.109 0.155 0.118 0.154 0.070
w4 0.106 0.062 0.065 0.080 0.118 0.076 0.094 0.095 0.049 0.151
w5 0.091 0.136 0.168 0.089 0.085 0.090 0.126 0.044 0.113 0.073
w6 0.098 0.087 0.147 0.144 0.099 0.159 0.117 0.102 0.087 0.131
w7 0.125 0.063 0.064 0.058 0.165 0.058 0.085 0.076 0.107 0.087
w8 0.049 0.134 0.109 0.144 0.075 0.076 0.097 0.072 0.105 0.090
w9 0.109 0.108 0.049 0.130 0.102 0.089 0.153 0.094 0.113 0.107
w10 0.185 0.107 0.146 0.074 0.055 0.145 0.069 0.091 0.064 0.065

Table 9. The criteria weights related to category (iii).

Set 1 Set 2 Set 3 Set 4 Set 5 Set 6 Set 7 Set 8 Set 9 Set 10
w1 0.048 0.098 0.034 0.033 0.049 0.049 0.032 0.053 0.037 0.069
w2 0.079 0.028 0.044 0.047 0.057 0.032 0.072 0.057 0.054 0.049
w3 0.056 0.036 0.038 0.038 0.038 0.051 0.046 0.027 0.051 0.047
w4 0.071 0.050 0.058 0.048 0.068 0.047 0.045 0.030 0.044 0.062
w5 0.028 0.045 0.038 0.092 0.036 0.054 0.046 0.071 0.053 0.057
w6 0.033 0.041 0.069 0.024 0.049 0.050 0.070 0.034 0.093 0.038
w7 0.057 0.050 0.061 0.051 0.048 0.046 0.043 0.037 0.044 0.055
w8 0.049 0.056 0.040 0.046 0.035 0.059 0.053 0.057 0.054 0.043
w9 0.048 0.045 0.060 0.069 0.053 0.075 0.051 0.043 0.046 0.052
w10 0.045 0.047 0.062 0.052 0.040 0.024 0.038 0.061 0.067 0.042
w11 0.062 0.036 0.042 0.071 0.053 0.051 0.035 0.071 0.029 0.054
w12 0.033 0.033 0.047 0.043 0.057 0.078 0.046 0.035 0.056 0.054
w13 0.063 0.077 0.063 0.066 0.047 0.049 0.080 0.054 0.048 0.074
w14 0.048 0.053 0.047 0.048 0.032 0.044 0.061 0.046 0.027 0.055
w15 0.077 0.066 0.056 0.048 0.072 0.058 0.074 0.065 0.070 0.041
w16 0.037 0.050 0.040 0.040 0.050 0.053 0.053 0.071 0.048 0.055
w17 0.055 0.043 0.043 0.038 0.066 0.044 0.052 0.045 0.045 0.032
w18 0.042 0.053 0.058 0.048 0.043 0.051 0.049 0.052 0.029 0.040
w19 0.034 0.040 0.057 0.039 0.059 0.035 0.028 0.047 0.063 0.041
w20 0.033 0.052 0.043 0.058 0.047 0.051 0.028 0.044 0.041 0.040

Like in the previous sub-section, to compare the results of MEREC in the different
categories of simulation with other methods, we use correlation coefficients in a significance
level of α = 0.05 (confidence level of 95%). Table 10 presents these values in the sets of each
category of simulation, and all the p-values are smaller than the significance level. Besides,
Figure 3 graphically depicts these values. In this figure, the degrees of relationship are
illustrated using different background colors in different classes defined by Walters [64],
i.e., red for r < 0.2 or very weak relationship, orange for 0.2 ≤ r < 0.4 or weak relationship,
yellow for 0.4 ≤ r < 0.6 or moderate relationship, blue for 0.6 ≤ r < 0.8 or strong
relationship, and green for 0.8 ≤ r or very strong relationship. Based on Table 10 and
Figure 3, we can see that all the average correlation coefficient values in all categories are
Symmetry 2021, 13, 525 14 of 20

more significant than 0.4, so the moderate relationship between the results can be concluded.
However, the correlation values decrease when the size of the problem increases. Therefore,
we cannot say that MEREC behaves like the other methods in all circumstances.

Table 10. The correlation values of comparison in different categories.

Category (i) Category (ii) Category (iii)

Set No. Standard Standard Standard
CRITIC Entropy CRITIC Entropy CRITIC Entropy
Deviation Deviation Deviation
Set 1 0.950 0.934 0.876 0.797 0.664 0.731 0.637 0.605 0.690
Set 2 0.851 0.778 0.923 0.959 0.944 0.952 0.675 0.608 0.651
Set 3 0.810 0.898 0.956 0.695 0.731 0.785 0.510 0.473 0.537
Set 4 0.914 0.973 0.959 0.747 0.699 0.774 0.759 0.698 0.734
Set 5 0.947 0.925 0.917 0.778 0.798 0.824 0.549 0.520 0.558
Set 6 0.675 0.751 0.843 0.817 0.711 0.770 0.675 0.732 0.802
Set 7 0.904 0.958 0.968 0.739 0.802 0.842 0.593 0.507 0.639
Set 8 0.867 0.941 0.969 0.680 0.818 0.784 0.653 0.562 0.651
Set 9 0.820 0.889 0.867 0.734 0.800 0.806 0.629 0.469 0.604
Set 10 0.698 0.915 0.903 0.829 0.761 0.824 0.607 0.550 0.601
Symmetry 2021, 13, x FOR PEER REVIEW 
Average 0.844 0.896 0.918 0.777 0.773 0.809 0.629 0.572 15  of  21 
0.647
 

 
Figure 3. The graphical representation of correlation values in different categories. 
Figure 3. The graphical representation of correlation values in different categories.

In  the  case

In the case  of
of  the
the  large-size
large‐size  problems,
problems,  toto  understand
understand  the
the  behavior
behavior  ofof  MEREC
MEREC  in in 
comparison to the other methods considered in this study, we can calculate the variance 
comparison to the other methods considered in this study, we can calculate the variance
(or  standard
(or standard  deviation)
deviation)  ofof  the
the  weights
weights  in
in  each
each set
set of
of category
category (iii).
(iii).  Table  11  shows
Table 11 shows  the
the 
results. 
results. Greater 
Greater values 
values of 
of the 
the variance 
variance show  that  the
show that the  method
method can
can distinguish
distinguish different 
different
criteria weights in a more efficient way. According to Table 11, the values of variance for 
criteria weights in a more efficient way. According to Table 11, the values of variance for
Entropy and MEREC are more than those of CRITIC and Standard Deviation. It shows that
Entropy and MEREC are more than those of CRITIC and Standard Deviation. It shows 
MEREC is relatively efficient in differentiating between the weights of criteria.
that MEREC is relatively efficient in differentiating between the weights of criteria.   

Table 11. The variance of the weights determined using each method in category (iii). 

Set No.  MEREC  CRITIC  Entropy  Standard Deviation 

Set 1  0.0002257  0.0000271  0.0001958  0.0000311 
Set 2  0.0002539  0.0000571  0.0003225  0.0000529 
Set 3  0.0001073  0.0000681  0.0002640  0.0000501 
Set 4  0.0002369  0.0000785  0.0003797  0.0000768 
Set 5  0.0001215  0.0000478  0.0002659  0.0000436 
Set 6  0.0001536  0.0000601  0.0003629  0.0000630 
Set 7  0.0002241  0.0000845  0.0003938  0.0000557 
Set 8  0.0001888  0.0000519  0.0002926  0.0000450 
 Symmetry 2021, 13, 525 15 of 20

Table 11. The variance of the weights determined using each method in category (iii).

Standard
Set No. MEREC CRITIC Entropy
Deviation
Set 1 0.0002257 0.0000271 0.0001958 0.0000311
Set 2 0.0002539 0.0000571 0.0003225 0.0000529
Set 3 0.0001073 0.0000681 0.0002640 0.0000501
Set 4 0.0002369 0.0000785 0.0003797 0.0000768
Set 5 0.0001215 0.0000478 0.0002659 0.0000436
Set 6 0.0001536 0.0000601 0.0003629 0.0000630
Set 7 0.0002241 0.0000845 0.0003938 0.0000557
Set 8 0.0001888 0.0000519 0.0002926 0.0000450
Set 9 0.0002362 0.0000538 0.0003744 0.0000476
Set 10 0.0001157 0.0000436 0.0002579 0.0000407
Average 0.0001864 0.0000572 0.0003110 0.0000507

As previously mentioned, to examine the stability of the results of MEREC, we use

ANOM for variances with Levene (ADM). This analysis helps us to have a graphical
representation for verifying the homogeneity of variances. The graphical representation
has three variables: the overall mean ADM is the centerline, upper decision limits (UDL)
and lower decision limits (LDL). If a group (set) standard deviation falls outside of the
decision limits, then that standard deviation is significantly different from the overall mean
ADM, and we have heterogeneity in variances. In other words, if the standard deviations
of all sets are between LDL and UDL, homogeneity of variances is confirmed. Tables 7–9
show the criteria weights for this analysis.
Symmetry 2021, 13, x FOR PEER REVIEW 
  Figures 4–6 demonstrate the results of the analysis for category (i), category (ii), and
category (iii).

 
Figure 4. The results of ANOM for variances for category (i). 
Figure 4. The results of ANOM for variances for category (i).
 Symmetry 2021, 13, 525 16 of 20
 
Figure 4. The results of ANOM for variances for category (i). 

 
Symmetry 2021, 13, x FOR PEER REVIEW  17  of 
  Figure 5. The results of ANOM for variances for category (ii). 
Figure 5. The results of ANOM for variances for category (ii).

 
Figure 6. The results of ANOM for variances for category (iii). 
Figure 6. The results of ANOM for variances for category (iii).

The  interpretation  of  Figures  4–6  (ANOM  techniques)  is  like  interpreting  contr
charts used in SPC (Statistical Process Control).   
As shown in these figures, all the mean ADM values in different simulation categ
ries  are  between  decision  limits  (LDL  and  UDL).  Accordingly,  the  simulation‐base
Symmetry 2021, 13, 525 17 of 20

The interpretation of Figures 4–6 (ANOM techniques) is like interpreting control charts
used in SPC (Statistical Process Control).
As shown in these figures, all the mean ADM values in different simulation categories
are between decision limits (LDL and UDL). Accordingly, the simulation-based analysis
shows the homogeneity of variances. This analysis also demonstrates the stability of the
results determined by MEREC.

5. Conclusions
Determination of criteria weights is a vital function in a multi-criteria decision-making
process. Researchers usually divide weighting methods into subjective and objective
methods. Direct judgements and opinions of decision-makers are the basis for determining
the subjective weights of criteria. Meanwhile, the initial data defined in the MCDM
problem-solving matrices support objective criteria weights. In this study, we focused
on objective weighting methods. A new objective weighting method, called MEREC, has
been introduced in this research. The idea of the proposed method is different from the
other objective weighting methods. Most of the methods for determining objective weights
use variations in criteria to calculate the weights. However, in the method introduced,
the removal effects of criteria on alternatives’ performances are considered a measure for
weighting. It is a novel perspective on the determination of objective criteria weights.
MCDM methods may yield different results for individual problems. However, their
various perspectives in defining problems and providing solutions can help researchers
and practitioners justify their application. While a method could be efficient for solving a
problem, it may be less efficient or inefficient for dealing with another. For example, the
SAW and TOPSIS methods use completely different decision-making ideas, and decision-
makers can use them depending on different circumstances and problems’ characteristics.
Although we defined the process of weighting criteria based on a new perspective
in the proposed method, the results were consistent with the other objective weighting
methods. The simulation-based analysis shows that the correlation between the MEREC
and the results of other methods decreases with increasing problem size. Therefore, we
cannot say that the MEREC behaves like the other methods in all circumstances. However,
the way we used for validation (i.e., comparative and simulation-based analyses) is the
most prevalent literature approach. It may be considered an essential limitation of this
study and many other studies addressing MCDM problems. We can claim that introducing
new MCDM methods based on novel perspectives could ensure the robustness of results.
Specifically, by integrating weighting methods, decision-makers can obtain weights that
are more reliable.
We have used a simple example to illustrate the steps of using the MEREC. The authors
used another numerical example to compare the results of the MEREC with the results
of the CRITIC, Entropy, and Standard Deviation methods. The values of the correlation
coefficients between the results have shown that MEREC yields credible weights for the
criteria. Finally, the authors performed a simulation-based analysis, generating MCDM
problems whose data follow a prevalent symmetric distribution (normal distribution). The
authors carried out two types of analyses based on the generated data. First, a comparison
verifies the reliability of the results of MEREC. Then, the use of the ANOM for variances
checked the stability of the results. The simulation-based analysis shows that the weights
resulted from the MEREC are relatively reliable and stable weights. Thus, the method
introduced is efficient and suitable to determine objective criteria weights. The performance
measure function plays an essential role in the determination of the weight by MEREC. We
used a logarithmic function to measure alternatives performances. Future research can
investigate using other functions like multiplicative functions, distance-based functions,
i.e., functions used in WASPAS, TOPSIS, and VIKOR.
Moreover, integrating MEREC with the other objective and subjective weighting meth-
ods like Entropy, IDOCRIW (Integrated Determination of Objective CRIteria Weights), WE-
BIRA (Weight Balancing Indicator Ranks Accordance), ACW (Adaptive Criteria Weights),
Symmetry 2021, 13, 525 18 of 20

SWARA and considering other methods of expert evaluation can be the focus in future re-
search [68–72]. Another area for future research could be to apply the proposed method to
real-world problems, such as designing geographic information systems, economic indica-
tors assessment, evaluations in tourism management, and pollution management [2,73–75].
Future research can be enriched by the proposed method in a vaguely determined environ-
ment, e.g., fuzzy and neutrosophic environments [3,76,77].

Author Contributions: Conceptualization, M.K.-G., E.K.Z., and Z.T.; methodology, M.K.-G., M.A.,
E.K.Z.; validation, M.K.-G. and J.A.; formal analysis, M.K.-G. and J.A.; investigation, M.K.-G., M.A.,
and Z.T.; writing—original draft preparation, M.K.-G. and M.A.; writing—review and editing, E.K.Z.,
J.A., and Z.T. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Abdulgader, F.S.; Eid, R.; Daneshvar Rouyendegh, B. Development of Decision Support Model for Selecting a Maintenance Plan
Using a Fuzzy MCDM Approach: A Theoretical Framework. Appl. Comput. Intell. Soft Comput. 2018, 2018, 9346945. [CrossRef]
2. Samanlioglu, F.; Burnaz, A.N.; Diş, B.; Tabaş, M.D.; Adıgüzel, M. An Integrated Fuzzy Best-Worst-TOPSIS Method for Evaluation
of Hotel Website and Digital Solutions Provider Firms. Adv. Fuzzy Syst. 2020, 2020, 8852223. [CrossRef]
3. Irvanizam, I.; Usman, T.; Iqbal, M.; Iskandar, T.; Marzuki, M. An Extended Fuzzy TODIM Approach for Multiple-Attribute
Decision-Making with Dual-Connection Numbers. Adv. Fuzzy Syst. 2020, 2020, 6190149. [CrossRef]
4. Muangman, J.; Krootsong, K.; Polrong, P.; Yukunthorn, W.; Udomsap, W. Fuzzy Multicriteria Decision-Making for Ranking Inter-
crop in Rubber Plantations under Social, Economic, and Environmental Criteria. Adv. Fuzzy Syst. 2020, 2020, 6508590. [CrossRef]
5. Zapolskytė, S.; Burinskienė, M.; Trepanier, M. Evaluation criteria of smart city mobility system using MCDM method. Balt. J.
Road Bridge Eng. 2020, 15, 196–224. [CrossRef]
6. Ueasin, N. Decision-making on Public Transportation Services Based on the Socio-economic, Psychological, and Environmental
Concern Factors. Open Transp. J. 2020, 14. [CrossRef]
7. Zagorskas, J.; Turskis, Z. Location preferences of new pedestrian bridges based on multi-criteria decision-making and GIS-based
estimation. Balt. J. Road Bridge Eng. 2020, 15, 158–181. [CrossRef]
8. Alakaş, H.M.; Gür, Ş.; Özcan, E.; Eren, T. Ranking of sustainability criteria for industrial symbiosis applications based on ANP. J.
Environ. Eng. Landsc. Manag. 2020, 28, 192–201. [CrossRef]
9. Keshavarz-Ghorabaee, M.; Govindan, K.; Amiri, M.; Zavadskas, E.K.; Antuchevičienė, J. An integrated type-2 fuzzy decision
model based on WASPAS and SECA for evaluation of sustainable manufacturing strategies. J. Environ. Eng. Landsc. Manag. 2019,
27, 187–200. [CrossRef]
10. He, T.; Wei, G.; Lu, J.; Wu, J.; Wei, C.; Guo, Y. A novel EDAS based method for multiple attribute group decision making with
Pythagorean 2-tuple linguistic information. Technol. Econ. Dev. Econ. 2020, 26, 1125–1138. [CrossRef]
11. Lin, P.-P.; Li, D.-F.; Jiang, B.-Q.; Yu, G.-F.; Wei, A.-P. Evaluating the comprehensive impacts of tourism in Hainan by intergrating
input-output model with MCDM methods. Technol. Econ. Dev. Econ. 2020, 26, 989–1029. [CrossRef]
12. Ban, A.I.; Ban, O.I.; Bogdan, V.; Popa, D.C.S.; Tuse, D. Performance evaluation model of Romanian manufacturing listed
companies by fuzzy AHP and TOPSIS. Technol. Econ. Dev. Econ. 2020, 1–29. [CrossRef]
13. Mi, X.; Liao, H.; Liao, Y.; Lin, Q.; Lev, B.; Al-Barakati, A. Green suppler selection by an integrated method with stochastic
acceptability analysis and MULTIMOORA. Technol. Econ. Dev. Econ. 2020, 26, 549–572. [CrossRef]
14. Tian, C.; Peng, J.; Zhang, W.; Zhang, S.; Wang, J. Tourism environmental impact assessment based on improved AHP and picture
fuzzy PROMETHEE II methods. Technol. Econ. Dev. Econ. 2020, 26, 355–378. [CrossRef]
15. Tabatabaei, M.H.; Amiri, M.; Ghahremanloo, M.; Keshavarz-Ghorabaee, M.; Zavadskas, E.K.; Antucheviciene, J. Hierarchical
decision-making using a new mathematical model based on the Best-Worst Method. Int. J. Comput. Commun. Control 2020, 14,
710–725. [CrossRef]
16. Roszkowska, E. Rank Ordering criteria weighting methods–a comparative overview. Optim. Studia Ekon. 2013, 5,
14–33. [CrossRef]
17. Alfares, H.K.; Duffuaa, S.O. Simulation-based evaluation of criteria rank-weighting methods in multi-criteria decision-making.
Int. J. Inf. Technol. Decis. Mak. 2016, 15, 43–61. [CrossRef]
18. Wang, J.-J.; Jing, Y.-Y.; Zhang, C.-F.; Zhao, J.-H. Review on multi-criteria decision analysis aid in sustainable energy decision-
making. Renew. Sustain. Energy Rev. 2009, 13, 2263–2278. [CrossRef]
19. Ginevičius, R. A new determining method for the criteria weights in multicriteria evaluation. Int. J. Inf. Technol. Decis. Mak. 2011,
10, 1067–1095. [CrossRef]
20. Zardari, N.H.; Ahmed, K.; Shirazi, S.M.; Yusop, Z.B. Weighting Methods and their Effects on Multi-Criteria Decision Making Model
Outcomes in Water Resources Management; Springer: New York, NY, USA, 2014.
Symmetry 2021, 13, 525 19 of 20

21. Kao, C. Weight determination for consistently ranking alternatives in multiple criteria decision analysis. Appl. Math. Model. 2010,
34, 1779–1787. [CrossRef]
22. Wu, H.-W.; Zhen, J.; Zhang, J. Urban rail transit operation safety evaluation based on an improved CRITIC method and cloud
model. J. Rail Transp. Plan. Manag. 2020, 16, 100206. [CrossRef]
23. Delice, E.K.; Can, G.F. A new approach for ergonomic risk assessment integrating KEMIRA, Best–Worst and MCDM methods.
Soft Comput. 2020, 24, 15093–15110. [CrossRef]
24. Freeman, J.; Chen, T. Green supplier selection using an AHP-Entropy-TOPSIS framework. Supply Chain Manag. Int. J.
2015. [CrossRef]
25. Du, Y.-W.; Gao, K. Ecological security evaluation of marine ranching with AHP-entropy-based TOPSIS: A case study of Yantai,
China. Mar. Policy 2020, 122, 104223. [CrossRef]
26. Du, Y.; Zheng, Y.; Wu, G.; Tang, Y. Decision-making method of heavy-duty machine tool remanufacturing based on AHP-entropy
weight and extension theory. J. Clean. Prod. 2020, 252, 119607. [CrossRef]
27. Chen, C.-H. A novel multi-criteria decision-making model for building material supplier selection based on Entropy-AHP
weighted TOPSIS. Entropy 2020, 22, 259. [CrossRef]
28. Liu, S.; Hu, Y.; Zhang, X.; Li, Y.; Liu, L. Blockchain service provider selection based on an integrated BWM-Entropy-TOPSIS
method under an intuitionistic fuzzy environment. IEEE Access 2020, 8, 104148–104164. [CrossRef]
29. Nguyen, P.-H.; Tsai, J.-F.; Nguyen, V.-T.; Vu, D.-D.; Dao, T.-K. A Decision Support Model for Financial Performance Evaluation of
Listed Companies in The Vietnamese Retailing Industry. J. Asian Financ. Econ. Bus. 2020, 7, 1005–1015. [CrossRef]
30. Sałabun, W.; Watróbski,
˛ J.; Shekhovtsov, A. Are MCDA methods benchmarkable? A comparative study of TOPSIS, VIKOR,
COPRAS, and PROMETHEE II Methods. Symmetry 2020, 12, 1549. [CrossRef]
31. Vavrek, R.; Bečica, J. Efficiency evaluation of cultural services in the Czech Republic via multi-criteria decision analysis. Sustain-
ability 2020, 12, 3409. [CrossRef]
32. Narayanamoorthy, S.; Annapoorani, V.; Kalaiselvan, S.; Kang, D. Hybrid Hesitant Fuzzy Multi-Criteria Decision Making Method:
A Symmetric Analysis of the Selection of the Best Water Distribution System. Symmetry 2020, 12, 2096. [CrossRef]
33. Liao, H.; Xu, Z. A VIKOR-based method for hesitant fuzzy multi-criteria decision making. Fuzzy Optim. Decis. Mak. 2013, 12,
373–392. [CrossRef]
34. Xu, Z.; Zhang, X. Hesitant fuzzy multi-attribute decision making based on TOPSIS with incomplete weight information. Knowl.
Based Syst. 2013, 52, 53–64. [CrossRef]
35. Li, Z.-H. An Extension of the MULTIMOORA Method for Multiple Criteria Group Decision Making Based upon Hesitant Fuzzy
Sets. J. Appl. Math. 2014, 2014, 527836. [CrossRef]
36. Anitha, J.; Das, R. Optimization of Process Parameters in Electro Discharge Machine Using Standard Deviation, MULTIMOORA
and MOOSRA Methods. In Innovative Product Design and Intelligent Manufacturing Systems; Springer: Berlin/Heidelberg, Germany,
2020; pp. 619–629.
37. Anitha, J.; Das, R. Optimization of EDM Process Parameters Using Standard Deviation and Multi-objective Optimization on the
Basis of Simple Ratio Analysis (MOOSRA). In Intelligent Manufacturing and Energy Sustainability; Springer: Berlin/Heidelberg,
Germany, 2020; pp. 655–662.
38. Maheshwari, N.; Choudhary, J.; Rath, A.; Shinde, D.; Kalita, K. Finite Element Analysis and Multi-criteria Decision-Making
(MCDM)-Based Optimal Design Parameter Selection of Solid Ventilated Brake Disc. J. Inst. Eng. (India) Ser. C 2021. [CrossRef]
39. Ramasamy, V.; Subramanian, Y.; Varadarajan, S.; Ramaswamy, K.; Kaliappan, K.; Arulmozhi, D.; Srinivasan, G.R.; Gubendiran,
R.K. Influence of process parameters on the optimisation of crystalline phase, size and strain of multiferroic Bismuth Iron Tri
Oxide (BiFeO3) nanoceramics: A MCDM based TOPSIS approach. Ceram. Int. 2020, 46, 1457–1471. [CrossRef]
40. Şahin, M. Hybrid Multiattribute Decision Method for Material Selection. Int. J. Pure Appl. Sci. 2020, 6, 107–117.
41. Yazdani, M.; Torkayesh, A.E.; Santibanez-Gonzalez, E.D.R.; Otaghsara, S.K. Evaluation of renewable energy resources using
integrated Shannon Entropy—EDAS model. Sustain. Oper. Comput. 2020, 1, 35–42. [CrossRef]
42. Afshar, M.R.; Shahhosseini, V.; Sebt, M.H. An interval type-2 fuzzy MCDM model for work package subcontractor prequalification.
Soft Comput. 2021, 25, 635–648. [CrossRef]
43. Arya, V.; Kumar, S. A new picture fuzzy information measure based on Shannon entropy with applications in opinion polls using
extended VIKOR–TODIM approach. Comput. Appl. Math. 2020, 39, 197. [CrossRef]
44. Li, H.; Wang, W.; Fan, L.; Li, Q.; Chen, X. A novel hybrid MCDM model for machine tool selection using fuzzy DEMATEL,
entropy weighting and later defuzzification VIKOR. Appl. Soft Comput. 2020, 91, 106207. [CrossRef]
45. Görçün, Ö.F. Efficiency analysis of Black sea container seaports: Application of an integrated MCDM approach. Marit. Policy
Manag. 2020, 1–28. [CrossRef]
46. Zha, S.; Guo, Y.; Huang, S.; Wang, S. A hybrid MCDM method using combination weight for the selection of facility layout in the
manufacturing system: A case study. Math. Probl. Eng. 2020, 2020, 1320173. [CrossRef]
47. Alao, M.A.; Ayodele, T.R.; Ogunjuyigbe, A.S.O.; Popoola, O.M. Multi-criteria decision based waste to energy technology selection
using entropy-weighted TOPSIS technique: The case study of Lagos, Nigeria. Energy 2020, 201, 117675. [CrossRef]
48. Salehi, V.; Zarei, H.; Shirali, G.A.; Hajizadeh, K. An entropy-based TOPSIS approach for analyzing and assessing crisis manage-
ment systems in petrochemical industries. J. Loss Prev. Process Ind. 2020, 67, 104241. [CrossRef]
Symmetry 2021, 13, 525 20 of 20

49. Torkashvand, M.; Neshat, A.; Javadi, S.; Yousefi, H. DRASTIC framework improvement using Stepwise Weight Assessment Ratio
Analysis (SWARA) and combination of Genetic Algorithm and Entropy. Environ. Sci. Pollut. Res. 2020. [CrossRef]
50. Wang, R.; Li, X.; Li, C. Optimal selection of sustainable battery supplier for battery swapping station based on Triangular fuzzy
entropy -MULTIMOORA method. J. Energy Storage 2021, 34, 102013. [CrossRef]
51. Mohamadghasemi, A.; Hadi-Vencheh, A.; Hosseinzadeh Lotfi, F. The multiobjective stochastic CRITIC–TOPSIS approach for
solving the shipboard crane selection problem. Int. J. Intell. Syst. 2020, 35, 1570–1598. [CrossRef]
52. Peng, X.; Krishankumar, R.; Ravichandran, K.S. A novel interval-valued fuzzy soft decision-making method based on CoCoSo
and CRITIC for intelligent healthcare management evaluation. Soft Comput. 2021. [CrossRef]
53. Xu, C.; Ke, Y.; Li, Y.; Chu, H.; Wu, Y. Data-driven configuration optimization of an off-grid wind/PV/hydrogen system based on
modified NSGA-II and CRITIC-TOPSIS. Energy Convers. Manag. 2020, 215, 112892. [CrossRef]
54. Abdel-Basset, M.; Mohamed, R. A novel plithogenic TOPSIS- CRITIC model for sustainable supply chain risk management. J.
Clean. Prod. 2020, 247, 119586. [CrossRef]
55. Li, S.; Wang, B. Research on Evaluating Algorithms for the Service Quality of Wireless Sensor Networks Based on Interval-Valued
Intuitionistic Fuzzy EDAS and CRITIC Methods. Math. Probl. Eng. 2020, 2020, 5391940. [CrossRef]
56. Görçün, Ö.F. Evaluation of the selection of proper metro and tram vehicle for urban transportation by using a novel integrated
MCDM approach. Sci. Prog. 2021, 104, 1–18. [CrossRef] [PubMed]
57. Wei, G.; Lei, F.; Lin, R.; Wang, R.; Wei, Y.; Wu, J.; Wei, C. Algorithms for probabilistic uncertain linguistic multiple attribute group
decision making based on the GRA and CRITIC method: Application to location planning of electric vehicle charging stations.
Econ. Res. Ekon. Istraživanja 2020, 33, 828–846. [CrossRef]
58. Mishra, A.R.; Rani, P.; Pandey, K. Fermatean fuzzy CRITIC-EDAS approach for the selection of sustainable third-party reverse
logistics providers using improved generalized score function. J. Ambient Intell. Humaniz. Comput. 2021. [CrossRef]
59. Piasecki, M.; Kostyrko, K. Development of Weighting Scheme for Indoor Air Quality Model Using a Multi-Attribute Decision
Making Method. Energies 2020, 13, 3120. [CrossRef]
60. Narayanamoorthy, S.; Annapoorani, V.; Kang, D.; Baleanu, D.; Jeon, J.; Kureethara, J.V.; Ramya, L. A novel assessment of
bio-medical waste disposal methods using integrating weighting approach and hesitant fuzzy MOOSRA. J. Clean. Prod. 2020, 275,
122587. [CrossRef]
61. Liaw, C.-F.; Hsu, W.-C.J.; Lo, H.-W. A Hybrid MCDM Model to Evaluate and Classify Outsourcing Providers in Manufacturing.
Symmetry 2020, 12, 1962. [CrossRef]
62. Keshavarz Ghorabaee, M.; Zavadskas, E.K.; Olfat, L.; Turskis, Z. Multi-criteria inventory classification using a new method of
evaluation based on distance from average solution (EDAS). Informatica 2015, 26, 435–451. [CrossRef]
63. Keshavarz Ghorabaee, M.; Zavadskas, E.K.; Amiri, M.; Esmaeili, A. Multi-criteria evaluation of green suppliers using an extended
WASPAS method with interval type-2 fuzzy sets. J. Clean. Prod. 2016, 137, 213–229. [CrossRef]
64. Walters, S.J. Quality of Life Outcomes in Clinical Trials and Health-Care Evaluation: A Practical Guide to Analysis and Interpretation;
Wiley: New York, NY, USA, 2009.
65. Keshavarz-Ghorabaee, M. Supplementary data for “Determination of objective weights using a new method based on the removal
effects of criteria (MEREC)”. Figshare 2021. [CrossRef]
66. Wludyka, P.S.; Nelson, P.R. An analysis-of-means-type test for variances from normal populations. Technometrics 1997, 39,
274–285. [CrossRef]
67. SAS Institute Inc. JMP® 11 Basic Analysis; SAS Institute Inc.: Cary, NC, USA, 2013.
68. Sitorus, F.; Brito-Parada, P.R. A multiple criteria decision making method to weight the sustainability criteria of renewable energy
technologies under uncertainty. Renew. Sustain. Energy Rev. 2020, 127, 109891. [CrossRef]
69. Zavadskas, E.K.; Podvezko, V. Integrated determination of objective criteria weights in MCDM. Int. J. Inf. Technol. Decis. Mak.
2016, 15, 267–283. [CrossRef]
70. Krylovas, A.; Kosareva, N.; Dadelo, S. European Countries Ranking and Clustering Solution by Children’s Physical Activity and
Human Development Index Using Entropy-Based Methods. Mathematics 2020, 8, 1705. [CrossRef]
71. Li, H.; Wang, Y.; Li, Y.; Xiao, G.; Hu, P.; Zhao, R.; Li, B. Learning adaptive criteria weights for active semi-supervised learning. Inf.
Sci. 2021, 561, 286–303. [CrossRef]
72. Chernyi, S.; Budnik, V. Methods for optimizing solutions when considering group arguments by team of experts. AIP Conf. Proc.
2017, 1899, 060002.
73. Saeed Iranmanesh, M.I.; Norallah Salehi, A.; Seyyed Abdolmajid Jalaee, B. Using Fuzzy Logic Method to Investigate the Effect of
Economic Sanctions on Business Cycles in the Islamic Republic of Iran. Appl. Comput. Intell. Soft Comput. 2021, 2021, 8833474.
74. Chumaidiyah, E.; Dewantoro, M.D.R.; Kamil, A.A. Design of a Participatory Web-Based Geographic Information System for
Determining Industrial Zones. Appl. Comput. Intell. Soft Comput. 2021, 2021, 6665959.
75. Phetpradap, P. A Fuzzy Soft Model for Haze Pollution Management in Northern Thailand. Adv. Fuzzy Syst. 2020, 2020,
6968705. [CrossRef]
76. Javaherian, N.; Hamzehee, A.; Sayyadi Tooranloo, H. Designing an Intuitionistic Fuzzy Network Data Envelopment Analysis
Model for Efficiency Evaluation of Decision-Making Units with Two-Stage Structures. Adv. Fuzzy Syst. 2021, 2021, 8860634.
77. Wei, G.; Wu, J.; Guo, Y.; Wang, J.; Wei, C. An extended COPRAS model for multiple attribute group decision making based on
single-valued neutrosophic 2-tuple linguistic environment. Technol. Econ. Dev. Econ. 2021. [CrossRef]