Available online at www.sciencedirect.

com
Available online at www.sciencedirect.com
ScienceDirect
ScienceDirect
Procedia Computer Science 00 (2017) 000–000
Available online at www.sciencedirect.com
Procedia Computer Science 00 (2017) 000–000 www.elsevier.com/locate/procedia
www.elsevier.com/locate/procedia
ScienceDirect
Procedia Computer Science 112 (2017) 2269–2279

International Conference on Knowledge Based and Intelligent Information and Engineering

International Conference on Knowledge
Systems, KES2017, Based and 2017,
6-8 September Intelligent Information
Marseille, Franceand Engineering
Systems, KES2017, 6-8 September 2017, Marseille, France
Construction and Use of the ANP Decision Model Taking into
Construction and Use of the ANP Decision Model Taking into
Account the Experts’ Competence
Account the Experts’ Competence
Jarosław Becker*a, Aneta Beckerb, Wojciech Sałabunc
a
Jarosław Becker*a, Aneta Beckerb, Wojciech Sałabunc
Jacob of Paradyz University in Gorzów Wielkopolski, Teatralna 25, 66-400 Gorzów Wielkopolski, Poland
ba
West
JacobPomeranian
of Paradyz University in
of Gorzów
Technology, Faculty ofTeatralna
Wielkopolski, Economics,
25,Janickiego 31, 71-210
66-400 Gorzów Szczecin,Poland
Wielkopolski, Poland
c
West
b
Pomeranian
West University
Pomeranian of Technology,
University Faculty
of Technology, of Computer
Faculty Science,
of Economics, Żołnierska
Janickiego 31, 49, 71-210
71-210 Szczecin,
Szczecin, Poland
Poland
c
West Pomeranian University of Technology, Faculty of Computer Science, Żołnierska 49, 71-210 Szczecin, Poland

Abstract
Abstract
The Analytic Network Process (ANP) is a subject of interest of the authors, which is included into methods of multi-criteria
The
decision-making
Analytic Network
approach.
Process
The article
(ANP) covers
is a subject
the issue
of interest
of supporting
of thedecisions
authors, involving
which is included
experts. Ainto
network
methods
model
of multi-criteria
structure was
decision-making
proposed, where approach. The article
the assessment of thecovers thevariants
decision issue ofonsupporting
the set ofdecisions involving
criteria depends notexperts.
only onA the
network model
experts’ structure
opinion, but was
also
proposed,
on the levelwhere the competence.
of their assessment ofThe the possibility
decision variants
of takingon into
the set of criteria
account depends not
the preferences foronly
the on the experts’
criteria set by: aopinion,
decisionbut also
maker,
on the level
experts of their competence.
or beneficiaries Thethe
representing possibility of takingoptions
decision-making into account
was alsotheconsidered.
preferencesThe
for model’s
the criteria set by: a decision
performance maker,
was verified on
experts or beneficiaries
the example of employeerepresenting
assessment.theThe
decision-making
results of the options was have
experiment also considered. The the
confirmed that model’s performance
decision was verified
model developed can on
be
the example
formally of employee
represented in theassessment. The results
computer decision of the
support experiment
systems have confirmed
and delivered that the
to the users decision
in the form ofmodel developedproposal
a ready-made can be
formally
(pattern).represented in the computer decision support systems and delivered to the users in the form of a ready-made proposal
(pattern).
© 2017 The Authors. Published by Elsevier B.V.
© 2017 The Authors. Published by Elsevier B.V.
©
Peer-review
2017 The under
Peer-review Authors.
under responsibility
responsibility
Published by ofElsevier
of KES International
KES International.
B.V.
Peer-review under responsibility of KES International.
Keywords: multiple-criteria decision-making (MCDM), model of ANP (analytic network process), experts opinions, competence, ranking;
Keywords: multiple-criteria decision-making (MCDM), model of ANP (analytic network process), experts opinions, competence, ranking;

1. Introduction
1. Introduction
The Information and Communication Technologies (ICT) which are developing intensely have become the
The Information
driving and Communication
force of the modern economy. TheyTechnologies
have created (ICT) which are developing
great opportunities for growth intensely haveattractiveness
and increased become the
driving force of the modern economy. They have created great opportunities for growth and increased attractiveness

* Corresponding author. Tel.: +0-000-000-0000 ; fax: +0-000-000-0000 .

E-mail address:author.
* Corresponding [email protected]
Tel.: +0-000-000-0000 ; fax: +0-000-000-0000 .
E-mail address: [email protected]
1877-0509 © 2017 The Authors. Published by Elsevier B.V.
Peer-review©under
1877-0509 2017 responsibility of KES International.
The Authors. Published by Elsevier B.V.
Peer-review under responsibility of KES International.

1877-0509 © 2017 The Authors. Published by Elsevier B.V.

Peer-review under responsibility of KES International
10.1016/j.procs.2017.08.145
2270 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279
Jarosław Becker et al. / Procedia Computer Science 00 (2017) 000–000

for many companies. The material resources are no longer the most important factor of the company’s functioning,
but the information, people and their knowledge, competence and skills. That is why, the electronic economy (e-
economy, e-economics) is often called the “knowledge-based economy”. Technological progress has made the
business world intensify its actions. Companies show adaptability to frequent changes occurring in the variable,
complex and uncertain external and internal environment. Their organizational structures become very flexible and
they are subject to frequent reconfigurations. Management decisions must be adapted to the changing circumstances,
determined by the complex and alternating environment. The complexity of the decision analyses (numerous
processes, criteria, opinions, data, dependencies) and the variety of available resources to solve them (numerous
methods, techniques from the field of, e.g., multi-criteria decision analysis, operational research, system analysis,
soft-computing, simulation, visualization, etc.) makes us seek new ways of conducting research which is the
adaptation or synthesis of the already available methods, models, techniques and tools.
The multifaceted nature of the decision analyses is a significant problem in the engineering (construction) of the
computer decision support systems. It appears when the addition off a complete description of the decision problem
is difficult, impractical and impracticable. For example, the information is not explicitly expressed in the form of
decision criteria, resulting only from the context of the given decision situation, it is difficult to model and include in
the calculation of multi-criteria methods. Relevant studies in this field are presented in the papers: [1], [2], [3].
Kersten [4] defined the category of decision supporting systems driven by models (Model-Driven DSS). In these
systems, the formally represented decision models are delivered to the user in the form of ready proposals (patterns)
or tools for designing own solutions. The methods of the decision analysis, optimization, stochastic modelling,
simulation, statistics and logical modelling are used to solve them. The example of the construction and use of this
type of system was presented in [5], [6].
The literature [7], [8], [9] includes the discussion of different procedures and methods of multi-criteria decision
analysis (MCDM). According to Greco et al. [8], they can be divided into methods based on the functional model
(American school) and the relational model (European school). The decision theory creates the methodological
foundations for the analysis and searching for the best (optimal or preferred) solutions, but it does not prejudge their
usability in practice. In fact, the management needs translate into fundamental determinants [10], which should be
taken into account when designing decision models, namely:
 multi-criteria aspect – the structure of the criteria can be simple (criteria vector) or complex (hierarchical or
network relationships),
 decision maker – defines the criteria and preferences,
 experts – they assess the decision-making variants on criteria as a group, they can also assist the decision maker
in selecting the evaluation criteria and express own preferences,
 expert competence – it constitutes the importance of an expert in group evaluation of decision-making variants,
 decision variants – they constitute the subject of the analysis, in mass problems (with a large number of variants)
there may be problems related to the computational efficiency.
The aim of the article is to develop a decision model with a network structure that takes into account the factors
defined in the work, including, in particular, the influence of opinions and experts’ competence on the evaluation of
decision-making variants. It was assumed that the model could be one of many ready-made proposals, patters to be
used (e.g. provided within the computer-based DSS)). The performance of the model has been verified on the bases
of employee evaluation. The main motivation for using ANP [11] is the ability to capture all dependencies, including
indirect ones between the elements taking part in the decision-making process (competence, experts, criteria,
preferences, decision variants).

2. Literature review

The distribution of the dedicated computer programs, which help build the AHP/ANP models, has contributed to
the increased interest in hierarchical and network models. Unlike the AHP (Analytic Hierarchy Process), the ANP
method provides a free form of element ordering. Presenting the structure of the problem in the network form allows
to take into account the interdependence and influence of elements placed on different levels. The person expressing
opinions (through the Saaty’s comparison scale [12]) answers the questions about the strength of the advantage of
 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279 2271
Author name / Procedia Computer Science 00 (2017) 000–000

comparative elements. This allows obtaining information on which of two provided elements is more dominant and
which one of two provided elements has a bigger impact on the third element in relation to the criterion [11].
Analyzing a specific criterion, e.g., economic, political, social or environmental, allows an expert to understand the
problem. Such a criterion can be directly incorporated into the structure and form the main objective of the study, as
a comparative and linking control criterion. Another suggestion is to make a comparative and influential role, and
this influencing the comparisons made in the network. The discussion of methodological features of the AHP/ANP
methods was presented in the papers by Whitaker [13], [14] and Saaty et al. [15].
The first mentions of competence appeared in the literature related to social sciences in the seventies and eighties
of the XX century through the publication of McClelland [16] and Boyatzis [17]. In subsequent years, there has been
a development of interest in this concept and attempts to construct its coherent definition. Many different
formulations were formed. Each one can distinguish three main concepts: knowledge, skills and responsibility.
The concept of competence comes from the Latin word competentia and it means ability and willingness of the
subject to perform tasks at the expected level [18]. Competence is a subjective category (of a specific person, or a
group of people) acquired through life experiences (circus knowledge, mother tongue, customs, fashion etc.) and in
the course of deliberate and thoughtful teaching and education process. A competent person is empowered to act and
decide in the field and is qualified to issue judgments and assessments.
By analyzing the competence of the organization, one can observe a division into: professional, position, real and
achievable, as well as: key, core, process, managerial, functional, cognitive, social and meta-competence (the ability
to learn and reflect, coping in conditions of uncertainty), etc. While it should be noted that the core and most
important resource of the organization is the competence of the employees. According to Kozłowska [19] another
objective accompanies the employee assessment, and a different one the modelling of the state of future
organizations and its desired competence potential. Competence models need to be forward-looking and should be
built towards the anticipated situations, resulting from the conditions of problem solving and difficulties of the
workplace. Patanakul and Milosevic [20] point to unique competence, especially of the management. They propose
the construction of a model that would include information concerning, e.g., the organizational experience, multi-
tasking, simultaneous team management and interdisciplinary process management. While Wang et al. [21]
examined the impact of core competence: marketing, technological and integration on the effectiveness of the
company’s operation, indicating their significant role.
The use of advanced competence assessment methods requires the participation of experts, whose opinions often
determine the final outcome of the research. Remeikiene and Gaspareniene [22] point out that an expert is a
specialist with sufficient knowledge in the given field. On the other hand, the group of experts should solve the
studied problem in a reliable and effective way. Hanson and Ramani [23] point out that the number of experts can be
reduced, depending on the nature of the study, if they have a high level of knowledge in the field. Remeikiene and
Gaspareniene [22] indicate the decisive role of competence and experience in the area of the study in the expert
selection. They also prove that with the equal weight, the precision of the decisions and evaluations done by a small
expert group is the same as the opinion of a large group. Saaty [24; 25] broadly discusses the advantages and
disadvantages of expert judgment and the method of selecting experts for the study. In the paper by Kupraszewicz
and Żółtowski [26] we can familiarize ourselves with the original proposition of the algorithm for selecting and
evaluating experts. According to the authors, the quality of feedback received depends on the level of expert
judgment generated. Therefore, it is important to carefully select and examine a team of experts in terms of its
competence and opinion compliance. People who are high-level professionals should be invited to the group, and
whose opinions are moderately consistent. The compliance of the experts’ answers can be checked based on the
multiple interviews of the respectively numerous and representative group of anonymous experts, with extensive
experience in the subject of the study. Due to the difficulties with gathering the appropriate number of authorities, it
is proposed to carry out the targeted selection, for example, while using the method of recommendation (nomination
or co-nomination). Then, people who already participate in the design work recommend specialists in this field.
However, this approach has many disadvantages, e.g., omitting people with different views or controversial
behaviour, despite their considerable expertise. Therefore, the open recruitment is often proposed as an alternative.
Then, all those who meet the criteria can report themselves, and become the participants of the study as experts.
Another solution may involve the combination of these procedures.
2272 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279
Jarosław Becker et al. / Procedia Computer Science 00 (2017) 000–000

An important step in the work of the expert group is to achieve the consistent assessments. For this purpose, the
variability measures can be used [27]. It should be noted that the use of the measure of conformity of experts’
assessments depends on the scale, which measures the experts’ answers. The average degree of compliance of the
opinions of all experts while using the ordinal scale can be examined with the Kendall and Smith concordance
coefficient. In the case of a strong scale, that is inter-sectional and quotient, the inter-quartile range is used for
assessing the conformity. While the dispersion coefficient of relative classification can be used to check the opinions
obtained while using the nominal scale. The results obtained as a result o the application of the weak scale should be
verified using, among others, the dispersion coefficient, variation, rank pair correlation or concordance [28].
Tsyganok et al. [29] conducted an experiment, the results of which confirmed the hypothesis about the
coefficients of experts’ competence in the studies using the AHP method. According to the authors, the individual
competencies of a large group of experts (over 50) can be omitted, due to their negligible impact on the whole
analysis. However, if this group is relatively small, all experts’ competence should be taken into account. The paper
by Ossadnik et al. [30] presented a comprehensive review of the literature, which described different possibilities of
aggregating individual preferences of the experts and the impact of individual decisions on achieving the final
consensus of the team. The latest publications, including the discussion of the basic aggregation procedures in the
AHP/ANP grasp and the presentation of appropriate case studies, include the works by Lin and Wang [31], and
Grošelj and Zadnik [32]. More advanced methods, requiring a great mathematical knowledge, due to the
assumptions, can be found in the papers by: Altuzarra et al. [33], Basak [34] and Dong et al. [35]. The fuzzy
approach to preferences seems interesting, used by Yang at al. [36] when designing a group decision support system
and the selection of investment strategies discussed in the paper by Wu et al. [37]. In turn, the methods of
aggregating experts’ opinions expressed by fuzzy numbers, were presented by Bardossy et al. [38] and Hsu, Hen
[39]. Examples of fuzzy modeling of behavior and user experience in social networking platforms are described in
[40], [41], [42] and [43]. Creating group preferences by combining AHP/ANP with other methods, e.g., the DEA
method or linear programming can be found in the papers by Grošelj et al. [44] and Hosseinian et al. [45]. The
publications of Aull-Hyde et al. [46] and Chen et al. [47] can be a supplement of the discussed considerations about
the aggregation of individual expert preferences. Kreitner and Cassidy [48] point out that the group decision-making
is associated with the problem of the domination of more assertive individuals over the quieter group members. That
is why, the decisions of the entire group should be surrounded with special support. An interesting description of the
stratification of the research sample and the selection of experts for modelling the decision process with the AHP
method can be found in the paper by Gawlik [49]. The research used the method of the layered random selection and
reasoning according to the Delphic method.

3. The network structure pattern of the decision-making model

The proposed network structure model reflects the relations, which take place in the decision-making process
between four groups of elements: the assessed decision variants (objects), evaluation criteria, assessing experts and
their competence. In the ANP method, the groups are called components, while their constituents are called
elements. The directed and coherent graph is the dependency network between the components.
The two-tier structure of expert competence is introduced to the model (fig. 1). The parent component contains
the key competences 𝐶𝐶𝑘𝑘 (k = 1, …, s), while the subordinate components take into account the specialist competences
𝑐𝑐𝑘𝑘𝑘𝑘 (l = 1, …, r). Key competences are of a general nature. The European standard can be a starting point
(Recommendation of the European Parliament [50]), which covers 9 categories, these are the competences: C1 –
mathematical, C2 – scientific and technical, C3 – informatics, C4 – civic, C5 – professional, C6 – cultural, C7 –
communication, C8 – learning and C9 – using foreign languages. Specialized competences are a subset for specific
categories of key competences. They are precisely defined to the needs resulting from the specificity of the given
decision problem, creating the so-called competence profile for the expert selection.
Estimating the competences of the evaluators is tantamount to establishing their power to influence the group
assessment of decision variants. The competence coefficients are determined in the form of the elements of the scale
vector w ̅ 𝑘𝑘𝑘𝑘 = [𝑤𝑤1𝑘𝑘𝑘𝑘 , 𝑤𝑤2𝑘𝑘𝑘𝑘 , … , 𝑤𝑤ℎ𝑘𝑘𝑘𝑘 ]. It is calculated on the basis of results of the pair comparisons of the experts 𝐸𝐸𝑔𝑔 (g =
1, …, h), on each specialised competence 𝑐𝑐𝑘𝑘𝑘𝑘 (using the Saaty’s evaluation scale). The scale vectors achieved are
captured in the initial super-matrix.
 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279 2273
Author name / Procedia Computer Science 00 (2017) 000–000

Non-mandatory competence can be prioritized for validity. In the case of key competence, this role is attributed to
beneficiaries, who represent decision variants (fig. 1, feedback: objects -> key competence). The preferences of key
competences 𝐶𝐶𝑘𝑘 (k = 1, …, s) has a group nature, they take the form of scale vectors w ̅ 𝑡𝑡 = [𝑤𝑤1𝑡𝑡 , 𝑤𝑤2𝑡𝑡 , … , 𝑤𝑤𝑠𝑠𝑡𝑡 ]attributed
to individual decision variants 𝑉𝑉𝑡𝑡 (t = 1, 2, …, z). If it is necessary to take into account the preferences of one person
(e.g. the decision maker), then, it is assumed that the scale vectors for all decision variants 𝑉𝑉𝑡𝑡 in the super-matrix are
the same. While for the specialized competences 𝑐𝑐𝑘𝑘𝑘𝑘 (l = 1, …, r) the scale vectors w ̅ 𝑘𝑘 = [𝑤𝑤1𝑘𝑘 , 𝑤𝑤2𝑘𝑘 , … , 𝑤𝑤𝑟𝑟𝑘𝑘 ] are
defined from the point of view of the related key competences 𝐶𝐶𝑘𝑘 .

Key competences

C1, C2, …, Cs

Specialist Specialist
competences C 1 competences C s
. ..
c11 , …, c1r cs1, …, cs r

Experts

E1, E2, …, Eh

Criteria E 1 Criteria E h
. ..
D11 , …, D1n Dh1, …, Dh n

Subcriteria D11 Subcriteria D1n Subcriteria Dh1 Subcriteria Dhn

. .. . ..
d111, …, d11m d1n1, …, d1n m dh11 , …, dh1m dh n1, …, dh n m

Variants of the
decision-mak ing
(alterna tives)
V1, V2, …, Vz

Fig. 1. Network structure of the decision-making model taking into account the influence of experts’ competence
on multi-criteria assessment of objects (alternatives)

In the model, each expert 𝐸𝐸𝑔𝑔 (g = 1, …, h) is assigned with the same structure of assessment criteria (fig. 1). It
is assumed that the non-mandatory main criteria 𝐷𝐷𝑔𝑔𝑔𝑔 (i = 1, …, n) can be decomposed by creating components for
sub-criteria. Individual experts express their preferences of criteria and sub-criteria using assessments from the
Saaty’s scale. This is done by comparing pairs of all elements in the given cluster. The scale vectors calculated on
𝑔𝑔 𝑔𝑔 𝑔𝑔 𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔
this basis w ̅ 𝑔𝑔 = [𝑤𝑤1 , 𝑤𝑤2 , … , 𝑤𝑤𝑛𝑛 ] for the main criteria 𝐷𝐷𝑔𝑔𝑔𝑔 and the scale vectors w ̅ 𝑔𝑔𝑖𝑖 = [𝑤𝑤1 , 𝑤𝑤2 , … , 𝑤𝑤𝑚𝑚 ] for sub-
criteria 𝑑𝑑𝑔𝑔𝑔𝑔𝑔𝑔 (j = 1, …, m) are recorded in the appropriate columns of the initial super-matrix. It is important to
mention that the possibility to define horizontal relationships between sub-criteria focused in different compartments
and relationships occurring inside the components has been planned. They are assumed to be non-compulsory.
Components that focus the main criteria 𝐷𝐷𝑔𝑔𝑔𝑔 (i = 1, …, n) can be expanded, by placing back links to decision
variants 𝑉𝑉𝑡𝑡 (fig. 1, connections marked with a dotted line). Preferences for the criteria specified on each 𝑉𝑉𝑡𝑡 , like in
the previous cases, are expressed in the form of scale vectors w ̅ ′𝑡𝑡 = [𝑤𝑤′1𝑡𝑡 , 𝑤𝑤′𝑡𝑡2 , … , 𝑤𝑤′𝑡𝑡𝑛𝑛 ] and placed in appropriate
places of the initial super-matrix The influence of the experts’ preferences on the criteria will decrease, when the
preferences of the objects 𝑉𝑉𝑡𝑡 are included at the same time and it will decrease along with the increase in the number
of objects. If the feedback is not used, then the values of elements of the scale vector w ̅ ′𝑡𝑡 in the super-matrix are
zeroed.
2274 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279
Jarosław Becker et al. / Procedia Computer Science 00 (2017) 000–000

The decision variants 𝑉𝑉𝑡𝑡 are related to the sub-criteria 𝑑𝑑𝑔𝑔𝑔𝑔𝑔𝑔 (j = 1, …, m). Each expert 𝐸𝐸𝑔𝑔 (g = 1, 2, …, h)
assesses objects 𝑉𝑉𝑡𝑡 (t = 1, 2, …, z) on the associated sub-criteria using the fixed point assessment scale. At this point,
the construction of the matrix of comparisons with the object pairs in relation to the given criterion was dropped,
because this method is useful only for their very small number (z < 11). The obtained vectors of point assessments
𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔
p̅ 𝑔𝑔𝑔𝑔𝑔𝑔 = [𝑝𝑝1 , 𝑝𝑝2 , … , 𝑝𝑝𝑧𝑧 ] are transformed into the scale vectors w ̅ 𝑔𝑔𝑔𝑔𝑔𝑔 = [𝑤𝑤1 , 𝑤𝑤2 , … , 𝑤𝑤𝑧𝑧 ] , where 𝑤𝑤𝑡𝑡 =
𝑔𝑔𝑔𝑔𝑔𝑔 𝑔𝑔𝑔𝑔𝑔𝑔
𝑝𝑝𝑡𝑡 / ∑𝑧𝑧𝑡𝑡=1 𝑝𝑝𝑡𝑡 . In order to standardize the vectors in the super-matrix, an assumption was adopted that the
assessments of the objects on the main criteria, which were not decomposed, are recorded in the super-matrix in the
block of sub-criteria, w ̅ 𝑔𝑔𝑔𝑔 = w
̅ 𝑔𝑔𝑔𝑔𝑗𝑗 (j=1). This treatment is purely technical, simplifying the configuration of the
model in the form of a super-matrix, does not affect the change of the results achieved.
The essence of the ANP method algorithm is the iterative process of increasing the weighed super-matrix
expressed in order to obtain the limited super-matrix. The times of operation of the matrix enhancement algorithms
– according to the asymptotic notation, in which the attention is not paid to the components of a lower order and
constant factors – is Θ(3), where  means the number of input data. Expansion of the super-matrix for the proposed
model structure with a very large number of assessed decision variants and a considerable number of experts (or
respondents of the survey) may be revealed in the extended calculation time. One solution is to move the
components: criteria, sub-criteria and decision variants to separate sub-nets, where each one is a super-matrix
assigned only to one expert. However, a large part of dependency is lost then, also the indirect ones, between the
elements of the decision process.
Assuming, according to the accepted notations, that s = 2, r = 6, n = 2, m = 6, g =10, t = 10, then in the first case
we obtain a super-matrix containing  = 11 664 elements, which translates into the performance of P1 = 1,6*1012
operations. While in the second case (for g = 10 experts), we will obtain ten separate super-matrices with  = 324
elements, which requires the performance of P2 = 10*(3,4*107) = 3,4*108 operations. Increasing the number of
variants to the level of t = 100 (with other unchanged parameters) will result in the performance of, in the first case,
P1 = 6*1013, and in the second case of P2 = 1,6*1013 operations. While when we restore the number of variants of t=
10, and we will increase only the number of experts g = 100, then P1 = 6*1017, and P2 = 3,4*109. The comparison of
results indicates that the inclusion of decision problems in separate sub-nets gives significant savings only for the
large group of people expressing their opinion (e.g. survey respondents).

4. Case study of the employee assessment

The proposed network structure of the decision model was verified on the basis of the multi-directional
assessment of the locksmith brigade (seven employees: V1, …, V7) of a company. The entire process was divided
into two main stages.
1. Preparation of the problem structure and data:
1.1. defining the assessment criteria by the decision maker and giving them preferences,
1.2. designating the competence profile by the decision maker and the selection of the assessing team members
(experts) on its basis,
1.3. identifying the individual preferences by the experts for the criteria and performing the assessment of
individual employees.
2. Generating rankings of employees by assuming three different sets of preferences: non-differentiated, imposed
by the decision maker and determined individually by each expert. In each case, four variants of the impact of
experts’ competence were included:
2.1. without taking into account the competences – the values of the competency coefficients are not
diversified,
2.2. taking into account the competences,
2.3. taking into account weighed competences, when the decision maker prefers a greater influence (on the
employee assessment) of competence related to:
a. the business effectiveness over the behavioural aspects,
b. behavioural aspects over effectiveness.
 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279 2275
Author name / Procedia Computer Science 00 (2017) 000–000

The first step (1.1) determined four main criteria for assessing employees (d1, …, d4) and the preferences of the
decision maker were assigned to them (weighting factors: W = [w1, …, w4]):
 professional qualifications (d1) – professional knowledge, e.g., knowledge of the technical drawing, knowledge
of welding methods, possession of professional qualifications, etc. (w1 = 0.35),
 efficiency and quality of work (d2) – efficiency, experience, accuracy, ability to solve atypical problems, etc. (w2
= 0.35),
 professional attitudes (d3) – behavioural criteria e.g.: accountability, initiative, discipline (weight: w3 = 0.20).
 additional skills (d4) – the extra skills criterion for example: driver's license, certificates, courses, and foreign
language skills (w4 = 0.10).
All the weighting factors were determined by the Saaty's method (the convergence coefficient CR for the criteria
was 0,012).
The next step (1.2) defined the expert competence profile appropriate to the decision problem, i.e., providing
substantive coverage for the determined set of criteria. In order to simplify the analysis of the decision model at the
general level, only one key competence was defined (C1 – expert competence), which was decomposed. It was
assumed that the team assessing employees should have three categories of specialized competence:
 interpersonal and communication (c11) – covering the behavioural aspects, i.e., the ability to observe and
analyse individual behaviours, attitudes and actions of the employees considered individually, as well as
cooperating in a team,
 strategic and organizational (c12) – concerning knowledge, mission, strategies and policies of the company, and
the legal aspects (e.g. labour law),
 professional (C13) – related to the possession of knowledge and professional skills and knowledge of the
specifics of the work organization at the given position.
Based on the competence profile, an assessment team was formed, consisting of three experts: a) human resources
director (E1) – representing the formal point of view, assessing employees mainly based on the documentation
gathered by the Human Resources department, b) direct superior (E2) –with a pragmatic view of his subordinates
resulting from the ongoing observation, supervision, instruction and motivation, c) work psychologist (E3) –
representing the behavioural perspective, assessing employees mainly on the basis of the results of psychological
research (observation, interview, tests). For each expert the coefficients were determined, which specify the level of
his competence (rank of importance) in the employee assessment. The estimation involved the pair comparison of
(11)
experts on the given competence. For example, within the c11 competence, the highest note of 𝑤𝑤3 = 0,71 was
awarded to expert no. 3 (fig. 2).

Expert 1 Expert 2 Expert 3

0,71

0,45

0,43
0,32

0,30

0,26

0,23
0,14

0,14

C11 C12 C13

Fig. 2. Expert competence coefficients determined subjectively and adequately to the merits of the decision problem

The last step in the data preparation (step 1.3) was to determine individual preference vectors for the criteria:
̅ (𝑔𝑔=1) = [0,30; 0,30; 0,25; 0,15], w
w ̅ (𝑔𝑔=2) = [0,27; 0,33; 0,27; 0,13], w
̅ (𝑔𝑔=3) = [0,24; 0,24; 0,35; 0,17] and the
assessment of employees. The experts expressed their own opinions using the ten-point assessment scale (1p., 2p.,
…, 10p.). The maximum value of the assessment sum, which could have been obtained by an employee from the
expert, was 40 points. The total numbers of points, which were obtained by employees under the criteria from
individual experts, are presented in fig. 3.
2276 Jarosław Becker
Jarosław et al. et
Becker / Procedia Computer
al. / Procedia Science
Computer 00 (2017)
Science 000–000
112 (2017) 2269–2279

Expert 1 Expert 2 Expert 3

36

35

34
34
32

32
31

31
30

30
26

26
25

25

25
23
23

23
22

21
20
V1 V2 V3 V4 V5 V6 V7

Fig. 3. Sums of points awarded by experts to individual employees

The test procedure, which consisted of steps 2.1-2.3, was carried out three times. As a result of the
implementation of individual stages, 12 employee rankings were obtained (tab. 1). The first test included the
analysis of the expert competence influence on the ranking result without the participation of preferences.
Comparison of the ranking results obtained in accordance with the procedures in step 2.1 and 2.2 indicated the
different ranking of employees. Changes in the location of the objects were clear. Only two employees did not
change the position (V1, V2), as illustrated in fig. 4 and tab. 1, test 1.
Consideration of the weighed competences w ̅ (𝑘𝑘=1) =[0,15; 0,46; 0,39] in the case (2.3.a), when the decision maker
prefers a greater influence of competence related to the classic, pro-effectiveness approach to management over the
behavioural aspects, made the final assessments in relation to the results of step 2.2 were very similar (there was a
change in position of employees V1 and V3). This was due to the fact that the adopted policy strengthened the
opinion of experts E1 and E2, who unanimously represented the company’s interests and represent the majority in the
assessing team. In the opposite case (2.3.b), when the decision maker has significantly increased the rank of
behavioural competence w ̅ (𝑘𝑘=1) =[0,6; 0,2; 0,2] (indirectly enhancing the impact force of expert E3), the obtained
results have significantly changed in relation to the results of step 2.3.a. Only two employees have not changed their
position in the ranking (V2 and V7, fig. 4). The proposed network structure allowed to change the impact power of
the experts representing the competence required in the decision problem. This was achieved by the appropriate
correction of the weight factors.

Table 1. Employee rankings

Tes t 1. Preferences non-di fferenti a ted Tes t 2. Preferences i mpos ed by the deci s i on ma ker Tes t 3. Preferences determi ned by ea ch expert
Scena ri o 1 Scena ri o 2 Scena ri o 3 Scena ri o 4 Scena ri o 1 Scena ri o 2 Scena ri o 3 Scena ri o 4 Scena ri o 1 Scena ri o 2 Scena ri o 3 Scena ri o 4
(s tep 2.1) (s tep 2.2) (s tep 2.3.a ) (s tep 2.3.b) (s tep 2.1) (s tep 2.2) (s tep 2.3.a ) (s tep 2.3.b) (s tep 2.1) (s tep 2.2) (s tep 2.3.a ) (s tep 2.3.b)
Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s Obj. Idea l s
V6 1,0000 V4 1,0000 V4 1,0000 V6 1,0000 V4 1,0000 V4 1,0000 V4 1,0000 V4 1,0000 V4 1,0000 V4 1,0000 V4 1,0000 V4 1,0000
V4 0,9970 V6 0,9993 V6 0,9989 V4 0,9986 V2 0,9472 V6 0,9286 V6 0,9264 V2 0,9403 V6 0,9713 V6 0,9601 V6 0,9562 V6 0,9674
V2 0,9551 V2 0,9334 V2 0,9235 V2 0,9483 V6 0,9352 V2 0,9259 V2 0,9166 V6 0,9328 V2 0,9494 V2 0,9257 V2 0,9157 V2 0,9416
V1 0,9050 V1 0,8387 V3 0,8299 V1 0,8838 V1 0,7977 V3 0,7562 V3 0,7533 V1 0,7750 V1 0,8531 V3 0,7925 V3 0,7879 V1 0,8287
V5 0,8960 V3 0,8315 V1 0,8090 V5 0,8673 V5 0,7951 V1 0,7286 V1 0,6987 V5 0,7671 V5 0,8230 V1 0,7792 V1 0,7476 V3 0,8026
V7 0,8408 V5 0,8072 V5 0,7681 V3 0,8363 V3 0,7674 V5 0,7103 V5 0,6742 V3 0,7632 V3 0,8086 V5 0,7348 V5 0,6976 V5 0,7938
V3 0,8390 V7 0,7742 V7 0,7431 V7 0,8198 V7 0,7384 V7 0,6668 V7 0,6342 V7 0,7154 V7 0,7611 V7 0,6935 V7 0,6632 V7 0,7392

The second study (tab.1, test 2) involved the analysis of the experts’ competence impact on the ranking result
taking into account the decision maker’s preferences. The experts’ preference vectors were on purpose assigned with
the decision maker’s weight vector values w ̅ (𝑔𝑔=1)= w
̅ (𝑔𝑔=2) = w
̅ (𝑔𝑔=3) = W = [0,35; 0,35; 0,20; 0,10]. The decision
maker’s views had a big impact. The disproportions in the employee assessments have been considerably deepened
(this is evident when comparing the ranking results in fig. 5 with the results in fig. 4). It should be added that the
dominance of the decision-maker’s views has strengthened as the number of experts increases, which is considered
as an undesirable phenomenon. In the example, with the assumption of a small number of assessing people, the
comparison of the ranking results obtained in accordance with the proceedings in step 2.1 and 2.2 indicated the clear
impact of the experts’ competence. Changes in the position of objects were visible (fig. 5, tab. 1, test 2). Only two
 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279 2277
Author name / Procedia Computer Science 00 (2017) 000–000

employees did not change their position (V4 – the first location and V7 – the last location). The comparison of the
rankings of step 2.3.a with step 2.2 indicated no significant changes in the employee ratings. Their order was
identical. This was due to the fact that the preferences determined by the decision-maker for the criteria and the
competence priorities were convergent (they expressed the pro-efficiency approach to management). While in step
2.3.b, when they were divergent, the ranking results were very similar to the initial state, that is the results from step
2.2. Analogously to the results from the first test in step 2.3, the impact power of expert V 3 was increased and the
effect of mutual weakening (drifting) of divergent priorities appeared.

Scenario 1 (step 2.1.) Scenario 2 (step 2.2.) Scenario 3 (step 2.3.a) Scenario 4 (step 2.3.b)

1,00

1,000
1,000

1,000

1,000
0,999
0,999
0,999
0,95

0,997
0,955

0,948

0,90
0,933
0,924
0,905

0,85

0,896
0,884

0,867
0,80

0,841
0,839
0,839

0,836
0,832
0,830

0,820
0,809

0,807
0,75

0,774
0,768
0,70

0,743
0,65
0,60
V1 V2 V3 V4 V5 V6 V7

Fig. 4. Employee rankings with the assumption of non-differentiated preferences for the criteria

Scenario 1 (step 2.1.) Scenario 2 (step 2.2.) Scenario 3 (step 2.3.a) Scenario 4 (step 2.3.b)

1,00
1,000
1,000
1,000
1,000

0,95
0,90
0,947

0,940

0,935

0,933
0,929
0,926
0,926
0,917

0,85
0,80
0,75
0,798

0,795

0,674
0,775

0,667
0,767

0,767
0,763
0,756
0,753

0,70
0,738

0,634
0,715
0,710
0,729
0,699

0,65
0,60
V1 V2 V3 V4 V5 V6 V7

Fig. 5. Employee rankings taking into account decision maker’s preferences for the criteria

Scenario 1 (step 2.1.) Scenario 2 (step 2.2.) Scenario 3 (step 2.3.a) Scenario 4 (step 2.3.b)

1,00
1,000

1,000
1,000
1,000

0,95
0,971

0,967
0,960
0,956
0,949

0,90
0,942
0,926
0,916

0,85
0,853

0,80
0,829

0,823
0,809

0,803

0,75
0,794
0,793
0,788
0,779

0,663
0,761

0,70
0,748

0,739
0,735
0,698

0,693

0,65
0,60
V1 V2 V3 V4 V5 V6 V7

Fig. 6. Employee rankings taking into account the experts’ preferences for the criteria
2278 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279
Jarosław Becker et al. / Procedia Computer Science 00 (2017) 000–000

In the third study (tab.1, test 3), the individual preferences of the experts were used (weight vectors were set in
step 1.3). Once again it was confirmed that the behaviour of the model is predictable. The ranking results in steps
2.1 and 2.3.b were very similar compared to the ones obtained in the second study. In step 2.2 and 2.3.a the same
order of employee rankings was achieved. Diversification of preferences on the criteria (understood as the lack of
dominance of the decision-maker’s dominance) resulted in disproportions in employee evaluations, which were not
as big as in the second study (this is evident when comparing fig. 6 with fig. 5).
The configuration of network impacts assumed in the third study, out of three considered studies, was the most
favourable. The decision maker has the opportunity to take into account “fuller” expert opinions (not only
assessments on criteria, but also priorities for the criteria), which rank depends on the competence level relevant to
the problem. At the same time, the decision maker does not lose the ability to express own views, setting priorities
for competence.

5. Conclusion

The model presented in the article with the network structure takes into account different configurations of
impacts in the network between the components and their constituents. The practical verification of the model has
confirmed that it can be applied in computer decision support systems and used by the users as a ready-made
proposal (pattern).
The developed model takes into account all dependencies, also the intermediate ones, between the competences,
experts, assessment criteria and decision variants. Saaty’s work [11] demonstrated that the network model can grasp
the interdependencies between its elements, increasing the weighed, stochastic matrix to a power, which determines
the depth of the considered dependencies (the enhancement process should be continued until all relevant
dependencies have been captured). In addition, the preferences for the criteria can be included in it from the point of
view of: the decision-maker, experts or beneficiaries representing the decision-making variants and the preferences
for the competences.

References

1. Zadeh LA. Toward a perception-based theory of probabilistic reasoning with imprecise probabilities. J of Statistical Planning and Inference
2002; 105:233-264.
2. Grzegorzewski P. The coefficient of concordance for vague data. Computational Statistics & Data Analysis 2006; 51:314-322.
3. Ma J, Ruan D, Xu Y, Zhang G. A fuzzy-set approach to treat determinacy and consistency of linguistic terms in multi-criteria decision making.
International J. of Approximate Reasoning 2007, 44:165-181.
4. Kersten GE. Decision Making and Decision Support. In: Kersten GE, Mikolajuk Z, Garon Yey A, editors. Decision Support Systems for
Sustainable Development. A Resource book of method and applications. Dordrecht, the Netherlands: Kluwer Academic Publishers; 2000.
5. Budziński R, Becker J. Transformations of knowledge sources in decision support system. J of Automation, Mobile Robotics & Intelligent
System 2015; 9(2):28-36.
6. Becker J, Jankowski J, Wątróbski W. Transformations of Standardized MLP Models and Linguistic Data in the Computerized Decision
Support System. In: Różewski P, Novikov D, Bakhtadze N, Zaikin O, editors. New Frontiers in Information and Production Systems
Modelling and Analysis. Intelligent Systems Reference Library, vol. 98, Berlin Heidelberg: Springer; 2016. p. 213-231.
7. Roy B, Bouyssou D. Aide multicritere a la decision: Methodes et cas. Paris: Economica; 1993.
8. Greco S, Matarazzo B, Słowiński R. Rough sets theory for multicriteria decision analysis. Euro. J. of Operatio. Research 2001; 129/1: 1–47.
9. Słowiński R. Ordinal regression approach to multiple-criteria ordering decision variants. In: Kulczycki P, Hryniewicz O, Kacprzyk J, editors.
Information techniques’ in systems research. (in Polish) Warsaw: Wyd. Naukowo-Techniczne; 2007. p. 315-337.
10. Becker J, Budziński R. Optimization Procedure of the Multi-parameter Assessment and Bidding of Decision-Making Variants in the
Computerized Decision Support System, In: Núñez M, Nguyen NT, Camacho D, Trawiński B, editors. Computational Collective Intelligence.
LNCS, vol. 9330, Heidelberg: Springer; 2015. p.182-192.
11. Saaty TL. The Analytic Hierarchy and Analytic Network Processes for the Measurement of Intangibles Criteria and for Decision-Making. In:
Figueira J, Greco S, Ehrgott M, editors. Multiple Criteria Decision Analysis: State of the Art Surveys. Dordrecht: Kluwer Academic
Publishers; 2004. p. 345-408.
12. Saaty TL. The analytic hierarchy process: Planning, priority setting, resource allocation. New York: McGraw-Hill Inter. Book Co.; 1980.
13. Whitaker R. Validation examples of the Analytic Hierarchy Process and Analytic Network Process. Mathematical and Computer Modelling
2007; 46(7-8):840-859.
14. Whitaker R. Criticisms of the Analytic Hierarchy Process: Why they often make no sense. Mathematical and Computer Modelling 2007;
46(7-8): 948-961.
 Jarosław Becker et al. / Procedia Computer Science 112 (2017) 2269–2279 2279
Author name / Procedia Computer Science 00 (2017) 000–000

15. Saaty TL, Vargas L, Whitaker R. Addressing with brevity criticism of the Analytic Hierarchy Process. Inter. J of the Analytic Hierarchy
Process 2009; 1(2):121-134.
16. McClelland D. Testing for competency Rather than for intelligence. American Psychologist 1973; 28:1-14.
17. Boyatzis R. The competent manager: a model for effective performance. New York: Wiley; 1982.
18. Kwiatkowska H., Pedeutology. (in Polish) Warsaw: Academic and Professional Publishing House; 2008.
19. Kozłowska D. Application of selected competency models in the improvement of tourism activities. (in Polish) Bialystok: Department of
Publications of the School of Physical Education and Tourism; 2014.
20. Patanakul P, Milosevic D. A competency model for effectiveness in managing multiple project. J of High Tech Man Res 2008; 18:118–131.
21. Wang YG, Lo HP, Yang YH. The constituents of core competencies and firm performance: Evidence from High-technology firm in China.
J of Engineering and Technology Management 2004; 21(4):249-280.
22. Remeikiene R, Gaspareniene L. Evaluation of the impact of the EU structural support on the competitiveness of Lithuanian economics.
Business, Management and Education 2016; 14(2):74-88, p. 78.
23. Hanson WH, Ramani N. Technology forecasting: a hydroelectric company experience. Techno. Management Publication 1988; 1(3):266-270.
24. Saaty TL. Multicriteria Decision Making. Pittsburgh PA: RWS Publications; 1996.
25. Saaty TL. Decision Making for Leaders: The Analytic Hierarchy Process for Decisions in a Complex World. 3rd ed. Pittsburgh PA: RWS
Publications; 1999.
26. Kupraszewicz W, Żółtowski B. The selection of the experts team to diagnose the condition of the machines. Diagnostyka 2002, 26:94-100.
27. Cieślak M, editor. Economic forecasting - methods and applications. (in Polish) Warsaw: PWN; 1997.
28. Wójciak M. Methods for assessing the compatibility of expert opinions for the needs of foresight researches. (in Polish) Economic Studies.
Science materials of University of Economics in Katowice 2015; 220:58-77.
29. Tsyganok VV, Kadenko SV, Andriichuk OV. Significance of expert competence consideration while group decision-making using AHP.
International J of Production Research 2012; 50(17):1-8.
30. Ossadnik W, Schinke S, Kaspar RH. Group Aggregation Techniques for Analytic Hierarchy. Process and Analytic Network Process: A
Comparative Analysis. Group Decis Negot 2016; 25:421.
31. Lin CW, Wang CH. A selection model for auditing software. Industrial Management & Data Systems 2011; 111(5):776–790.
32. Grošelj P, Zadnik S. Acceptable consistency of aggregated comparison matrices in analytic hierarchy process. European J of Operational
Research 2012; 223(2):417–420.
33. Altuzarra A, Moreno-Jiménez JM, Salvador M. Consensus Building in AHP-Group Decision Making: A Bayesian Approach. Operations
Research 2010; 58(6):1755–1773.
34. Basak I. An Alternate Method of Deriving Priorities and Related Inferences for Group Decision Making in Analytic Hierarchy Process. J of
Multi-Criteria Decision Analysis 2011; 18(5/6):279–287.
35. Dong Y, Zhang G, Hong WC, Xu Y. Consensus models for AHP group decision making under row geometric mean prioritization method.
Decision Support Systems 2010; 49(3):281–289.
36. Yang WZ, Ge YH, He JJ, Liu B. Designing a group decision support system under uncertainty using group. Fuzzy analytic network process
(ANP). African J of Business Management 2010; 4(12):2571–2585.
37. Wu W, Kou G, Peng Y, Ergu D. Improved AHP-Group Decision Making for Investment Strategy Selection. Technological and Economic
Development of Economy 2012; 18(2):299–316.
38. Bardossy A, Duckstein L, Bogardi I. Combination of fuzzy numbers representing expert opinions. Fuzzy Sets and Systems 1993; 57:173-181.
39. Hsu HM, Chen CT. Aggregation of fuzzy opinions under group decision making. Fuzzy Sets and Systems 1996; 79:279-285.
40. Jankowski J, Kazienko P, Wątróbski J, Lewandowska A, Ziemba P, Zioło M. Fuzzy multi-objective modeling of effectiveness and user
experience in online advertising. Expert Systems with Applications 2016; 65:315-331.
41. Jankowski J, Kolomvatsos K, Kazienko P, Wątróbski J. Fuzzy Modeling of User Behaviors and Virtual Goods Purchases in Social
Networking Platforms. J of Universal Computer Science 2016; 22(3):416-437.
42. Wątróbski J, Jankowski J, Ziemba P. Multistage Performance Modelling in Digital Marketing Management. Economics & Sociology 2016;
9(2):101.
43. Jankowski J, Ziemba P, Wątróbski J, Kazienko P. Towards the tradeoff between online marketing resources exploitation and the user
experience with the use of eye tracking. In: Nguyen NT, Trawiński B, Fujita H, Hong TP, editors. Intelligent Information and Database
Systems. ACIIDS 2016. Lecture Notes in Computer Science, vol 9621, Berlin Heidelberg: Springer; 2016. p. 330-343.
44. Grošelj P, Pezdevšek Malovrh S, Zadnik Stirn L. Methods based on data envelopment analysis for deriving group priorities in analytic
hierarchy process. Cent Eur J of Operations Research 2011; 19(3):267–284.
45. Hosseinian S, Navidi H, Hajfathaliha A. A New Linear Programming Method for Weights Generation and Group Decision Making in the
Analytic Hierarchy Process. Group Decision and Negotiation 2012; 21(3):233–254.
46. Aull-Hyde R, Erdogan S, Duke JM. An experiment on the consistency of aggregated comparison matrices in AHP. Eur J of Operational
Research 2006; 171(1):290–295.
47. Chen S, Jiang Y, Liu Y, Diao C. Cost Constrained Mediation Model for Analytic Hierarchy Process Negotiated. Decision Making. J of Multi-
Criteria Decision Analysis 2012; 9(1/2):3–13.
48. Kreitner R, Cassidy C. Management. 12th ed. Mason: Cengage Learning; 2011.
49. Gawlik R. Stratification of research target group and selection of experts for AHP based decision making model. (in Polish) Studies and
Materials. Miscellanea Oeconomicae 2016; 20(3/2):193-200.
50. Recommendation of the European Parliament and of the Council of the European Union of 18 December 2006 on competence in the learning
process throughout their lives (2006/962/WE); Official Journal of the European Union 2006; L394/10:10-18, http://eur-lex.europa.eu.