Available online at www.sciencedirect.

com

RAI Revista de Administração e Inovação 13 (2016) 211–220 http://www.revistas.usp.br/rai

Organizational performance evaluation in intangible criteria: a model based

on knowledge management and innovation management
Deise Graziele Dickel ∗ , Gilnei Luiz de Moura
Universidade Federal de Santa Maria, Santa Maria – UFSM, Brasil
Received 7 January 2016; accepted 6 May 2016
Available online 21 June 2016

Abstract
Today innovation and knowledge management are determining factors for success and continuity of organizations. However, because they are
considered intangibles, their measurement becomes a challenge. Therefore, this study aimed to develop a model to measure organizational
performance with a focus on knowledge management and innovation management. To be able to do that, we used a quantitative research study,
characterized as a multi-case study applied to three companies in the metal-mechanic sector in southern Brazil. The methodology uses the
assumptions of well-known methods such as the Key Performance Indicators, the Swing Weighting and Simple Attribute Rating Technique. With
the results, it could be seen that the proposed model can be an effective tool for assessing organizational performance and that, in its application,
the surveyed organizations could already identify their main weaknesses and use the results reported to improve its management.
© 2016 Departamento de Administração, Faculdade de Economia, Administração e Contabilidade da Universidade de São Paulo – FEA/USP.
Published by Elsevier Editora Ltda. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

Keywords: Knowledge management; Innovation; Competitive advantage; KPIs

Introduction It is precisely this context that this paper seeks to explore.

The era of knowledge as an important transformation of orga-
In a world of constant change and where organizations com- nizations, society and professionals, the management of that
pete with literally everyone in the global network, there are many volume of knowledge following the changes and the importance
studies on how to differentiate amid increasingly constant inno- of innovation as competitive advantage.
vations, increasingly improved techniques and knowledge ever This paper adopts a broader definition of innovation in line
wider. The need for the organizations to adapt grows, given the with studies of Schumpeter (1984), focused not only on the
discontinuities created by the globalization level, high volatil- product, but the phenomenon that goes beyond the dimension of
ity, hyper-competition, demographic changes and explosion technology. Moreover, it is emphasized that this article is geared
of knowledge (Porter, 2009). The media, continuously faster, to the firm, i.e., an internal dimension and not the systemic
changes the business climate and every day it becomes more evi- capacity of an economy\society to innovate.
dent that organizational learning and knowledge management, It is evident that the ability to innovate is considered one
as well as innovation, are prerequisites to face this kind of global of the most important features of competitive organizations.
trend (Easterby-Smith, Burgoyne, & Araujo, 2001; Nonaka & Because of this, the systematic search for radical innovations,
Takeuchi, 2008). i.e. those able to create new markets and provide rapid economic
growth and production expansion and for incremental innova-
tion, identified as continuous improvement processes, to “do
Peer Review under the responsibility of Departamento de Administração, Fac- better what was already being done”, is critical to the survival
uldade de Economia, Administração e Contabilidade da Universidade de São of businesses (Carnongia, Santos, Santos, & Zachiewicz, 2004;
Paulo – FEA/USP.
∗ Corresponding author. Machado, Carvalho, & Heinzmann, 2012).
E-mails: [email protected] (D.G. Dickel), However, how to evaluate whether an organization is or is
[email protected] (G.L. Moura). not competitive and innovative? How to measure the results

http://dx.doi.org/10.1016/j.rai.2016.06.005
1809-2039/© 2016 Departamento de Administração, Faculdade de Economia, Administração e Contabilidade da Universidade de São Paulo – FEA/USP. Published
by Elsevier Editora Ltda. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
212 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220

of the management of its knowledge? Innovation and knowl- their processes to the managers (Parmenter, 2012; Samsonowa,
edge management are now considered intangible assets and, 2012).
therefore, their measurements become a big challenge for orga- Fernandes (2006) highlights an important topic about the per-
nizations. Thus, in view of the presented topic, the objective of formance evaluation, to clarify that the expected results may
this study was to develop a model to measure organizational differ between the various stakeholders in the performance of
performance with a focus on knowledge management and inno- an organization. Notably, the owners seek maximum return on
vation management. Therefore, it was considered necessary to investment (ROI), employees seek maximum payment and cus-
build a measurement tool; to apply the proposed tool to evaluate tomers call for innovative high quality products at the lowest
its effectiveness; to analyze the performance index obtained in price, so the main goal is often a conflict between these groups.
the surveyed organizations; and to compare the results obtained Thus, it is important to outline to whom the performance mea-
from the companies surveyed to identify key areas for perfor- surement system is destined and to which strategic vision it
mance improvement. aligns.
This research is justified by the imminent growth on the Several models are available in the scientific literature related
issue of knowledge management and also the importance of to performance measurement, each one with features that seek
the subject associated with innovation. In the same vein, with to track the rapidly changing global market. This concern was
competition increasing and intensifying the race to get ahead, demonstrated by Neely (2002), which notes the growing expan-
innovation becomes an important strategy for growth and even sion on researching this theme.
survival for organizations. This work is also justified by the con- Amidst all these proposals, a compilation made by
tribution to the business world as it seeks to explain and solve, Neuenfeldt Júnior (2014) presents in a summary form some of
through the scientific method, phenomena that are part of the the models considered most relevant to the performance mea-
daily routine of companies. It is also important to mention the surement as well as their main features, as it can be seen in
subjectivity involved in the constructor of innovation and knowl- Table 1.
edge management because they are intangible. In this line, it is a This list of possibilities, however, should not be understood
very big challenge to measure the performance of organizations as isolated models but as flexible options able to adjust the best
in these respects. This study contributes to a tool that enables possible way to the reality intended to be modeled, leaving to
this measurement. the user of the tool the responsibility to be sensible enough to
The work is divided into four sections besides this introduc- do that, since even the scientific literature does not present a
tion. The second section offers a brief review of the literature consensus of which method is most appropriate.
on the concepts involving knowledge management and innova- Adding to this, publications that are intended to iden-
tion management. In the third, there is a complete explanation tify desirable attributes in performance measurement sys-
of the methodology used for the study. In the fourth, the results tems such as the study by Figueiredo, Macedo-Soares,
achieved by applying the proposed method are reported. The Fuks, and Figueiredo (2005) stand out, which identified the fol-
fifth section seeks to make an overview of the work, ending it lowing nine characteristics based on the analysis of different
with the book references. bibliographic sources: organizational learning; critical analysis;
balancing; clarity; dynamism; integration; alignment; participa-
Systems for organizational performance measurement tion; and causal relationship. The author also lectures on each of
these attributes, in an attempt to guide the reader in the choice
The process of performance measurement is considered one of an evaluation model. Accordingly, Simons (2009) argues on
of the key elements of strategic management, being able to iden- four points of view that should support the construction of a
tify the gap between the current situation of an organization and performance measurement system:
the level of excellence to be considered, by proposing goals that
are aligned with strategic planning and the use of indicators
(Hill & Jones, 2012; Kaplan & Norton, 2008). The proposal of (a) Its function should be to transmit basic information about
using indicators is based on the fact that tangible and intangi- the case either having economic focus or not;
ble factors, such as innovation, can always be measured, as long (b) It must contain routines and standard procedures;
as they use well-defined metrics, routines that operationalize the (c) It should promote cross-checks that allow the systemic view
data collection and standardized measurement scales, translating of the business, not the exact representation of processes’
scattered data into useful information for managing production data.
units (Hubbard, 2009; Olson & Slater, 2002). (d) It should focus on improving the efficiency and effectiveness
Takashina and Flores (1996) say that the use of indicators of processes, directed to the goals.
plays essential role in planning and control activities, since
they enable the establishment of quantifiable goals that help in
anticipating future events and monitoring of current processes, In light of the desirable characteristics for a performance
assisting in decision-making and in the pursuit of operational measurement system, the Key Performance Indicators (KPIs)
excellence. Consequently, the provision of these tools con- stand out. Parmenter (2012) states that there is a general
tributes to both innovation and knowledge managements when misunderstanding about the tool because many organizations
promoting mechanisms that bring back robust information on use measurements that, despite returning valuable information,
 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220 213

Table 1
Methods for performance measurement.
Method Key features

Management by objectives (APO) Effort directed technique through planning and administrative control, in which the goals are set together between
manager and his superior and responsibilities are specified for each position according the expected results.
KPI Tool used to assess the status of certain activity, so that the levels of an organization understand how their jobs affect the
business.
Balanced Scorecard (BSC) Translates the organization’s strategy into a set of measures that carry out the measurement of its performance in order
to achieve the main strategic goals set.
Three Levels of Performance Considers the establishment of three levels (organization, process, and performer) of performance, so that a company or
system can be assessed from the implementation of these vertices.
Mckinsey 7-S Management model developed to understand seven factors considered determining for effective change in an
organization.
Baldrige It aims to provide assistance to companies when it comes to stimulating the improvement of their quality and
productivity by providing the necessary information to reach a high level of qualification of their processes
Quantum Model proposed in order to associate mission, strategy, goals and processes within the organization, working with an
array in three dimensions: quality, cost and time, seeking balance between these.
Performance Prism It is a methodology that aims to integrate the processes in order to create value for the stakeholders in the system,
starting from indicators that refer the status in which the management is.

Source: Adapted from Neuenfeldt Júnior (2014).

cannot be considered KPIs. To define which are and which are services (Friedman, 2005). The world is moving from an
not KPIs, the author frames the indicators under four groups: industrial age, based on natural resources, to an era of knowl-
edge, based on skills, education and research and development.
(a) Key Results Indicators (KRIs) express the performance Knowledge has emerged as a key source of jobs and economic
achieved in a perspective of the Balanced Scorecard or growth in the global economy because it is the basis for innova-
critical success factors; tion (Gulbranson & Audretsch, 2008).
(b) Result Indicators (RIs) express any result achieved; Organizational knowledge provided improvements in the
(c) Performance Indicators (PIs) express what should be done; course of processes, activities, competitiveness and growth of
(d) Key Performance Indicators (KPIs) express what should be organizations. The characterization of knowledge as the most
done to boost performance dramatically. important production asset of the organizations is the central
aspect of the twenty-first century society, standing out from the
As it can be seen, KPIs are a set of special indicators that traditional assets such as hand labor, capital and technology.
reflect in a quantitative and condensed way the performance of Before, the central value was the mass production of goods, val-
a specific sector of the organization as a whole, affecting not ued for its materiality. Today the central position is occupied
just one but multiple perspectives of the BSC or critical suc- by ideas, information and digital codes, valued in its materi-
cess factors (CSF) (Dransfield, Fischer, & Vogel, 1999; Meyer, ality producer of innovation, creativity and service (Nicolás &
2003; Parmenter, 2012; Samsonowa, 2012). Thus, the use of Cerdán, 2012; Zabot & Silva, 2002).
KPI assumes to establish a strategy with a target they want to Although there are numerous approaches to conceptualize
reach (objective KPI), and through it unfold the CSF, where the knowledge management, there is a consensus that it is a struc-
correct identification of these corresponds mainly to the suc- tured approach to creation, codification, use, exchange and
cessful implementation of the methodology (Parmenter, 2012; retention of knowledge to meet the organizational challenges
Samsonowa, 2012). and to create additional value (Rowlei, 2000; Tobin, 1998). Con-
Finally, the use of KPIs as a performance measurement sys- temporary organizations are a result of the knowledge they and
tem can be considered as an updated tool because of the theme other individuals and groups have built in the past and continue
recurrence in scientific works. Recent publications by Janes and to build through the experience they have and the changes that
Faganel (2013), Flipse et al. (2013), Dombrowski, Schmidtchen, occur all the time (Rodan & Galunic, 2004; Zabot & Silva, 2002).
and Ebentreich (2013), May et al. (2014), Sánchez (2014) and Knowledge can provide sustainable advantage, since over
Galar, Berges, Sandborn, and Kumar (2014) are some of the lat- time competitors can usually even the price or quality of the
est examples that address KPIs and show this subject is being products offered by a company. Meanwhile, the company that
widely explored by the scientific community today. is rich in knowledge will be able to reach a new level of quality,
creativity and efficiency. The advantage of knowledge is sus-
Knowledge management tainable because it generates increasing returns by using tools
that competition does not know (Chou, Wang, & Tang, 2015).
Although manufacturing still have fundamental importance The relevance of knowledge as a base for innovation requires
for development, globalization has changed the concept of exploration and interaction of different sources for its achieve-
competitiveness of developed economies, moving away the stan- ment. With all the resources available now and the speed with
dardized manufacturing activities from the knowledge-based how changes are happening, there is a growing demand for
214 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220

combination of sources, information and knowledge, facilitated 2008; Yam et al., 2011; Forsman, 2011; Alves et al., 2011). How-
by the proper management of these resources. This led to a ever, other inputs are still needed to consolidate the concepts of
substantial increase in the degree of interaction between organi- innovation, since this mapping out is very complex and involves
zations (Chen & Fong, 2015). all areas of the organization.
The current meaning of competitiveness covers not only per- The contribution of Schumpeter (1984) proved to be very rich
formance excellence and technical efficiency of companies or in understanding the importance of innovation in organizations.
products, it also covers the ability to develop systematic pro- The author points to a form of holistic innovation in order to
cesses to search for new opportunities and overcome technical determine dimensions for innovation, arguing that it can come
and organizational obstacles via production and application of in the dimension of a new product, a new process, in the search
knowledge. Innovation management seeks to bring together the for new markets, developing new sources of raw materials or
mechanisms and instruments as well as the methodologies and new market structures. He also says that innovation involves
forms of organization that can guarantee the ability to innovate combining different types and parts of knowledge and turning
in organizations based on knowledge acquired inside and outside them into new useful products and services to the market and
the company (Carnongia et al., 2004). society.
However, there is still a lot of difficulty in the adoption of From that, other authors also advocate for innovation mod-
knowledge management practices by organizations and part of els that not only run from the concept of product innovation.
the reason for the possible failure of knowledge management The models by Utterback (1970), Pugh (1991), Thomas (1993)
initiatives is justified by skepticism because of the inability and Levy (1998) emphasize the market as a source of ideas for
to develop metrics to measure the success of these practices. the development of new products and processes. Cooper (1993,
Knowledge management deals with intangible assets and for 1994, 2008), Khurana and Rosenthal (1998), Goffin and Mitchell
that reason it might be difficult to measure the benefits, but (2010), Rozenfeld et al. (2006) and Coral et al. (2008) highlight
management needs to know in depth the benefits in terms of the organizational strategy as driving element to the beginning
added value, derived from knowledge management initiatives of the process, related to organizational strategy. Furthermore,
(Liebowitz, 2013; Poyhonen & Hamalainen, 2001; Roper & models such as by Khurana and Rosenthal (1998), Goffin and
Dundas, 2015). Mitchell (2010) and Bessant et al. (2005) emphasize that the
Mills and Smith (2011) led a study that sought to find a way to strategy must be the guiding principle of the whole process, giv-
measure knowledge management. The study provides evidence ing a systemic meaning to the concept (Silva, Bagno, & Salerno,
linking some knowledge resources to organizational perfor- 2014).
mance and the results show that some knowledge resources In the current business world, innovation as discipline has not
(e.g., organizational structure and the application of knowledge) yet reached the stage of development able to satisfy the need to
are directly related to organizational performance, while oth- innovate. It appears that in many companies where innovation
ers (e.g., technology and conversion of knowledge), although is considered important, the need exceeds the capacity (Bruce
important preconditions for knowledge management, are not & Birchall, 2009; De Bes & Kotler, 2011; Sigala & Chalkiti,
directly related to organizational performance. Those insights 2015). This is due to the innovation process being characterized
can help companies to better target their investments and as discontinuous and irregular, with concentration of innovation
increase the success of their knowledge management initiatives. outbreaks, which will influence differently the various sectors
of the economy in certain periods. In addition to not follow-
Innovation ing a linear pattern, continuous and regular, innovations also
have a considerable degree of uncertainty, since the solution
Industry has gone through periods of intense competitiveness of the problems and consequences of resolutions are a priori
in globalized economy, characterized by increasingly more effi- unknown. It reveals, however, a cumulative basis, given that the
cient and qualified processes and technology, so that the adoption ability of a company to make changes and improvements within
of innovative strategies becomes crucial in the management pro- an established standard is strongly influenced by the character-
cess (Hitt, Ireland, & Hoskisson, 2012; Porter, 2009). Thus, to istics of the technologies used and the experience accumulated
ensure the survival of any business, it is necessary that the activ- in the past, which shows a strong influence of knowledge man-
ities create value, not only within the organization’s boundaries, agement in the process of innovating (Rowlei, 2000; Song, Zhu,
but also in any competitive environment (Certo & Peter, 2005; & Rundquist, 2014).
Di Serio & Vasconcellos, 2009). Clark and Wheelwright (1993) proposed a classic model that
Innovation management is a contribution to the companies for seeks to understand the process of innovation as a key to the
making them more competitive in the market through the use of acquisition, development and application of technology for com-
new concepts or improving existing concepts in the organiza- petitive advantage, the development funnel. The premise of this
tional context. Innovation is no longer a differential but became model is based on the theory of selectivity (Silva et al., 2014),
a determining factor for the continuity of businesses (Wang et al., in which many ideas go through phases of selecting and cut-
2008; Forsman, 2011). ting, and only the most promising ones become products in the
There are still many studies about the innovativeness that aim market.
to develop the concept of innovation itself, to try to identify the In the same line of thought, aiming project selection, the
specific skills needed to make it possible to innovate (Wang et al., model by Chesbroug (2006) arose, highlighting the idea of open
 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220 215

Determination of the Construction of Evaluation criteria of

Bibliographic research
evaluation criteria collection instruments measurement

Definition of individual Calculating weighted

Conclusion Analysis of results
performance weights

Fig. 1. Methodological research steps.

Source: Authors.

organization. This author defended with his model the vision Methodology
that innovation comes not only from people, knowledge, pro-
cesses and internal capabilities of the organization, but also that This research is classified as quantitative as to the nature,
the boundaries must be broken so that the knowledge and oppor- descriptive and exploratory about the objectives and, on the tech-
tunities come to add to the internal process, in the same extent nical procedures, we opted for a multi-case study with three
as it seeks new markets (Friesike, Widenmayer, Gassmann, & companies, all of these large organizations of the mechanical
Schildhauer, 2014). engineering sector of Rio Grande do Sul, located in the Serra
In the paradigm of Open Innovation, organizations become Gaucha, Central and South Regions, now called only A, B and
able to respond quickly and flexibly to changes in the environ- C.
ment, remain competitive and do not lose the market time of For the development of the evaluation of knowledge man-
products and technologies’ life cycle. Cooperation with univer- agement and innovation management factors and organizational
sities, research centers and new entrepreneurs is a great asset performance determination, eight methodological steps were
to improve and expand innovation strategies in a variety of mapped, assessing internal aspects of business management and
organizations (Chesbroug, 2006). also external factors, as shown in Fig. 1.
More focused on the bias of the processes that culminate in Then, we assigned variables that would be able to demon-
innovations, Sawhney, Wolcott, and Arroniz (2006) presented strate, in the end, the degree of innovation performance of each
his model named radar innovation, in which it is understood of the companies, being structured in two depth levels: factors
that the innovation of a business is not only the innovation of a (Fi ) and criteria (Cu ), as shown in Fig. 2. To survey these factors
product, but mainly of its values. Given that, the innovation of a and criteria, the principles defended by authors such as Rodan
business can be divided into four quadrants: product, customer, and Galunic (2004), Zogbi (2008), Freitas Filho (2013), Song
process and place. et al. (2014) were used.

Incentives for the generation of

knowledge
People (C1) Functions oriented for knowledge
management
Recruitment and selection based
on skills
Search engine knowledge
Knowledge management Intranet
(F1) Structure (C2)

Internal expertise available to decision

Knowledge documentation

Processes (C3) Performance measurement

KPI’s

Knowledge transfer between

Organizational sectors
performance
Clear focus on innovation
Organizational
alignment (C4) Well communicated organizational
goals

Financial resources for innovation

Support and organizational
resursos (C5) Adequate infrastructure to generate
Innovation management value
(F2)
Clear process for innovation
Innovation process (C6)
Ideas from various sectors

Behavior and Innovative organizational culture

organizational model (C7)
Partnership with universities

Fig. 2. Hierarchical structure for evaluation of development.

Source: Authors.
216 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220

To get the value that measures the organizational performance criterion of synthesis, it is first necessary to prepare the over-
of each evaluated company, two factors will be used: Knowledge all objective function (BOBJ) in order to be able to express the
Management (F1 ) and Innovation Management (F2 ) that can company’s situation in relation to the context. The same was
be determining factors for the competitiveness of organizations established from a mathematical model provided by Eq. (1),
(De Bes & Kotler, 2011; Terra, 2012). The criteria used are able i=1
to gather the key factors of performance (KPIs) to measure the Fi
VObj = n (1)
results obtained in each of the companies separated by evaluation 2
teams. which is necessary for both checking the condition of the two
Knowledge management variables (F1 ) measure the organi- factors (knowledge management and innovation management),
zation’s ability to use internal and external expertise to improve (Fi ) considered for the measurement of context, as shown in Eq.
its performance. The people criterion (C1 ) addresses how the (2)
organization acts in people management to improve knowledge
i=1
management, the structure criterion (C2 ) measures how the com- Cu
pany invests and uses the necessary infrastructure for knowledge F i = Wi ∗ n → ∀u ⊂ i (2)
NCu
management and the process criterion (C3 ) measures if the com-
pany processes are focused on knowledge management. where Wi is the relevance of each criterion in relation to the
In this same logic, the variables of innovation management whole; NCu is the total amount of characteristic criteria of i and
(F2 ) measure the organization’s ability to turn their efforts Cu , which are the criteria disposed in the second level of the
toward innovation as well as the results it obtains focusing on hierarchical structure, measured from the definitions proposed
that. Organizational alignment criteria were raised (C4 ) in this by Eq. (3)
factor, whose function is to measure if the organization has its n
 KPIi
strategy aligned with innovation objectives; support and organi- C u = Wu ∗ → ∀i ⊂ u (3)
zational resources (C5 ) seeks to measure whether the resources s
i=1
allocated in the organization are sufficient for the innovation
development; innovation process (C6 ) measures whether the Therefore, the determination of dos Cu is directly related to
innovation process is well defined and efficient; and the behav- the result obtained by measuring the KPIi , ∀i ∈ {1, 2, . . ., n},
ior and mental model (C7 ) aims to understand how the company generated according to the metrics established at the time of
culture favors innovation. definition of KPIs and following the math proposal described
Given the seven criteria for modeling, it is possible to deter- by Eq. (4), designed using the α range, based on the Likert,
mine KPIs capable to measure performance as explained in which used a 5-point Likert Scale, starting with a scale with a
Table 2, using the principles espoused by Davenport and Prusak minimum of 1 and a maximum of 5 with intermediate values 2, 3
(2003), Prahalad (2008), Zogbi (2008), Santos, Basso, and and 4 capable to transmit to the interviewee’s opinion regarding
Kinura (2012), Oliveira et al. (2012), and Ferreira (2012). the indicators, ∀e ∈ {1, 2, . . ., n},
Regarding the development of modeling, developed math-
KPIi = βi → βi ∝ α (4)
ematical equations based on multiple criteria methodology to
support the decision, as explained by Gomes and Gomes (2012), being the opinions expressed from the values assumptions in
will be presented to follow. From the approach to the only βe , according to the variation limits proposed by α. As the
Table 2
Description and purpose of KPIs.
KPIi Definition Purpose

KPI1 Incentives for knowledge generation To measure how much incentive the organization provides for people to generate knowledge
KPI2 Roles centered in knowledge management To measure whether the company uses specific roles to handle knowledge management
KPI3 Recruitment and selection based on skills To represent how people’s skills are taken into consideration when hiring
KPI4 Knowledge searching mechanisms To show that the company has mechanisms to search the knowledge generated internally and externally
KPI5 Intranet To check the use of tools such as intranet in the dissemination of knowledge
KPI6 Internal knowledge available for decision To check that the internal knowledge is available and organized to assist managers in decision-making
KPI7 Knowledge documentation To represent the formalization level of knowledge
KPI8 Performance measurement To observe if the company has the habit to measure the results of its actions
KPI9 Knowledge transference between departments To measure the level of knowledge transference between departments and areas
KPI10 Clear focus on innovation To show if the organization has well-defined focus and if it is oriented to innovation
KPI11 Widespread organizational objectives To check how widespread the organization’s objectives are
KPI12 Financial resources for innovation To measure if the financial resources for innovation are sufficient
KPI13 Adequate infrastructure to generate value To measure if the infrastructure for innovation is suitable
KPI14 Clear process for innovation To demonstrate the existence of a clear flow for innovation development
KPI15 Ideas from several departments To check that the ideas of the different teams are moving forward
KPI16 Innovative organizational culture To observe if the organizational culture is focused on innovation
KPI17 Partnership with universities Represent the degree of involvement with universities and research centers

Source: Elaborated by the authors.

 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220 217

determination of weights Wy , according to Eq. (5), it was pro- Table 3

posed to use standard techniques to faithfully represent the Weights assigned by managers.
preference of the decision maker, such as Swing Weighting Factors Company A Company B Company C Total
and Simple Attribute Rating Technique (SMART), based on the F1 50% 35% 45% 43%
assumptions of Guitouni and Martel (1998), Clemen and Reilly F2 50% 65% 55% 57%
(2001) and Poyhonen and Hamalainen (2001), so that the first,
laid down by Edwards (1971), performs this process by the Criteria Company A Company B Company C Total
direct decreasing order of importance for each one of them, C1 10% 15% 25% 17%
for which is usually assigned for the worst placed a value of C2 20% 5% 10% 12%
10 and, from this, values are listed increasingly, according to C3 5% 15% 10% 10%
the degree of discrepancy in the existing behavior between them C4 20% 35% 10% 22%
C5 30% 10% 15% 18%
(Figueira, Greco, & Ehrgott, 2005; Galarza-Molina et al., 2015). C6 5% 10% 10% 8%
As for the technique named Swing Weighting, the logic of C7 10% 10% 20% 13%
value assignment occurs in reverse, starting from the same order-
Source: Elaborated by the authors.
ing system, identifying which has greater relevance, adopting the
value 100 and performing this same process to the remaining as
far as reaching a value able to resume the less relevant item executive directors, also considered in this research as decision
in relation to the hierarchical system level in question, as the makers. Together, they attributed the necessary deliberations for
difference found in each range is set again according to the carrying out the relativization of its values to the factors and crite-
characteristics of both (Gomes & Gomes, 2012), ria. The data obtained through the diagnosis were transferred to
i=1 a database through Microsoft Office Excel® software.
(Ry1 RY 2 )
Wy = n (5)
n
where W is relative to each of the companies to be verified, ∀x ∈ Results and discussions
{1, 2, ..., n} and y = {i, u} related to a generic representation
of the levels considered for the representation of weights of the In order to test the proposed methodology, the research was
elements in the hierarchical structure, which were determined by applied in mechanical engineering companies of Rio Grande
obtaining the values related to the calculation of multi-criteria do Sul, given that in this industry innovation and knowledge
Swing Weights methods (Ry1 ) and SMART (Ry2 ), based on the management have become essential for the competitiveness and
opinion of decision makers, according to Eqs. (6) and (7), also because of this sector’s relevance, which is rather evident
in the economy.
Fy1
Ry1 = y=1 → Fy1 In Table 3, there is the position of managers in relation to
n Fy1 the weights assigned to each of the factors (Fi ) and criteria (Cu )
= 16.67b − 16.67 → Fmax = 100 → Fmin = 1 (6) raised, getting to the objective function. Therefore, the man-
agers observed the proposed variables for the development of
the research, diminishing the importance of each one for the
Fy2 whole in the measurement.
Ry2 = i=1 → Fy2 A sort of balance was evident between the weights of the
n Fy2 two factors, with a slight emphasis on the factor (F2 ), Inno-
= F(y−1)2 + FRy2 ∝ scale factor ∂ → Fmin 2 = 10 vation Management, which can happen because of the greater
(7) dissemination of concepts related to innovation, since knowl-
edge management is still not consolidated as a management
being Fy1 and Fy2 related to proportional scores obtained, b cor- focus of many organizations.
responding to the order of each factor given by the interviewed In the criteria relativization, we can highlight the (C4 ) crite-
and FRy2 directly related to the scale factor ∂, based on the deter- rion, Organizational Alignment, which had the highest relative
mination of values for each of those according to a Likert scale importance in the managers’ opinion, demonstrating a concern
ranging from a difference minimum of 1 to a maximum of 10. about the importance of the strategic issues involving innova-
And to become possible to perform comparisons of the results tion. The criteria (C5 ), Organizational Support and Resources,
in all the methodological steps, it is necessary to set the goal for and (C1 ) also stood out positively from the rest, while crite-
each of these according to the determination of a value that rion (C6 ) had the lowest relevance rate for the performance of
has as a behavior the increasing or the decreasing proportional organizations surveyed.
variation, related to the expected level of rigor to measure that; From the data obtained through the diagnosis applied to the
in this case, it was suggested based on the agreement with the proposed model for the KPIs, it was possible to get the result
company managers as equal to 90%. for each factor. Initially, we observed the performance of each
For the companies’ diagnostic step, a diagnosis instrument company by the evaluated criteria as it can be seen in Table 4,
was applied through a structured closed interview consisting of given the weights for each factor. The criterion (C4 ), Organiza-
17 questions, each one related to the KPIs and held with the tional Alignment, was the one that performed better, pointing
218 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220

Table 4 innovation management factors. The application of this method-

Outcome of the criteria. ology will be able to help company B to find the criteria and
Criteria Company A Company B Company C KPIs with lower rates in order to act on these and thus increase
C1 73.3 80.0 100.0
its performance and become more competitive.
C2 86.7 93.3 86.7 By analyzing the average performance of each researched
C3 73.3 93.3 73.3 factor, it is understood that they are balanced but do not reach
C4 100 100 90 the goal, which can demonstrate that the researched sector still
C5 60 80 100 has to improve its practice and outcomes in the areas of knowl-
C6 90 90 90
C7 80 60 90
edge management and innovation management, as it shows that
company C is above average and ahead of the others.
Source: Elaborated by the authors.
Conclusion
to a strategic mobilization of the surveyed companies regarding
management of innovation. Amid the new economic context that is characterized by
The worst performance was observed in criterion (C7 ), related the increasing competition in various sectors, the changes in
to Behavior and Organizational Model, pointing to a certain customer attitude, society, competitors, employees and other
fragility of the surveyed companies in building corporate culture stakeholders have contributed to increase competitiveness in
focused on innovation and the dissemination of organizational organizations and innovation has become the key factor for com-
knowledge. panies’ survival. Similarly, the management of organizational
Finally, we came to the overall result regarding the perfor- knowledge has proved to be decisive for achieving objectives and
mance for each of the companies surveyed, characterized as competitive advantage, since the knowledge of organizations
objective function, as shown in Table 5. and their experiences have made the decision-making easier,
Observing the results obtained from the application of the precise and assertive.
proposed methodology, it can be observed that among the three In this context, this study showed that it is possible to mea-
organizations surveyed, only company C obtained performance sure aspects taken as intangible, such as innovation management
above the stipulated goal of 90%. In both factors researched, and knowledge management, so that we can know more pre-
company C managed to get positive results, showing effective- cisely on which competitive level the company is, through a
ness in knowledge management practices and commitment to specific methodology that takes into account the key indicators
innovation management, numbers that reflect its competitive of measuring that performance.
position in the market. The application of the proposed method was performed with
Carvalho et al. (2015) observed various industry sectors in the 3 big companies from different regions of the state of Rio
light of the Innovation Radar and found results that placed the Grande do Sul, all belonging to the mechanical engineering
sectors of civil construction, agribusiness and mechanical metal sector. It showed the different levels of importance given by
as less innovative among those surveyed, which may explain the managers for each item of the methodology and also pointing out
fact that only one of the companies surveyed has reached the set the most important index for the criterion (C4 ), Organizational
target. Alignment, whose function is to measure the level of alignment
Company B, despite not having reached the designated goal, between the strategy adopted by the organization dedicated to
shows itself quite close to it, with 88.66% of overall performance innovation.
in the research questions. It is observed that in the innovation It was evidenced by the study that only company C reached
management factor, the company has a higher index than in the performance goal proposed by the methodology, placing
the knowledge management one and almost reaches the goal, itself in front of the others when it comes to innovation man-
demonstrating that small adjustments in the company’s man- agement and knowledge management aspects. The study also
agement can put its performance above the goal. showed that the companies A and B are still on their way to
On the other hand, company A had the lowest overall perfor- achieve the proposed goal and may use the survey results to
mance index, further from the goal, and hence less competitive guide actions to improve their indicators.
than the other researched in the knowledge management and The main limitation of this research is conditional approach
to the management only in the internal and structural levels, not
Table 5 being contemplated, therefore, systemic variables. The absence
Results. of systemic aspects of modeling, such as taxes, laws, culture
and social aspects is due to the fact that these conditions are
Knowledge Innovation Objective Goal
management management function very similar in all companies of the sector researched, with lit-
tle or no distinction between them. In addition, the systemic
Company A 82.55% 82.55% 82.55% 90%
aspects cannot be controlled by the companies, which prevents
Company B 87.61% 89.23% 88.66% 90%
Company C 91.11% 92.72% 92.00% 90% organizations to formulate strategies or earmark resources to
increase innovation and knowledge management in these fac-
Average 87.09% 88.17%
tors, leaving them only monitoring the external situation. Thus,
Source: Elaborated by the authors. the fact that the evaluation has taken place only in the business
 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220 219

and structural levels alleviates the effects of possible economic Dombrowski, U., Schmidtchen, K., & Ebentreich, D. (2013). Bal-
downturns, political or social on the results presented, as these anced Key Performance indicators in product development. Inter-
national Journal of Materials, Mechanics and Manufacturing, 1(1),
occur in the system level.
27–31.
It is necessary to say that the performance measurement Dransfield, S. B., Fischer, N. I., & Vogel, N. J. (1999). Using statistics and statisti-
methodology proposed could be used in future studies and is cal thinking to improve organizational performance. International Statistical
intended to serve as a management tool for companies from the Review, 67(2), 99–150.
mechanical engineering sector as well other sectors in which Easterby-Smith, M., Burgoyne, J., & Araujo, L. (2001). Aprendizagem Organi-
zacional e Organização de Aprendizagem. São Paulo: Atlas.
innovation management and knowledge management are shown Edwards, W. (1971). Social utilities. Engineering Economist, 6, 119–129.
determinants. Fernandes, B. H. R. (2006). Competências e desempenho organizacional: o que
há além do Balanced Scorecard. São Paulo: Saraiva.
Ferreira, E. M. (2012). Diagnóstico para inovação. São Paulo: Qualitymark.
Conflicts of interest Figueira, J., Greco, S., & Ehrgott, M. (2005). Multiple criteria decision analysis:
State of art surveys. New York: Springer.
The authors declare no conflicts of interest. Figueiredo, M. A. D., Macedo-Soares, T. D. L. A., Fuks, S., & Figueiredo, L.
C. (2005). Definição de atributos desejáveis para auxiliar a auto-avaliação
dos novos sistemas de medição de desempenho organizacional. Gestão &
References Produção, 12(2), 305–315.
Flipse, S. M., Sanden, M. C. A., Velden, T., Fortuin, F. T. J. M., Omta, S.
Alves, A. C., Zen, A. C., & Padula, A. D. (2011). Routines, capabilities and inno- W. F., & Osseweijer, P. (2013). Identifying key performance indicators in
vation in the Brazilian wine industry. Journal of Technology Management food technology contract R&D. Journal of Engineering and Technology
and Innovation, 6(2), 128–144. Management, 30, 72–94.
Bessant, J., Lamming, R., Noke, H., & Phillips, W. (2005). Managing innovation Forsman, H. (2011). Innovation capacity and innovation development in small
beyond the steady state. Technovation, 25(12), 1366–1376. http://dx.doi.org/ enterprises. A comparison between the manufacturing and service sectors.
10.1016/j.technovation.2005.04.007 Research Policy, 40(5), 739–750.
Bruce, A., & Birchall, D. (2009). Via expressa para o sucesso em inovação. São Freitas Filho, F. L. (2013). Gestão da inovação: teoria e prática para
Paulo: Bookman. implantação. São Paulo: Atlas.
Carnongia, C., Santos, D. M., Santos, M. M., & Zackiewicz, M. (2004). Fore- Friedman, T. L. (2005). The world is flat: A brief history of the twenty-first
sight, competitive intelligence and knowledge management as innovation century. London: Lane.
management tools. Gestão e Produção, 11(2), 231–238. http://dx.doi.org/ Friesike, S., Widenmayer, B., Gassmann, O., & Schildhauer, T. (2014).
10.1590/S0104-530X2004000200009 Opening science: Towards an agenda of open science in academia
Carvalho, G. D. G., Silva, W. V., Póvoa, A. C. S., & Carvalho, H. G. (2015). and industry. The Journal of Technology Transfer, http://dx.doi.org/
Radar da inovação como ferramenta para o alcance de vantagem competitiva 10.1007/s10961-014-9375-6
para micro e pequenas empresas. Revista de Administração e Inovação-RAI, Galar, D., Berges, L., Sandborn, P., & Kumar, U. (2014). The need for aggregated
São Paulo, 12(4), 162–186. indicators in performance asset management. Maintenance and Reliability,
Certo, S. C., & Peter, J. P. (2005). Administração estratégica: planejamento e 16(1), 120–127.
implementação da estratégia. São Paulo: Makron Book., 320 pp. Galarza-Molina, S. L., Torres, A., Moura, P., & Lara-Borrero, J. (2015). Cride: A
Chen, L., & Fong, P. S. W. (2015). Evaluation of knowledge management perfor- case study in multi-criteria analysis for decision-making support in rainwater
mance: An organic approach. Information & Management, 52(4), 431–453. harvesting. International Journal of Information Technology & Decision
http://dx.doi.org/10.1016/j.im.2015.01.005 Making, 14(01.).
Chesbroug, H. (2006). Open business models: How to thrive in the new innova- Goffin, K., & Mitchell, R. (2010). Innovation management: Strategy and imple-
tion landscape. Boston, Massachusetts: Harvard Business School Press. mentation using the Pentathlon framework (2nd ed.). Basingstoke: Palgrave
Chou, C. H., Wang, Y., & Tang, T. (2015). Exploring the determinants of Macmillan.
knowledge adoption in virtual communities: A social influence perspec- Gomes, C. F., & Gomes, L. F. A. M. (2012). Tomada de decisão gerencial:
tive. International Journal of Information Management, 35(3), 364–376. Enfoque Multicritério (4th ed.). São Paulo: Atlas.
http://dx.doi.org/10.1016/j.ijinfomgt.2015.02.001 Guitouni, A., & Martel, J. M. (1998). Tentative guidelines to help choosing an
Clark, K. B., & Wheelwright, S. C. (1993). Managing new product and processes appropriate MCD-A method. European Journal of Operational Research,
development: Text and cases. New York: Free Press. 109(2), 501–521.
Clemen, R. T., & Reilly, T. (2001). Making hard decisions with decisions tools Gulbranson, C. A., & Audretsch, D. B. (2008). Proof of concept cen-
(2nd ed.). Pacific Grove: Duxbury. ters: Accelerating the commercialization of university innovation. The
Cooper, R. G. (1993). Winning at new products: Accelerating the process from Journal of Technology Transfer, 33, 249–258. http://dx.doi.org/10.1007/
idea to launch (1st ed.). Massachusetts: Perseus Publishing. s10961-008-9086-y
Cooper, R. G. (1994). Third-generation new product processes. Journal of Prod- Hill, C. W., & Jones, G. R. (2012). Strategic management theory: An integrated
uct Innovation Management, 11(1), 3–14. approach. Independence: Cengage Learning., 560 pp.
Cooper, R. G. (2008). Perspective: The Stage-Gate (R) idea-to-launch process- Hitt, M. A., Ireland, R. D., & Hoskisson, R. E. (2012). Strategic management:
update, what’s new, and NexGen systems. Journal of Product Innovation Competitiveness and globalization. Independence: Cengage learning., 472
Management, 25(3), 213–232. pp.
Coral, E., et al. (2008). Visão geral da metodologia NUGIN. In E. Coral, A. Hubbard, G. (2009). Measuring organizational performance: Beyond the triple
Ogliari, & A. F. Abreu (Eds.), Gestão Integrada da Inovação: Estratégia, bottom line. Business Strategy and the Environment, 18(3), 177–191.
Organização e Desenvolvimento de Produtos. São Paulo: Atlas. Janes, A., & Faganel, A. (2013). Instruments and methods for the integration
Davenport, T. H., & Prusak, L. (2003). Conhecimento Empresarial: como as of company’s strategic goals and key performance indicators. Kybernetes,
organizações gerenciam o seu capital intelectual. Rio de Janeiro: Elsevier. 42(6), 928–942.
De Bes, F. T., & Kotler, P. (2011). A bíblia da inovação: princípios fundamentais Kaplan, R. S., & Norton, D. P. (2008). The execution premium. Harvard Business
pata levar a cultura da inovação contínua às organizações. São Paulo: Lua School.
de papel. Khurana, A., & Rosenthal, S. R. (1998). Towards holistic “front ends” in new
Di Serio, L. C., & Vasconcellos, M. A. (2009). Estratégia e competitividade product development. Journal of Product Innovation Management, 15(1),
empresarial: inovação e criação de valor. São Paulo: Saraiva., 364 pp. 57–74.
220 D.G. Dickel, G.L. Moura / RAI Revista de Administração e Inovação 13 (2016) 211–220

Levy, N. S. (1998). Managing high technology and innovation. New Jersey: Rowlei, J. (2000). From learning organisation to knowledge entrepreneur. Jour-
Pearson Education. nal of Knowledge Management, 4(1), 7–15.
Liebowitz, J. (2013). Developing metrics for determining knowledge manage- Rozenfeld, H., Forcellini, F. A., Amaral, D. C., Toledo, J. C., Silva, S. L.,
ment success: A fuzzy logic approach. Information System Journal, 6(2), Alliprandini, D. H., et al. (2006). Gestão de desenvolvimento de produtos:
36–42. uma referência para a melhoria do processo. São Paulo: Saraiva.
Machado, D. P. N., Carvalho, L. C., & Heinzmann, L. M. (2012). Ambi- Samsonowa, T. (2012). Industrial research performance management: Key Per-
ente favorável ao desenvolvimento de inovações e cultura organizacional: formance Indicators in the ICT industry. Berlin: Physica-Verlag.
integração de duas perspectivas de análise. Revista de Administração (São Sánchez, M. A. (2014). Integrating sustainability issues into project manage-
Paulo), 47(4). São Paulo. ment. Journal of Cleaner Production, no prelo.
May, A., Anslow, A., Wu, Y., Ojiako, U., Chipulu, M., & Marshall, A. (2014). Santos, D. F. L., Basso, L. F. C., & Kinura, H. (2012). A estrutura da capacidade
Prioritization of performance indicators in air cargo demand management: de inovar das empresas brasileiras: uma proposta de construto. Revista de
An insight from industry. Supply Chain Management, 19(1), 108–113. Administração e Inovação. São Paulo, 9(3), 103–128.
Meyer, M. W. (2003). Rethinking performance measurement: Beyond the bal- Sawhney, M., Wolcott, R. C., & Arroniz, I. (2006). The 12 different ways for
anced scorecard. Cambridge: Cambridge University Press., 220 pp. companies to innovate. MIT Sloan Management Review, 47(3), 75–81.
Mills, A. M., & Smith, T. A. (2011). Knowledge management and organizational Schumpeter, J. A. (1984). The theory of economic development: An inquiry into
performance: A decomposed view. Journal of Knowledge Management, profits, capital, credit, interest, and the business cycle. New York: Oxford
15(1), 156–171. http://dx.doi.org/10.1108/13673271111108756 University Press.
Neely, A. (2002). Business performance measurement: Theory and practice. Sigala, M., & Chalkiti, K. (2015). Knowledge management, social media and
Cambridge: Cambridge University Press., 369 pp. employee creativity. International Journal of Hospitality Management, 45,
Neuenfeldt Júnior, A. L. (2014). Modelagem para a mensuração de desem- 44–58.
penho dos sistemas BRT no Brasil. Dissertação (Mestrado em Engenharia Silva, D. O., Bagno, R. B., & Salerno, M. S. (2014). Modelos para a gestão da
de Produção). Santa Maria: Universidade Federal de Santa Maria. inovação: revisão e análise da literatura. Production, 24(2), 477–490.
Nicolás, C. L., & Cerdán, A. L. M. (2012). Strategic knowledge management, Simons, R. (2009). Performance measurement & control systems for implemen-
innovation and performance. International Journal of Information Manage- ting strategy: Text and cases. New Jersey: Prentice Hall., 792 pp.
ment, 31(6), 502–509. http://dx.doi.org/10.1016/j.ijinfomgt.2011.02.003 Song, N., Zhu, J., & Rundquist, E. J. (2014). Knowledge Transfer Mechanisms
Nonaka, I., & Takeuchi, E. H. (2008). Gestão do conhecimento. São Paulo: and Global R&D Operations in MNCs. International Journal of Innovation
Bookman. and Technology Management, 11(06).
Oliveira, H. P., Gonçalves, C. A., Paula, E. A. M., & Santos, K. A. (2012). Takashina, N. T., & Flores, M. C. X. (1996). Indicadores da qualidade e do
Gestão do conhecimento orientada para a estratégia de inovação de produtos desempenho: como estabelecer metas e medir resultados. Rio de Janeiro:
tecnológicos: o caso da Invent Vision. Revista de Administração e Inovação, Qualitymark.
São Paulo, 9(4), 153–176. Terra, J. C. (2012). 10 Dimensões da Gestão da Inovação. São Paulo: Elsevier.
Olson, E. M., & Slater, S. F. (2002). The balanced scorecard, competitive strat- Thomas, R. J. (1993). New Product Development: Managing and forecasting
egy, and performance. Business Horizons. for strategic success. New York: John Wiley & Sons.
Parmenter, D. (2012). Key performance indicators for government and nonprofit Tobin, D. R. (1998). Networking your knowledge. Management Review, 46–48.
agencies: Implementing winning KPIs. New Jersey: John Wiley and Sons Utterback, J. M. (1970). Process of innovation – A study of origination and
Inc. development of ideas for new scientific instruments. IEEE Transactions on
Porter, M. E. (2009). Competição on competition: estratégias competitivas Aerospace and Electronic Systems, Aes6(5).
essenciais. Rio de Janeiro: Campus Elsevier. Wang, C. H., Lu, L. Y., & Chen, C. B. (2008). Evaluating firm technological
Poyhonen, M., & Hamalainen, R. P. (2001). On the convergence of multiattribute innovation capability under uncertainty. Technovation, 28, 349–363.
weighting methods. European Journal Operational Research, 129, 569–585. Yam, R. C. M., Lo, W., Tang, E. P. Y., & Lau, A. K. W. (2011). Analysis of sources
Prahalad, C. K. (2008). A Nova era da Inovação (1st ed.). São Paulo: Elsevier. of innovation, technological innovation capabilities, and performance: An
Pugh, S. (1991). Total design: Integrated methods for successful product engi- empirical study of Hong Kong manufacturing industries. Research Policy,
neering. Harlow: Addison Wesley. 40, 391–402.
Rodan, S., & Galunic, C. (2004). More than network structure: How knowledge Zabot, J. B. M., & Silva, L. C. M. (2002). Gestão do Conhecimento. São Paulo:
diversifies influences managerial performance and innovativeness. Strategic Atlas.
Management Journal, 25(6), 541–562. Zogbi Edison. (2008). Competitividade através da gestão da inovação. São
Roper, S., & Dundas, N. H. (2015). Knowledge stocks, knowledge Paulo: Atlas.
flows and innovation: Evidence from matched patents and innovation
panel data. Research Policy, 44(7), 1327–1340. http://dx.doi.org/10.1016/
j.respol.2015.03.003