Review

Innovation Metrics: A Critical Review

Lyubomir Todorov * , Margarita Shopova , Iskra Marinova Panteleeva and Lyubomira Todorova

Department of Statistics and Applied Mathematics, Faculty of Economic Accounting, “D. A. Tsenov” Academy of
Economics, 5250 Svishtov, Bulgaria; [email protected] (M.S.); [email protected] (I.M.P.);
[email protected] (L.T.)
* Correspondence: [email protected]

Abstract: Innovations are complex phenomena with important impacts on firms, regions, the economy
as a whole, society, and the environment. Measuring innovation is a challenging and time-consuming
task with many problems ranging from the conceptual framework to data collection and interpretation.
The development of the produced variety of single indicators and multidimensional metrics covers
one or more innovation characteristics—inputs, stages, sources, mechanics, outputs, and impacts.
While the abundance of metrics allowed measurement of many innovation aspects, it also created
problems with comparability, coverage, timeliness, and reliability, making it difficult for academics,
businesses and policymakers to efficiently use the information, perform correct analysis and make
adequate decisions. To address this problem, this article aimed to review the literature, develop
instruments for the structuring and assessment of the innovation measurements, systematize the
variety of metrics, and evaluate their compliance with the requirements of users’ needs and the
quality of statistical information. The literature review identified 23 innovation metrics and helped
create a classification scheme with 11 attributes and a criteria checklist with seven criteria groups.
The results from the application of the instrument for the identified metrics revealed that they could
be divided into three groups: appropriate, needing refinement, and unsuitable, with the best ones
being the European Innovation Scoreboard and Global Innovation Index. They too showed some
data gaps, connected with cultural environment, sustainability, open innovations, structural changes,
and regional development, thus reinforcing the necessity for further advancement of theory and
methodology for innovation measurement to augment the high-quality macro-information that is
readily available with firm-level qualitative data of the innovation at the place where they emerge.
Citation: Todorov, Lyubomir,
Keywords: innovation metrics; innovation measurement; innovation indicators; comparative analysis
Margarita Shopova, Iskra Marinova
Panteleeva, and Lyubomira Todorova.
2024. Innovation Metrics: A Critical
Review. Economies 12: 327. https://
doi.org/10.3390/economies12120327 1. Introduction

Academic Editor: Tsutomu Harada

Innovations are a major factor influencing modern economic development. Propagat-
ing innovation-based economic growth, creating adequate conditions for the development
Received: 29 October 2024 of new products, new processes, and new forms of organization, was established as a
Revised: 22 November 2024 key goal in the European policy (European Commission: Directorate-General for Com-
Accepted: 26 November 2024 munication 2019) on the path of creating an advanced and sustainable knowledge-based
Published: 28 November 2024 economy (European Commission: Directorate-General for Communication 2023). Achiev-
ing such a goal requires effective implementation of the government-introduced policies. It
is connected with the accurate and timely measurement, assessment and monitoring of the
Copyright: © 2024 by the authors.
innovation process, results and consequences for individuals, enterprises, society, and the
Licensee MDPI, Basel, Switzerland. environment (El Bassiti and Ajhoun 2016; Taques et al. 2021).
This article is an open access article Measuring complex and multidimensional phenomena like innovations was always a
distributed under the terms and difficult, time-consuming and challenging task for economists, statisticians and engineers
conditions of the Creative Commons (Viso 2013). The problems varied from simple, precise and widely accepted definitions
Attribution (CC BY) license (https:// (Gault 2018), through broad conceptual models (Björk et al. 2023), to individual indicators
creativecommons.org/licenses/by/ and their responses—both as formulated and perceived (Cirera and Muzi 2020). The
4.0/). challenge has produced many attempts to measure innovation—from simple indicators for

Economies 2024, 12, 327. https://doi.org/10.3390/economies12120327 https://www.mdpi.com/journal/economies

Economies 2024, 12, 327 2 of 28

patents (Griliches 1990; Ponta et al. 2021) or citations (Coombs et al. 1996) to the elaborated
system of the European Innovation Scoreboard (EIS), which covers several areas with many
indicators (European Commission: Directorate-General for Research and Innovation 2024).
The abundance of innovation metrics has ambiguous effects—on one side, it made
it possible to measure different aspects of the innovation phenomena (Erdin and Çağlar
2023; Taques et al. 2021), the innovation process (Viso 2013), and the actors in this process
(Virkkala and Mariussen 2021). On the other side, though, the data comparability, cov-
erage, and quality limited the extent to which the information could be used efficiently
in scientific research and government policy implementation (Brenner and Broekel 2010;
Gericke 2013; Reeb and Zhao 2020). Without high-quality information, the studies of the
innovations and innovation process could lead to biased results and inaccurate conclusions,
limiting, complicating, and even hindering the research in the area; the lack of comparable
data would prevent longitudinal or international comparisons, while different concepts
and methodologies could prevent the pooling of the data or combining the different data
sources. The insufficient and untimely data would restrict the policymakers from react-
ing adequately and in a timely way to the problems that arise in the development of the
innovations, could lead to wrong decisions, contradicting policies, and hamper the inno-
vation management process at the national or transnational level. Policy interventions
could be misdirected or miss important areas that needed support, it would be difficult
to objectively assess the efficiency and efficacy of the implemented measures, and the
complexity and systematicity of the planning process would suffer from the limited data
coverage or lack of representativeness across economic activities or institutional units. The
diverse unstructured data could negatively impact business, making investment decisions
costlier, preventing proper diffusion of knowledge, and increasing the risks of missing
opportunities. All this highlights the importance of high-quality data about innovations
and their impact on economic and social development.
The increasing need for accurate and timely information about the innovation char-
acteristics, suitable for the needs of researchers, business and public administration, and
the limitations of the existing data sources, provokes strong and persistent interest in
the methodological and practical issues of innovation measurement. In this regard, this
article aims to help advance the research and methodology on the subject by reviewing
the scientific development of innovation metrics and drawing conclusions regarding the
systematization of the variety of metrics, their evaluation and the establishment of pos-
sibilities for their complementation in terms of scope (both for the statistical subject and
subject/predicate), and the quality and relevance of the data. The literature review focuses
on studies that were at least partially methodological, suggesting new indicators and
metrics, or modifying and upgrading existing ones.
The paper aims to make contributions in three directions:
First, to identify the main innovation metrics used in practice by official statistical
institutions or researchers, systematizing similar ones and outlining their main attributes
related to data quality and relevance.
Second, to develop a system of criteria that would allow evaluation of the innovation
metrics in terms of their relevance to satisfy the needs of the users for high-quality, timely
and adequate information for studying and managing the innovation processes.
Third, to assess the identified innovation metrics, present their limitations, and identify
the gaps in the data coverage.
Section 2. of the paper is focused on the existing research that presents methodol-
ogy, broad areas for measurement, and concrete indicators of the innovations. Section 3.
discusses the development of the instruments that facilitated the assessment process for
the innovation metrics. Section 4. presents the outcomes of implementing the proposed
instrument for the identified metrics. The main implications and impact of the results are
presented in Section 5., while Section 6. discusses the contributions, practical implications,
limitations, and further research.
Economies 2024, 12, 327 3 of 28

2. Literature Review
Recently, innovations have become a priority for corporate managers, national govern-
ments, and the EU. The use of different metrics not only contributed to the understanding of
the innovation itself but was regarded as a management tool that supported R&D decisions,
provided information on the strengths and weaknesses of the innovation activities, and
assisted in the monitoring and evaluation of institutional policies.
To have a real impact on innovation policies both at the EU level and in individual
Member States, an objective assessment of the state of innovation was needed about
the investments, results, and impact. The measurement of innovation was a subject of
continuous interest among researchers. Scientific publications discussed the evaluation
indicators, the application of different analytic techniques, and the data provided by the
different surveys. The development of the indicators was closely linked with the proper
conceptual frameworks, and the identification of the key areas that describe not just the
characteristics of the innovation object or the corresponding actors, but also the additional
features concerning the efficiency, optimization and sustainability of the innovation process
(Banu 2018).
One of the first innovation measurement attempts was based on the number of patents
(Griliches 1990). The single indicator proved insufficient and inaccurate for the researchers
(Reeb and Zhao 2020). It addressed only one aspect of the outcomes of the innovation
process, while there were other important areas linked with innovation inputs, actors,
impact, and implementation or diffusion mechanics. Its accuracy was also subject to debate,
as not all patents were adopted in practice, and their economic benefits were not considered.
In that regard, McAleer and Slottje (2005) developed an additional indicator—the patent
success ratio (PSR), as a tool to track the success of the innovative activity, and Ponta et al.
(2021) suggested the Innovation Patent Index (IPI) for quantitative insight into the various
aspects of firms’ innovations.
The LBIOI approach was related to the patent indicators. It was based on a literature
review to assess innovation outcomes (Coombs et al. 1996), and allowed the study of
relationships specifically between innovation and performance, which was important for
the promotion of innovation in the public sector (Walker et al. 2002). This approach enabled
the creation of longitudinal data sets, leading to improved communication in sharing good
practice and the use of evidence in public policy, management and research (Van der Panne
2007). It was also subject to various improvements, such as the Technology Impact Factor
(TIF), which is based on the Journal Impact Factor (JIF), used to evaluate journals in terms
of practical innovation in connection with the patents. The JIF factor examines the impact
of journal articles on patents by calculating the number of patents cited in a journal divided
by the number of articles published in that particular journal. The aim is to measure the
impact of academic publications on practical innovation (Huang et al. 2014).
To measure innovation performance, the World Intellectual Property Organization
(WIPO) created the Global Innovation Index (GII). The index included around 80 indicators,
including measures of the political environment, education, infrastructure and knowledge
creation for each economy, and the results are published annually (World Intellectual
Property Organization 2024). Erdin and Çağlar (2023) used the input and output sub-indices
of the GII in their input–output model. They constructed an innovation performance matrix
that presented the relative positions of countries to support the exploration of countries’
strengths, weaknesses, and potential based simultaneously on innovation performance
and productivity.
At the European level, the Summary Innovation Index (SII) has been developed and
implemented (European Commission: Directorate-General for Enterprise and Industry
2008). A tool for its visualization was the European Innovation Scoreboard (EIS), an
initiative of the European Commission. The current version of the SII includes four sub-
indicators, each with eight indicators, equally weighted, that characterize the dimensions
of the innovation process—main drivers, investments, innovation activities and impacts
(European Commission: Directorate-General for Research and Innovation 2024). A variant
Economies 2024, 12, 327 4 of 28

of the EIS was the European Public Sector Innovation Scoreboard or EPSIS (Sandor 2018).
Its creation was aimed at improving the ability to compare the innovation results of the
public sector.
The possibilities for assessing organizational innovation and the problems of using
existing large sample surveys in European countries were analyzed by Armbruster et al.
(2008). They recommended adapting the research methodology to the specifics of organi-
zational innovation. Nǎstase et al. (2009) examined the different options for measuring
innovation and their potential to complement each other for research and policy to stimulate
innovation. Gericke (2013) identified four areas that were important in the management of
innovation in the specific segment of small and medium-sized enterprises: the company’s
innovation strategy; the internal innovation culture; the combination of internal research
and development with open external sources; and the management of innovation cycles.
El Bassiti and Ajhoun (2016) considered the assessment of the entire innovation process
in three dimensions: Scale and Detail (“Innovation Granularity Scales”) related to the
participants, new knowledge and the context of the innovation; Stages and Innovation
Capacity (“Innovation Capability Stages”), related to the successive activities from the idea
to the exploitation of the innovation; and “Innovation Maturity Levels”, characterizing the
management of the innovation process.
To create a conceptual framework for the evaluation of innovations, Gault (2018) sug-
gested using a systemic approach, including a general definition, a typology of innovations,
innovation activities, and the institutional sectors, where innovations were implemented
or their effects were manifested. The unification of definitions was a step towards the
creation of adequate statistical information not only in the business sector but also in other
institutional sectors, which would create conditions for completeness and comparability
of data. Serrano et al. (2017) explored the problem of knowledge complementarity from
different sources for product, process, organizational and commercial innovations. The in-
dicators for innovation activities had three directions: internal (the performance of internal
training and research and development activities), external (the acquisition of material and
immaterial assets of an innovative nature, external training for innovation activities) and
cooperation for the development of innovations.
Cirera and Muzi (2020) analyzed the problem of information quality when conducting
specialized surveys on innovation in enterprises. They found that in many cases of self-
reporting, there was a discrepancy in responses (a bias) to an increase in the level of
innovation caused by questionnaire formulation as well as cognitive problems. Similarly
referring to input and output dimensions Taques et al. (2021) analyzed 26 indicators
of innovation grouped into three areas: input, intermediate, and output. They found
that multidimensional indicators connected in a system gave better information about
innovations than single ones, and were more comprehensive. Grouping the indicators in
three fields: incoming (for the innovation potential); ongoing (on the status of innovation
activities) and outcome (on the benefits of innovation), Björk et al. (2023) analyzed the
measurement of innovation in firms. They conducted 39 studies over six years and based
on this, they derived recommendations for adequately measuring the innovation efficiency
of companies.
Ivanov and Avasilcai (2014) performed a comparative analysis of four models in
measuring innovation: “Balanced Scorecard”, “Malcolm Baldrige”, “Performance Prism”
and “European Foundation for Quality Management”. They proposed a new model,
uniting the characteristics of these models, in five main directions: Strategy, Processes,
Leadership, Competencies, and Organizational Culture, of which Strategy was considered
to be fundamental. Lopes and Farinha (2018) developed a metric of the innovation activity
of innovation and business networks based on the conceptual model “multi-helix ecosystem
for sustainable competitiveness”. The metric consists of four areas: network cooperation,
economy area, social area, and environment. To study the structure and dynamics of
innovation networks and synergistic effects, Virkkala and Mariussen (2021) generated by
factor analysis four indicators: significance, expectations, experience and gap. Li et al.
Economies 2024, 12, 327 5 of 28

(2023) developed a composite index of innovation system development capacity based on

three indicators: coordinated development capacity, evolutionary development capacity,
and sustainable development capacity.
There were also numerous specialized indicators referring to various aspects of en-
terprise management or specific economic sectors. The measurement of innovativeness
in services was carried out through service marks, which were used as an indicator to
establish a relationship between services and innovation (Schmoch 2003). Another way
to measure innovativeness is the pairing approach, which provides information to the
firm based on which it can assess the quality of its innovation and the sustainability of
innovation (Shapiro 2006). TEPSIE was created as a tool for measuring social innovation
(Murray et al. 2010). The model was based on screening at different analytical levels that
help define dimensions and individual indicators. Analytical levels were related to different
stages of the innovation process. By combining different indicators and using multiple
outputs and inputs Pan et al. (2010) measured the differences in the effectiveness of the
National Innovation Systems.
Brenner and Broekel (2010) considered the problems in measuring and benchmarking
innovation activities in a territorial aspect—between individual countries or regions. They
analyzed four approaches: the overall impact of the territorial unit; the attractiveness of
innovation; supporting innovation; and assessment of the impact of specific factors, and
recommended that the measurement be carried out by individual industrial sectors and
branches to adjust for regional characteristics. Camison and Monfort (2012) analyzed the
problems in the individualized evaluation of innovations in the field of tourism. According
to them, there were serious inaccuracies in the evaluation of the innovation activities of
tourism companies, caused by the specific activities of the companies and their heteroge-
neous innovative behavior, which made international comparisons difficult. There was a
need to complement existing studies with data for the technological and organizational
innovation in tourism companies and their specific innovative capabilities.
Viso (2013) considered the possibilities of measuring innovation not from a techno-
logical, but from a social point of view—as an action that takes place in a certain cultural
environment with its specifics. A variety of actions and innovative combinations were
possible, connected with the diversity of innovators in different aspects (professional, reli-
gious, linguistic, of different generations, gender, economic, organizational and product);
the stability of the working environment, and the microclimate (lack of conflicts); the
freedom of thought and decision-making and the availability of free time. The evaluation of
innovation in the software industry was the subject of research by Edison et al. (2013). The
authors combined a literature review with a survey of researchers and software developers
to create a conceptual model of the main elements of innovation: determinants, inputs,
outputs and performance.
Cruz-Cázares et al. (2013) advocated the thesis of the relationship between innovation
and efficiency. They monitored annual changes in technological innovation performance
through the global Malmquist index, examining two inputs and two outputs of the in-
novation process. Problems in measuring innovation diffusion related to scale, time of
diffusion, organizational demographics, publications, and collaboration were analyzed by
Nelson et al. (2014). The authors suggested combining different metrics to achieve better
results in measuring innovation diffusion. Ivan and Despa (2014) suggested the use of tools
based on statistical indicators in measuring innovation in IT management, one of the main
ones being the Innovation Factors Indicator (IFI). IFI was used to measure innovation in IT
project management based on three aspects of software development—planning, research
and development.
The management innovation (MI) measurement tool was used to assess the rela-
tionships between MI and the organization’s performance or its impact on technological
innovativeness, and as a diagnostic tool for defining the management innovativeness of
the company, as well as when comparing it with other organizations operating in the same
industry (Kraśnicka et al. 2017). Iddris (2016) examined the development of innovation
Economies 2024, 12, 327 6 of 28

capacity within supply chains based on a sample study and identified four dimensions of
innovation capacity: idea management, idea implementation, collaboration, and learning.
The conceptual framework for measuring innovation proposed by Janger et al. (2017)
identified structural change and structural refinement as two key dimensions in both man-
ufacturing and services. The modified metric was aimed at a narrowly defined high-tech
understanding of innovation outcomes. Gamito and Madureira (2019) proposed a tool for
measuring innovation introduced by organizations in rural areas—the Rural Innovation
Indicator System (RIIS). This tool compared the performance of different organizations
through a multidimensional set of indicators that monitored the innovative behavior of
the diverse types of rural organizations. RIIS included the three main dimensions re-
lated to the stages of innovation: input resources; processes for achieving innovation; and
innovation outputs.
The measurement of innovation openness was conducted through the ATOM (Ag-
gregated Openness Measurement Technique) approach, which helped to characterize and
measure innovation openness, based on the concepts of knowledge supply (KS) and inno-
vation practice (IP). ATOM enabled the identification and measurement of the criticality of
knowledge supplies, evaluation of the openness of the adopted innovation practices, and
support for subsequent training in the way the project was managed (Bellantuono et al.
2021). Kalapouti et al. (2020) examined the influence of patent applications, the level of
development, the level of employment and the degree of technological diversity. The thesis
of the authors was that the effectiveness of innovation was determined by the ratio of input
and output resources, measured by the costs of research and development activity and
human capital, and the technological knowledge diffusion that comes from the spatial and
technological neighborhood.
Indicators used for other purposes could be adapted to analyze the performance of
micro-level innovations. Such was the Balanced Scorecard (BSC), which was an indicator of
performance and included the measurement of four main aspects of business: learning and
growth, business processes, customers and finances. According to Gama et al. (2007), the
BSC was a tool for measuring business results, but it could not measure their added value.
The authors proposed the use of innovation indicators from the BSC, organized in a system,
with which to measure precisely the added value. A Business Intelligence dashboard could
be used as a visualization tool, which a company could create to present its results. It
was necessary to specify their goals to select appropriate key indicators and collect the
necessary data for them—both from ad hoc surveys and from official sources. Using the
capabilities of Business Intelligence dashboards, Aimiuwu and Bapna (2011) proposed an
innovation index to track innovation capacity, which measured innovation by firm size,
market power, motivation to innovate, availability of resources, as well as decentralization.
The literature review distinguished two important groups of indicators. The first
group covered indicators that were concerned with and interpreted at the micro-level—in
individual enterprises, reflecting the specific conditions for the emergence of innovations
(product, process, organizational and marketing), as well as the innovation activities in
the company that lead to innovations (Aimiuwu and Bapna 2011; Bellantuono et al. 2021;
Gama et al. 2007; OECD/Eurostat 2018). These indicators were related to the place of
innovation origin and reflected in detail the characteristics of the innovation itself, the
participants in the innovation process, and its specific conditions and limitations. They were
connected to technological, technical and administrative processes, the development of
knowledge, and its sharing. Regarding the usefulness of the data, these indicators have the
greatest cognitive importance. They can provide in-depth information about innovations
and innovation activities, about potential and real difficulties, as well as about measures
that would support the accelerated development of innovations and knowledge transfer
(Dziallas and Blind 2019). On the other hand, precisely because of the micro character
of the individual indicators, they were dominated by specifics and concreteness, which
made it difficult to establish general regularities and patterns, sometimes gave misplaced
estimates and inaccuracies, and—due to the lack of unification—restricted implementation
Economies 2024, 12, 327 7 of 28

of comparative and dynamic analysis, planning, and implementation of a targeted policy to

support innovation (Ponta et al. 2021; Taques et al. 2021). Furthermore, low-level indicators
could not capture all the effects of innovation in firms, as they may have an impact on
larger geographic, economic or institutional spheres (Hoelscher et al. 2015).
In contrast, indicators at a high level of aggregation provided a generalized picture
of innovation processes in a particular region, institutional sector or country. They were
suitable for planning and reporting policies and strategies and could be extensively used
for statistical and econometric analysis, as they meet the requirements for large data
samples, information quality, completeness, comparability and (OECD/Eurostat 2018).
However, with them, a large part of the detailed, specialized information was lost, as the
heterogeneous picture was presented in a small number of generalizing characteristics,
which limited the possibilities for in-depth analysis.
Between these two extreme positions, there were a certain number of indicators that
could be evaluated both at the micro and macroeconomic levels. This included indicators
for patents, trademarks and citations, some financial indicators of companies, R&D ex-
penditures, employed persons and their qualifications, and high-tech production. They
provided information on the general state, structure and trends, allowing disaggregation
to enrich the analysis and showing domain-specific regularities (Erdin and Çağlar 2023;
Virkkala and Mariussen 2021). At the same time, some indicators were specific only to
the relevant level of aggregation, group or territory, reflecting its specific features and its
specific contribution to innovation development (Banu 2018; Gamito and Madureira 2019;
Gericke 2013). Sometimes, this contribution could not be established and allocated precisely
in the lower levels and was lost (neutralized) at a higher level of aggregation.
The process of gathering information on innovation at the firm level was accompanied
by problems related to low comparability, especially internationally; narrow coverage of
the population, covering only part of the companies or economic activities; subjectivity of
answers; lack of or limited representation; extended periods of observation and processing
of the results; and limited scope, since the firm is observed, not the innovation (Rammer
and Es-Sadki 2023). Big data offered new possibilities to solve some of those problems and
collect accurate and reliable data about innovations in enterprises (Kinne and Axenbeck
2020). They posed certain advantages over traditional data collection: timeliness, low costs,
better coverage, good accessibility, and flexibility, while at the same time, some limitations
existed, related to information bias, unknown accuracy and consistency, outdated information,
language problems, and the need for interpretation (Rammer and Es-Sadki 2023).
Big data differed from the data collection methods, as they rely on the data generated
for other purposes, and could produce results in a short time. The data are not structured,
and the processing requires new, advanced methods and computational power. Devel-
opment in the field of information technologies and the techniques of data mining, text
mining, machine learning, web crawling, web content analysis, labelling, keyword search
and the application of artificial intelligence facilitated the use of big data as an additional
source of information on a firm’s innovation output. While big data have the potential
to revolutionize data collection not only for innovations but in many areas, and present
opportunities to reach a large segment of the population, they are restricted to the data
published on the internet. They are useful as a supplement to official data, providing
additional aspects and details for innovation at the company level (Gök et al. 2015), and
could be combined with other information in databases (Ashouri et al. 2024) and official
surveys (Daas and van der Doef 2020; Kinne and Lenz 2021), even if some results were
controversial (Nelhans 2020).
Further development of big data potential requires advances in both statistical method-
ologies for data collection and information technologies for processing unstructured data
while showing the potential to enhance firm-level innovation measurement and the de-
velopment of new indicators about knowledge sharing and diffusion. To do so, some
conditions were to be satisfied—established coverage, bias correction, validation/reliability,
Economies 2024, 12, 327 8 of 28

transparency, and comparability (those were named guiding principles by Rammer and
Es-Sadki 2023).
The main conclusion from the literature review was related to the variety of forms of
innovation metrics (a list of metrics is presented in Appendix A). They combined individual,
quantitative, qualitative, absolute and relative indicators, as well as statistical and non-
statistical indicators in a list or connected in a system, with its corresponding structure,
and methodology of weighing and summarizing in a composed indicator. Assessing the
adequacy, applicability and suitability of the diverse and heterogeneous innovation metrics
presented was a complex task. Solving it required, on the one hand, the creation of a
classification scheme for their grouping and summarization, and on the other hand, the
development of a system of criteria through which to establish both the statistical and
cognitive characteristics of individual innovation metrics.

3. Methodology
Information on the individual indicators or groups of indicators developed, proposed
and used in practice, which were called innovation metrics in the study, was collected
through bibliographic research of scientific publications in the specialized Scopus and
Web of Knowledge databases. Those reference databases were chosen, as they cover a
large number of journal titles—Scopus included more than 47,000 in 2024, and Web of
Knowledge included more than 25,000 titles. A sizeable share of those titles were open-
access journals—respectively about 7900 in Scopus and about 6500 in Web of Knowledge.
Papers in both databases underwent double-blind peer review, warranting high-quality
standards. Databases have strong coverage of science, technology, and medicine, with
a wide range of subjects that were closely connected with innovations and innovation
measurement. The search engines allowed for elaborated conditions and abstracts, titles,
author names, keywords, and citations.
The process of publication selection for the study of the innovation metrics was im-
plemented in two steps. First, a basic keyword search was performed (date: 8 April 2024)
that included the “(Innovation OR Innovations) AND (Measurement OR Measure OR Indi-
cators) in (Author Keywords)”. The Web of Knowledge results were restricted by “Open
Access” and “Business Economics” criteria and produced 211 documents. The Scopus
results were restricted by “Open Access” and “Business, Management and Accounting”
criteria, producing 223 documents. The second step was manual screening to eliminate
the duplication of papers that existed in both databases and to outline the studies relevant
for the current research. The papers chosen had to be at least partially methodological
to present new concrete indicators or a broad scheme that could be operationalized with
indicators, or to suggest modifications and refinements in the existing indicators or met-
rics of innovation activities. The papers that covered only empirical analysis of existing
indicators and metrics were excluded from the review, as were the papers that discussed
indicators concerning the technical or technological aspects of particular innovations. The
papers published before 2000 were also excluded with two notable exceptions—a Griliches
(1990) paper for patent indicator, and a Coombs et al. (1996) paper for literature-based
innovation output indicator. The manual screening produced a total of 53 papers for the
literature review. In addition, the latest methodological issues of the World Intellectual
Property Organization for Global Innovation Index (World Intellectual Property Orga-
nization 2024), European Commission for European Innovation Scoreboard (European
Commission: Directorate-General for Research and Innovation 2024), and OECD/Eurostat
Oslo Manual (OECD/Eurostat 2018) were also included in the review, as they presented
important conceptual notes about the innovation measurement process.
The large number of innovation metrics required systematization and structuring
in the first place, and in the second place, to develop evaluation criteria and subsequent
comparative analysis.
Systematization was performed during the literature review of the publications, and
indicators or metrics that have common elements, cover the same aspects of innovation
Economies 2024, 12, 327 9 of 28

or use the same source data were combined as one metric (patent indicators, innovation
management indicators). Structuring of the innovation metrics was achieved by outlining
their important features or attributes. Some classification features could be found in the
Oslo Manual, specifically object/subject approach, qualitative/quantitative measurement;
survey type; data sources and purpose; aggregation; measurement dimensions and hierar-
chy. In addition, three more aspects of innovation metrics were examined in the current
study—data coverage, variable type, and variable scaling.
The measurement object attribute answers the question “What was measured?”, with
possible outcomes being ‘object’ when innovation’s features were observed, and ‘subject’
when actor’s characteristics were measured. The measurement type answers the question
“How was it measured?” with possible outcomes ‘qualitative’, ‘quantitative’ or ‘mixed’.
The survey type determined the data collection target, with ‘census’ used for surveys that
encompassed all units, ‘representative’ for random sample surveys, and ‘case study’ for all
others. Case study survey type includes situations when indicators were developed only for
a limited number of subjects, for a particular innovation type, or selected units in a specific
territory or economic activities. Case studies were thus neither complete nor representative
methods for data collection. The data source types were defined as ‘official’, ‘unofficial’ and
‘mixed’, respectively covering data supplied by official institutions (statistical or others),
by unofficial sources (like big data, or projects), or when indicators used a combination
of both. The data purpose was connected with the sources, as official institutions usually
collect data for administrative needs, while businesses collect data for commercial purposes.
When both were used in indicator construction and evaluation, the ‘mixed’ category was
chosen. The data aggregation attribute was linked with the question “What was the level
of aggregation?”, with two possible outcomes—‘individual’ and ‘summarized’. Individual
indicators were computed at low levels (key performance indicators, R&D expenditures),
while summarized indicators were computed at higher—regional or national—levels of
aggregation. Some of the indicators could be used at both levels, and in the process of
structuring, the metric would be classified based on their implementation—when the
indicators or scheme were developed as a tool for enterprise innovation management, they
were considered individual; when they were developed for the evaluation of the national
innovations, they were considered summarized.
The data coverage feature depended on the inclusion of all subjects in the target
population. When all units were included, data coverage was complete, while when only
part of the units were included, the data coverage was partial. This attribute was connected
but not identical to the survey type, as some census surveys excluded units when defining
their target population (e.g., CIS 2018 excluded small firms with less than 10 employed
persons, following the Commission Regulation No 995/2012).
The types of variables used as indicators were defined as ‘absolute’ when absolute
values were used, ‘relative’ when indicators were evaluated as shares, ratios, or percentages,
and ‘mixed’ when both types were used in the metrics. The scaling of the variables was an
important feature, as the absolute variables often increased with the size of the subject, and
were not suitable for the comparative analysis. The scaled variables were transformed in
some way and were comparable for different subjects, regions or countries.
The measurement dimensions determined whether the indicator assessed only one
aspect of the innovations (‘single’) or many (‘multiple’). When the multiple indicators were
weighted and combined into one summary variable, metrics were considered ‘synthetic’.
The presence of indicator structure was connected to the multiple-type metrics when they
consisted of groups, subgroups and individual indicators. When metrics covered more aspects
but indicators were not structured in some hierarchy, the metrics were classified as ‘list’.
In total, 11 attributes were included in the proposed classification scheme. The general
form of the classification scheme and the individual groups are presented in Appendix B.
The criteria for evaluating the indicators were based only broadly on the six criteria of
the Oslo Manual. Some of those criteria were too abstract or generalized (‘Serve the needs of
actual and potential users’, OECD/Eurostat 2018, p. 215) and needed contextualization related
Economies 2024, 12, 327 10 of 28

to the nature of the innovations, their interaction with the environment, implementation,
diffusion and impact. The importance of theoretical background for innovation data quality
was considered in the manual (OECD/Eurostat 2018, p. 187), with its key role for data
collection—in survey planning, definition of units, scope, variables, and development of
the questionnaire. In the present study, other important theoretical aspects were included
concerning data processing, results interpretation, and connection with innovation practice
and other scientific disciplines, forming their criterial group.
Thus, seven groups of criteria were formed, and in each group, additional oper-
ationalized criteria were established, logically derived from scientific research on the
issues of innovation measurement. The groups were as follows: Relevance, Accuracy,
Reliability, Timeliness, Accessibility and Clarity, Comparability and Coherence, and Theo-
retical/Conceptual Soundness.
The Relevance of the indicators was broadly expressed as the requirement that they suc-
cessfully serve the needs of actual and potential users of innovation data (OECD/Eurostat
2018). This requirement was too abstract, especially considering the numerous directions in
which innovation must be observed and reported (Erdin and Çağlar 2023; Kraśnicka et al.
2017; Pan et al. 2010; Reeb and Zhao 2020; Spircu and Matei 2009; Taques et al. 2021).
Relevance could be interpreted as thematic relevance, i.e., to cover inputs, processes
and mechanisms, outputs, impact, and sources of innovation. In addition, one could
consider target relevance when the purpose of the derived indicators was to carry out a sub-
sequent analysis of factor influences, relationships and dependencies (Roszko-Wójtowicz
and Białek 2016; Walker et al. 2002), for augmenting the measurement of input and output
with an estimate of their ratio, i.e., efficiency (Cruz-Cázares et al. 2013). Thirdly, a specific
relevance could be indicated when separate but important aspects of the innovation activity
were taken into account, such as the achievement of positive results (McAleer and Slottje
2005), sustainability (Shapiro 2006), conditions and cultural features of the environment
(Viso 2013), system connections and learning (Manzini 2015), structural changes and up-
grading (Janger et al. 2017), synergy (Virkkala and Mariussen 2021), and open innovation
(Carrasco-Carvajal et al. 2022).
The Accuracy criterion was related to ensuring an objective and unbiased representa-
tion of innovation phenomena (OECD/Eurostat 2018), achieving a low level of error and
lack of bias, both as overestimation (Sandor 2018) and underestimation (Cirera and Muzi
2020; Taques et al. 2021). The ways of collecting information and the derived indicators
should consider the heterogeneity of the phenomenon or observed population (Camison
and Monfort 2012).
The criterion of Reliability was related to obtaining stable data with little noise
(OECD/Eurostat 2018), identical results in repeated observations as well as in observations of a
different group of units. Data from different sources must complement each other successfully
(Serrano et al. 2017). Samples in partial surveys should be representative (Iddris 2016).
The Timeliness criterion considers the speed of reporting and publication of results,
which increases the possibilities and usefulness of indicators in monitoring and decision-
making (OECD/Eurostat 2018). The simplicity of the computational procedures and the speed
of obtaining primary data also improved the timeliness of information (Ponta et al. 2021).
Accessibility and Clarity were linked to the comprehensibility of indicators. They
should be easy to interpret and have developed metadata and interpretation guidelines
(OECD/Eurostat 2018). An additional benefit would be the availability of ready-made
data from past periods or collected for other purposes, but also suitable for calculating
innovation indicators (Nagaoka et al. 2010).
Comparability and Coherence were related to the possibilities of summarizing individ-
ual indicators into more general ones, as well as the reverse process—their decomposition
by individual characteristics and levels of aggregation (OECD/Eurostat 2018). Important
directions in this regard were the individual or company level (Aimiuwu and Bapna 2011;
Bellantuono et al. 2021; Gama et al. 2007), special target groups such as small and medium
enterprises (Banu 2018; Gericke 2013), or rural firms (Gamito and Madureira 2019). Com-
Economies 2024, 12, 327 11 of 28

parability should be ensured for longitudinal (OECD/Eurostat 2018) and comparative

analysis, including between sectors and regions (Chang et al. 2012; Lopes and Farinha 2018)
or countries, including international comparability.
Conceptual/theoretical soundness was based on the creation of a conceptual model
and the adoption of common, standardized and unambiguous definitions (Edison et al.
2013) to define on the one hand the object of the study—definitions not only of product
innovation but also of the other types: technological, process, organizational innovations.
On the other hand, the entities at the different levels of aggregation must also be clearly
defined. At the micro level, these were the individual enterprises, scientific organizations,
and academic institutions, and at the macro level, it was necessary to adopt the already
created unified definitions of the institutional sectors, to create a connection of the metric
with the indicators of the system of national accounts (Gault 2018).
The system of indicators should be flexible, allowing for the addition of new indicators
(Nǎstase et al. 2009) to reflect new or existing but acquired aspects of innovation and/or
actors. At the same time, the system of indicators must be balanced concerning the
individual measurement directions (Stek and van Geenhuizen 2015), thus giving a correct
assessment of the innovation processes, without shifting the focus on one or another
aspect, just because it was easier to measure or was an agenda of economic policy at a
particular time.
The system of indicators should be aligned with not only theory, but also with the
innovation practice (Hoelscher et al. 2015), thereby accounting for and supporting the
validation of good practices in knowledge sharing and innovation implementation. It is
necessary that it correctly reflects innovations in a regional aspect (Brenner and Broekel
2010), allowing one to distinguish innovations that arose in a certain region from those
diffused in the region, as well as being able to distinguish that part of the beneficial effects
that arise in the region after the innovation, from the overall benefits to the firms expressed
in more regions.
Unification of the operationalization of innovation was needed to eliminate subjective
perception in surveys (Cirera and Muzi 2020). This would facilitate the use of standardized
questionnaires with qualitative indicators in different countries and regions, as they would
be perceived in the same way, and responses would be processed more easily.
Based on the seven criteria groups, the specific requirements for the individual indica-
tors and indicator systems, a criteria checklist was developed (Appendix C), which enabled
an objective assessment of the specific innovation metrics.

4. Results
The performed literature review and the derived list of the metrics presented in the
publications with their main characteristics were subject to classification and assessment.
The classification described the various external features of metrics, i.e., characterized their
formal side. The criteria checklist, on the other hand, described their nature, content and
measurement qualities.
The full classification of the 23 metrics is presented in Appendix D, and the summary is
presented in Table 1. The attribute concerned with the target of measurement (“Measuring
Object/Subject”) showed prevailing metrics that covered both the object (innovation itself)
and subject (innovation actors). Twelve metrics related to only one aspect of the innovation
process—six served to measure the properties of innovations, and six measured the char-
acteristics of the participants. According to the “Qualitative/Quantitative Measurement”
feature, most were metrics (11) that offer only a numerical result, i.e., quantification of in-
novations by calculating some indicator. Verbal or mixed presentation of the measurement
result was found for three and nine metrics, respectively. It can be summarized that most
of the metrics used a quantitative result, which is a prerequisite for comparability over time
and between participants.
Economies 2024, 12, 327 12 of 28

Table 1. Classification Summary 1 .

Classification Attribute Groups Metrics

1.1. Object 6
1. Measuring Object/Subject 1.2. Subject 6
1.3. Mixed 11
2.1. Quantitative 11
2. Qualitative/Quantitative
2.2. Qualitative 3
Measurement
2.3. Mixed 9
3.1. Census Survey 16
3. Survey Type 3.2. Representative Survey -
3.3. Case Survey 7
4.1. Official 8
4. Data Sources 4.2. Unofficial 15
4.3. Mixed -
5.1. Administrative 4
5. Data Purpose 5.2. Commercial 15
5.3. Mixed 4
6.1. Summarized 11
6. Data Aggregation
6.2. Individual 12
7.1. Complete 12
7. Data Coverage
7.2. Partial 11
8.1. Absolute 2
8. Variable Type 8.2. Relative 4
8.3. Mixed 17
9.1. Scaled 6
9. Scaling
9.2. Unscaled 17
10.1. Single 4
10. Measurement Dimensions 10.2. Multiple 13
10.3. Synthetic 6
11.1. Hierarchical 18
11. Indicator Structure
11.2. List 5
1 Authors’ assessment.

In “Survey Type”, the census was prevalent (used by 16 metrics), while in the other
seven, the survey was conducted for specific cases and units. Under “Data Sources”, eight
metrics used official data sources, while data for 15 metrics were collected from unofficial
sources, which might pose an issue for the reliability of the results and the conclusions
that could be drawn from them. The metrics under the attribute “Data Purpose” were
distributed among three groups. For 15 metrics, measurement data were collected for
various business needs, while only four were using data collected for administrative
purposes. In four cases, the data originated from both administrative and business sources.
Most of the data used for the metrics came from business sources. This could affect
both their scientific soundness and could pose a limitation on the conclusions that follow
from the applied measurement to only one economic unit, industry, or region. The “Data
Aggregation” and “Data Coverage” attributes showed a balanced division of the metrics.
Half of them were summarized, and half were using individual data. Half offered full
coverage, while the other half covered part of the target population.
In 17 cases, both absolute and relative values were used under the “Variable Type”
attribute. This meant that by applying the appropriate descriptive variable, the various
aspects of innovation were more fully captured. The application of only absolute or only
relative variables was limited to two, respectively four metrics. For the “Scaling” attribute,
the metrics were not adjusted for population size or properties (17 metrics). Adjusted
variables (per person, per sq. km, in %) were used primarily in the composite indicators
(six metrics). According to the “Measurement Dimensions” feature, the complex nature
of the innovations was captured by a group of indicators, which provided the study of
different innovation aspects (19 metrics), and in six of those, the composed indicator
was provided. Only four metrics measured a single aspect. According to the “Indicator
Economies 2024, 12, 327 13 of 28

Structure”, in 18 metrics, the indicators were characterized by structure and subordination,

i.e., they were organized into a system. Only in five metrics were there individual indicators
or a list of indicators not connected.
The assessment of the metrics was performed via the criteria checklist. The full results
are presented in Appendix E. In Table 2 a summary is shown.

Table 2. Criteria Checklist Summary 1 .

Criteria Performance Score Count

I. Relevance
Served the needs of the users 23
Covered the main characteristics of innovations
13
(inputs, actors, processes, results, impact)
Suitable for factor analysis, and analysis of
18
relationships
Reflected cultural characteristics, system
-
connections and learning
Reflected sustainability 2
Reflected open innovations 2
Reflected efficacy 7
Reflected efficiency 9
Reflected synergy 9
Reflected structural changes and upgrades 3
II. Accuracy
Ensured objectivity and unbiasedness 19
Detected overestimation 2
Detected and corrected bias 1
Reflected heterogeneity 4
III. Reliability
Presented stable results with less noise 11
Suitable for data integration 13
Representative samples 2
IV. Timeliness
Timely calculation and delivery to users 16
Simplicity and speed of measurement 12
V. Accessibility and Clarity
Self-explanatory, had metadata and
9
interpretive guidelines
Data are readily available for the calculation 14
VI. Comparability and Coherence
Can be calculated at different levels (micro,
macro, industries, economic activities, 9
institutional sectors)
Can be calculated for important groups/areas
5
(SMEs, rural areas)
Time Comparability 20
Can be used for international comparisons 13
VII. Theoretical/Conceptual Soundness
Logical relationship/structure of indicators 15
Flexible system for adding new indicators 13
Balanced system in measurement aspects 7
Aligned with theory and innovation practice 11
Considers regional development theory 1
Based on commonly accepted definitions and
15
concepts
Based on the unification of the operationalized
concepts and giving a uniform subjective 6
perception of the issues
1 Authors’ assessment.
Economies 2024, 12, 327 14 of 28

The Relevance of the analyzed metrics revealed that all of them were serving some
purpose for the potential users, as they covered at least one but usually more aspects of
the innovation process. Eighteen metrics were found suitable for analyzing the factors
influencing innovation and the relationships and dependencies due to the prevailing
quantitative nature of the indicators. In about half of the metrics (13), all important topics
were covered—the conditions for implementation as input, the innovation activities as
process, the actors, and the results/impacts as output. Less attention was paid to the specific
aspects reflecting efficiency, effectiveness, and synergistic effects of innovations, while open
innovations, sustainability, cultural elements, and structural changes were neglected.
From the point of the “Accuracy” criterion, it could be noted that most metrics pro-
vided objectivity and validity of the assessment. The detection and correction of over or
underestimation of actual achievement were less presented due to neglect of the possible
heterogeneity in the data. As much as “Reliability” was concerned, only half of the metrics
presented stable results or were suitable for data integration. The representative samples
were used only in official data collection, and only for some of the indicators, rendering the
reliability of others in question, especially when based upon a small number of case studies.
Better was the performance of the metrics in the “Timeliness” criterion, where half of them
were simple and easy to record, while two-thirds were delivered to the users on time.
Most of the metrics did not possess either guides or metadata, but this was partially
compensated for by the data readily available for long periods, or large populations. The
lack of metadata and common methodology reflected the metrics’ performance in the
“Comparability and Coherence” criterion. The time comparability was easiest to achieve
but less than half (nine) allowed for various levels of aggregation, and even less (five) could
focus on important groups like SMEs or rural areas.
With the established logical relationship between the constituent indicators in the
metrics (15) and the availability of a flexible system for adding new indicators (13) that are
characteristics under the “Theoretical Soundness” criterion, researchers received useful
guidance for the development and improvement of the methodologies used. The metrics
were consistent with the theory and the innovation policy implemented at various levels,
and the majority were based on commonly accepted definitions and concepts, which made
them relevant and understandable to a wide range of users and facilitated their easy
implementation. What was lacking, however, was the balancing of the indicator system,
the difficulties in the perception and operationalization, and especially the connection with
regional development.
Based on the criteria results for the analyzed metrics, we made a ranking in descending
order (using the total number of characteristics possessed by each of them). The highest
rank was found to be 26, and it was obtained for the European Innovation Scoreboard.
The second place was occupied by the Global Innovation Index. Given that the maximum
possible score was 32, no metrics offered a complete and comprehensive assessment of
innovation. Based on their scores, we divided all 23 metrics into three groups (Table 3).

Table 3. Innovation Metrics Summary 1 .

Appropriate Needing Refinement Unsuitable

Summary Innovation Index Software Innovation Model Pyramidal Model
Global Innovation Index EPSIS Tri-axial Model
Reduced EIS Model Helix Ecosystem Scoreboard Capacity Index
Structural Features Model Patent Indicators LBIOI
Pairing Approach Networks Structure & Dynamics Model Patent Efficiency Model
Adapted BSC Model TEPSIE Service Marks
Business Intelligence Dashboards NIS Metric Four-Component Model
Innovation management Indicators
AТОМ
1 Authors’ assessment.
Economies 2024, 12, 327 15 of 28

In the first, we put all metrics with a rank above 16, i.e., they corresponded to at least
half of the criteria considered. In the third group, we singled out nine metrics with a rank
of or below 10. All but one of them lacked scores in two or more criteria groups, making
them the least suitable for measuring innovation. The middle group included seven metrics
with a rank between 11 and 15. They required refinement of the characteristics in terms
of accuracy of assessment, accessibility of interpretation and consistency at various levels
or groups. In summary, the official innovation metrics performed better when met with
the criteria requirements, while the specialized or broad models lacked concretization,
theoretical foundation, efficient and reliable data collection and coherence.

5. Discussion
Innovation is among the most important sources of competitive advantage, recognized
by researchers as a powerful means of realizing polyvalent outcomes and multidirectional
benefits. Like the broad discussions on the intrinsic aspects of innovation, the scientific
community and business were concerned with the possibilities of measuring, evaluating
and—on this basis—using them as a benchmark for gaining market position, scope of
impact, tools for subsequent competitive struggle, etc. The literature review showed
sustained interest, decades of effort and numerous attempts to capture the complex and
multivariate nature of innovation; to incorporate certain aspects of it into evaluation systems
of indicators at different levels; to model relationships, dependencies and evaluation
mechanisms, to provide scientific explanations and obtain empirical evidence on why
and how innovation drove the development of economies and contributed to societal
well-being. In just over a century (since 1912), scientific thought had come a long way
from Schumpeter’s creative destructions and the thesis of their growth as ever-expanding
clusters that brought benefits, impacts, and provoked innovations and changes, to the
modern complex, multidimensional evaluation systems of metrics that sought to measure
and prove the importance and power of innovation as an economic category, technological
lever and engine of growth.
Innovations were studied as a process from a temporal point of view, as statistical
phenomena, and in the last decade, attempts have also been made to evaluate them as
a key competence characteristic integrated into the structure of a certain object, system,
or network. Researchers used various metrics to define their performance characteristics,
parameters, properties, components, etc. Their multi-dimensional and complex nature
and wide range of performance indicators presupposed the immense variety of metrics
used. This diversity of metrics was also provoked by different research interests, specific
perspectives, and target orientations. There was a collective understanding that innovations
could not or could hardly be measured and evaluated by one universal indicator (be it
an integrative indicator) since their in-depth evaluation screening covered aspects that
were too diverse (Erdin and Çağlar 2023; Taques et al. 2021). Despite this weakness, their
observation produced objective outcome metrics and provided valuable information on
the potential of innovation output (e.g., patent output and efficiency data) to provide
benefits, impact competitiveness at all hierarchical levels, and indicate the capacity of a
given structure, system, or country to develop on a technological basis. On this foundation,
prognostic assessments could be indicatively made for the possibilities of generating
subsequent innovation and value flows, processes, and trends.
The complexity of innovation processes and the multi-component nature of innova-
tion results led to the complication of the systematization and classification of the ever-
increasing variety of innovation results. To distinguish, measure and report innovations,
not only attributes and features of individual types of innovations were used, but clustering
of innovations and approaches were introduced for their classification (oriented to the indi-
vidual, structure, interactions, and innovation systems), etc. A conceptual and etymological
level of abstraction was reached to denote innovation varieties, i.e., the systematization of
the innovation objects based on a two-level classifier, distinguishing the basic attributes for
classifying the innovation objects and the typological concepts according to those attributes
Economies 2024, 12, 327 16 of 28

(Gault 2018). This made it difficult to develop a universal metrication system, as on the one
hand, single indicators were unreliable, and on the other, it was difficult for the selected
single indicators to distinguish between several types of innovation. Identifiable were the
needs for the identification of common standards and the development of metrics based
on quantitative indicators. In practice, these metrics prevailed, but it was also essential to
assess states, structures, and processes that were difficult to measure quantitatively. In this
case, it was recommended to use or supplement with qualitative indicators (for example,
evaluation of sources of information and innovations, application of specific mechanisms,
consideration of psychological aspects, etc.).
There was a certain contradiction or duality related to the fact that the dominant part
of innovations was carried out at the micro level, the level of a separate entity (enterprise,
organization, institution), to which the relevant evaluation metrics were applied. At the
same time, many of the indicators used in statistical observations referred to macro-level
measurements. Abstractly speaking, regardless of the size/scope of the object within which
the innovations were carried out, the widespread use of the systems approach allowed the
identification of these primary objects as small systems, for which it was appropriate to
apply the relevant system metric. However, a critical point was the correct construction
of the assessment methodologies at a higher hierarchical level (economic sector, region,
country, or a wider system). In the inclusion of the primary data, it was necessary to
structurally and cumulatively achieve reliable generalizing metrified evaluation results
that allowed comparability of state and dynamics on the maximum possible basis. A key
point was also the correct choice of the main thematic structural units, the primary objects,
and the statistical units, for which the opinion and behavior would be analyzed, and the
evaluations would bring useful information with the potential to be used as effectively as
possible at the managerial and prognostic level.
In practice, both simplified models for metrification as well as complex and wide-
ranging systems have been established. They underwent their natural development, given
the need to maintain a certain correspondence with the needs of science and practice for
the maximum degree of measurability, comparability, and permissible scope, to cover the
major areas of evaluation and to give an objective view, explanations, and evidence for
certain statements. Regardless of the efforts to upgrade and enrich, there were also certain
criticisms of their imperfections, limitations, gaps, etc. For example, SII was not a good
enough measure of innovation performance—its value increased even if the innovation
output resulting from additional input was zero (Edquist et al. 2018). Some authors use
the constituent SII indicators, which they analyzed most often through DEA, and found
overestimation as well (Cirera and Muzi 2020; Roszko-Wójtowicz and Białek 2016; Sandor
2018; Taques et al. 2021).
For the modern innovation process, the emphasis was placed on characteristics related
to increased risk and the need for its diversification, the need for significant investments
(of course, depending on the level of innovation radicality) and the efforts of many people.
Since the beginning of the 21st century, the Open era had begun, in which innovations were
increasingly the result of collaboration between different entities, whose formal boundaries
were becoming increasingly “blurred” (Carrasco-Carvajal et al. 2022). Ever-increasing
competition has put significant pressure on the intensification of processes based on the
sharing of time, resources, activities, efforts, and risk. The wide development of information
technologies provided an opportunity to accelerate the network approach’s implementation
(Virkkala and Mariussen 2021). In this context, the interest of researchers in measuring and
evaluating networked innovation processes, the contribution of individual participants and
the mechanisms of interrelationships and interactions that lead to successful innovations
was growing. Specialized software has increasingly been used to help with metrics and
assessments. Processes of including artificial intelligence for their improvement were
also considered.
The strong connection between the several types of participants in innovation pro-
cesses and the open nature of their interactions directed the focus to efforts to identify
Economies 2024, 12, 327 17 of 28

opportunities more clearly and to objectify the degree of suitability of existing metrics
to be used to evaluate innovation networks. Not all could be used; not all that could be
used would be a reliable objective way of measuring and evaluating. However, there
were positive indications that based on existing metrics, and with the help of information
technology and artificial intelligence, improvements could be made, or existing metrics
could be combined in a different way to reach the desired level of reliability, suitability and
objectivity of the metrication and related evaluation processes.

6. Conclusions
The innovation metrics were an important part of monitoring the innovation pro-
cess, policy development, implementation, and assessment, as well as establishing a rich
database for scientific research and business management. To successfully comply with
the needs of all users, innovation metrics have to satisfy certain conditions about their
reliability, accuracy, timeliness, and relevance.
The extensive literature review performed in the paper led to several results concerning
the advancement of scientific research on the topic of innovation measurement. The findings
could be summarized in the following directions. First, the exploration of the scientific
papers found in Scopus and Web of Knowledge allowed the identification of 23 innovation
metrics ranging from broad outlining areas for measurement to the elaborated hierarchical
systems with operationalized concrete variables, and from single indicators that measured
only the output aspect of the innovation process to multidimensional schemes that covered
inputs, outputs, actors, conditions, impacts and interactions with the environment. In the
second place, the literature survey helped to produce two instruments that facilitated the
analysis of the innovation metrics: a classification scheme with 11 attributes that allowed
each metric to be described in detail, and a criteria checklist of seven criteria groups
that allowed metrics to be evaluated versus the needs of the users (relevance, timeliness,
accessibility and clarity, comparability and coherence), and data quality (accuracy, reliability,
theoretical soundness).
The practical implications of the produced instruments were connected with the possi-
bility of other users or researchers applying them when assessing existing or newly created
metrics, including at the stages of their design and development. Such analysis could
help improve the metrics and avoid the deficiencies arising from conceptual problems
or insufficient information for important areas. The performed ranking of the 23 metrics
identified in the literature could be used by policymakers or business managers when
choosing the appropriate indicators to suit their needs, and avoiding metrics that were ei-
ther undeveloped, narrow, unreliable, or untimely. The ranking of the metrics also revealed
several gaps existing in the majority of them, connected with the cultural environment,
sustainability, open innovations, structural changes, and regional development. Those
deficiencies need to be addressed in the future development of methodology and indicators
to enhance the usefulness of the innovation metrics.
When interpreting the results of the study, several limitations should be considered.
The selection of the publications for the literature review was limited by the choice of
databases, keywords, time, open access and subject area. It produces many publications,
yet the narrow search criteria could miss some important research and the corresponding
innovation indicators or metrics. Additional exploration of the publications in other
scientific databases, with more keywords and broader subject areas, is advised to find
more indicators and metrics, as well as the methodological issues especially concerning
possibilities presented by big data, panel surveys, and their combination. At the same time,
the structuring of the innovation metrics was performed from a somewhat statistically
inclined point of view. Other classifications were also possible, from the functional point, or
in connection with the stages of the innovation process. The exploration of the metrics from
different sides would allow a better understanding of the innovation measurement and
enhance the clarity and usefulness of the collected data. The generated criteria checklist
was limited to the features and needs identified in the literature review. More research in
Economies 2024, 12, 327 18 of 28

the area, including pilot surveys with representatives from different institutional sectors,
could bring more light to the potential needs of academics, businesses and policymakers.
The ranking of the 23 identified metrics should also be considered with caution, as it
was performed with the subjective judgment of the authors. More research in the field
of innovation indicators and metrics could help refine the criteria and decrease the level
of subjectivity.
The processing of the metrics through the instruments revealed that the Summary
Innovation Index and Global Innovation Index were the best metrics at the moment,
although they too showed some deficiencies, particularly in the absence of information
about the mechanisms and sources, that is “how innovations happen?”. On the other hand,
the business metrics, like the Adapted Business Scorecard Model or Business Intelligence
Dashboards, presented some individual data about the innovations at the firm level but
lacked the aggregation and representation for the sector, region, or country as a whole. The
solution to this problem is not an easy one and would require more efforts from business,
administration, academic circles and statistical organizations.

Author Contributions: All authors contributed equally to the conception and design of the study. All
authors have read and agreed to the published version of the manuscript.
Funding: This research as well as the APC was funded by the Research Fund of the Ministry of Educa-
tion and Science in Bulgaria, project № KΠ-06-H65/10 “Formal and Informal Innovation Networks”.
Acknowledgments: The authors want to thank two anonymous reviewers for their helpful comments
to significantly improve the manuscript. All remaining shortcomings are the sole responsibility of
the authors.
Conflicts of Interest: The authors declare no conflicts of interest.

Appendix A. List of Identified Innovation Metrics

No Innovation Metrics Reference(s)
1 Pyramidal Model Ivanov and Avasilcai (2014)
2 Tri-axial Model El Bassiti and Ajhoun (2016)
3 Four-Component Model Gericke (2013)
4 Helix Ecosystem Scoreboard Lopes and Farinha (2018)
5 Software Innovation Model Edison et al. (2013)
6 Networks Structure & Dynamics Model Virkkala and Mariussen (2021)
7 Capacity Index Li et al. (2023)
8 LBIOI Walker et al. (2002); Van der Panne (2007)
9 Patent Indicators Reeb and Zhao (2020); Ponta et al. (2021)
10 Innovation Management Indicators Ivan and Despa (2014); Kraśnicka et al. (2017)
11 Service Marks Schmoch (2003)
12 Pairing Approach Shapiro (2006)
13 AТОМ Bellantuono et al. (2021)
14 TEPSIE Murray et al. (2010)
15 Patent Efficiency Model Kalapouti et al. (2020)
Spircu and Matei (2009); European Commission:
16 Summary Innovation Index
Directorate-General for Research and Innovation (2024)
17 Reduced EIS Model Roszko-Wójtowicz and Białek (2016)
18 Structural Features Model Janger et al. (2017)
19 EPSIS Sandor (2018)
Erdin and Çağlar (2023); World Intellectual Property
20 Global Innovation Index
Organization (2024)
21 Adapted BSC Model Gama et al. (2007)
22 Business Intelligence Dashboards Aimiuwu and Bapna (2011)
23 NIS Metric Pan et al. (2010)
Economies 2024, 12, 327 19 of 28

Appendix B. Innovation Metrics Classification Scheme

Classification Attribute Groups
1.1. Object
1. Measuring Object/Subject 1.2. Subject
1.3. Mixed
2.1. Quantitative
2. Qualitative/Quantitative Measurement 2.2. Qualitative
2.3. Mixed
3.1. Census Survey
3. Survey Type 3.2. Representative Survey
3.3. Case Survey
4.1. Official
4. Data Sources 4.2. Unofficial
4.3. Mixed
5.1. Administrative
5. Data Purpose 5.2. Commercial
5.3. Mixed
6.1. Summarized
6. Data Aggregation
6.2. Individual
7.1. Complete
7. Data Coverage
7.2. Partial
8.1. Absolute
8. Variable Type 8.2. Relative
8.3. Mixed
9.1. Scaled
9. Scaling
9.2. Unscaled
10.1. Single
10. Measurement Dimensions 10.2. Multiple
10.3. Synthetic
11.1. Hierarchical
11. Indicator Structure
11.2. List

Appendix C. Innovation Metrics Criteria Checklist

Criteria Performance Score

I. Relevance __/10
Served the needs of the users
(The metric provides concrete indicators and ways to compute their
[ ]

values)
Covered the main characteristics of innovations (inputs, actors,
processes, results, impact)
[ ]

(The metric covers major aspects of innovations)

Suitable for factor analysis and analysis of relationships
(The metric provides concrete indicators; they are quantitative or
[ ]

qualitative, but with variability)

Reflected cultural characteristics, system connections and learning
(The metric provides information about the cultural environment,
[ ]

social capital, knowledge transfer, and connections between innovation

actors)
[ ]

Reflected sustainability
(The metric provides information about sustainability of innovations)
Economies 2024, 12, 327 20 of 28

Reflected open innovations

[ ]
(The metric provides information about the open innovations)
Reflected efficacy
(The metric provides information about the results from its

[ ]
implementation)
Reflected efficiency
(The metric provides information about the relation between

[ ]
innovation inputs and outputs)
Reflected synergy
(The metric provides information about the synergic impact of the
innovations on the economy, society and environment in the region or

[ ]
country)

Reflected structural changes and upgrades

[ ]
(The metric provides information about structural changes or upgrades
after its implementation)
II. Accuracy __/4
Ensured objectivity and unbiasedness

[ ]
(The metric indicators were not subjective, and data are unbiased)
Detected overestimation
(The metric provides mechanism to detect overestimation of innovation

[ ]
data)

Detected and corrected bias

(The metric provides mechanism for detection and correction of the

[ ]
bias in the data)
Reflected heterogeneity
(The metric provides indicators that account for the diversity of subject

[ ]
and object)
III. Reliability
Presented stable results with less noise __/3
(Populations or samples are large, questions are clear and answered

[ ]
without guessing)
Suitable for data integration
(The indicators are additive and could be aggregated or decomposed at
different levels or by characteristics of the subject)
[ ]

Representative samples
(The metric data are collected by random samples or complete census
[ ]

survey)
IV. Timeliness
__/2
Timely calculation and delivery to users
[ ]

(The indicators are computed and delivered to the users quickly, with a
small lag after the end of the observation period)
Simplicity and speed of measurement
(The indicators are computed quickly and easily with simple
[ ]

mathematical operations and without the need for sophisticated

econometric modelling)
__/2
V. Accessibility and Clarity
Self-explanatory, had metadata and interpretive guidelines
[ ]

(The metric is clear and easy to interpret, metadata exist)

Data are readily available for calculation

[ ]

(The information database is available, either collected for the

innovation or for other purpose)
Economies 2024, 12, 327 21 of 28

VI. Comparability and Coherence

__/4
Can be calculated at different levels (micro, macro, industries,

[ ]
economic activities, institutional sectors)
(The metric indicators cover different aggregation levels)
Can be calculated for important groups/areas (SMEs, rural areas)
(The metric indicators could be used to present information about

[ ]
important groups, sub-groups, territories)
Time comparability
(The indicators are time-comparable and could be used in longitudinal

[ ]
analysis)
Can be used for international comparisons
(The metric is methodologically unified, and the data are comparable

[ ]
across different countries)
VII. Theoretical/Conceptual Soundness
__/7
Logical relationship/structure of indicators
(Indicators in the metric are structured in accordance with their relation

[ ]
to the innovation process)
Flexible system for adding new indicators

[ ]
(The metric provides possibilities for the inclusion of new indicators)
Balanced system in measurement aspects
(The different groups of indicators that form the metric are balanced,

[ ]
without emphasizing one aspect over the others)
Aligned with theory and innovation practice
(The metric is consistent with the innovation theory and practice in

[ ]
terms, relations, mechanisms, factors)
Considers regional development theory

[ ]
(The metric is aligned with the theory of regional development)
Based on commonly accepted definitions and concepts
(The metric data are collected with the widely accepted definitions of

[ ]
innovation, innovation type, factors, actors, stages)
Based on the unification of the operationalized concepts and giving a
uniform subjective perception of the issues
[ ]
(The questions are unambiguous and are not dependent on the
respondents’ subjective perceptions of the terms and concepts)

Appendix D. Innovation Metrics Classification Results

Classification
Groups 1* 2 3 4 5 6 7 8 9 10 11 12
Attribute
1.1. Object X X X
1. Measuring
1.2. Subject X X X X X X
Object/Subject
1.3. Mixed X X X
2. Qualita- 2.1. Quantitative X X X
tive/Quantitative 2.2. Qualitative X X
Measurement 2.3. Mixed X X X X X X X
3.1. Census Survey X X X X X X X X
3. Survey Type 3.2. Representative
Survey
3.3. Case Survey X X X X
4.1. Official
4. Data Sources 4.2. Unofficial X X X X X X X X X X X X
4.3. Mixed
Economies 2024, 12, 327 22 of 28

5.1. Administrative
5. Data Purpose 5.2. Commercial X X X X X X X X X X X X
5.3. Mixed
6.1. Summarized X X X
6. Data Aggregation
6.2. Individual X X X X X X X X X
7.1. Complete X X X X
7. Data Coverage
7.2. Partial X X X X X X X X
8.1. Absolute X X
8. Variables Type 8.2. Relative X X
8.3. Mixed X X X X X X X X
9.1. Scaled
9. Scaling
9.2. Unscaled X X X X X X X X X X X X
10.1. Single X X X
10. Measurement
10.2. Multiple X X X X X X X
Dimensions
10.3. Synthetic X X
11.1. Hierarchical X X X X X X X X
11. Indicator Structure
11.2. List X X X X
Classification
Groups 13 14 15 16 17 18 19 20 21 22 23
Attribute
1.1. Object X X X
1. Measuring
1.2. Subject
Object/Subject
1.3. Mixed X X X X X X X X
2. Qualita- 2.1. Quantitative X X X X X X X X
tive/Quantitative 2.2. Qualitative X
Measurement 2.3. Mixed X X
3.1. Census Survey X X X X X X X X
3. Survey Type 3.2. Representative
Survey
3.3. Case Survey X X X
4.1. Official X X X X X X X X
4. Data Sources 4.2. Unofficial X X X
4.3. Mixed
5.1. Administrative X X X X
5. Data Purpose 5.2. Commercial X X X
5.3. Mixed X X X X
6.1. Summarized X X X X X X X X
6. Data Aggregation
6.2. Individual X X X
7.1. Complete X X X X X X X X
7. Data Coverage
7.2. Partial X X X
8.1. Absolute
8. Variable Type 8.2. Relative X X
8.3. Mixed X X X X X X X X X
9.1. Scaled X X X X X X
9. Scaling
9.2. Unscaled X X X X X
10.1. Single X
10. Measurement
10.2. Multiple X X X X X X
Dimensions
10.3. Synthetic X X X X
11.1. Hierarchical X X X X X X X X X X
11. Indicator Structure
11.2. List X
* Numbers correspond to Innovation metrics in Appendix A.
Economies 2024, 12, 327 23 of 28

Appendix E. Innovation Metrics Criteria Checklist Results

Groups 1* 2 3 4 5 6 7 8 9 10 11 12
I. Relevance 3/10 3/10 1/10 3/10 4/10 3/10 3/10 2/10 2/10 3/10 2/10 4/10
Served the needs of the users X X X X X X X X X X X X
Covered the main characteristics of innovations (inputs, actors, processes,
X X X X
results, impact)
Suitable for factor analysis and analysis of relationships X X X X X X X X X X X
Reflected cultural characteristics, system connections and learning
Reflected sustainability X
Reflected open innovations
Reflected efficacy X X X
Reflected efficiency X
Reflected synergy X
Reflected structural changes and upgrades
II. Accuracy 1/4 1/4 1/4 1/4 1/4 -/4 1/4 1/4 1/4 1/4 -/4 1/4
Ensured objectivity and unbiasedness X X X X X X X X X X
Detected overestimation
Detected and corrected bias
Reflected heterogeneity
III. Reliability 1/3 1/3 1/3 1/3 1/3 1/3 1/3 1/3 2/3 -/3 -/3 2/3
Presented stable results with less noise X X X X X X X X X
Suitable for data integration X X X
Representative samples
IV. Timeliness -/2 -/2 -/2 2/2 2/2 -/2 2/2 2/2 2/2 -/2 2/2 2/2
Timely calculation and delivery to users X X X X X X X
Simplicity and speed of measurement X X X X X X X
V. Accessibility and Clarity -/2 -/2 -/2 1/2 1/2 -/2 -/2 2/2 2/2 -/2 1/2 2/2
Self-explanatory, had metadata and interpretive guidelines X X X
Data are readily available for the calculation X X X X X X
VI. Comparability and Coherence 1/4 1/4 2/4 1/4 1/4 2/4 3/4 1/4 3/4 -/4 2/4 4/4
Can be calculated at different levels (micro, macro, industries, economic
X X
activities, institutional sectors)
Can be calculated for important groups/areas (SMEs, rural areas) X X X X X
Time comparability X X X X X X X X X X X
Can be used for international comparisons X X X
Economies 2024, 12, 327 24 of 28

VII. Theoretical/Conceptual Soundness 4/7 4/7 2/7 4/7 5/7 5/7 -/7 1/7 1/7 3/7 1/7 1/7
Logical relationship/structure of indicators X X X X X X
Flexible system for adding new indicators X X X X X X
Balanced system in measurement aspects X X X
Aligned with theory and innovation practice X X X X X
Considers regional development theory
Based on commonly accepted definitions and concepts X X X X X X X X X
Based on the unification of the operationalized concepts and giving a uniform
X X
subjective perception of the issues
Total Score 10/32 10/32 7/32 13/32 15/32 11/32 10/32 10/32 13/32 7/32 8/32 16/32
Groups 13 14 15 16 17 18 19 20 21 22 23
I. Relevance 3/10 4/10 5/10 7/10 7/10 7/10 4/10 5/10 4/10 5/10 2/10
Served the needs of the users X X X X X X X X X X X
Covered the main characteristics of innovations (inputs, actors, processes, results,
X X X X X X X X X
impact)
Suitable for factor analysis and analysis of relationships X X X X X X X
Reflected cultural characteristics, system connections and learning
Reflected sustainability X
Reflected open innovations X X
Reflected efficacy X X X X
Reflected efficiency X X X X X X X X
Reflected synergy X X X X X X X X
Reflected structural changes and upgrades X X X
II. Accuracy -/4 -/4 1/4 2/4 2/4 2/4 2/4 2/4 2/4 2/4 1/4
Ensured objectivity and unbiasedness X X X X X X X X X
Detected overestimation X X
Detected and corrected bias X
Reflected heterogeneity X X X X
III. Reliability 1/3 1/3 -/3 3/3 1/3 1/3 1/3 3/3 1/3 1/3 1/3
Presented stable results with less noise X X
Suitable for data integration X X X X X X X X X X
Representative samples X X
IV. Timeliness -/2 1/2 -/2 2/2 2/2 2/2 2/2 2/2 1/2 1/2 1/2
Timely calculation and delivery to users X X X X X X X X X
Simplicity and speed of measurement X X X X X
Economies 2024, 12, 327 25 of 28

V. Accessibility and Clarity -/2 -/2 -/2 2/2 2/2 2/2 2/2 2/2 2/2 1/2 2/2
Self-explanatory, had metadata and interpretive guidelines X X X X X X X
Data re readily available for the calculation X X X X X X X X
VI. Comparability and Coherence -/4 2/4 3/4 3/4 2/4 2/4 3/4 3/4 3/4 3/4 2/4
Can be calculated at different levels (micro, macro, industries, economic activities,
X X X X X X X
institutional sectors)
Can be calculated for important groups/areas (SMEs, rural areas)
Time comparability X X X X X X X X X
Can be used for international comparisons X X X X X X X X X X
VII. Theoretical/Conceptual Soundness 3/7 3/7 1/7 7/7 4/7 4/7 1/7 6/7 3/7 3/7 2/7
Logical relationship/structure of indicators X X X X X X X X X
Flexible system for adding new indicators X X X X X X X
Balanced system in measurement aspects X X X X
Aligned with theory and innovation practice X X X X X X
Considers regional development theory X
Based on commonly accepted definitions and concepts X X X X X X
Based on the unification of the operationalized concepts and giving a uniform subjective
X X X X
perception of the issues
Total Score 7/32 11/32 10/32 26/32 20/32 20/32 15/32 23/32 16/32 16/32 11/32
* Numbers correspond to Innovation metrics in Appendix A.
Economies 2024, 12, 327 26 of 28

References
Aimiuwu, Ehi, and Sanjay Bapna. 2011. Measuring Innovation Using Business Intelligence Dashboards. Paper presented at 17th
Americas Conference on Information Systems, Detroit, MI, USA, August 4–8; Detroit: AMCIS, pp. 1–9.
Armbruster, Heidi, Andrea Bikfalvi, Steffen Kinkel, and Gunter Lay. 2008. Organizational innovation: The challenge of measuring
non-technical innovation in large-scale surveys. Technovation 28: 644–57. [CrossRef]
Ashouri, Sajad, Arash Hajikhani, Arho Suominen, Lukas Pukelis, and Scott Cunningham. 2024. Measuring digitalization at scale using
web scraped data. Technological Forecasting and Social Change 207: 123618. [CrossRef]
Banu, Geanina. 2018. Measuring innovation using key performance indicators. Procedia Manufacturing 22: 906–11. [CrossRef]
Bellantuono, Nicola, Pierpaolo Pontrandolfo, and Barbara Scozzi. 2021. Measuring the Openness of Innovation. Sustainability 13: 2205.
[CrossRef]
Björk, Jennie, Johan Frishammar, and Louise Sundström. 2023. Measuring Innovation Effectively—Nine Critical Lessons. Research-
Technology Management 66: 17–27. [CrossRef]
Brenner, Thomas, and Tom Broekel. 2010. Methodological Issues in Measuring Innovation Performance of Spatial Units. Industry and
Innovation 18: 7–37. [CrossRef]
Camison, César, and Vicente Monfort. 2012. Measuring Innovation in Tourism from the Schumpeterian and the Dynamic-Capabilities
Perspectives. Tourism Management 33: 776–89. [CrossRef]
Carrasco-Carvajal, Omar, Mauricio Castillo-Vergara, and Domingo Garcia-Perez-de-Lema. 2022. Measuring open innovation in SMEs:
An overview of current research. Review of Managerial Science 17: 397–442. [CrossRef]
Chang, Yuan-Chieh, Ming-Huei Chen, Yuan-Po Lin, and Yu-Shiang Gao. 2012. Measuring Regional Innovation and Entrepreneurship
Capabilities. Journal of the Knowledge Economy 3: 90–108. [CrossRef]
Cirera, Xavier, and Silvia Muzi. 2020. Measuring innovation using firm-level surveys: Evidence from developing countries. Research
Policy 49: 103912. [CrossRef]
Coombs, R., P. Narandren, and A. Richards. 1996. A literature-based innovation output indicator. Research Policy 25: 403–13. [CrossRef]
Cruz-Cázares, Claudio, Cristina Bayona-Sáez, and Teresa García-Marco. 2013. You can’t manage right what you can’t measure well:
Technological innovation efficiency. Research Policy 42: 1239–50. [CrossRef]
Daas, Piet, and Suzanne van der Doef. 2020. Detecting innovative companies via their website. Statistical Journal of the IAOS 36: 1–13.
[CrossRef]
Dziallas, Marisa, and Knut Blind. 2019. Innovation Indicators throughout the Innovation Process: An Extensive Literature Analysis.
Technovation 80–81: 3–19. [CrossRef]
Edison, Henry, Nauman bin Ali, and Richard Torkar. 2013. Towards innovation measurement in the software industry. Journal of
Systems and Software 86: 1390–407. [CrossRef]
Edquist, Charles, Jon M. Zabala-Iturriagagoitia, Javier Barbero, and Jose L. Zofío. 2018. On the meaning of innovation performance: Is
the synthetic indicator of the Innovation Union Scoreboard flawed? Research Evaluation 27: 196–211. [CrossRef]
El Bassiti, Lamyaa, and Rachida Ajhoun. 2016. Towards Innovation Excellence: Why and How to Measure Innovation Performance?
Paper presented at 6th International Conference on Information and Communication Technology for the Muslim World, Jakarta,
Indonesia, November 22–24; pp. 99–104.
Erdin, Ceren, and Mehmet Çağlar. 2023. National innovation efficiency: A DEA-based measurement of OECD countries. International
Journal of Innovation Science 15: 427–56. [CrossRef]
European Commission: Directorate-General for Communication. 2019. Towards a Sustainable Europe by 2030: Reflection Paper.
Publications Office. Available online: https://data.europa.eu/doi/10.2775/676251 (accessed on 1 November 2024).
European Commission: Directorate-General for Communication. 2023. The European Green Deal: Delivering the EU’s 2030 Climate
Targets. Publications Office of the European Union. Available online: https://data.europa.eu/doi/10.2775/783179 (accessed on 1
November 2024).
European Commission: Directorate-General for Enterprise and Industry. 2008. European Innovation Scoreboard 2007: Comparative
Analysis of Innovation Performance. Publications Office. Available online: https://data.europa.eu/doi/10.2769/15023 (accessed
on 1 November 2024).
European Commission: Directorate-General for Research and Innovation. 2024. European Innovation Scoreboard 2024. Publications
Office of the European Union. Available online: https://data.europa.eu/doi/10.2777/779689 (accessed on 1 November 2024).
Gama, Nelson, Miguel Mira da Silva, and José Ataíde. 2007. Innovation Scorecard: A Balanced Scorecard for Measuring the Value
Added by Innovation. In Digital Enterprise Technology. Edited by P. Cunha and P. Maropoulos. Boston: Springer. [CrossRef]
Gamito, Teresa, and Lívia Madureira. 2019. Shedding light on Rural Innovation: Introducing and applying a comprehensive indicator
system. Regional Science Policy & Practice 11: 251–78. [CrossRef]
Gault, Fred. 2018. Defining and measuring innovation in all sectors of the economy. Research Policy 47: 617–22. [CrossRef]
Gericke, Markus. 2013. Performance measurement of innovation. In Finance and the Performance of Firms in Science, Education and
Practice. Mostni: Univerzita Tomáše Bative Zlíně, Fakulta Managementu a Ekonomiky, pp. 243–54.
Gök, Abdullah, Alec Waterworth, and Philip Shapira. 2015. Use of web mining in studying innovation. Scientometrics 102: 653–71.
[CrossRef]
Griliches, Zvi. 1990. Patent Statistics as Economic Indicators: A Survey. Journal of Economic Literature 28: 1661–707. [CrossRef]
Economies 2024, 12, 327 27 of 28

Hoelscher, Michael, Georg Mildenberger, Eva Bund, and Ulrike Gerhard. 2015. A Methodological Framework for Measuring Social
Innovation. Historical Social Research/Historische Sozialforschung 40: 48–78. [CrossRef]
Huang, Mu-Hsuan, Wei-Tzu Huang, and Dar-Zen Chen. 2014. Technological impact factor: An indicator to measure the impact of
academic publications on practical innovation. Journal of Informetrics 8: 241–51. [CrossRef]
Iddris, Faisal. 2016. Measurement of innovation capability in supply chain: An exploratory study. International Journal of Innovation
Science 8: 331–49. [CrossRef]
Ivan, Ion, and Mihai Despa. 2014. Statistical Indicators for Measuring Innovation in IT Project Management. Procedia Economics and
Finance 10: 167–77. [CrossRef]
Ivanov, Cristian-Ionuţ, and Silvia Avasilcai. 2014. Performance Measurement Models: An Analysis for Measuring Innovation Processes
Performance. Procedia-Social and Behavioral Sciences 124: 397–404. [CrossRef]
Janger, Jürgen, Torben Schubert, Petra Andries, Christian Rammer, and Machteld Hoskens. 2017. The EU 2020 innovation indicator: A
step forward in measuring innovation outputs and outcomes? Research Policy 46: 30–42. [CrossRef]
Kalapouti, Kleoniki, Konstantinos Petridis, Chrisovalantis Malesios, and Prasanta Dey. 2020. Measuring efficiency of innovation using
combined Data Envelopment Analysis and Structural Equation Modeling: Empirical study in EU regions. Annals of Operations
Research 294: 297–320. [CrossRef]
Kinne, Jan, and Janna Axenbeck. 2020. Web mining for innovation ecosystem mapping: A framework and a large-scale pilot study.
Scientometrics 125: 2011–41. [CrossRef]
Kinne, Jan, and David Lenz. 2021. Predicting innovative firms using web mining and deep learning. PLoS ONE 16: e0249071. [CrossRef]
[PubMed]
Kraśnicka, Teresa, Wojciech Głód, and Martyna Wronka-Pośpiech. 2017. Management Innovation and Its Measurement. Journal of
Entrepreneurship, Management and Innovation 12: 95–122. [CrossRef]
Li, Mingqiu, Heng Chen, Jinqiu Li, and Yan Li. 2023. Measuring the Development Capability of the Innovation Ecosystem from a
Quadruple Helix Perspective—An Empirical Analysis Based on Panel Data for Chinese High-Tech Industries. Systems 11: 338.
[CrossRef]
Lopes, João, and Luís Farinha. 2018. Measuring the Performance of Innovation and Entrepreneurship Networks. Journal of the
Knowledge Economy 9: 402–23. [CrossRef]
Manzini, Sibusiso. 2015. Measurement of innovation in South Africa: An analysis of survey metrics and recommendations. South
African Journal of Science 111: 8. [CrossRef] [PubMed]
McAleer, Michael, and Daniel Slottje. 2005. A new measure of innovation: The patent success ratio. Scientometrics 63: 421–29. [CrossRef]
Murray, Robin, Julie Caulier-Grice, and Geoff Mulgan. 2010. The Open Book of Social Innovation. NESTA, The Young Founda-
tion. Available online: http://youngfoundation.org/wp-content/uploads/2012/10/The-Open-Book-of-Social-Innovationg.pdf
(accessed on 1 November 2024).
Nagaoka, Sadao, Kazuyuki Motohashi, and Akira Goto. 2010. Patent Statistics as an Innovation Indicator. In Handbook of the Economics
of Innovation. Edited by B. Hall and N. Rosenberg. Amsterdam: North-Holland, vol. 2, pp. 1083–127. [CrossRef]
Nǎstase, Gabriel, Florina Bran, and Adrian Panǎ. 2009. Measurement approaches for innovation. Metalurgia International 14: 114–17.
Nelhans, Gustaf. 2020. Analysis of Company, EU Projects, Policy Documents, Clinical Guideline and Social Media/Media Data
Report, Deliverable 5.2 of the Project “Big Data Approaches for Improved Monitoring of Research and Innovation Performance
and as-Sessment of the Societal Impact in the Health, Demographic Change and Wellbeing Societal Challenge” (Data4Impact).
Available online: https://ec.europa.eu/research/participants/documents/downloadPublic?documentIds=080166e5d1e1ff02&
appId=PPGMS (accessed on 1 November 2024).
Nelson, Andrew, Andrew Earle, Jennifer Howard-Grenville, Julie Haack, and Doug Young. 2014. Do innovation measures actually
measure innovation? Obliteration, symbolic adoption, and other finicky challenges in tracking innovation diffusion. Research
Policy 43: 927–40. [CrossRef]
OECD/Eurostat. 2018. Oslo Manual 2018: Guidelines for Collecting, Reporting and Using Data on Innovation, 4th ed. The Measurement of
Scientific, Technological and Innovation Activities. Luxembourg: OECD Publishing, Paris/Eurostat. [CrossRef]
Pan, Ta-Wei, Shiu-Wan Hung, and Wen-Min Lu. 2010. Dea performance measurement of the national innovation system in Asia and
Europe. Asia-Pacific Journal of Operational Research 27: 369–92. [CrossRef]
Ponta, Linda, Gloria Puliga, and Raffaella Manzini. 2021. A measure of innovation performance: The Innovation Patent Index.
Management Decision 59: 73–98. [CrossRef]
Rammer, Christian, and Nordine Es-Sadki. 2023. Using big data for generating firm-level innova-tion indicators—A literature review.
Technological Forecasting and Social Change 197: 122874. [CrossRef]
Reeb, David, and Wanli Zhao. 2020. Patents Do Not Measure Innovation Success. Critical Finance Review 9: 157–99. [CrossRef]
Roszko-Wójtowicz, Elżbieta, and Jacek Białek. 2016. A multivariate approach in measuring innovation performance. Zbornik Radova
Ekonomskog Fakultet au Rijeci 34: 443–79. [CrossRef]
Sandor, D. 2018. Measuring Public Sector Innovation. Transylvanian Review of Administrative Sciences 14: 125–37. [CrossRef]
Schmoch, Ulrich. 2003. Service marks as novel innovation indicator. Research Evaluation 12: 149–56. [CrossRef]
Serrano, Ana, María López-Fernández, and Gema Piqueres. 2017. Complementarity between innovation knowledge sources: Does the
innovation performance measure matter? Business Research Quarterly 21: 53–67. [CrossRef]
Economies 2024, 12, 327 28 of 28

Shapiro, Amram. 2006. Measuring Innovation: Beyond Revenue from New Products. Research-Technology Management 49: 42–51.
[CrossRef]
Spircu, Liliana, and Monica Matei. 2009. Measuring the Innovation Process. Paper presented at 6th International Conference on the
Management of Technological Changes, Alexandroupolis, Greece, December 15–17; pp. 725–28.
Stek, Pieter, and Marina van Geenhuizen. 2015. Measuring the dynamics of an innovation system using patent data: A case study of
South Korea, 2001–2010. Qualily & Quantity 49: 1325–43. [CrossRef]
Taques, Fernando, Manuel López, Leonardo Basso, and Nelson Areal. 2021. Indicators used to measure service innovation and
manufacturing innovation. Journal of Innovation & Knowledge 6: 11–26. [CrossRef]
Van der Panne, Gerben. 2007. Issues in Measuring Innovation. Scientometrics 71: 495–507. [CrossRef]
Virkkala, Seija, and Åge Mariussen. 2021. Networks of Innovation: Measuring Structure and Dynamics between and within Helices
Regions and Spatial Levels. Empirical Evidence from the Baltic Sea Region. Triple Helix 8: 1–47. [CrossRef]
Viso, Armando. 2013. Innovación y cultura: La medida social de la innovación. Isegoría 48: 185–96. [CrossRef]
Walker, Richard, Emma Jeanes, and Robert Rowlands. 2002. Measuring Innovation—Applying the Literature-Based Innovation Output
Indicator to Public Services. Public Administration 80: 201–14. [CrossRef]
World Intellectual Property Organization. 2024. Global Innovation Index 2024: Unlocking the Promise of Social Entrepreneurship. Geneva:
WIPO. [CrossRef]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.