Humanities and Social Sciences

2024; 12(1): 1-12

http://www.sciencepublishinggroup.com/j/hss
doi: 10.11648/j.hss.20241201.11
ISSN: 2330-8176 (Print); ISSN: 2330-8184 (Online)

Training or Desactivación? Measuring Training

Effectiveness Evidence from Ethiopian Ministry of
Revenues
Moges Mengstu Kassaw
Department of Research and Development, Ministry of Revenues, Addis Ababa, Ethiopian

Email address:

To cite this article:

Moges Mengstu Kassaw. (2024). Training or Desactivación? Measuring Training Effectiveness Evidence from Ethiopian Ministry of
Revenues. Humanities and Social Sciences, 12(1), 1-12. https://doi.org/10.11648/j.hss.20241201.11

Received: November 23, 2023; Accepted: December 26, 2023; Published: January 23, 2024

Abstract: If employees of the tax office receive inadequate training to fill knowledge and skill gaps, it will have a serious of
negative impact on revenue collection. As a result, in addition to assessing the potential benefits and difficulties of the training,
the tax office should be provide the training by identifying the gaps in knowledge, attitudes and skills among the employees of
the tax office. This study attempts to investigate the effectiveness of the training provided by the Ministry of Revenues
Training Center as well as to identify the gaps seen in the training and to indicate the solutions to solve them. The research was
done through three Training Effectiveness (TREF) measures including satisfaction, learning performance, and behavioral
change measures. Data were collected from employees who took training in 2021 through structured questioners and it was
analyzed using structural equation modeling (SEM), confirmatory factor analysis (CFA). The study suggests that the were not
given enough time to impart the necessary knowledge and skills; consequently, the institution ought to devote sufficient time
and the training center must ensure that the training documents and modules it ought to be prepared in a way that can be
utilized for trainees working after the preparation and the training place must be clear and contented. The TREF developed in
this study can be utilized as a tool to evaluate the effectiveness of private and public sector training programs.
Keywords: Training Effectiveness, Reaction, Learning, Behavior, Structural Equation Modeling, CFA

boosts the motivation, self-assurance, and morale of

1. Introduction employees. 2) Individuals are able to reduce waste, which
The process of learning specific industry techniques is reduces production costs. 3) It fosters a sense of safety,
called training. It is the most important part of growing the which in turn lowers employee turnover and absenteeism. 4)
business, and it helps the employees as well as the business By providing employees with the skills they need to adapt to
grow and produce better products. An employee's skills, new and challenging circumstances, it increases their
mindset, and behavior are systematically altered through involvement in the change process. 5) It opens the door to
training in order to perform a specific job [6]. promotion, recognition, and increased pay. 6) It contributes
Training entails teaching a particular behavior or skill. to the organization's improvement in staff availability and
When employees in a company need or want to learn new quality. It is important to remember that training and
skills, they receive training [19]. development programs improve people's skills and abilities,
One of the most important ways to help people and which leads to increased productivity [4].
businesses achieve their short- and long-term goals and Human resources play a significant role among the
objectives is through employee development and training. In resources required to achieve the institution's purpose. The
addition to enhancing knowledge, skills, and attitudes, institution should, to the greatest extent possible, provide the
training and development offers a number of other employee with task-oriented training, a pleasant working
advantages. According to [22], the following are typical environment, and various benefits to enable the workforce to
advantages of employee training and development: 1) it be effective. From this vantage point, it can be seen that
2 Moges Mengstu Kassaw: Training or Desactivación? Measuring Training Effectiveness Evidence from
Ethiopian Ministry of Revenues

organizations use a variety of research methods to determine employees will gain specific knowledge and be able to train
the degree to which their employees are satisfied with their skills that can later be used in the workplace if they receive
work and implement continuous improvement strategies to training.
find and keep skilled, competitive, and professional Training has become one of the crucial strategies to
employees. improve the performance of the workforce. Improved
Inland Revenue contributes significantly to the overall capabilities, knowledge, and skills of the talented workforce
economic growth of a country; and so, the government is have proven to be a major source of capacity building to
making a lot of efforts to continue the rapid development of human resources for organizational productivity and
the country. Therefore, the Ministry of Revenue of Ethiopia, competitive advantage in a global market, [20].
to collect the national revenue required by the government Helping employees feel satisfied and productive at work is
and to prevent illegal trade, provides continuous training and one of the most important tasks any employer must perform.
various lessons to the employees. Employers are bound by the efficiency and quality of the
In order to collect the national revenue required by the employee's performance to achieve organizational goals;
government and to prevent illegal trade, the Ministry of therefore, it is expected employers to meet various needs and
Revenue provides its employees with ongoing training and create motivation for employees to feel satisfied with their
special education related to taxation and tax policy. work. [25]
One employee is estimated to collect an average of two Training is provided for four basic reasons, first new
hundred fifty five thousand seven hundred birr (255,707) per employees joining institutions or organizations are given
day when we divide the institution's daily goal of collecting training. The training provided to these employees is aimed
one billion one hundred and fifty million birr (1,150,684,931) at making employees familiar with the institution's mission,
by the total number of employees. vision, rules and regulations, and the institution's operating
However, revenue collection will be seriously impacted if systems. Secondly, existing employees will be trained to
employees receive inadequate training to fill their knowledge refresh and enhance their knowledge. Thirdly, as and when
and skill gaps. Since, it is essential to conduct this survey any improvements in technology are made, employees are
because it is necessary to know that the institution's training trained to be aware of and adapt to these technological
helps to collect income and builds employees' capacity. improvements. On the last, when promotion and career
This study attempts to investigate the effectiveness of the development are necessary, training is provided to prepare
training provided by the Ministry of Revenues Tax Center as employees for higher job responsibilities and leadership.
well as to identify the gaps seen in the training and to
indicate the solutions to solve them. And the following 2.1.1. Training Effectiveness
research queries will be answered by the study: The training According to [18], monitoring employee training and
provided to the employees was effective? What should the measuring training effectiveness is a key objective of any
Ministries and the training center do to make the training training department. One of the ways to evaluate the
effective? effectiveness of employee training is by establishing Key
Performance Indicators. When these institutional indicators
are properly implemented and monitored, they serve as
2. Literature Review benchmarks for measuring and improving progress toward
2.1. Theoretical Review broader fundamental goals or objectives.
The effectiveness of training can be measured by the
Human capital is considered a major asset in most response level of trainees and/or the knowledge and learning
developed countries and organizations; this is because it is that trainees gain from the training program; measuring the
believed that the skills, performance, and experience of effectiveness of the training provided by institutions is one of
employees are important assets in determining the present the key elements in indicating how a job can be achieved
and future of the institution. appropriately. If the training program is proven to be
Different authors define the term training using different effective, it will be more effective than the
terms [21], training is the process by which people acquire institution/organization needs. However, measuring training
skills to perform tasks and the focus of the training is work or effectiveness is one of the biggest challenges facing
activity. Such as [15], description, training as an organized institutions.
process in which people learn knowledge or skills for a Overall training effectiveness is the extent to which the
specific purpose. Training is an ongoing and consistent effort training objectives are achieved and benefited the Ministry
by an organization or institution to expand education and and trainees, which can be evaluated using the combination
knowledge among employees, enhance employee capabilities, of satisfaction (reaction), learning performance, and
and improve performance [24]. behavioral change finding from the trainee’s feedback.
The process of shaping and equipping employees through Effectiveness is evaluated on different four levels. All the
the addition of their skills, abilities, knowledge, and behavior results of the four levels are already merged and recoded into
is known as training, [13]. This enables work to be completed 'strongly agree', "agree", "neutral", “disagree” and “strongly
more quickly, effectively, and rationally. According to [2], disagree”. This section will summarize the result of all levels
 Humanities and Social Sciences 2024; 12(1): 1-12 3

and this will provide a result of whether the provided training 2.1.2. The Kirkpatrick Evaluation Model
has been effective or not. Although there are various measurement methods and
Measuring the effectiveness of training has many models to measure the effectiveness of training, many human
advantages [9], the first reason to conduct a training resource development experts and researchers tend to By Dr.
effectiveness survey is to measure how well the training went. Donald Kirkpatrick (Dr. Donald Kirkpatrick), the model
Questions asked in the survey may reveal what participants developed in 1959 (Kirkpatrick model) the main preferred
liked or disliked about the training. It can show what model.
improvements need to be made to make future training better Kirkpatrick's model was a 4-step evaluation method
or more relevant. Second, conducting training effectiveness developed in the late 1950s to measure the effectiveness of
surveys plays a vital role in making future trainings better institutional training programs. It is preferred over other
and more effective. The omissions identified in the study and similar training measurement models to evaluate training
observed during the training will prevent trainers and effectiveness. This training effectiveness assessment
organizers from making the same mistakes over and over framework is aimed at facilitating business globally and
again. A training effectiveness survey helps identify training leveraging the knowledge gained from training in a cost-
gaps and areas and topics where employees need actual effective and time-efficient way to achieve better business
training. And a training effectiveness survey helps to results. Such as Kirkpatrick (Kirkpatrick model) According
determine whether the knowledge expected from the given to the research theme, there are four steps to evaluate the
training has been imparted to the trainees. effectiveness of training; these are:

Source: Kirkpatrick Training Evaluation Model, 1979

Figure 1. Kirkpatrick model.

Level 1. Satisfaction (Reaction): Behavior change is normally the immediate supervisor or

The purpose of this evaluation point is straightforward, thirdly side reviewer or by the individual as if it can be filled.
asking questions that confirm the trainees' opinions, allowing Level 4: Result:
individuals to assess the relevance and usefulness of the It answers the question of what is the result or goal of the
training. training. This phase focuses on evaluating the actual results
To measure and collect data, you can collect feedback of the training. This level, which is typically considered the
from trainees about the training situation, fill out post- main goal of the program, is to reduce training costs, achieve
training questionnaires, and ask for reports from employees' high returns on training costs, increase productivity, and
immediate supervisors. reduce risk in the workplace, enabling more efficient
Level 2: Learning: operations and this scale is used to measure high work
It helps to measure the skills and knowledge acquired as a morale.
result of the training. Learning is the assessment point we use This is the effect in the workplace because of attending the
to determine whether trainees have acquired the knowledge training courses. Following the result, we asked whether the
and skills intended by the training in line with the learning prepared training documents are helpful, was the training
objectives set. The study methods varied from formal to provided is directly related to your work, how relevant those
informal tests and from self-assessment to group assessment. contents of on workplace, and another important question
An assessment or evaluation is done to find out how much was; has the training program helped you to do your job or
knowledge and understanding the trainees had about the any part of it better than before training?
subject pre-training and post-training.
Level 3: Behavior: 2.2. Empirical Review
To measure the extent to which the training has affected The impact of training on employee job performance was
the performance or attitude of the trainee or participant, it investigated by [5], the research was carried out at a small
helps to assess how well and accurately the participants apply and medium-sized business (SME) in Malaysia. The study
the knowledge gained from the training on the job. looked at how the company's employees' job performance
This level of measurement is evaluated by comparing the was affected by training. The study's functional area, which
difference in the behavior of the trainee before and after the includes employee job performance and training, has been
training after the completion of the training program. examined in relation to the company's efficient human
4 Moges Mengstu Kassaw: Training or Desactivación? Measuring Training Effectiveness Evidence from
Ethiopian Ministry of Revenues

resource management practices. The questionnaire-based

survey was completed by 85 company employees from their
respective production units. Effective training has a
significant impact on employees' job performance, according
to the study's findings.
A study conducted by [3], with the primary objective of
statistically examining the connection between a training
strategy and employee performance. The theoretical portion
of the study identified key personnel's potential effect on
performance as the study's goal. From Al-Bayt University's
651 administrative department staff, a sample of 120
employees was selected. The study demonstrated that Al-
Bayt University employees' training needs must be
determined using scientific methods, and training programs Source: Author
must be developed that focus on practice obstacles and work
Figure 2. Training Effectiveness conceptual framework.
to overcome them. The study made a number of
recommendations, one of which was to create training
programs that take into account global developments and 3. Research Methodology
offer trainees training opportunities to prepare them for what
might come from global changes, which may necessitate This study adopts a quantitative approach. The targeted
periodic retraining. group of study for this research was Ministry of Revenue
An integrated research model used by [7], combined the employees attending training at the central level in
primary factors that have been shown to be related to training 2020/2021;
transfer with an examination of the relationship between The population of this study is 377 employees who have
training transfer and operational performance to examine the received training on tax laws and regulations from the
medium to long term effects of training programs on branches of the Ministry of Revenue, and the sample was
businesses. The study selected the training design, trainee taken from the branch based on stratified sampling.
self-efficacy, and work environment as transfer factors. The sample size is a figure that is taken from the
Using data from 126 employees who have participated in population or among the research elements to conduct a
various training programs in a number of Greek study and represents the majority. The overall sample size
organizations, the validity of this model is evaluated using and sample of each stratum of the study were 194.
the structural equation modeling approach. The findings It will be difficult to obtain the required sample size using
indicate that, in addition to trainees' self-efficacy and post- random sampling because of the workload in some
training behavior, the design of a training program has the departments, employees who are moving to educate
greatest impact on job performance after training. taxpayers, and employees who remain outside the office for
The primary objective of [8], research was to investigate monitoring, control, and various tasks. In order to obtain the
the impact of training and development on employee required sample size for the study, convenience sampling,
performance and the competitive advantage of an also known as non-probability sampling, was used to collect
organization in the Nigerian banking sector. Two hundred the data.
and twenty-three valid questionnaires were filled out by Table 1. Sample size.
selected banks in Lagos State, South-West Nigeria, using a
simple random sampling technique for the descriptive Tax Center Population Sample
LTO 46 24
research method. To meaningfully represent the raw data, MTO 42 22
descriptive statistics were used to analyze the collected data. EAST 45 23
The findings demonstrated a strong connection between WEST 49 25
employee performance, competitive advantage, and training NORTH WEST 33 17
and development. ADAMA 35 18
Head Quarter 127 65
2.3. Conceptual Model Total 377 194

Source: MOR training center and Author Compilation

This study is based on Kirkpatrick's research model to
explore the effectiveness of training; this conceptual model is 3.1. Data Collection Method
used to say whether the study is effective or not based on the
three criteria of measurement (satisfaction or reaction, Convenience Sampling is a cost-effective, convenient, and
learning performance, and attitude and behavioral change easily accessible method of collecting data from
measures of the training). representative data sources. This method of sample collection
is quick, uncomplicated, and economical, making it the most
common sampling method in many studies. The convenient
 Humanities and Social Sciences 2024; 12(1): 1-12 5

sampling method was used to conduct this research, due to Behavioral change (shaping attitudes of the employees).
the nature of the work of the Ministry, it is difficult to find The questionnaire was prepared as closed-ended and open-
the selected employees in order to collect information at the ended, and the respondents were asked to fill in the provided
desired time. answer space. The closed-ended section of the questionnaire
A written questionnaire to collect sufficient information is a quick and easy way for the informants and the researcher.
regarding the primary source of information was prepared. The Most of the close-ended questionnaires are prepared using a
closed-ended section of the questionnaire is a quick and easy five-scored Likert scale. Accordingly, "1" indicates strongly
way for the informants and the researcher. Most of the close- disagree with the question or statement and "5" indicates
ended questionnaires are prepared using a five-scored Likert strongly agree.
scale. Accordingly, "1" indicates strongly disagree with the
question or statement and "5" indicates strongly agree. 3.2. Data Processing and Analyzing
In this study, the questionnaire has two parts; Section one Since training effectiveness is a latent variable i.e., a
general information about the respondents focuses on variable we can’t directly measure but gives rise to observe
questions like gender, age, education level, and work patterns in things we can observe, and then we chose the
experience. Section two was developed to measure the appropriate model Confirmatory Factor Analysis (CFA).
variables of satisfaction (reaction), learning performance, and The TEF was developed and modified from the
behavioral change. It comprised 20 closed-ended questions combination of [1, 16], for evaluation of the training
which utilized a five-point Likert scale. The items in the effectiveness provided by Ethiopian Ministry of Revenues
questionnaire were connected to respondents' perspective on Training Center. The TEF comprises three different sections
training delivery, which consists of 3 question parts based on to measure reaction (satisfaction), learning performance, and
training measurement requirements; These are satisfaction behavioral change; each dimension used in the TEF was
(training preparation and delivery method), learning operationally defined (see Table 2).
performance (improving knowledge and skills), and

Table 2. Latent Variable Definition.

Latent Variables Item Measures Description

TREF (Overall Training Effectiveness) R1-B6 Overall training effectiveness is the extent to which the training objectives are achieved
Satisfaction (reaction) evaluation is the overall fulfilment towards the training design,
Satisfaction (Reaction) R1 – R10
material, delivery and quality, and personal satisfaction.
Learning performance evaluation is the improvement and acquired knowledge and skills
Learning Performance L1 - L4
in theoretical or practical.
Behavioural Change B1 – B6 Behaviour was measured by relationships with co-workers and customers.

Source: Author

In the first development of TREF, 10 positive statements the study.

were constructed for the satisfaction (reaction) dimension, 4 Prior to the actual data collection exercise, necessary
statements were constructed for learning performance and 6 adjustments were made on the basis of feedback given by
statements were constructed for behavioral change measures, certain employees and Ministries training center officials.
in sum, 20 items were developed. After this, 190 questionnaires were collected from the
Items for satisfaction were developed using training expected sample (97.9%) out of the total questionnaires
quality, design, content, and personal satisfaction, items for distributed.
learning performance were developed using acquired There are various measures of reliability to determine the
knowledge and skill from the training, and behavioral change reliability of a study. One of the most commonly used and
was measured as attitude change, being part of a team, and common reliability measures is Cronbach's Alpha.
relationship with others. Cronbach's Alpha is a measure used to assess the reliability
or internal consistency of scales.
3.3. Validity and Reliability of the Research According to [10], construct validity can be tested using
Data was collected at least three months after the convergent validity (the degree to which two measures of the
completion of training. Content validity was applied to same concepts are correlated), and nomological validity (all
determine the validity of the questionnaires, and the constructs are significantly correlated with each other).
distributed data collection questionnaires were reviewed by Hence, AVE and CR were assessed in the measurement
the training center department director and used as model. Assessing validity and reliability, defined reliability
modifications to the final questionnaires applicable to the as an assessment of the degree of consistency between
study. multiple measurements of a variable.
In order to include all employees who takes training, the This study assesses the consistency of the entire scale with
questionnaires were prepared in Amharic language, since it is Cronbach’s alpha and its overall reliability of each factor of
official language of the country and translate into English for productivity values. All values yielded an alpha coefficient that
6 Moges Mengstu Kassaw: Training or Desactivación? Measuring Training Effectiveness Evidence from
Ethiopian Ministry of Revenues

exceeded the values of 0.70 suggested by (Hair Jr, J. F., et.al. Variables Freq. Valid Percent
[10]. from this result of Cronbach’s alpha coefficient value, MA/MSc & above 46 24.21
this questionnaire was accepted and admissible. To validate the Source: Author
instrument, this study also considered construct validation
using analysis of structural model software (Stata and Jamovi) If the average result of the measuring scale (questionnaire)
with maximum likelihood (ML) to analyze the data. is below 3.39, it is a low result (Low), the average result is
Jamovi and Stata version 15 statistical software package more than 3.4. If it is up to 3.79, it is considered moderate
was used to analyze the collected data and significant and if the average score is 3.8 and above, it is considered
findings. All the questionnaires were prepared in Amharic high. [23]
language and the data collected from the questionnaires were The respondents' average scores on the ten in-depth
translated into English to be able to analyze the data by questions about the training and how it was prepared ranged
computer and the analysis was done. from 2.73 to 4.03. The survey received the lowest score of
Research ethics refers to the type of agreement that the 2.73. The four items (R7, R8, R9, and R10) that designate
person/group conducting the research enters into with the "the time given for the training," "training place,"
parties involved in the research. Since the study is based on "accommodation," and "trainees overall satisfaction" have a
the willingness of the employees, the purpose of the study is low level of satisfaction (average score below 3.39).
being explained to the participants, and if they do not want to The average response score on the satisfaction level was
give an opinion, it is not binding. 3.6, which is considered to be a medium level of satisfaction.
The average result of the time allotted to the training (2.73)
4. Results and Discussion demonstrates that it is impossible to assert that not enough
time is allotted to the training and that the time allotted to the
4.1. Respondent’s Profile pieces of training is sufficient. According to the trainees'
responses to the training delivery questionnaires, which were
As shown in Table 3 below, 107, or 56.3%, of the trainees prepared, the average score was 3.6 (medium level),
who participated in this study were male, while the remaining indicating that the trainings had many strengths as well as
83, or 43.7%, were female. Concerning the trainees' ages, 125, many gaps.
or 66.1% of the total workforce, are young workers between The questionnaires that were administered to the trainees
the ages of 18 and 35, followed by 48, or 25.4%, who are to determine whether or not they had acquired sufficient
between the ages of 36 and 45. It employs 8.4% of workers knowledge and skills from the training program revealed that,
over the age of 46. This suggests that the majority of ministry on average, they had been able to transfer the expected
employees are young men. In terms of education, 98.4% of the knowledge and skills. The participants in the study believe
190 employees have a first degree or higher, 74.2% have a first that the knowledge and skill gap they had as a result of
degree, and 24.21 percent have a second degree or higher. receiving training has been reduced to some extent,
Table 3. Respondent’s profile. regardless of whether the trainees have acquired the
necessary knowledge and skills, according to the findings of
Variables Freq. Valid Percent the study. The respondents who stated that the training
SEX
female 83 43.68
helped me become familiar with new procedures and
male 107 56.32 technology received an average score of 3.32 and verbal
AGE interpretation at a low level in the detailed questions about
18-35 125 66.14 the trainings' ability to impart knowledge and skills. The
36-45 48 25.40 training that is provided typically has a medium level of
46-55 14 7.41
>55 2 1.06
knowledge and skill transfer (3.52). The survey found that
EDUCATION the study's participants believe that the knowledge and skill
TVET & Diploma 3 1.58 gap they had as a result of receiving training has been
BA/BSc 141 74.21 reduced to some extent.

Table 4. Measuring TREF using Weighted Average.

TREF Measures Variable symbol Average Interpretation

R1 3.8388 High
R2 3.7842 Medium
R3 3.9973 High
R4 3.9699 High
R5 3.6694 Medium
Satisfaction R6 4.0355 High
R7 2.7295 Low
R8 3.3443 Low
R9 3.2322 Low
R10 3.3798 Low
Average 3.59809 Medium
 Humanities and Social Sciences 2024; 12(1): 1-12 7

TREF Measures Variable symbol Average Interpretation

L1 3.6421 Medium
L2 3.7186 Medium
Learning performance L3 3.3251 Low
L4 3.4071 Medium
Average 3.523225 Medium
B1 3.7213 Medium
B2 3.6831 Medium
B3 3.7678 Medium
Attitude and Behavioral change B4 3.7705 Medium
B5 3.7213 Medium
B6 3.6311 Medium
Average 3.71585 Medium

Source: Author

The training demonstrates that the trainees' average behaviors, as evidenced by the average score of 3.72 for
response scores range from 3.63 to 3.77, as shown in the attitudes and behavior shaping.
table 4. The question, "The training has helped me feel
satisfied with my work," received the highest mean score 4.2. Validity and Reliability
(3.77), while the question, "The training has developed a The assessment of Average Variance Explain (i.e. AVE)
culture of teamwork," received the lowest mean score (3.33). for each construct was above.50, as well as both reliability
The trainings played a moderate role in shaping employee tests (i.e. Composite Reliability and Cronbach’s Alpha) for
attitudes and behaviors, as evidenced by their average score each targeted construct being above.70 [11]. Hence, the
of 3.72 in influencing attitudes and behaviors. In general the measurement model meets the validity criteria concept.
training had a moderate impact on employees' attitudes and

Table 1. Validity and Reliability of Training Effectiveness.

AVE CR alpha Satisfaction Learning Behavioral

Satisfaction 0.57 0.62 0.89 1.0000
Learning 0.66 0.85 0.82 0.7074 1.0000
Behavioral 0.73 0.91 0.83 0.6908 0.8018 1.0000

Source: Author

The average AVE was 0.57 for satisfaction with ten items, baseline model. Second, the standardized root mean square
0.66 for learning performance with four items, and 0.73 for residual (SRMR) and the root mean square error of
behavioral change measure with six items, as shown in table approximation (RMSEA), which measure the model's
5 above. As a result, there is convergent validation for each absolute fit to the data [12, 17].
training effectiveness dimension with sufficient AVE and The best-fitting model was identified as the one with the
sufficient significance. Additionally, the construct reliability lowest RMSEA and SRMR values but the highest CFI and
(CR) for behavioral change was 0.91, the CR for learning TLI estimates. We highlighted all models with the simplest
performance was 0.85, and the CR for satisfaction (reaction) fit for data sets in which they fit equally well. To put it
was.62. This indicates that construct reliability is sufficient. another way, all models were chosen when the fit statistics of
The measurement model contained 20 items, all of which the competing models were either equally high (i.e., CFI, TLI)
demonstrated an acceptable level of construct validity and or low (i.e., RMSEA, SRMR).
construct reliability. Model1: The first three-factor model with 20 items was
represented (satisfaction was represented by items R1, R2,
4.3. Confirmatory Factor Analysis (CFA) R3, R4, R5, R6, R7, R8, R9, and R10; Learning performance
The CFA model looks at how well three observed was represented by items L1, L2, L3 and L4; Behavioral
variables—satisfaction, learning performance, and change was represented by item B1, B2, B3, B4, B5, and B6)
behavior—measure the latent variable of training was analyzed by Stata Version15 revealed a poor fit with the
effectiveness. This is important because the level of training X2 (df) = 477 (167), p < 0.001, SRMR = 0.058, CFI = 0.894,
effectiveness is a difficult-to-measure complex latent concept. TLI = 0.879, RMSEA = 0.0988, indicating needs for further
We recorded four approximate model fit indices that each modification based on the Modification indices (M.I); a
measure fits in a unique way to determine which of the three modification index (M.I) refers to the predicted decrease of
measurement models fits each data set best. First, the Tucker- the χ2 statistic when a fixed parameter is freely estimated or
Lewis index (TLI) and the comparative fit index (CFI), an equality constraint is relaxed.
which measure the improvement in fit in comparison to a
8 Moges Mengstu Kassaw: Training or Desactivación? Measuring Training Effectiveness Evidence from
Ethiopian Ministry of Revenues

Table 6. Model 1 Fit Measures. Fit statistic Value

RMSEA 0.114
Fit Measures
CFI 0.979
χ² Df P CFI TLI SRMR RMSEA
TLI 0.957
477 167 < .001 0.894 0.879 0.058 0.0988
SRMR 0.026
CD 0.907
Model 2: Five items (i.e. items R3, R6, R8, R9, and B4)
were removed based on the Modification Indices of model 1. Learning:
The three-factor model with 14 items (satisfaction was
represented by items R1, R2, R4, R5, R7, R8, R9, and R10;
Learning was represented by items L1, L2, and L4; ε1 L1 4.2 .87
Behavioral was represented by items B1, B2, B3, B5, and B6)
was analyzed and showed a poor fit with the latent constructs ε2 L2 4.4 .89 Learning
(X2 (df) = 209 (74), p < 0.001, CFI = 0.929, TLI = 0.913,
SRMR = 0.0488, and RMSEA = 0.0981), indicating further 1
.8
modification was necessary to improve the model fitness. ε3 L4 3.5

Table 2. Model 2 Fit Measures. Figure 4. Learning construct.

Fit Measures Table 9. Learning Fit Measures.

χ² Df P CFI TLI SRMR RMSEA
209 74 < .001 0.929 0.913 0.0488 0.0981 Fit statistic Value
p > chi2 0.000
Source: Author RMSEA 0.000
CFI .9999
Model 3: Another two items (i.e. items R7 and L3) were TLI .9999
removed based on the modification indices of model 2. The SRMR 0.000
three-factor model with 13 items (satisfaction was CD 0.897
represented by item R1, R2, R4, R5 and R10; Learning was
Behavior:
represented by item L1, L2 and L4; Behavior was
represented by item B1, B2, B3, B5, and B6) was analyzed
and found a good fit with the latent (X2 (df) = 137.8 (62), p < ε1 B1 4.4
0.001, CFI = 0.962, TLI = 0.953, RMSEA = 0.0764 and
SRMR = 0.032) and CD (coefficient of determination) .88
ε2 B2 4.1
=0.997. Standardized factor loadings showed that all the .92
items in model 3 were well loaded on each latent construct.
ε3 B3 4.2 .81
Satisfaction:
Behavior
ε4 B5 3.9 .88 1

.86
ε5 B6 3.7

Figure 5. Behavior construct.

Table 10. Behavior Fit Measures.

Fit statistic Value

chi2_ms(5) 33.567
p > chi2 0.000
RMSEA 0.175
Figure 3. Reaction. Construct. CFI 0.966
TLI 0.932
Table 8. Reaction Fit Measures. SRMR 0.025
CD 0.945
Fit statistic Value
chi2_ms(5) 16.804 Source: Stata-15 output and Author Compilation
p > chi2 0.005

Table 11. Model 3 Fit measures.

Fit Measures
χ² df p CFI TLI SRMR RMSEA pclose CD
137.83 62 0.000 0.962 0.953 0.032 0.0764 0.003 0.997

Source: Stata-15 output and Author Compilation

 Humanities and Social Sciences 2024; 12(1): 1-12 9

The research results show that the composite indicator

from training effectiveness measures consisted of 3 main 4.4. Measurement Model for Training Effectiveness
components and 13 indicators were fitted with the empirical The result of CFA analysis using correlated model in
data, determined from the Chi-square values = 137.83, Figure 4 and Table 12 shows good results of GOF, i.e.,
comparative fit index (CFI) =0.962, Tucker-Lewis index χ2/df=137.83, root mean square error of approximation
(TLI) =0.953, standardized root mean square residual (RMSEA) = 0.083, standardized root mean square residual
(SRMR) =0.032, root mean square error of approximation (SRMR) =0.032, comparative fit index (CFI) =0.962,
(RMSEA) =0.0764, and coefficient of determination (CD) Tucker-Lewis index (TLI)=0.953, and coefficient of
=0.997. determination (CD)=0.997.

ε1
R1 4.4
.84
ε2
R2 4.4 .77

ε3 R4 4.8 .81
React
ε4 .82
R5 4.1
.83
1
ε7
R10 3.6
.8

ε5 L1 .88
4.2

ε6 Learning
L2 4.4 .87 .75

ε8 .8 1
L4 3.4

ε9 B1 .87
4.4
.89
ε10 B2 4.2 .92

ε11 .81
B3 4.2 Behavior

ε12 B5 .88
3.9 1
.86
ε13 B6 3.7

Source: Author
Figure 6. TREF Measurement Model.

Table 12. Model 3 Fit measures.

Fit Measures
χ² df p CFI TLI SRMR CD RMSEA pclose
137.829 62 0.031 0.962 0.953 0.032 0.997 0.083 0.003

Source: Stata-15 output and Author Compilation

The CFI, TLI, and SRMR do not find the models for RMSEA includes a drawback for model complexity, which
complexity so it compares models in absolute terms. We will result in penalizing the multiple correlated factors model
expect that as model complexity increases (i.e., more more than the higher-order model because the correlated
indicators per factor), the performance of fit indices that factors model is less miserly (i.e., requires more parameters
include a penalty for model complexity will deteriorate. The to be estimated) than the higher-order model.
10 Moges Mengstu Kassaw: Training or Desactivación? Measuring Training Effectiveness Evidence from
Ethiopian Ministry of Revenues

4.5. Second-Order Training Effectiveness Model model fit.

The 2nd Order CFA test is carried out by looking at the Table 13. 2nd order TREF Model Fit measures.
factor load value (>0.5). A factor load of 0.50 or more is Fit statistic Value
considered vital enough to explain the latent construct [11]. chi2_ms(62) 137.829
The model shown in the Figure below is a figural p > chi2 0.062
representation of the structure of the training effectiveness as RMSEA 0.063 (pclose 0.003)
proposed by the author. For this “All Parameters Invariant” CFI 0.971
TLI 0.964
model, all aspects of the CFA model first and second-order
SRMR 0.032
factor loadings, factor variances, and unique and error CD 0.974
variances were constrained to be equal across groups. The
difference between the two models tests whether the factor Source: Stata-15 with Author Compilation
structure imposed does or does not lead to degradation in the

Source: Stata-15 output

Figure 7. 2nd order TREF Model.

The structural model for TREF with acceptable Goodness

of fit (GOF). The structural model shows that the most 5. Conclusions and Recommendations
important construct of training effectiveness was learning
performance (0.97), followed by behavioral change (0.91), The objective of this study was to evaluate the
and satisfaction or reaction measures (0.83). The finding is in effectiveness of the training provided to employees of the
line with the result of Diamantidis, A., & Chatzoglou, P. [7] Ministry of Revenue in 2020/2021, and construct a valid and
and Falola, H., et al. [8], the training has an impact on reliable instrument to measure training effectiveness stretches
employee’s job performance and learning performance. by the Ministry of Revenues Ethiopia. The questionnaire was
the main data collection method to get compulsory
 Humanities and Social Sciences 2024; 12(1): 1-12 11

information for the study from employees and immediate included the evaluation of individual and organizational
supervisors. To determine the overall effectiveness of the performance, TREF should be used at least three to five
training, Kirkpatrick's first three stages (reaction or months after the completion of training. Specifically, this
satisfaction, learning performance, and behavioral change) effort can support the improvement of quality human
were studied based on 20 detailed questions: resources in the public sector for effective national policy
The research results show that most of the respondents implementation.
were men aged between 18-35 years, and 98 percent and Additionally, TREF can also benefit the public service by
above of the total respondents have a bachelor's degree or offering a self-evaluation instrument to determine the effect
higher. Out of the total employees who take training, 57% of the training that they have provided. For researchers,
were male, and the remaining 43% are female TREF can be used in a survey to determine the relationship
employees.78.8% of all trained workers are in the age group between training effectiveness and other variables, such as
of 18-35. training place, accommodations, and employee norms to
To determine the overall effectiveness of the training, increase the effectiveness of training. It can also benefit the
Kirkpatrick's first three stages (satisfaction or response, students that need to study training since there is a limited
learning performance, and behavioral change) were studied instrument to measure general training effectiveness.
based on 20 detailed questions, the measurement level In light of the study's findings, the following
(satisfaction or response) has a total average score of 3.6 recommendation was made: Instead of being merely
(medium level), In terms of imparting knowledge and skills theoretical, the trainings that are provided ought to be hands-
to employees (learning performance) it is a medium level on, It is recommended that employees be able to attend the
(3.52), and the training in terms of shaping attitude and relevant topics for their work, and the training center should
behavior (behavioral change) has the same medium result ensure that the right employee receives the training in terms
(3.72). In general, the result shows that the effectiveness of of work behavior, It is recommended that the training be
the training is at a medium level in the effort to evaluate the given by experts in the field who are close to the work and
training effectiveness provided by the institution. have practical knowledge and experience, to fill the trainees'
AVE and CR indicators from development consisted of 3 knowledge and skill gaps, the training time should be
main components and 13 indicators were fitted with the increased.
empirical data, determined from the Chi-square values =
137.83, and no statistical significance (p = 0.062) at zero Declaration
degrees of freedom (df = 98) indicated that model not
different from the empirical data. In addition, found the Availability of Data and Materials
RMSEA = 0.063, CFI = 0.97, TLI = 0.96 and SRMR =
0.032. The data used during the study are available with the
In general, it shows that all three levels (satisfaction, author and they are ready to make it available upon a
learning, and behavior) have moderate effectiveness reasonable request.
according to the standards set to measure the effectiveness of
training; the three-factor model with 13 items of the training Conflicts of Interest
effectiveness had a good fit and showed good maneuver
values. It is a valid and reliable measurement to identify the The authors declare no conflicts of interest.
effectiveness of training provided to tax officials across the
Ministry of Revenue in Ethiopia.
The training facilities (training hall, cafeteria, and dorms References
and so on) where the trainings are given are not comfortable, [1] AA Aziz, et. al. (2012). Developing a Structural Model of
so the training places should be made comfortable for the Assessing Students’ Knowledge-Attitudes towards
next trainings. Sustainability,. Social and Behavioral Sciences,
It demonstrates that the trainings were not given enough (https://doi.org/10.1016/j.sbspro.2012.09.683.), 515-519.
time to impart the necessary knowledge and skills; [2] Adnyani, N. L. P. R., & Dewi, A. A. S. K. (2019). Pengaruh
consequently, the institution ought to devote sufficient time Pengalaman Kerja, Prestasi Kerja Dan Pelatihan Terhadap
to the various types of trainings; Pengembangan Karier Karyawan. E-Jurnal Manajemen
Since the modules and documents preparing for training Universitas Udayana, 8(7), 4073.
was inappropriate and lack of clarity, so the training center https://doi.org/10.24843/ejmunud.2019.v08.i07.p03.
must ensure that the training documents and modules it ought [3] Al-Awawdeh, W. (2011). Impact of Using Training Strategy
to be prepared in a way that can be utilized for trainees on the Performance of Employees at the Administrative
working after the preparation; Departments at Al Al-Bayt University. Al-Manarah for
Human Resource Development practitioners from public Research and Studies, 78-106.
sectors are encouraged to use the TREF in training evaluation. [4] Bapna, R., et. al. (2013). Human capital investments and
TREF can be used to determine training effectiveness for any employee performance. An analysis of IT services industry.
training program as a general evaluation. Since TREF Management Science, 641-658.
12 Moges Mengstu Kassaw: Training or Desactivación? Measuring Training Effectiveness Evidence from
Ethiopian Ministry of Revenues

[5] Bin Atan, J., et. al. (2015). Impact of Training on Employees' [15] Jones, GR, and George, JM. (2005). Contemporary
Job Performance: A Case Study of Malaysian Small Medium Management. New York: McGraw Hills.
Enterprise. Review of Management, 5(1/2), 42-48.
[16] Kirkpatrick, D. L. (1959). Techniques for evaluation training
[6] commercemates. (2022, January 15). commercemates.com. programs. Journal of the american society of training directors,
Retrieved from commercemates: 21-26.
https://commercemates.com/what-is-training-types-methods-
importance/ [17] Kline, R. (2011). Principles and Practice of Structural
Equation Modeling, 3rd ed. New York, NY, USA: Guilford.
[7] Diamantidis, A., & Chatzoglou, P. (2014). Employee Post-
training Behavior and Performance: Evaluating the Results of [18] Lilly, A. (2011). Tracking Employee Training: Measuring
the Training Process. International Journal of Training & training effectiveness with KPIs. Canada: Robert
Development, 159-166. Bacal/Bacal&Associates.

[8] Falola, H., et. al. (2014). Effectiveness of Training and [19] marketbusinessnews. (2023, January 18).
Development on Employees' Performance and Organization (https://marketbusinessnews.com/financial-glossary/training/)
Competitiveness in the Nigerian Banking Industry. Bulletin of
the Transilvania University of Braşov, 165-170. [20] Martin, R. (2015). How to begin with employee Training.
Retrieved from cleverism: https://www.cleverism.com/ways-
[9] Ganesh M. & R. Indradevi. (2015). Importance and training-can-drive-employees-productivity/
Effectiveness of Training and Development. Mediterranean
Journal of Social Sciences. doi: 10.5901/mjss.2015.v6n1p334, [21] Mathis, RL, & Jackson, JH. (2011). Human Resource
334-337. Management, 13th edition. United States of America: South-
western Cengage learning: West Publishing Company.
[10] Hair Jr, J. F., et. al. (2017). PLS-SEM or CB-SEM: updated
guidelines on which method to use. International Journal of [22] Nassazi, N. (2013). Effects of training on employee
Multivariate Data Analysis, 107-123. performance: Evidence from Uganda (Unpublished doctoral
dissertation). Finland: University of Applied Sciences.
[11] Hair, J. F., et. al. (2014). Multivariate data analysis. Pearson
new international edition. Essex: Pearson Education Limited. [23] Pihie, Z. A. L., & Bagheri, A. (2013). Self-efficacy and
entrepreneurial intention: The mediation effect of self-
[12] Hoyle, R. (2000). In Handbook of Applied Multivariate regulation. Vocations and Learning, 385-401.
Statistics and Mathematical Modeling;. New York, NY, USA:
Academic Press. [24] Scott, G., et. al. (2004). The effectiveness of creativity training:
A quantitative review. Creativity research journal, 16(4), 361-
[13] Ichsan, R. N. (2020). Pengaruh Pelatihan terhadap Prestasi 386.
Kerja Karyawan pada PDAM Tirtanadi Cabang Padang Bulan
Medan. Jurnal Ilmiah METADATA, 2(1), 72–75. [25] Tyson, S., and York, A. (2009). The Asia Foundation Training
Needs Assessment Report. Essentials of HRM. 4th edition.
[14] Jayan, M., et. al. (2016). Investigating the relationship of Vietnam: National Library of Vietnam.
adaptive leadership and leadership capabilities on leadership
effectiveness in Sarawak schools. Procedia-Social and
Behavioral Sciences, 540-545.