Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

Inter-rater Reliability Assessment for Motivating Factors in

Blood Donation Using Cohen's Kappa Analysis

Aspalilla Main
Politeknik Merlimau
[email protected]

Rodzah Hj Yahya
Politeknik Merlimau
[email protected]

Hafidah Mahat
Politeknik Merlimau
[email protected]

Abstract
The identification of motivating factors for blood donation, especially for new donors, is
important for donor recurrence and retention to ensure that blood stock is always
maintained. The aim of this study is to identify the inter-rater reliability of the motivating
factors for blood donation, in order to verify that the identified factors are reliable and can be
used for further research work. The result from the reliable item has the potential to be used
to discover possible blood donation candidates, particularly among the younger generation.
It involved two professionals called experts from health industry to give the rating. Therefore,
this research performed the Cohen Kappa analysis in order to obtain the mutual agreement
value among raters. The result shown the Kappa value is 0.7 which is considered to be in
substantial agreement, with 12 out of 16 pairs had perfect agreement, 3 had perfect
disagreement while there was one item, which does not have the same level of consensus
among raters. The reliability evidence from previously identified factors was provided by these
findings and significantly proven for creating appropriate incentive schemes for individual
potential donors, which could help raise donation volumes without jeopardising blood
supplies. The agreed-upon parameters can then be utilised to create any promotional scheme
to ensure that blood is always available and never runs out.

Keywords: blood donation, appreciation, notification

1.0 Introduction
Blood Donation are essential to saving one's life. The need for blood is
growing in tandem with the improvement of modern medical treatment and
the rise in human life expectancy. However, only a small percentage of eligible
blood donors, on the other hand, option to donate blood on a regular basis
(Vimal & Nishanthi, 2020). As a result, hospital and institutional blood banks
are always on the lookout for new donors in order to meet their blood product
needs while also maintaining current donors.
Donor recruitment, donor invitation, donor selection, donation
processes, and donor retention are all part of the donor cycle idea (Pallavi,
Vijaya & Rashmi B, 2017). Therefore, along with the rapid pace of digital
technology, many initiatives have recently been made around the world to
leverage social media and smartphone applications to make blood donation
easier, provide additional services, and establish a community around the
blood donation centre. As a result, advertising urging healthy persons to
13
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

donate blood to patients who require rapid blood transfusions are common
on social media. This is in response to a shortage of blood at the blood bank
(Mostafa, Youssef & Alshorbag, 2014).
Identifying the motivation factors and targets who have the potential to
donate blood especially the young is one of the most exciting endeavours, as
these donors are usually in good health and may have a very long donation
period. However, this demographic is under-represented among blood donors
in general (Yuan, Hoffman, Lu, Goldfinger & Ziman, 2011). Therefore, better
understanding of the factors that deter and attract potential blood donors to
donate blood will enable to build more effective recruitment and retention
program. In fact, any factors that are significantly connected with blood
donation behaviour, blood donor status, or the intention to contribute or
become a blood donor are referred to as motivation. Thus, more information
and observations regarding what prevents and motivates people from
donating blood are needed to improve successful retention.
While most countries have various blood donation and supply systems,
the barriers, motivators, and facilitators may differ. Previous research on
barriers and motivators has been summarised but not explored in depth
(Klinkenberg, Huis, Wit, Dongen, Daams, Kort & Fransen, 2019). This will
complicate the process of designing campaigns focused on promoting blood
donation especially targeted for new donor. Therefore, the goal of this research
is to evaluate the inter-rater reliability of the motivational factors to donate
blood in order to verify that the identified factors are reliable and can be used
for further research work.
To get insight into the factors that inspire blood donation, particularly
among young people, we applied the Cohen Kappa analytical technique to
obtain mutual agreement between two experts. Both experts would then rate
each item to verify its reliability. The level of agreement between raters is
referred as dependability between evaluators or inter-rater agreement in
statistics. The judges/raters assign a score based on how much homogeneity,
agreement, or consensus there is. To establish inter-rater reliability, the
components' dependability will be assessed using the Cohen Kappa
agreement index. Elements with mutual agreement will be taken into account
while elements that are not agreed upon will be removed from the list of
factors. In this case, items that identified from two categories of motivators
can be used to formulate strategies and actions to help promote blood
donation campaigns.
This paper is organized as follows: Section I and II presents introduction
of motivating factor on blood donation followed by the proposed conceptual
framework, Section III presents the methodology; Section IV presents all the
obtained results based on mutual agreement between rater; and finally, we
close discussing of each aspect by suggesting strategies and actions that
might be taken to ensure the country's blood supply.

2.0 Background of the blood donation

In terms of the demand for blood transfusions, it has been found that
blood supply is quite important, particularly for victims of car accidents who
require blood urgently. In Malaysia, the long holiday season also contributes

14
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

for seasonal blood shortages, particularly around holiday periods (such as

Eid-al-Fitr and Chinese New Year), school breaks, and the month of
Ramadhan (Tey, Arsil, Brindal, Lee, & Teoh, 2020). Consequently, a source of
obtaining blood during the season is desperately needed. In these conditions,
a huge volume of blood is necessary to address such a wide range of needs.
Accordingly, many countries are also facing an increase in demand for blood
or blood components. Because it is collected from people who voluntarily
donate blood, blood is a valuable source of health. As a result, health facilities
have to run constant promotions to encourage voluntary blood donations in
order to satisfy demand.
In study, young adults have the potential in terms of being repeated
blood donors. If individuals begin donating blood in their 20s, they have the
ability to donate for up to 40 years. Based on statistic, nearly a quarter of
Malaysia's blood supply originates from college student donors in their
twenties, though critical blood shortages continue as an ongoing concern in
Malaysia (Jaafar, Chong & Alavi, 2017).
Based on previous study, individuals that donate on a regular basis
have been universally proven to be the basis for a safe source of blood (Pal,
Sengupta & Shenga, 2008). This activity is seen as a voluntary act to support
blood needs, hence, the act of donating blood by the donor contribute to
sustain a continuous supply of blood over time. Decision-making by the donor
ultimately determines the stability of the blood bank supply. However, the
recruitment and retention of recurring donor volunteers is a major challenge
for health care systems around the world. Therefore, identifying the
motivation factors that contribute to the recurrence of blood donation is
crucial to ensure the blood donation cycle continues.

2.1 Motivation factors associated to blood donation among donor

In order to meet the demands of the blood transfusion process and
ensure that the blood supply is always adequate and continuous in all
hospitals, it is important to understand the driving factors to blood donation
for the formulation and implementation of effective donor recruitment
programs. Hence, the implementation of strategies to retain first -time donors
as voluntary permanent donors can also be devised if such factors can be
identified. Individuals will be motivated to donate blood when they are well
informed about the process and benefits of saving someone’s life (Mohammed
& Essel, 2018) and have positive experience during blood donation process
(Vimal & Nishanthi, 2020. This shows that, individuals are motivated to
donate if they have knowledge in the matter and will be more aware of blood
donation if they are educated about it. In study, social continues support to
encourage blood donation and experience with knowledgeable and skilled of
blood collection staff were factors that contributed to imparting knowledge to
donors as well as associated with repeat blood donors (Jaafar et al., 2017).
There are several factors to impart knowledge and give a positive
experience to blood donors. Evidence from previous researchers shows that
among the factors that contribute towards the positive experience among the
blood donor is appreciation (Suemnig, Konerding, Hron, Lubenow, Alpen,
Hoffmann & Greinacher, 2017; Cicolini, Comparcini, Alfieri, Zito, Marta,
Tomietto & Simonetti, 2019; Domingos Lima, Messias, Feijo, Diniz & Soares,
15
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

2016). The elements involved in this appreciation are incentive, certification,

star, medical preference, praise and greeting, material reward, interaction and
recognition.
Moreover, factors that can convey knowledge among blood donor is
through notification (Raza, Qureshi & Qureshi, 2018; Basnotra & Sidhu,
2019; Pallavi et. al., 2017). Notification can be used as a medium for screening
information, donation locator, advertisement, campaign, reminder and
awareness. Figure 1 shows the conceptual framework of motivation to donate
blood.

Appreciation
Motivation

Notification

Figure 1: Conceptual framework of motivation to donate blood

3.0 Methodology
This study involves two professionals called experts from health
industry to give the rating on the factors which have been identified as a
motivator and contributor to donate blood. One of the rater is a practitioner
engaged in collecting the blood at blood collection center while another rater
is a medical officer whom actively involved with Malaysian Red Crescent
Malaysia. Both raters were given directions on how to provide the rating with
the use of a checklist sheet. 16 items from two categories which are
appreciation and notification with two ranks of agreements, which are Agree
(1) and Disagree (0).
The 16 items were identified based on previous studies on motivational
factors as shown in Figure 1. Therefore, from the conceptual framework,
Ratings Agreement Table (Table 1) is constructed to be assessed its consensus
by two experts called rater. Rater will indicate which items they agree to be as
motivating factor in blood donation based on their experience involved in
previous blood donation campaigns.
The result then will be analysed using the Cohen Kappa values to
establish its reliability (Cohen, 1960). Jacob Cohen established the Cohen
Kappa index analysis as a way for measuring the reliability of qualitative data.
As a result, it's also to figure out how much agreement there is on the theme
of the constructs being researched. Mahamod & Ishak, (2003) stated that in
order to determine the high value of reliability for each item used to
characterise a theme, agreement between raters is required. These items
whose reliability has been measured can be used in developing
questionnaires.
Thus, to determine whether the agreement's significance is strong or
weak, Landis & Koch (2013) proposed the method of calculation is based on
the Cohen Kappa agreement scale presented in Table 2.

16
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

Table 1 : Ratings Agreement Table Based on Category

Category Items Rater Rater

1 2
Appreciation Incentive
Certification
Star
Medical Preference
Praise And Greeting
Material Reward
Material incentive (allowance)
Message to think
Interaction
Recognition
Notification Screening Information
Donation Locator
Advertisement
Campaign
Reminder
Awareness

Table 2 : Cohen Kappa Agreement Scale

Kappa Value Degree of consensus

Below 0.00 Poor
0.01-0.20 Slight
0.21-0.40 Fair
0.41-0.60 Moderate
0.61-0.80 Substantial
0.81-1.00 Almost perfect

4.0 Results and discussion

The findings of the study indicate that the 16 items previously identified
as motivators to donate blood are beneficial. All of the items have been
evaluated and are deemed valid cases. The items from two categories of data
provided in Table 3 are being rated.
Table 3 shows that there were 16 valid cases, with 12 of 16 pairings
(75%), having perfect agreement, three items having prefect disagreement
(18.75%) while one item did not have the same agreement across raters. Based
on the evaluation process by experts, the calculation process is carried out
using the formula below:

K = fa – fc
N – fc
Fa = no of agree units
Fc = 50% expected agreement
17
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

N = No of valid cases

Table 3 : Assessment of agreement by experts

Category Items
Expert Expert
1 2
Appreciation Incentive / /
Certification / /
Star / x
Medical Preference / /
Praise And Greeting / /
Material Reward x x
Material incentive (allowance) x x
Message to think / /
Interaction / /
Recognition / /
Notification Screening Information x x
Donation Locator / /
Advertisement / /
Campaign / /
Reminder / /
Awareness / /

Table 4 shows the total value of agreement between experts based on

each category. After computing the agreement value for both experts, the
researcher has obtained the Cohen Kappa Coefficient Index at the level of 0.5
for category 1 and 0.7 for category 2.

Table 4 : Value of agreement between experts based on Cohen Kappa

analysis

Categories Expert 1 Expert 2 Total agreement value

Appreciation K = 8-5 K = 7-5 = 0.6 + 0.4
10-5 10-5 2
= 0.6 = 0.4 = 0.5
Notification K = 5-3 K = 5-3 = 0.7 + 0.7
6-3 6-3 2
= 0.7 = 0.7 = 0.7

According to Cohen Kappa Agreement Scale, result of this study shows

moderate agreement between raters for appreciation and substantial
agreement for notification, as presented in Table 4.

Table 4 : Symmetric Measure

Value Degree of consensus
Appreciation 0.5 Moderate
Notification 0.7 Substantial
18
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

Based on Mahamod & Ishak, (2003), no specific value is appropriate to

indicate the level of agreement between experts, though according to Fleiss &
Cohen (1973), a kappa of 1.0 indicates complete agreement, a kappa of -1.0
indicates consistent disagreement, and a kappa of 0 indicates a random level
of agreement/disagreement between the two raters. While, Wood (2007)
argues that the value indicates to be strong agreement if value of kappa
should be at least 0.60 or 0.70.
Therefore, based on the findings from this analysis, the factors obtained
were found to be reliable for use in developing questionnaire items related to
individual motivation to donate blood for research purposes.
The above finding consistent with Vimal et al., (2020), stated that
voluntary action of blood donors deserves special consideration and should
be appreciated. Appreciations such as the awarding of certificates for
voluntary blood donations, stars for each donation, priority in medical
examinations will encourage more voluntary donors. Mohammed & Essel
(2018) reported similar finding among South African which appreciation in
the form of gift items, time off work, material support such as money and
recognition, among most validated motivator. In addition, feeling warm or
satisfied after donating explains positive emotions from the act of donating
are important to attract repeat donors. The feeling of satisfaction after
donating can be explained through the appreciation of caring staff and thank
you note after going through the blood donation process (Suemnig et al.,
2017).
Similar to this study, the sign of the appreciation given by the blood
collection staff has a significant positive impact on the donor experience. They
found a simple “thank you” greeting was more meaningful than a material
reward. Respondents reported feeling appreciated after donating blood with
such kind treatment thus encouraging them to donate again (Jaafar et al.,
2017). Therefore, such feelings should be nurtured and stimulated in blood
donation campaigns.
In addition, the driving factor towards blood donation is contributed by
information factors through notification. First-time donors especially young
donors are attracted to interactive websites such as social media platforms to
get information regarding blood donation campaigns. Therefore, Sümnig et.
al., (2018) developed communication system through social media to recruit
new donors and to motivate repeat donors as the use of social media among
the young population has increased significantly. This is also agreed upon by
Domingos et. al., (2016), where one of the methods to keep new donors from
becoming frequent donors is being to keep in touch with them. Notification of
the new location of blood donation and the nearest location where they can
do so are among the information that can be updated. Social networking is
an innovative type of communication for blood donation circle since it allows
donors and donation centers to communicate.
Furthermore, providing relevant information on blood donation needs
may enable prospective donors to make a clear decision to volunteer in blood
donation especially among young people (Tanye & Ahorlu, 2015). This is
agreed by Vimal et. al., (2020) where the information through media and
peers, are influential in determining a donor’s decision to donate blood in
19
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

future. Information on blood donation will create awareness among donors

about blood donation campaigns.
Therefore, it can be concluded that the factors that have been identified
and tested the reliability are seen to be consistent with some findings from
previous studies and earlier research work.

4.0 Conclusion
This study has focusing on verifying the factors that motivate individual
to donate blood which are appreciation and notification. Finding from this
study is proven reliable and can be used as a variable to assess the level of
motivation among donor to donate blood as well as a repeat donor for future
research. Identifying the factors that motivate individuals to donate can help
in comprehensive coordination and cooperation activities from various parties
especially the government and non-governmental organizations. This is
important to increase the encouragement of blood donation in the community
as well as educating the younger generation on blood donation practices.
Hence, it will help to recruit more voluntary donors and also to achieve
retention of existing blood donors.
Every blood bank should use a strategy to get individuals to voluntarily
donate blood on an ongoing basis. Regular blood donation awareness
campaigns and promotion through technology need to be expanded to
strengthen social support as well as to recruit new blood donors on a
voluntary basis.

References

Basnotra, R. M., & Sidhu, M. D. (2019). Donor notification in reactive donors

and their response to communication. International Journal of Research
in Medical Sciences, 7(4), 1088–1092.

Cicolini, G., Comparcini, D., Alfieri, S., Zito, E., Marta, E., Tomietto, M., &
Simonetti, V. (2019). Nursing students’ knowledge and attitudes of blood
donation: A multicentre study. Journal of Clinical Nursing, 28(9–10),
1829–1838.

Cohen, J. (1960). A Coefficient of agreement for nominal scales. Educational

and Psychological Measurement, 20(1), 37–46.

Domingos, D. C. L., Lima, L. F. S. G., Messias, T. F., Feijo, J. V. L., Diniz, A.

A. R., & Soares, H. B. (2016). Blood hero: An application for encouraging
the blood donation by applying gamification. Proceedings of the Annual
International Conference of the IEEE Engineering in Medicine and Biology
Society, EMBS, 2016-Octob, 5624–5627.

Fleiss, J. L., & Cohen, J. (1973). The Equivalence of Weighted Kappa and the
Intraclass Correlation Coefficient as Measures of Reliability. Educational
and Psychological Measurement, 33(3), 613–619.

20
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

Jaafar, J. R., Chong, S. T., & Alavi, K. (2017). Experience as factors associated
with repeat blood donation among university students in Malaysia.
Southeast Asian Journal of Tropical Medicine and Public Health, 48(5),
1118–1126.

Klinkenberg, E. F., Huis In’t Veld, E. M. J., de Wit, P. D., van Dongen, A.,
Daams, J. G., de Kort, W. L. A. M., & Fransen, M. P. (2019). Blood
donation barriers and facilitators of Sub-Saharan African migrants and
minorities in Western high-income countries: a systematic review of the
literature. Transfusion Medicine, 29(S1), 28–41.

Landis, J. R., & Koch, G. G. (2013). The Measurement of Observer Agreement

for Categorical Data. Biometrics, 33(1), 159–174.

M.Mostafa, A., E.Youssef, A., & Alshorbag, G. (2014). A Framework for a

Smart Social Blood Donation System Based on Mobile Cloud Computing.
Health Informatics - An International Journal, 3(4), 1–10.

Mahamod, Z., & Ishak, N. H. M. (2003). Analisis Cohen Kappa Dalam

Penyelidikan Bahasa: Satu Pengalaman. Seminar Penyelidikan
Pendidikan Guru Peringkat Kebangsaan, 1–7. Retrieved from
http://www.ipbl.edu.my/portal/penyelidikan/seminarpapers/2003/za
mriUKMkk1.pdf

Mohammed, S., & Essel, H. B. (2018). Motivational factors for blood donation,
potential barriers, and knowledge about blood donation in first-time and
repeat blood donors. BMC Hematology, 18(1), 1–9.

Pal, R., Sengupta, S., & Shenga, N. (2008). Behavior disparities towards blood
donation in Sikkim, India. Asian Journal of Transfusion Science, 2(2), 56.

Pallavi, P., Vijaya, B., & Rashmi B, K. (2017). Blood donor Notification: Boon
for the community, bane for blood donors and blood Centers? Global
Journal of Transfusion Medicine, 2(2), 124–129.

Raza, N., Qureshi, H., & Qureshi, A. (2018). Reactive blood donor notification;
their responses and perceptions: Experience From Southern Pakistan.
Journal of Ayub Medical College, Abbottabad  : JAMC, 30(3), 351–355.

Suemnig, A., Konerding, U., Hron, G., Lubenow, N., Alpen, U., Hoffmann, W.,
Greinacher, A. (2017). Motivational factors for blood donation in first-time
donors and repeat donors: a cross-sectional study in West Pomerania.
Transfusion Medicine, 27(6), 413–420.

Sümnig, A., Feig, M., Greinacher, A., & Thiele, T. (2018). The role of social
media for blood donor motivation and recruitment. Transfusion, 58(10),
21
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875
 Inter-rater Reliability Assessment for Motivating Factors in Blood Donation Using Cohen's Kappa Analysis

2257–2259.

Tanye, S., & Ahorlu, C. S. (2015). Perceptions of Voluntary Blood Donation

Among the Youth in La Dadekotopon.

Tey, Y. S., Arsil, P., Brindal, M., Lee, S. K., & Teoh, C. T. (2020). Motivation
structures of blood donation: a means-end chain approach. International
Journal of Health Economics and Management, 20(1), 41–54.

Vimal, M., J, B., & Nishanthi, A. (2020). Motivating and deterring factors for
repeat blood donation among donors of blood bank. IP Journal of
Diagnostic Pathology and Oncology, 5(1), 53–56.

Wood, J. M. (2007). Understanding and Computing Cohen’s Kappa: A

Tutorial. In WebPsychEmpiricist (pp. 1–18).

Yuan, S., Hoffman, M., Lu, Q., Goldfinger, D., & Ziman, A. (2011). Motivating
factors and deterrents for blood donation among donors at a university
campus-based collection center. Transfusion, 51(11), 2438–2444.

22
Politeknik & Kolej Komuniti Journal of Social Sciences and Humanities, Vol. 6, No. 1, 2021
eISSN 0128-2875