Proceedings of the Fifth International Conference on Communication and Electronics Systems (ICCES 2020)

IEEE Conference Record # 48766; IEEE Xplore ISBN: 978-1-7281-5371-1

Machine Learning based Sarcasm Detection

on Twitter Data
Neha Pawar Sukhada Bhingarkar
School of Computer Engineering and Technology School of Computer Engineering and Technology
M IT World Peace University M IT World Peace University
Pune, India Pune, India
[email protected] [email protected]

Abstract: S arcasm is a subtle type of irony, which can be widely Therefore, social networks are usually widely used, in
used in social networks. It is usually used to transmit hidden
information to criticize and ridicule a person and to recognize. The
particular microblogging sites such as Twitter. Thus, the
sarcastic reorganization system is very helpful for the improvement modern approach to sentiment analysis and opinion
of automatic sentiment analysis collected from different social analysis usually performs lower indicators when
networks and microblogging sites. S entiment analysis refers to analyzing collected data, such sites. Maynard and
internet users of a particular community, expresse d attitudes and Greenwood [4] show that the effectiveness of sarcastic
opinions of identification and aggregation. In this paper, to detect analysis can be significantly improved when sarcasm is
sarcasm, a pattern-based approach is proposed using Twitter data.
Four sets of features that include a lot of specific sarcasm is
detected in sarcastic statements. Therefore, effective
proposed and classify tweets as sarcastic and non-sarcastic. The means of detecting sarcasm are required.
proposed feature sets are studied and evaluate its additional cost Identifying sarcasm helps with the task of analyzing mood
classifications. when it is performed on microblogging sites such as
Twitter.
Keywords: Sarcasm detection, Twitter, Sentiment analysis, Mood analysis and opinion mining rely on emotional
Machine learning. words to detect their polarity in a text (that is, whether it
relates to "positivity” or “ negativity” in its thread).
I. INTRODUCTION However, the text appearance can lead to confusion. [5]
Today the twitter has been a very biggest network, by [6]. The aim of this paper is to propose a system to
using peoples share their opinions and thoughts. Twitter automatically detect a sarcastic tweet.
has been an official site which contain active 288 million
users and sent 500 million tweets are daily. [1]. II. RELATED WORK
However, due to the limitations of the unofficial In recent years, attention paid to analyzing twitter's
language and characters used by Twitter (that is, 140 mood by researchers and a number of current documents
characters per tweet), it is very difficult to understand the have been applied to classifying tweets. Sriram [7]
opinions of users and conduct such an analysis. In classifies tweets into a predefined set of General classes,
addition, the presence of sarcasm is even more difficult: including events, opinions, transactions and private
sarcastic when a person says that they are not what they messages, non-contextual features used, such as the
mean [2]. presence of slang, phrases about temporary events,
Oxford dictionary express sarcasm as "the use of opinion by word, and information about Twitter users.
sarcasm to Express or convey contempt". Free Dictionary The author of the article [8] [9] proposed a method for
also describes sarcasm as irony intended to convey identifying emotional and verbal patterns in Twitter data.
contempt. [3]. Sarcasm detection is very difficult in real However, most of the work was done to classify the
life. tweets according to the polarity of user sentiment towards
As a rule, people use sarcasm in everyday life, not only the specific topics, focusing on the content of the tweet.
jokes and humor, but also criticism or comments, ideas, Various functions have been proposed. They include
types and effects. presence of diagrams [10], frequency and non-text
features such as emoticons [11] [12]. The author [13],
defines framework that learns to classify the words and
that words emotions of context.
Sarcasm has been used in everyday conversation in a very
long time. Therefore, sarcasm in terms of psychological
[14] and neurobiological [15] is the subject of deep
research.
However, it has been studied as a language behavior
that characterizes a person. In [16], Burfoot and Baldwin
introduced a set of attributes, including profanity and
slang use, and they are certified "semantic action" and

978-1-7281-5371-1/20/$31.00 ©2020 IEEE 957

Authorized licensed use limited to: Texas Tech University. Downloaded on May 24,2022 at 13:52:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Fifth International Conference on Communication and Electronics Systems (ICCES 2020)
IEEE Conference Record # 48766; IEEE Xplore ISBN: 978-1-7281-5371-1

also used SVM classifier to classify satirical articles. In

[17], studying the contextual components that are used to
convey sarcastic ironic words, suggest that sarcasm
requires the presence of four people. Tepperman [18], the
proposed approach will find automatically sarcasm in
conversational dialogue.
In [19] proposed to classify this text as political,
humorous, sarcastic and sarcastic. In paper [20] formulate
the task of detecting sarcasm as a task of removing
ambiguity in the sense that the word can have a literal
meaning or irony, and therefore the irony of the word.
Kings et al. [21], this suggests that instead of trying to
decide if a tweet is sarcastic, it make sense to understand
context, but the majority of snippets tend to identify the
typeset clearly as sarcastic and non-sarcastic.
Davidov [22] and Tsur [23], a semi-controlled
algorithm for sarcasm identification is proposed. They
experimented with two sets of data: one is amazon and the Fig. 1. Block diagram of the proposed system
other is Twitter. The results were important and their
approach depended on the frequency of word detection, The system uses the SVM, KNN, and Random forest
but the training sets were related to the topic and this classifier.
approach treated what are called contextual words,
A. Dataset
regardless of grammatical features. It also does not
distinguish between sentimental and unconscious words. Sarcastic tweets, 9,104 tweets containing #sarcasm,
Samples that do not take into account words of emotional and #not is used. The link on which dataset is available:
content or cancel them low in order to be able to reduce https://www.kaggle.com/welkin10/airline-sentiment and
will use the Twitter English and Hindi tweets dataset to
the probability approximation.
validate the Framework. The data preparation step is the
In [24], Twitter users relied on the hashtags they were
most important and time-consuming stage of the
employing to identify the irony of the tweet. Riloff et al.
implementation of the problem. Since the data is
[25], when a positive mood contrasts with a negative
considered as the input of the project, the higher accuracy
situation, a method of detecting a certain type of irony is of this input leads to a more accurate output.
proposed. They use the single first word "love" and a The tweets are also remove that contain descriptions
series of sarcastic tweets to automatically detect or URLs to get information about productive media or
expressions that indicate a positive mood or phrase URLs. The tweet words are very noisy e.g, Yeah, right!
quoting a negative situation. And #sarcasm. The 9104 sarcastic tweets after
In [26] introduced behavioral modeling to detect irony preprocessing remove #sarcasm and #not hashtags from
on Twitter. They identified a variety of forms of irony and the tweets before proceeding to the rating. The twitter
its manifestations on Twitter, showing the importance of sample API is used for the collection of not sarcastic
historical statistics gathered from the past tweets and tweets. These tweets received the same for mentioned
identifying irony. It proved to be very effective, but this pre-processing techniques.
approach is less effective in the absence of previous The Hindi tweets are handled by following steps:
knowledge of the user. It is to determine the tweet from
the collected data relying on the best feature extraction. In ƒ The document is split into word. Such as split
the real-time stream of tweets posted by random users, the into the whitespace and punctuation and
approach is difficult to implement, and the knowledge removes punctuation and spaces to resize.
base size is very fast. The author in [27], uses machine ƒ Vocabulary building. All document words
learning to detect and classify sarcastic tweets, to vocabulary is collect and number them in
investigate the impact of vocabulary and practical factors alphabetical order.
on the performance, and to classify positive and negative ƒ Encoding. Count the words appear in the
tweets. vocabulary (Encode in machine language that
is binary (0, 1)
III. PROPOSED SYSTEM
This given a set of tweets, our goal is to rank each one
according to whether it is sarcastic or not. Fig 1, shows
the block diagram of the proposed system.

978-1-7281-5371-1/20/$31.00 ©2020 IEEE 958

Authorized licensed use limited to: Texas Tech University. Downloaded on May 24,2022 at 13:52:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Fifth International Conference on Communication and Electronics Systems (ICCES 2020)
IEEE Conference Record # 48766; IEEE Xplore ISBN: 978-1-7281-5371-1

3) The features related to syntactic and Semantic:

In addition to the functions associated with punctuation,
some widely used expressions are usually used in a
sarcastic context. Associate these expressions in
punctuation to determine if what is being said sarcastic or
not. In addition, in other cases, people as a rule, complex
sentences or use unusual words to hide the listener /
reader to make a clear answer. When the sarcasm is used
this is very common as an "evasion", and the person aims
to hide their true feelings and opinions with sarcasm.
Thus, the following characteristics that reflect these
aspects:

ƒ Usage of unusual words

ƒ Amount of interjections
ƒ Number of unusual words
ƒ Existence of usual sarcastic expressions
ƒ Number of laughing expressions

Fig. 2. Flow of handling of Hindi tweets Eg: ‘‘You are incredibly funny -_- ’’

When the algorithm is used on data, the machine learns 4) The feature related to pattern
on the basis of type of data, like provide input tweets, and The selection pattern of the previous subsection and
there output is either positive, negative or neutral. So, qualified “general ironic expression “is very common and
when the machine learns itself, there is not any issue, for even in conversation. However, their number is small,
they are not unique, and our training and test seals largely
which language input is provided, the only matter is their
do not include them.
output.
In this approach, the words are classified according to two
categories: high-frequency words and content words that
B. Feature Extraction are based on its data, the frequency of the frequency, and
Next, the system feature extraction is carried out for the determining the sample as high-frequency words and slots
data. Four features are extracted are as follows: in the ordered sequence of contextual words.

1) The features related to sentiment: C. Training and Testing Data

Opinion mining or sentiment analysis is the process by
which an individual determines the emotions expressed in Training data output available on the model while the
his or her writing. A very common type of irony, widely testing data is unseen data for which predictions must be
used in both normal conversations, this type of combat done. The K-fold cross-validation is used 10 times for
qualifies as "whimper", the management of the social training and testing datasets. To provide solution to this
network "twitter". (e.g., ‘‘All the time I enjoy being problem, K-fold Cross Validation is used that divide the
ignored). data into folds and making sure that some stage each fold
is used as a test set. Train and test the extracted data, and
2) The features related to Punctuation: perform an SVM, KNN, and Random forest algorithm for
For detecting any type of sarcasm the feature related to prediction of sarcastic or not and calculate the accuracy of
sentiments are not enough and that all components of each algorithm. This method will train about 70%of the
tweets are not used. Sarcasm is a sophisticated type of given data set, and the remaining 30% will be used for
expression, as already mentioned, not only does it play a testing purposes.
word or meaning, it converts these aspects into
punctuation or repeated use of vowels when a message is
D. Classification
written, such as a low-tone facial gesture. After detecting
these aspects, the set of qualified properties are extracted. The classification are run using the classifiers such as
The values for each tweet is calculates: Support Vector Machine (SVM), k Nearest Neighbors
(KNN), and Random forest. The result section presents
ƒ Number of all-capital words the performance of classifiers on the dataset.
ƒ Number of dots
ƒ Number of exclamation marks IV. RESULT AND DISCUSSION
ƒ Number of question marks In the approach to assessment used key performance
ƒ Number of quotes indicators (KPIs). Accuracy, Precision, and recall are
interesting ratings in classification problems. Statistics

978-1-7281-5371-1/20/$31.00 ©2020 IEEE 959

Authorized licensed use limited to: Texas Tech University. Downloaded on May 24,2022 at 13:52:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Fifth International Conference on Communication and Electronics Systems (ICCES 2020)
IEEE Conference Record # 48766; IEEE Xplore ISBN: 978-1-7281-5371-1

from the point of view of a recall are the ratio of the

number of true positives (TP) and the sum of true
positives and negative positives (TN).
ƒ Accuracy: It shows the overall accuracy of the
instances which are correctly classified to the
total number of the instances. It is calculated by
the following formula:

= (1)

Where, TP = true positive, TN = true negative,

FP = false positive, FN = false negative.

ƒ Precision: It represents the percentage of

relevant searched sarcastic tweets. That is, it
measures the amount of tweets categorized as
sarcasm against the total amount of tweets
classified as sarcasm. It is calculated by the Fig. 3. Accuracy of Classifiers
following formula:
The random forest obtains 81 % of accuracy and the F1-
= (2) score is 79.00%. SVM is presented 77.99 % of precision
for a low F1-score equals to 22.15%. The SVMs can
ƒ Recall: It represents the percentage of relevant detect sarcasm with high precision, and can use output to
sarcastic tweets that have been searched. That is, narrow down sentiment analysis, but the actual results of
against the total amount of sarcastic tweets, all sarcastic tweets can be very different. The classifiers
measured the amount of tweets that are normally Random Forest presents the highest accuracy and F1
classified as sarcastic. It is calculated by the score as compared to SVM and KNN classifiers.
following formula:
V. CONCLUSION
= (3)
In this work, a system is proposed which detects
sarcasm on English as well as on Hindi tweets in Twitter.
Finally, the F1 score is a measure of accuracy that can be Sarcasm is very dependent and highly contextual;
interpreted as a weighted average of accuracy and recall: therefore, sentiment and other contextual clues to help
detect the sarcasm text. The system uses sarcastic tweets,
1 = 2.
.
(4) 9,104 tweets containing #sarcasm, and #not dataset. The
system uses the SVM, KNN, and Random forest
classifier. The approach has shown good results and it is
The paper presents the comparative results of the four observed that Random forest classifier has more accuracy
classifiers utilized. than other classifier. All patterns for sarcastic detection
are not covered in the extracted patterns. So, in future
Table 1 and Fig 3, show the accuracy, recall, F1-score and Neural Network, Genetic Algorithm and Pattern-based
precision, of classification using different classifiers. approach can be combined for more accuracy.
Table I shows the accuracy of Random Forest, SVM and
KNN.

Table- I: Accuracy of Random Forest, SVM and KNN REFERENCES

Random 1. Yi T ay, Mondher Bouazizi And T omoaki Otsuki (Ohtsuki), “ A
SVM KNN
Fore st
Pattern-Based Approach For Sarcasm Detection On T witter” in
Accuracy 81% 74 % 58% August 24, 2016
2. Yi T ay†, Luu Anh T uan, Siu Cheung Huiφ, JianSuδ, “ Reasoning
with Sarcasm by Reading In-between”arXiv:1805.02856v1
Precision 82% 77.99% 59.63% [cs.CL] 8 May 2018
3. Aditya Joshi, Pushpak Bhattacharyya, Mark J. Carman.2018,
Recall 63.34% 64% 51% “ Sarcasm Detection Using Incongruity within T arget T ext”, In:
Investigations in Computational Sarcasm”,Cognitive Systems
Monographs, vol 37. Springer
F1-Score 79.00% 22.15% 66.8% 4. . Aditya Joshi, Pushpak Bhatta charyya,Mark J. Carman.2018,
“ Sarcasm Detection Using Contextual Incongruity. . . In:
Investigations in Computational Sarcasm”, Cognitive Systems
Monographs, vol 37. Springer

978-1-7281-5371-1/20/$31.00 ©2020 IEEE 960

Authorized licensed use limited to: Texas Tech University. Downloaded on May 24,2022 at 13:52:13 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Fifth International Conference on Communication and Electronics Systems (ICCES 2020)
IEEE Conference Record # 48766; IEEE Xplore ISBN: 978-1-7281-5371-1

5. Shubhadeep Mukherjee, Dr. Pradip Kumar Bala, “ Sarcasm

Detection in MicroblogsUsing Naïve Bayes and Fuzzy Clustering”
In Proceedings of T echnology in Society,2017 pages 19 -27
6. Nishant Nikhil, Muktabh Mayank Srivastava, “ Binarizer at
SemEval-2018 T ask 3:Parsing dependency and deep learning for
irony detection”arXiv preprintarXiv:1805.01112
7. B. Sriram, D. Fuhry, E. Demir, H. Ferhatosmanoglu, and M.
Demirbas, “ Short text classification in twitter to improve
information filtering,” in Proc. 33rd Int. ACM SIGIR Conf. Res.
Develop. Inf. Retr., Jul. 2010.
8. C. G. Akcora, M. A. Bayir, M. Demirbas, and H. Ferhatosmanoglu,
“ Identifying breakpoints in public opinion,” in Proc. 1st Workshop
Soc. Media Anal., Jul. 2010.
9. M. W. Berry, Ed., Survey of T ext Mining: Clustering,
Classification, and Retrieval. New York, NY, USA: Springer-
Verlag, 2004.
10. B. Pang, L. Lillian, and V. Shivakumar, “ T humbs up: Sentiment
classification using machine learning techniques,” in Proc. ACL
Conf. Empirical Methods Natural Lang. Process., vol. 10. Jul.
2002
11. M. Boia, B. Faltings, C.-C. Musat, and P. Pu, ``A Is worth a
thousand words: How people attach sentiment to emoticons and
words in tweets,'' in Proc. Int. Conf. Soc. Comput., Sep. 2013.
12. K. Manuel, K. V.Indukuri, and P. R. Krishna, “ Analyzing internet
slang for sentiment mining, ”in Proc. 2nd Vaagdevi Int. Conf.
Inform. T echnol. Real World Problems, Dec. 2010.
13. L. Dong, F.Wei, C. T an, D. T ang, M. Zhou, and K. Xu, “ Adaptive
recursive neural network for target-dependent T witter sentiment
classi cation,” in Proc. 52nd Annu. Meeting Assoc. Comput.
Linguistics, vol. 2. Jun. 2014,
14. F. Stringfellow, Jr., T he Meaning of Irony: A Psychoanalytic
Investigation. New York, NY, USA: State Univ. New York, 1994.
15. S. G. Shamay-T soory, R.T omer, and J. Aharon-Peretz, “ T he
neuroanatomical basis of understanding sarcasm and its
relationship to social cognition,” Neuropsychology, vol. 19, , May
2005.
16. C. Burfoot and T . Baldwin, “ Automatic satire detection: Are you
having a laugh?” in Proc. ACL-IJCNLP, Aug. 2009.
17. J. D. Campbell and A. N. Katz, “Are there necessary conditions for
inducing a sense of sarcastic irony?” Discourse Process. vol. 49,
no. 6, Aug. 2012.
18. J. T epperman, D.Traum, and S. S. Narayanan, “Yeah right': Sarcasm
recognition for spoken dialogue systems,” in Proc. InterSpeech,
Sep. 2006
19. T . Veale and Y. Hao, “ Detecting ironic intent in creative
comparisons,” in Proc. ECAI, Aug. 2010,
20. D. Ghosh, W. Guo, and S. Muresan, “ Sarcastic or not: Word
embeddings to predict the literal or sarcastic meaning of words,” in
Proc. EMNLP,Sep. 2015
21. Z. Wang, Z. Wu, R. Wang, and Y. Ren, “Twitter sarcasm detection
exploiting a context-based model,” in Proc. Web Inf. Syst. Eng.
(WISE), Nov. 2015
22. O. T sur, D. Davidov, and A. Rappoport, “ ICWSM-A great catchy
name: Semi-supervised recognition of sarcastic sentences in online
product reviews,” in Proc. AAAI Conf. Weblogs Soc. Media, May
2010,
23. D. Davidov, O. T sur, and A. Rappoport, “ Semi-supervised
recognition of sarcastic sentences in T witter and Amazon,” in
Proc. 14th Conf. Comput. Natural Lang. Learn., Jul. 2010
24. D. Maynard and M. A. Greenwood, “ Who cares about sarcastic
tweets? Investigating the impact of sarcasm on sentiment
analysis,” in Proc. 9th Int. Conf. Lang. Resour. Eval., May 2014
25. E. Riloff, A. Qadir, P. Surve, L. De Silva, N. Gilbert, and R. Huang,
“ Sarcasm as contrast between a positive sentiment and negative
situation,”, in Proc. Conf. Empirical Methods Natural Lang.
Process., Oct. 2013
26. A. Rajadesingan, R. Zafarani, and H. Liu, ``Sarcasm detection on
T witter:A behavioral modeling approach,'' in Proc. 18th ACM Int.
Conf. Web Search Data Mining, Feb. 2015
27. S. Muresan, R. Gonzalez-Ibanez, D. Ghosh, and N. Wacholder,
“ Identification of nonliteral language in social media: A case study
on sarcasm,” J. Assoc. Inf. Sci. T echnol., Jan. 2016.

978-1-7281-5371-1/20/$31.00 ©2020 IEEE 961

Authorized licensed use limited to: Texas Tech University. Downloaded on May 24,2022 at 13:52:13 UTC from IEEE Xplore. Restrictions apply.