Fake Content Detection

Information Processing and Management 59 (2022) 102740
Contents lists available at ScienceDirect
Information Processing and Management

journal homepage: www.elsevier.com/locate/ipm
What the fake? Probing misinformation detection standing on the

shoulder of novelty and emotion
Rina Kumari 1 , Nischal Ashok ∗,1 , Tirthankar Ghosal, Asif Ekbal
Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihta, Bihar, 801106, India
ARTICLE INFO ABSTRACT
Keywords: One of the most time-critical challenges for the Natural Language Processing (NLP) community
Fake news detection is to combat the spread of fake news and misinformation. Existing approaches for misinfor-
Novelty prediction mation detection use neural network models, statistical methods, linguistic traits, fact-checking
Emotion prediction
strategies, etc. However, the menace of fake news seems to grow more vigorous with the advent
Deep learning
of humongous and unusually creative language models. Relevant literature reveals that one
major characteristic of the virality of fake news is the presence of an element of surprise in
the story, which attracts immediate attention and invokes strong emotional stimulus in the
reader. In this work, we leverage this idea and propose textual novelty detection and emotion
prediction as the two tasks relating to automatic misinformation detection. We re-purpose textual
entailment for novelty detection and use the models trained on large-scale datasets of entailment
and emotion to classify fake information. Our results correlate with the idea as we achieve
state-of-the-art (SOTA) performance (7.92%, 1.54%, 17.31% and 8.13% improvement in terms
of accuracy) on four large-scale misinformation datasets. We hope that our current probe will
motivate the community to explore further research on misinformation detection along this line.
The source code is available at the GitHub.2
1. Introduction
Misinformation on the web and social media, deceptive websites is a real problem nowadays and has a profound social, economic,
and political impact resulting in unwanted consequences like election interference, polarization, violence, etc. (Scheufele & Krause,
2019). The problem appears more intense in this situation of a global health crisis, and misinformation on the COVID-19 pandemic
could result in an unprecedented health disaster. We already saw several myths are surfacing up on social media regarding medicines
for COVID-19, the virality of the infection spread, inflammatory articles targeting marginal communities, etc. (Brennen, Simon,
Howard, & Nielsen, 2020). For example, a rumor claiming that alcohol kills coronavirus led many people to drink counterfeit
alcohol containing toxic methanol in Iran. Because of this rumor, over 300 people died, around 1000 required hospitalization,
and many people are expected to have permanent vision loss. Similar misinformation has been spread that chloroquine (CQ)
and hydroxychloroquine (HCQ) can cure coronavirus. Further, World Health Organisation (WHO) has stated that these drugs are
generally prescribed for malaria patients and are not effective in curing the COVID-19. The problem seems profound in developing
countries because literacy levels are low, understanding and exposure to technology for fake news detection are limited, but
increasing access to cheap internet makes the mass more susceptible to believing and acting upon misinformation. Research on fake
∗ Corresponding author.
E-mail addresses: [email protected] (R. Kumari), [email protected] (N. Ashok), [email protected] (T. Ghosal), [email protected] (A. Ekbal).
1 First two authors contributed equally.
2 https://github.com/Nish-19/Misinformation-Novelty-Emotion.
https://doi.org/10.1016/j.ipm.2021.102740
Received 31 October 2020; Received in revised form 6 August 2021; Accepted 28 August 2021
Available online 13 October 2021
0306-4573/© 2021 Elsevier Ltd. All rights reserved.
R. Kumari et al. Information Processing and Management 59 (2022) 102740
news and misinformation (Hsu, Ajorlou, & Jadbabaie, 2019; Vosoughi, Roy, & Aral, 2018) shows that ‘‘Novelty’’ is a key attribute
for misinformation or fake news3 and contributes significantly to its viral spread and penetration in society. Novelty attracts human
attention and acts as a stimulus for information sharing and decision-making. The first task of fake news is to catch our attention,
and for this reason, novelty is the key. Psychologists suggest that one of the reasons hyperpartisan claims are so successful is that
they tend to be outlandish (Pennycook & Rand, 2019). Findings from a massive study on Twitter (Vosoughi et al., 2018) suggest
that news categorized as false or fake was 70 percent more likely than true news to receive a retweet. False news was more novel
than true news, and users were far more likely to retweet a tweet that was measurably more novel.
The ability of fake news to grab our attention and then hijack our learning and memory circuitry goes a long way explaining its
success. But its most vital selling point is its ability to appeal to our emotions. Studies of online networks (Brady, Wills, Jost, Tucker,
& Van Bavel, 2017) show text spreads more virally when it contains a high degree of moral emotion, which drives everything we do.
According to findings in the earlier study by (Vosoughi et al., 2018), false stories inspired fear, disgust, and surprise in replies and
true stories inspired anticipation, sadness, joy, and trust. The novelty and emotional conviction of fake news and how these properties
interact with the framework of our memories exceed our brains’ analytical capabilities (Barr, 2019).
Inspired by these findings, we set out to explore the role of textual novelty and emotions from a Natural Language Processing
(NLP) perspective to classify misinformation content automatically. To the best of our knowledge, this is the very first attempt
in this direction. We use the large-scale pre-trained models on Textual Novelty Detection and Emotion Prediction tasks and design a
transfer learning-based deep neural architecture to detect misinformation/fake news. Our approach is based on a solid grounding on
earlier psychological findings that outperform the existing models by a wide margin. It is found to effectively combat the menace
of fake news in society.
The remainder of this paper is organized as follows. In Section 2, we present a brief survey of the related works. Section 3
explains the research objectives. Section 4 depicts the methodology in details. In Section 5, we first describe the datasets used for
our experiments and demonstrate the experiments performed, followed by a detailed analysis. Section 6 explains the theoretical and
practical implications of our work. Finally, Section 7 concludes our work along with some road maps for future direction.
2. Related work
Fake news is a challenging task in natural language processing. Exploration of allied tasks such as rumor detection (Akhtar,
Ekbal, Narayan, Singh, & Cambria, 2018), satire detection (Liu, Shabani, Balet, & Sokhn, 2019) and spam detection (Jain, Sharma,
& Agarwal, 2019) give intuition on how to deal with the fake news and misinformation. Our paper explores the role of two new
tasks: textual novelty detection and emotion recognition, for fake news detection. In this section, we discuss the recent literature
on viz. Novelty detection, Emotion prediction and Fake news detection.
2.1. Novelty detection
The novelty that is also called an anomaly, outlier, or exception is a pattern in the data because of which the performance of any
system exhibits unexpected behavior. Novelty detection, nowadays, has gained popularity in different domains such as Healthcare
informatics (An, Huang, & Wang, 2020), Industrial monitoring (Jose, Yosimar, & Alejandro, 2020), Image processing (Kerner et al.,
2019), Text mining (Kumar & Bhatia, 2020), etc. This task is very much related to the Textual Entailment Recognition (Xiong et al.,
2020) task. Nowadays, a novelty detection mechanism is frequently applied to detect whether a document is novel or non-novel
based on its content. Previous studies tried to determine the novelty either at the lexical level or sentence level (Lee, 2015), which
cannot resolve the semantic-level redundancy. Researchers have implemented these techniques using a rule-based or traditional
feature-based machine learning framework that relies on handcrafted features extracted from the documents.
Recent studies focus on automatic feature extraction using deep learning and neural networks. A study (Ghosal, Edithal, Ekbal,
Bhattacharyya, Chivukula, & Tsatsaronis, 2020) introduced document-level novelty detection tasks to identify the novelty of a
document concerning the source document. This work presents a neural attention mechanism to detect document-level novelty
without any feature engineering. It demonstrates that the simple alignment of texts between source and target document could
determine the state of a target document’s novelty.
Textual Entailment, which is closely related to novelty detection also discovers the relation between the text fragments. The
research reported in (Saikh, Ghosal, Ekbal, & Bhattacharyya, 2017) proposed several machine learning algorithms, such as Support
Vector Machine (SVM), Random Forest (RF), and Multilayer Perceptron (MLP) that identify the document level novelty using textual
entailment. Apart from these general domains, researchers have proposed the novelty detection task in some specific domains. The
work in (Rohit, Deshmukh, & Jagdale, 2018) proposed novelty detection tasks in the news stream due to the high proliferation
of misleading and redundant information. It finds the news documents similarities using distance measurement techniques such as
cosine similarity, Euclidean distance, Jaccard distance, binary distance, etc.
Novelty detection is also beneficial for non-redundant document retrieval tasks. To explore the novel document retrieval task,
the authors in (Breja, 2015) demonstrated a novel approach for novelty detection in web documents to retrieve relevant and
non-redundant information. Further, (He, Meng, Wu, Chan, & Pang, 2020) has given a mechanism that can be appended with
a web crawler to avoid redundancy and retrieve the most relevant documents. This novelty detection mechanism is based on
3 Throughout the paper, we use the terms fake news detection or misinformation detection interchangeably.
2
semantic similarity and text summarization that first summarizes the text based on ontology and calculates semantic similarity
from the obtained summary. With the help of semantic similarity, it detects whether the document is novel or non-novel. In another
study (Amplayo, Hong, & Song, 2018), the authors have presented a network-based approach for the novelty detection of scholarly
literature. This mechanism introduces an auto-encoder network as the novelty detection model, which is trained using graph features.
High novelty increases the chance of the news post being fake. Inspired by this assumption, the authors in (Qin, Wurzer, Lavrenko,
& Tang, 2016) have presented a mechanism for rumor detection using novelty detection tasks. In this article, the authors used to
find the similarity between a new news article and existing rumors and use this similarity information as a novelty feature for rumor
detection. The authors in (Kumari, Ashok, Ghosal, & Ekbal, 2021) have investigated the role of textual novelty in the news article
and proved that it is a key attribute that plays a vital role in fake news detection.
2.2. Emotion prediction
Emotions are a vital part of our lives that reflect feelings such as happiness, sadness, joy, and fear, which affect our daily life
activities. Nowadays, social networking websites such as Twitter, Facebook, Instagram, etc., are the most popular platforms to
express and share emotions and opinions. Data such as news headlines, social media posts, blog posts, and song lyrics have been
efficiently utilized for the extensive study and design of emotion detection computational models (Abdul-Mageed & Ungar, 2017;
Liew & Turtle, 2016). The Emotion detection task is similar to other classification tasks that consider each emotion category as a
class label. Researchers study fine-grained emotions and their distributions in the news posted on various social media platforms.
(Zhou, Zhang, Zhou, Zhao, & Geng, 2016) presents a novel methodology based on emotion distribution learning to predict several
emotions with varying intensities in a single sentence. In (Wang & Pal, 2015), the proposed mechanism gives the emotion distribution
as output based on a dimensionality reduction technique using non-negative matrix factorization. It designs a constraint optimization
framework by combining different constraints such as topic correlations, emotions bindings, and emotion lexicons. The framework
proposed in (Becker, Moreira, & dos Santos, 2017) investigates textual emotion classification in a multilingual context using
NaiveBayes, SVM, and Radial Basis Function(RBF). Further, it combines these classifiers’ results to form meta classifiers, namely
Bagging and Boosting, for the emotion prediction. (Ren & Hong, 2019) proposed an emotion classification approach to extract the
discrete emotions from online reviews and investigated the differential effects of three discrete emotions(fear, anger, and sadness)
on perceived review helpfulness.
Recently, emotion prediction started to emerge in the fake news detection domain. The emotional response (e.g., like, comment,
share) on a news post plays an inevitable role in fake news dissemination. The studies in (Ghanem, Rosso, & Rangel, 2020; Zarrabian
& Hassani-Abharian, 2020) show that the fake news generated response contains a high probability and intensity of different
emotional effective information such as fear, disgust, and surprise. On the other hand, real news includes a high probability and
intensity of joy, sadness, and trust. Another literature (Giachanou, Rosso, & Crestani, 2019) investigates the role of emotional signals
in fake news detection. In this literature, the author proposed a Long Short Term Memory (LSTM) network model that incorporates
emotional signals extracted from the claims’ text to determine the credibility of the claims. (Giachanou, Rosso, & Crestani, 2021)
have also explored the extraction of emotional signals from claims to investigate their effects on credibility assessment. A deep
learning based mechanism presented in FakeFlow (Ghanem, Ponzetto, Rosso, & Rangel, 2021) learns the flow of affective information
that affects the reader’s emotions. It uses the combination of topics and the extracted affective information to analyze this flow.
Previous studies only leverage the emotion-aware feature representation of claims or actual news contents to detect the fake news,
but (Guo, Cao, Zhang, Shu, & Yu, 2019) has proposed a mechanism that learns the emotion-aware representation of contents and
comments both for publishers and users, respectively. It exploits the dual emotion representations simultaneously for fake news
detection. (Kumari, Ashok, Ghosal, & Ekbal, 2021) have proposed multitask model for fake news in which emotion prediction has
been performed as an auxiliary task. The result has proved the effectiveness of emotion in fake news prediction.
2.3. Fake or misinformation detection
Although fake news is not new, it has gained much attention in recent years due to the rapid dissemination of fake information
on online social media platforms, such as Facebook, Twitter, and microblogs. Rumor detection is very similar to fake news detection
because fake news is a type of false rumor. The method proposed in (Alkhodair, Ding, Fung, & Liu, 2020) detects breaking news
rumors rather than long-lasting rumors. It demonstrates an approach that simultaneously learns word embeddings and designs a
recurrent neural network with multiple objectives to identify breaking news rumors automatically. The work proposed in (Liu,
Jin, & Shen, 2019) tries to capture dynamic changes of news contents, news spreaders, and diffusion structures. In this paper, the
authors have implemented an LSTM based early rumor detection model to identify rumors in the initial stage. An article depicts that
false information can be in the form of either Misinformation or Disinformation. Disinformation is false information, which is shared
intentionally to cause harm. In contrast, the false information that does not lead to any harm is called Misinformation (Claire Wardle,
2017).
In another work (Zubiaga et al., 2018), authors have shown that the sequential classifier’s performance increases if it exploits
the discourse features extracted from social media interactions. The methods proposed in (Da San Martino, Yu, Barrón-Cedeno,
Petrov, & Nakov, 2019) extract the linguistic features of propaganda text by analyzing how the language pattern of real news is
different from the text containing propaganda. In recent times authors leverage contextual information as discussed in (Lu & Li,
2020), which depicts Graph-aware Co-Attention Networks (GCAN) for fake news detection. All the existing works describe how
we can stop the dissemination of fake news, but the authors in (Preslav, 2020) show how we can detect fake news before it
3
was even written. They developed media profiles that indicate the degree of propagandistic content, hyper-partisanship, leading
political ideology, and stance concerning various topics and claims. In (Slovikovskaya & Attardi, 2020), authors treated the fake
news detection task as similar to the stance detection task. They proposed various BERT models for the fake news Classification
task. Apart from textual and linguistic features, social context also provides useful evidence for fake news detection. Recently,
in (Shu, Wang, & Liu, 2019), authors have depicted how the social contexts are utilized in fake news detection. This paper proposed
a framework for tri-relationship embedding named TriFN that simultaneously models user-news interactions and publisher-news
relations for fake news classification. (Shu, Cui, Wang, Lee, & Liu, 2019) explains why a particular piece of news or post is detected
as fake. The authors have developed a sentence-comment co-attention sub-network that exploits news contents along with user
comments to jointly learns and captures check-worthy sentences and user comments for explainable detection of fake news.
Misinformation detection has experienced more popularity, especially after the COVID-19 infodemic. The coronavirus disease
(COVID-19) has caused a significant challenge for health systems, numerous rumors, hoaxes, and misinformation regarding
outcomes, prevention, and cure of the disease. As any misinformation regarding this pandemic has become a matter of life and
death, many researchers, journalists, and data scientists have taken an interest in investigating fake news and misinformation
related to this disease. Authors in (Pennycook, McPhetres, Zhang, Lu, & Rand, 2020) have discovered the reason for the belief
and spreading of fake news content about COVID-19. In another study (Kouzy et al., 2020), the authors have investigated the
importance of early interventions and provided an initial quantification of the magnitude of misinformation. (Cuan-Baltazar, Muñoz
Perez, Robledo-Vega, Pérez-Zepeda, & Soto-Vega, 2020) presents a study to evaluate the quality and readability of online information
about COVID-19 disease using validated instruments and relating the quality of information to its readability. This study in (Tasnim,
Hossain, Mazumder, et al., 2020) has shown the impact of rumors and misinformation on COVID-19 in social media.
3. Research objective
This section presents the specific objectives for investigating the role of novelty and emotional information for fake news
detection. Our research’s prime objective is to analyze a news post’s content spread on social media and detect whether it is fake
or real. Our current work presents an analysis of the extent to which a news post’s novelty and emotion boost the classification
performance. We set forth the following primary research objectives:
RO 1. Design a novel fake news detection framework leveraging novelty and emotion information that classifies the news posts as fake
or real.
This research’s main objective is to design a novel fake news detection framework that leverages novelty and emotional
information of the source and target news pairs and decides whether this news is fake or real. We perform this by implementing
Multilayer Perceptron (MLP) over the novelty+emotion shared representation. We evaluate the consistency of the proposed model
across different datasets. Further, we perform a detailed analysis to show the effectiveness of the proposed fake news detection
model.
4. Methodology
Research on fake news and misinformation shows that ‘‘Novelty’’ could be a key attribute for misinformation or fake news
and contribute significantly to its virality and penetration in society. Novelty attracts human attention and acts as a stimulus for
information sharing and decision-making. Fake news generated replies showing different emotional affecting information such as
fear, disgust, and surprise. Real news, on the other hand, inspired anticipation, sadness, joy, and trust emotion. In this paper,
we devise a deep learning and machine learning based technique for fake news detection. It leverages novelty and emotion-aware
information to decide whether a news article is fake or real. Fig. 1 depicts the overall architecture of the proposed model. It includes
three components, viz. Novelty detection model, Emotion prediction model and Fake news detection model. We discuss each
of these sub-models as below:
4.1. Novelty detection model
Novelty yields a vital aspect of a news post, and the task of novelty detection requires a high-level understanding of semantic
relationships within texts. We utilize Natural Language Inference (NLI) to exhibit such complex semantic interactions between the
text pairs. This part of the proposed architecture extracts the novelty-aware textual feature representations from the news headline
or article pairs.
Encoding Layers: Given a text pair that includes source as 𝐴𝑊1 , 𝐴𝑊2 , … , 𝐴𝑊𝑛 and target as 𝐵𝑊1 , 𝐵𝑊2 , … , 𝐵𝑊𝑛 ; where n is
the number of words in a sentence. Following (Yang, Zhang, Gao, Ji, & Chen, 2019), we take a pre-trained GloVe embedding
of 300-dimensions for each word and employ a sentence encoder based on a simple multi-channel Convolution Neural Net-
work(CNN) (Sotthisopha & Vateekul, 2018). We encode each sentence 𝑆𝑖 into the hidden representations ℎ𝐴𝑆 𝑎𝑛𝑑ℎ𝐵𝑆 , where 𝑆𝑖
𝑖 𝑖
is just the concatenation of word embeddings. We compute the hidden representations of source sentences using Eqs. (1) and (2).
ℎ𝐴𝑆 𝑝 = 𝑀𝑎𝑥𝑝𝑜𝑜𝑙(𝐶𝑜𝑛𝑣2𝑑(𝑆𝐴 (𝑝, 𝑝))) (1)

𝑖
ℎ𝐴𝑆 = 𝑐𝑜𝑛𝑐𝑎𝑡(𝑚𝑎𝑥(ℎ𝐴𝑆 𝑝 )) (2)

𝑖 𝑖
4
Fig. 1. Proposed overall fake news detection model leveraging novelty and emotion information. This model contains three components the final fake news
model, the novelty model, and the emotion model. The novelty and emotion model extracts the novelty and emotion-aware features, respectively, which are
further passed to the fake news detection (FND) model for the final fake news classification.
Similarly, we define the hidden representation of the target sentence as in Eq. (3).
ℎ𝐵𝑆 = 𝑐𝑜𝑛𝑐𝑎𝑡(𝑚𝑎𝑥(ℎ𝐵𝑆 𝑝 )) (3)

𝑖 𝑖
Here, 𝑝 ∈ [2, 3, 4]. To avoid information loss and better semantic representation of the sentence we make a residual connection
from the sentence representation(𝑆𝑖 ) to the hidden vectors (ℎ𝐴𝑆 ) and (ℎ𝐵𝑆 ) that gives a concatenated representation like Eq. (4).
𝑖 𝑖
𝑎 = [𝑆𝑖 ; ℎ𝐴𝑆 ]
𝑖
(4)
𝑏 = [𝑆𝑖 ; ℎ𝐵𝑆 ]
𝑖
We use the pre-trained GloVe embeddings (i) to extract novelty-aware textual feature extraction in line with the prior work (Yang,
Zhang et al., 2019); and (ii) There are evidences that GloVe achieves a better generalization for different tasks (Ajit Rajasekharan,
2020).
Alignment Layer: We encode the feature representations obtained from the encoding layer for both source and target pairs into
the alignment layer that finds the semantic interactions between source and target side sentences. It first projects the sentences into
a latent space as Eq. (5) and finds the dot product of the source–target pair using Eq. (6). We apply a softmax function over the
resultant vector using Eq. (7).
𝑒𝑎𝑖 = 𝑡𝑎𝑛ℎ(𝑊𝑎 𝑎𝑖 + 𝑏𝑠𝑎 )
(5)
𝑒𝑏𝑖 = 𝑡𝑎𝑛ℎ(𝑊𝑏 𝑏𝑖 + 𝑏𝑠𝑏 )
𝑑𝑎𝑏 = 𝑒𝑎𝑖 𝑒̇ 𝑏𝑖 (6)

𝑇 𝑑 )
𝑒𝑥𝑝(𝑑𝑎𝑏 𝑤
𝑆𝑎𝑏 = 𝑇 𝑑 )
(7)
𝛴𝑡 𝑒𝑥𝑝(𝑑𝑎𝑏 𝑤
5
Here, 𝑒𝑎𝑖 , 𝑒𝑏𝑖 are the projected vectors of source and target sentences respectively, 𝑑𝑎𝑏 is the dot product of both the vectors,
𝑑𝑤 is the weight, bs is the bias and 𝑆𝑎𝑏 represents the softmax output. This softmax function’s output is the attended and semantic
representation of the target sentence concerning the source sentence. We further encode this output into the fusion layer.
Fusion Layer: The fusion layer compares aligned representations and the local representations in three different perspectives
(shown in Eq. (8)) and fuses them. The output of the fusion layer is computed separately for both source and target sentences.
𝐹𝑎1 = 𝐺1([𝑎; 𝑆𝑎𝑏 ])
𝐹𝑎2 = 𝐺2([𝑎, (𝑎 − 𝑆𝑎𝑏 )]) (8)
𝐹𝑎3 = 𝐺3(𝑎; (𝑎◦𝑆𝑎𝑏 ))
𝐹𝑎 = 𝑚𝑎𝑥𝑝𝑜𝑜𝑙(𝐺([𝐹𝑎1 ; 𝐹𝑎2 ; 𝐹𝑎3 ])) (9)

Similarly,
𝐹𝑏 = 𝑚𝑎𝑥𝑝𝑜𝑜𝑙(𝐺([𝐹𝑏1 ; 𝐹𝑏2 ; 𝐹𝑏3 ])) (10)

We employ a pooling layer over the representation obtained from the fusion layer (described in Eqs. (9) and (10)) and encode
these pooled representations of both source and target sentence pairs into the feature extraction layer. Here G1, G2, G3, and G are
the single-layer feed-forward neural networks with independent parameters. 𝐹𝑎 and 𝐹𝑏 are the pooled representation of the source
and the target, respectively. ◦ denotes the element-wise multiplication. The multiplication finds the similarity between two vectors,
while subtraction highlights the difference.
Feature Extraction Layer: In this layer, we encode the pooled representations, and we concatenate the different aspects of
interactions between source and target sentences using Eq. (11). We project this combined representation into 200-dimensional
space using (12) and further pass it to an output layer with two neurons, according to Eq. (13). We employ a softmax activation
function over this output using Eq. (14).
𝐹 ′ = [𝐹𝑎 , 𝐹𝑏 , |𝐹𝑎 − 𝐹𝑏 |, 𝐹𝑎 ∗ 𝐹𝑏 ] (11)

𝑁𝐹 = 𝑡𝑎𝑛ℎ(𝑊𝐹′ ∗𝐹 ′
+ 𝑏𝑠′𝐹 ) (12)
𝑁𝐹′ = 𝑡𝑎𝑛ℎ(𝑊𝑁𝐹 ∗ 𝑁𝐹 + 𝑏𝑠𝑁𝐹 ) (13)
𝑒𝑥𝑝(𝑁𝐹′𝑇 𝑒𝑤 )
𝑒𝑎𝑏 = (14)
𝛴𝑡 𝑒𝑥𝑝(𝑒𝑇𝑎𝑏 𝑒𝑤 )
Here F’ is the interaction vector of source and target representations (obtained from the fusion layer), 𝑁𝐹 is the projected
representation, 𝑁𝐹′ is the output layer, and 𝑒𝑎𝑏 is the softmax output. The output of this softmax function gives the final novelty
labels (novel or duplicate). We pass these labels and the projected vectors into the Addition layer that guides the model to perform
the specific task.
4.2. Emotion prediction model
BERT model is a sequential composition of bidirectional transformer encoders based on the original transformer model (Vaswani
et al., 2017). Since the misinformation datasets ByteDance and FNC are a collection of news headline and/or article pair, they
do not contain rich emotional words and do not exhibit explicit emotion content. The GloVe embeddings are static, which does
not extract implicit emotional textual features, whereas BERT gives the contextual embedding, extracting the implicit emotional
features. This is the reason why we use BERT for emotion prediction. Given an input token sequence w = (𝑤1 , .., 𝑤𝑛 ), the concatenated
representation of word embeddings, positional embeddings, and the segment embeddings of this word sequence is given as the input
to the BERT model. A special token ([CLS]) is inserted as the first token, and another special token ([SEP]) is added as the final token
for each sentence. We use pre-trained 𝐵𝐸𝑅𝑇𝐵𝑎𝑠𝑒 model (Layer=12, Hidden size=768, Attention heads=12) (Devlin, Chang, Lee, &
Toutanova, 2019) context-dependent sentence representation for our emotion prediction task. The output of BERT is 𝐻 = (ℎ1 , .., ℎ𝑛 ).
We employ average polling over the contextual sentence representation vector H and apply Principle Component Analysis (PCA) (Bro
& Smilde, 2014) to obtain the 200-dimensional latent representation of emotion-aware feature representation of both source and
target sentences. We describe it in Eq. (15).
𝐸𝐹 𝑠 = 𝑃 𝐶𝐴(𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑝𝑜𝑜𝑙𝑖𝑛𝑔(𝐻𝑠 ))
(15)
𝐸𝐹 𝑡 = 𝑃 𝐶𝐴(𝐴𝑣𝑒𝑟𝑎𝑔𝑒𝑝𝑜𝑜𝑙𝑖𝑛𝑔(𝐻𝑡 ))
We further add the 200-dimensional features of both source and target and generate the combined emotion-aware representation
of the sentence pairs following Eq. (16).
𝐸𝐹 = 𝐸𝐹 𝑠 + 𝐸𝐹 𝑡 (16)
Here, 𝐸𝐹 𝑠 , 𝐸𝐹 𝑡 and 𝐸𝐹 are the emotion feature representation for the source, emotion feature representation for the target and
combined emotion feature representation, respectively. We pass the 𝐸𝐹 𝑠 and 𝐸𝐹 𝑡 into a dense layer and a softmax function for the
computation of emotion labels of the source following Eqs. (17) and (18).
𝐸𝐹′ 𝑠 = 𝑡𝑎𝑛ℎ(𝑊𝐸𝐹 𝑠 ∗ 𝐸𝐹 𝑠 + 𝑏𝑠𝐸𝐹 𝑠 ) (17)
6
𝑒𝑥𝑝((𝐸𝐹′𝑇𝑠 𝑒𝑤 ))
𝑒𝐸 ′ = (18)
𝐹𝑠 𝛴𝑡 𝑒𝑥𝑝((𝐸𝐹′𝑇𝑠 𝑒𝑤 ))
We compute the emotion labels for the target sentence similar to the source sentence and represent it as 𝑒𝐸 ′ . We then pass the
𝐹𝑠
scaffold labels along with the combined emotion-aware representation to the Addition layer. The Addition layer adds the combined
emotion-aware representation and the scaffold labels.
4.3. Fake news detection model
After we obtain the 200-dimensional novelty aware representation, novelty scaffold label representation, emotion-aware
representation, and emotion scaffold label representation, we add these four representations and pass them to a logistic regression
model that predicts whether the news is fake or real. Here, we use logistic regression because the model complexity is very low.
𝐹𝑁𝐸 = [𝑁𝐹 + 𝑒𝑎𝑏 + 𝐸𝐹 + 𝐹𝑙𝑎𝑏𝑒𝑙 ] (19)
Now the logistic function is,

1
𝑃 𝑟𝑜𝑏(𝐹𝑁𝐸 ) = 𝑃 (𝐹𝑁𝐸 ) = (20)
1 + 𝑒−𝐹𝑁𝐸
Here, 𝐹𝑁𝐸 is the Addition layer output, 𝑒𝑎𝑏 is the novelty label, 𝐹𝑙𝑎𝑏𝑒𝑙 is the emotion label, and Prob(𝐹𝑁𝐸 ) is the probability of
a news post being fake or real.
The misinformation datasets are not well-tuned for novelty and emotion detection tasks. The model may not extract better
novelty and emotion-aware textual feature representations if we train the novelty and emotion model using these misinformation
datasets. To avoid this limitation, we use the pre-trained novelty detection model using Quora Question Pair(QQP) dataset. We
consider the representation before the softmax layer as the novelty-aware textual feature representation and use it for fake news
detection. We show the results in Table 2. Further, we analyze that the model does not perform well because it does not learn the
dataset characteristics in the fake news detection context. So we append some part of the misinformation data with the QQP dataset
for proper learning. We train the novelty model using this new data and save the weights. Similarly, we save the weights of the
emotion model. Further, we use the representation obtained from the first dense layer (in the novelty model of Fig. 1) as novelty-
aware textual representation and the representation received from the PCA layer (in emotion model of Fig. 1) as the emotion-aware
textual representation for the fake news detection. So in this context, we use the term pre-trained for novelty and emotion model.
5. Datasets and experiments
This section presents the different datasets used for training the novelty, emotion, and fake news detection model. In real
applications, we have to provide the instances pair-wise (i.e. premise-hypothesis). The premise of this pair could be anything like
a piece of verified news, claim, or statement. All the discussions related to that premise over social media and/or news websites
could be hypotheses. For example, the WHO has posted verified news on Twitter, then this could be the premise and the comments,
retweets new post related to that news could be the hypothesis. According to the characteristics of the datasets, we are already aware
of the facts in the premises. So, we assume that if the premise is false and the hypothesis agrees with the premise, the hypothesis
must be false/fake and vice versa. Following this assumption, we decide that the hypothesis that agrees with the false premise is
fake and disagrees with the false premise can be real. Similarly, if the premise is true/real, the hypothesis agrees with the real
premise is real and disagrees with the real premise is fake.
We utilize publicly available real-world benchmark datasets to perform all the experiments of our work. We utilize Quora question
pair dataset4 (Imtiaz et al., 2020) to train the novelty model. To train the emotion model, we take the Unified dataset5 (Klinger
et al., 2018). Finally, we train our fake news detection model using ByteDance dataset6 (Xiaoye, 2019), Fake News Challenge-1.07
dataset (Hanselowski et al., 2018), Covid-Stance Dataset8 (Mutlu et al., 2020) and LIAR-PLUS Dataset9 (Alhindi, Petridis, & Muresan,
2018). These datasets are the collection of news articles from different sources and domains. All the datasets are of different sizes.
We use the quora question pair dataset for novelty detection because it is trendy and best suited to extract the novelty information
for our model. We use the unified dataset to pre-train the emotion model because we want to work with eight emotion labels (six
basic emotions + trust and anticipation). This dataset perfectly fits our requirements. We use ByteDance, FNC-1, Covid-stance, and
LIAR-PLUS datasets to train our fake news detection model because we need a dataset that includes premise and hypothesis pairs,
and only these datasets fulfill our requirements. We give a brief description of these datasets in two different subsections.
4 http://qim.ec.quoracdn.net/quora_duplicate_questions.tsv
5 https://github.com/sarnthil/unify-emotion-datasets
6 https://www.kaggle.com/wsdmcup/wsdm-fake-news-classification
7 https://github.com/FakeNewsChallenge/fnc-1
8 https://github.com/eceveco/COVID-CQ
9 https://github.com/Tariq60/LIAR-PLUS
7
Table 1
Dataset statistics.
Dataset Type Source Size Labels
Quora-QP Novelty Quora 4,04,290 3
Unified Emotion Various 2,33,665 10+N
ByteDance FND ByteDance 4,00,893 3
FNC FND FNC-1 49,972 4
Covid-Stance FND Twitter 14,374 3
LIAR-PLUS FND Politifact 12,836 6,2
Fig. 2. This figure depicts the graphical representations of QQP, Unified and ByteDance datasets respectively from left to right.
5.1. Pre-trained datasets for supporting tasks
Quora Question Pair Dataset: This dataset is the first released Quora question pair dataset. Each instance in the dataset includes
ID, Question1ID, Question2ID, Question1, Question2, and isduplicate attributes where ‘‘isduplicate’’ is either 0 or 1, indicating
whether the two questions are duplicate. This dataset contains 4,04,290 question pairs.
Unified Dataset: This unified dataset is the aggregation of 13 existing emotion classification datasets. The datasets have been
collected from different sources and domains; and are of different characteristics. The unified dataset includes a total of 2,33,665
sentences with corresponding emotion labels. Since it is a combination of different datasets, the instances have various emotion
labels. According to Ekman’s emotion label (Ekman, 1992), the authors have further mapped the instances into the labels (joy,
anger, sadness, disgust, fear, and surprise) mapping.
5.2. Target datasets for main task
ByteDance Dataset: ByteDance dataset has been released by ByteDance (a China-based global Internet technology company)
as the competition dataset of Task: Fake News Classification. It is the composition of training and the testing dataset. The training
dataset consists of 320,767 news pairs with 3 class labels, i.e., agreed, disagreed, and unrelated. The testing dataset contains 80,126
news pairs without any labels. These news pairs are available in both Chinese and English.
Fake News Challenge-1 Dataset: Fake News Challenge-1 has provided this dataset and derived from Emergent (a digital
journalism project for rumor debunking). There are 49,972 headline-body pairs in total, with stances labeled by expert journalists.
FNC1 dataset has a headline and a body text pair, either from the same news article or from two different articles, and the
corresponding stance labels (Agrees, Disagrees, Discusses, Unrelated).
Covid-Stance Dataset: Covid-Stance is a stance detection dataset that includes user-generated content on Twitter in the context
of COVID-19. It is a collection of approximately 14 thousand tweets. It contains manually annotated opinions of the tweet initiators
regarding the use of ‘‘chloroquine’’ and ‘‘hydroxychloroquine’’ to prevent or treat COVID-19. The instances of this dataset have three
different classes as Neutral, Against, and Favor.
LIAR-PLUS Dataset: LIAR-PLUS dataset is the extension of LIAR dataset (Wang, 2017). It includes the 12,836 claims taken
from POLITIFACT10 along with the justification used by humans to give the label of a claim. It introduces the six and two-way
classification mechanism. For the six-way classification, the labels are pants-fire, false, barely-true, half-true, mostly-true, and
true. For two-way classification, the labels are only true and false.
Table 1 shows the complete data distribution for all the datasets, and Figs. 2 and 3 depict the graphical representation of the
class distribution of all the datasets.
10 http://www.politifact.com/
8
Fig. 3. This figure depicts the graphical representations of FNC-1, Covid-Stance and LIAR-PLUS datasets respectively from left to right.
5.3. Experimental setup
This section discusses the embedding mechanism, text pre-processing, different hyper-parameters, and implementation details.
We perform all the experiments in a python environment with the required libraries. We evaluate the performance of the system in
terms of accuracy, precision, recall, and F-score. For better comparison with other works, we use the weighted-accuracy evaluation
measure to evaluate our model on the ByteDance dataset and set the weight of agreed and disagreed samples to 1/15 and 1/5,
respectively, as specified by the ByteDance competition. We train the novelty and emotion model separately on the datasets and
use the generated representations and scaffold labeling for the final fake news classification. We generate co-occurrence matrices of
the original dataset with the predicted novelty and emotion labels to visualize the distribution of samples and use scaffold-labeling.
We referred to the FNC dataset’s agree and disagree classes as agreed and disagreed for uniformity with the ByteDance dataset.
Also, while reporting results, we consider the against class of the Covid-Stance dataset as disagreed and the favor class as agreed
for uniformity with the other labels.
If we consider misinformation datasets labeled with real and fake classes, the dataset becomes very small, which is not sufficient
to train and validate our model. If we utilize the novelty and emotion models trained using the QQP and Unified datasets, the models
do not precisely learn the characteristics of the final fake news datasets. For this, some portion of the misinformation datasets are
appended with the novelty and emotion task-specific datasets, and hence this model generalizes well for fake news detection. For
the training of the novelty model, we append the original datasets with the Quora-Question pair dataset. For the ByteDance and
Covid-Stance datasets, we augment the entire datasets consisting of all the agreed and disagreed samples with the Quora-Question
pair dataset. We append 400 agreed samples, and 500 disagreed samples into the original dataset for the FNC dataset. We augment
the fake news datasets with the Quora-Question pair dataset in the following manner. We append the agreed classes of the fake
news dataset with the duplicate novelty label and disagreed class with the novel novelty label. The reason we do this is that the
agreed class of fake news datasets means that the source and target semantically agree with each other. Hence, they can be thought
of as being a duplicate of each other. In contrast, the disagreed class means that the source and the target of the dataset differ from
each other. Hence, we consider the disagree class as the novel one.
We train the novelty model 9 times for 30 epochs to capture any deviation in each run’s results. Then, we consider the final
results obtained by taking the best of the two runs, which we choose from the nine runs. According to our hypothesis, we obtain the
best results that agree samples are duplicate and disagree samples are novel. The margin of error in the runs is about 0.1%. For the
emotion part, we use a BERT-based model pre-trained on (Klinger et al., 2018) dataset. While constructing the dataset, we combine
anticipation, sadness, joy, trust as one label and calling it emotion true and fear, disgust, surprise, anger as another label calling it
emotion false from the original dataset. We obtain the novelty aware and aware emotion representations for the complete datasets
from the above models and their predictions that we use for the final prediction. The dimension specification for each layer of the
proposed model are as follows:
Shape of masks: ([bs, 34, 1]), Shape of embedding: ([bs, 34, 300]), Shape after encoding: ([bs, 34, 200]), Shape of alignment:
([bs, 34, 500]), Shape after fusion: ([bs, 34, 200]) and Shape after pooling: ([bs, 200]). Here, bs represents the batch_size which is
equal to 64 for all the experiments in our paper.
We explain more specific experimental details of our proposed model for ByteDance, FNC-1, Covid-Stance, and LIAR-PLUS
datasets below:
5.3.1. Bytedance dataset

For the novelty representations, we add the scaffold label of the word vector duplicate to all the predicted duplicate classes and
original to all the predicted novel classes. For the emotion representations, we add scaffold label of emotion true and emotion false
formed by adding the word vectors of their constituent emotions to disagreed samples predicted as source false–target true and
agreed samples predicted as source false–target false, respectively.
9
5.3.2. FNC-1 dataset

For the novelty part, to improve the distribution of samples in the co-occurrence matrix, we combine the results of two such
runs and choose the best result for each entry of the training set and test set such that they comply with our hypothesis. After
obtaining the final novelty representations, we add scaffold label of the word vector duplicate to all the predicted duplicate classes
of the original agreed class and original to all the predicted novel classes of the original disagreed class. After obtaining the emotion
representations, we add scaffold label of emotion true to agreed samples predicted as source true–target true, and source false–target
false. We add the scaffold label of emotion false to disagreed samples predicted as source true–target false and source false–target
true predicted samples.
5.3.3. Covid-stance dataset

We split the dataset into 80% train and 20% test and report all the results on the test set. For the novelty representations, we
add the scaffold label of the word vector duplicate to all the predicted duplicate classes and original to all the predicted novel classes.
For the emotion representation, we add the scaffold label of emotion true to the original against and favor classes that are predicted
as emotion true and add the scaffold label of emotion false for the favor classes predicted as emotion false.
5.3.4. LIAR-PLUS dataset

We combine the six labels into two labels as specified by the original LIAR-PLUS paper (Alhindi et al., 2018). We combine false,
barely-true, and pants-fire as false and mostly-true, half-true, and true as true. We split the dataset into 80% train and 20% test
and report all the results on the test set. For novelty module training, we follow the procedure similar to the FNC dataset. For the
emotion model, we only consider the emotion of the ‘‘claim (i.e. premise)’’ and not the justification as here we are not aware of the
fact of the premise.
Finally, we add these novelty-aware and emotion-aware representations and pass them to a simple logistic regression model to
formulate fake news detection’s final task on the datasets. We only consider the agreed and disagreed samples in all our experiments
as they convey information about fake or not fake for the final classification.
5.4. Baselines
We implement a few baseline models for validating the effectiveness of our proposed model. We define the following baselines
for the comparisons:
5.4.1. Siamese LSTM network

We use the Siamese LSTM network, which uses shared weights for generating representations on both the source and target of
the fake-news datasets. For the ByteDance and Covid-Stance datasets, we use a simple LSTM followed by a dense layer and softmax
prediction, whereas for FNC, we use two stacked Bi-directional LSTMs. We concatenate the original features with the features of
the second LSTM before passing to a dense layer for classification. We observe that our final model performs better compared to
the baseline.
5.4.2. Multi-channel CNN

We use multi-channel CNN with two channels of CNNs with filter sizes 2 and 3, respectively. We pass the premise through the
first filter and the hypothesis through the second filter and obtain the max-pooled output. The obtained results from the channels
are concatenated and then passed through a dense layer with softmax for classification. We observe that our final model performs
better than the baseline in terms of the final F1 score and the precision and recall of the individual classes.
5.5. Results and analysis
Table 2 shows the evaluation results of the proposed model, baselines, and ablation studies on three different datasets. We
perform some experiments using the LIAR-PLUS dataset to show how our model generalizes. We also apply the SVM (Pisner &
Schnyer, 2020) classifier for the fake news classification and observe that the results are almost similar. However, precision and F1-
score for false class vary by less margin, and the SVM accuracy beats LR by 0.4 points. We also implement FakeFlow (Ghanem et al.,
2021) for the better comparison of our model. For the implementation of this model, we concatenate the premise and hypothesis
pair of all the misinformation datasets to follow the same implementation pattern in the original paper. We consider the same
hyper-parameters as the original FakeFlow paper for our experiments. The results of FakeFlow is comparable but not better than
our proposed model. It is because, the FakeFlow model does not consider the novelty information for fake news detection and all
the datasets do not include emotional reach content. We show these results in Table 3. We report the accuracy, precision, recall,
and F1-score of the models for both the agreed and disagreed classes. The results show that our proposed approach yields better
performance than the existing SOTA and baseline models. The model leveraging novelty and emotion information delivers better
performance compared to the other models.
10
Table 2
Results of baselines, ablation studies, and proposed model.
Dataset Model Agreed Disagreed Acc
Prec Rec F-score Prec Rec F-score
Baselines
ByteDance Siamese LSTM 0.89 0.98 0.93 0.74 0.34 0.46 87.83
FNC-1 Siamese BiLSTM 0.74 0.85 0.79 0.33 0.21 0.25 67.5
Covid-Stance Siamese LSTM 0.85 0.89 0.87 0.82 0.77 0.79 83.92
ByteDance Multi-Channel CNN 0.89 0.98 0.94 0.81 0.35 0.49 88.57
FNC-1 Multi-Channel CNN 0.74 0.95 0.83 0.44 0.11 0.18 72.31
Covid-Stance Multi-Channel CNN 0.86 0.90 0.88 0.84 0.78 0.81 85.16
Ablation studies
Novelty (ESIM) + 0.91 1.00 0.95 0.99 0.48 0.65 91.81
Emotion (KlingBin)
ByteDance
Novelty (Quora) + 0.91 1.00 0.95 0.96 0.49 0.65 91.74
Emotion (KlingBin)
Novelty (Quora-BD) 0.95 0.99 0.97 0.92 0.73 0.82 94.83
Emotion (KlingBin) 0.93 1.00 0.96 1.00 0.61 0.75 93.83
Novelty (ESIM) + 0.89 0.96 0.92 0.89 0.49 0.63 88.19
FNC-1 Emotion (KlingBin)
Novelty (Quora-FNC) 0.91 1.00 0.95 1.00 0.73 0.84 92.77
Novelty (Quora) + 0.90 0.90 0.90 0.85 0.85 0.85 87.49
Covid-Stance Emotion (KlingBin)
Novelty (Quora-CS) 0.86 0.86 0.86 0.80 0.79 0.79 83.44
Proposed model
ByteDance Fake-News 0.96 0.99 0.98 0.95 0.79 0.86 96.11
(QuoraBD+KlingBin)
FNC-1 Fake-News 0.93 1.00 0.97 1.00 0.80 0.89 94.73
(Quora-FNC+KlingBin)
Covid-Stance Fake-News 0.95 0.96 0.95 0.93 0.92 0.93 94.14
(Quora-CS+KlingBin)
Table 3
Result of the proposed model using LIAR-PLUS dataset and result of the proposed model while using SVM classifier instead of Logistic Regression
(LR).
Result of the SVM model
Dataset Agreed Disagreed Accuracy
ByteDance 0.96 0.99 0.98 0.95 0.80 0.87 96.12
FNC-1 0.95 1.00 0.97 0.99 0.85 0.91 95.73
Covid-Stance 0.94 0.92 0.93 0.94 0.96 0.95 94.26
Result of the FakeFlow model
Dataset Agreed Disagreed Accuracy
ByteDance 0.93 0.92 0.93 0.87 0.86 0.87 92.18
FNC-1 0.73 0.99 0.84 0.42 0.20 0.40 73.26
Covid-Stance 0.90 0.79 0.84 0.73 0.86 0.79 82.43
Result of the LIAR-PLUS dataset
False True Accuracy
LIAR-PLUS (LR) 0.68 0.93 0.79 0.93 0.66 0.77 78.13
LIAR-PLUS (SVM) 0.71 0.93 0.80 0.93 0.66 0.77 78.5
Here, Prec and Rec represent the Precision and Recall respectively.
11
Fig. 4. ByteDance ROC curve.
5.6. Ablation study
We perform some ablation studies using all the datasets to propose a robust fake news detection model. We include the results
of these ablation studies in Table 2.
(a) Novelty(ESIM) + Emotion(KlingBin):
Our first baseline obtains its novelty representation from ESIM (Chen, Zhu, Ling, Inkpen, & Wei, 2018) model trained on the
SNLI corpus for textual entailment. This baseline gets its emotion representations from the Bert-based model trained on Klinger
binary dataset, as mentioned in our experimental setup. For ByteDance and FNC, we model the representations in the same way as
mentioned in the experimental setup and consider that the label entailment is synonymous with duplication and contradiction to
be with the novel. We observe that on both datasets, this model gives an accuracy comparable to our best models. However, our
best model shows better recall and F1-score values on the disagreed class, underrepresented in the datasets.
(b) Novelty(Quora) + Emotion(KlingBin):
We develop our second baseline model in the same way as our original model. However, we train the novelty model only on
the Quora-Question-Pair dataset without taking any samples from the ByteDance dataset, the Covid-Stance dataset, or the FNC
dataset. We again observe that the overall accuracy is comparable. However, our best (final) model yields better performance for
the underrepresented disagreed class.
(c) Novelty(Quora with respective datasets):
We carry ablation studies on all the datasets, viz. ByteDance, FNC, and Covid-Stance using the novelty detection model trained
on a mixture of Quora and the respective dataset, as mentioned in the experimental setup. The fake-news model leveraging only
novelty representations and alone gives comparable results as to the final part.
(d) Emotion(KlingBin):
Like the novelty component, our emotion model trained on (Klinger et al., 2018) dataset attains the accuracy comparable to
our proposed model on ByteDance, FNC, and the Covid-Stance datasets. The results of this baseline model are comparable to the
novelty and the proposed model results. However, the disagreed class shows the space for improvement.
Here, the model with combined novelty and emotion features does not perform well in comparison to the setting where novelty
and emotion models are used in isolation. This might be attributed to the fact that for this particular case, the model has not
learned the characteristics of the ByteDance dataset. The novelty model’s independent results are on the Quora-BD dataset, whereas
the combined model in the ablation study uses only the Quora dataset for its novelty part. Since the combined model has not learned
the ByteDance dataset’s characteristics, it does not perform up to the mark of the independent novelty model on Quora-BD, which is
aware of the characteristics of the Quora dataset. The result analysis section shows that the proposed model performs better than the
novelty and emotion models individually for the ByteDance dataset. During these ablation studies, we found that after combining
the best novelty and emotion models in our final model, we obtain an improvement in performance concerning all the evaluation
metrics on both the agreed and disagreed classes of all the datasets.
5.7. Error analysis
We carry out an error analysis of the classification results obtained on each dataset. We plot the Receiver Operating Character-
istics (ROC) Curve (see Figs. 4–6) and calculate the value of Area Under the Curve (AUC) of the corresponding graph obtained. We
obtain an AUC value of 0.98, 0.99, and 0.99, respectively, on the ByteDance, FNC, and the Covid-Stance datasets. This shows that
our final model performs well on all three datasets. We further carry out the qualitative and quantitative analysis on the datasets.
12
Fig. 5. FNC ROC curve.
Fig. 6. Covid-Stance ROC curve.
5.7.1. Qualitative analysis

We observe that our model operates as a kind of transfer learning from the knowledge of novelty and emotion to fake news
detection. We observe that FNC novelty plays a more significant role than emotion in the final fake news classification. This is
because FNC data is the collection of news article texts that are mainly objective and lack strong emotions. Novelty plays a more
important role than emotion in the Covid-Stance dataset as well. Due to some data being long and semantically complicated, the
emotion model may not capture the emotion of the data very well.
Our final classification results show that the agreed class’s precision and recall are excellent as it forms a significant share of the
dataset. However, our model also gives good precision and recall on the disagreed class that proves the successful learning of the
model on the underrepresented classes as well. Our architecture is specifically able to capture the entailment relationship between
the source and the target of both the datasets. For this reason, specifically, our model performs well on the disagreed class of the
FNC dataset that conveys essential information about fake-news. We present the analysis of some real example in Tables 4–6.
5.7.2. Quantitative analysis

We closely analyze the outputs of the classifiers to get better insights using the co-occurrence matrices and show it in Fig. 7. We
obtain these matrices by considering the co-occurrence of the predicted classes by the novelty and emotion model with the original
labels. Here, a(True, True), b(True, False), c(False, True) and d(False, False) are the different combinations of predicted emotion
labels on the source and the target. We analyze these co-occurrence matrices to check if they comply with our novelty and emotion
hypothesis. We add scaffold labels to the representations to improve the final accuracy of the model based on these results. In the
novelty co-occurrence matrices for each of the datasets, ByteDance, FNC, and Covid-Stance, we observe that the agreed samples are
majorly classified as duplicates. In contrast, the disagreed samples are majorly classified as novel.
The emotion co-occurrence matrix shows the distribution of the actual fake-news labels of the dataset with the emotion true and
emotion false predicted labels. For the Bytedance and FNC dataset, we can observe that agreed samples have more source–target
pairs classified as emotion true–emotion true and emotion false–emotion false and disagreed samples have a significant distribution of
the source–target as emotion true - emotion false and emotion false - emotion true. For the Covid-Stance dataset since the premise is
considered false we expect the favor class to be majorly represented as emotion false. As per our expectation, we can observe that the
13
Table 4
Error analysis on ByteDance.
Id Source Target BD_Label NLP/NLE ELP/ELE
373760 There is been a deadly carjacking in Bad carjacking incident in chuanchong, Disagreed Novel/Novel c/c
Lungcheong. the police defends rumor: it is a
relationship dispute.
334598 Tse tzu-feng was reported to have TSE Tse-feng replies Rumor of Breakup Agreed Novel/ Duplicate b/d
broken up? said zhuo wei and zhao with Wong Fei!
jiahai compete, song zuode how
come out?
Here, NLP - Novelty_Label_Predicted; NLE - Novelty_Label_Expected; ELP - Emotion_Label_Predicted; ELE - Emotion_Label_Expected. The
representation, a(True, True), b(True, False), c(False, True) and d(False, False) are the different combinations of predicted emotion labels on the
source and the target. The entry with id 373760 has correct predictions for both novelty and emotion. The entry with id 334598 has incorrect
predictions for both novelty and emotion. The semantics and complicated structure of the sentences are the main reason for the wrong prediction.
Table 5
Error analysis on FNC dataset.
Body_id Body Headline Stance NLP/ NLE ELP/ ELE
897 So here’s the latest in wild internet rumors. Everyone Argentina’s President Cristina Disagree Novel/ Novel b/b
thinks that the President of Argentina Christina Kirchner Adopts Jewish Godson To
Fernández de Kirchner adopted a young Jewish man Prevent HimTurning Into A Were
in order to stop him from turning into a were wolf. wolf
The story was reported by The Independent and
others. Well, the story’s kind of true — Fernández de
Kirchner did adopt the young man as her godson —
but not to keep him from turning into a were wolf.
1793 Hewlett-Packard is officially splitting in two. Hewlett-Packard to split into two Agree Novel/ Duplicate c/a
Following rumors over the weekend, HP is public companies
announcing today that it will separate its PC and
printer division from its enterprise and services
business. The split means current CEO Meg Whitman
will become the chairman of the PC and printer
operation, and continue as CEO of the split-off
enterprise business. Dion Weisler, an executive at
HP’s PC business, will take over as CEO of the
company’s PC and printer operation.
Here, NLP - Novelty_Label_Predicted; NLE - Novelty_Label_Expected; ELP - Emotion_Label_Predicted; ELE - Emotion_Label_Expected. The representation,
a(True, True), b(True, False), c(False, True) and d(False, False) are the different combinations of predicted emotion labels on the source and the target.
Table 6
Error analysis on Covid-Stance dataset.
Id Text Stance NLP/NLE ELP/ELE
1252709370540040000 I guess that’s why @realDonaldTrump has suddenly Against Duplicate/Novel False/True
shut his big mouth about #Hydroxychloroquine.
@DonaldJTrumpJr @kayleighmcenany
@KellyannePolls WHY @CNN @MSNBC @ABC
should NOT air Trumps rally of lies! Only experts,
scientists & doctors should be talking about
#COVID19 https://t.co/lwdpcvOSBh
1255986254157450000 I’m pissed for the PUSH for #remdesivir the Favour Duplicate/Duplicate False/False
EXPENSIVE drug (abt $1000/pill) to fight
#ChinaVirus while at the same trying to trash the
QUITE INEXPENSIVE Hydroxychloroquine And
Azithromycin( $.63/pill & $.84/pill respectively)
As *IF* #WeThePeople cant’ SEE THIS!
Here, NLP-Novelty_Label_Predicted; NLE-Novelty_Label_Expected; ELP-Emotion_Label_Predicted; ELE-Emotion_Label_Expected.
agreed samples (favor) are majorly classified as emotion false and the disagreed samples (against ) have significant samples classified
as emotion true.
5.8. Comparison to the SOTA models
We compare our results with the SOTA models on ByteDance, FNC-1, Covid-stance, and LIAR-PLUS datasets. Our model achieves
the accuracy gain over these models by wide margins. It outperforms the existing models by 8, 1.54, 17.31 and 8.3 points for
ByteDance, FNC-1, Covid-Stance and LIAR-PLUS datasets, respectively. We show the comparative analysis of our proposed model
with the existing SOTA models using the different datasets as shown in Table 7.
14
Fig. 7. Co-occurrence matrix for the test set of ByteDance, FNC and Covid-Stance datasets.
All the previous SOTA results on ByteDance are reported for the Chinese source and target pairs. In contrast, we report our
predictions of the corresponding English pair provided in the original dataset. The SOTA (Pham, 2019) proposed a blended prediction
of 18 neural networks, nine tree-based models, and a logistic regression model and evaluated this model on the ByteDance dataset.
We obtain an 8.82% and 7.6% increase in the weighted accuracy compared to them on the ‘‘Public’’ and ‘‘Private’’ part of the
ByteDance test set, respectively. Similarly, we compare our results with (Liu, Jin, & Shen, 2019; Yang, Niven, & Kao, 2019) and
find that we get a similar increase in the weighted accuracy compared to them.
For the FNC dataset, we compare our results with (Attardi & Slovikovskaya, 2020), which uses transfer learning from the
transformer to model the fake news task of FNC. We obtain an increase of 1.54% on the overall accuracy concerning their best model
on the FNC dataset. We also observe that our proposed model performs better for the agreed and disagreed classes by significant
margins. The model achieves an 8 point increase in the overall F1-score of the agreed class and 6 points increase in the F1-score of
the disagreed class. In terms of accuracy, we also gain comparable performance with (Chaudhry, Baker, & Thun-Hohenstein, 2017)
and obtain an 8 point increase in the F1-score of the agreed class and 11 points increase in the F1-score of the disagreed class.
For the Covid-Stance dataset, we compare our results with the models provided in the dataset paper (Mutlu et al., 2020). Our
final model gives an improvement of 17.31% compared to their best model that uses tf-idf features with Logistic Regression
(LR) as the classification model.
For the LIAR-PLUS dataset, we compare our results with the models provided in (Alhindi et al., 2018). Our final model gives an
improvement of 8.3 points compared to their best model that uses parallel bidirectional-LSTM(P-BiLSTM) as the classification
model.
6. Theoretical and practical implications
False, misleading, or fake news on social media can have significant adverse societal effects. Fake news detection on social media
has recently attracted the attention of researchers and practitioners. Some literature reveals that ‘‘Novelty’’ contributes significantly
to the virality of fake news. Some existing literature shows that another significant characteristic of the virality of fake news is
the presence of an element of surprise in the story, which attracts immediate attention and invokes strong emotional stimulus in the
reader. The investigations of this study imply both theoretical and practical implications.
Although there exists a fair amount of prior research for detecting fake news on social media, these methods do not investigate
the role of emotion and novelty in spreading fake news. Some of the existing ways only utilize emotion-aware representation for
the fake news detection task. As novelty and emotion attributes in news posts create much attraction and attention that can quickly
compel the population to believe, nowadays, people create and share fake news with novel information and some emotional contents.
This provides a good motivation for detecting fake news by leveraging novelty and emotion information.
Theoretically, our study describes a new dimension by incorporating novelty and emotion information for fake news detection.
It introduces a mechanism that demonstrates how to extract novelty and emotion features from the news headline or body pairs
and uses those features to obtain the common novelty and emotion-aware representation that finally classifies news as fake or real.
15
Table 7
Comparative analysis of the proposed model with the existing SOTA models.
ByteDance dataset
Model Agreed Disagreed Accuracy
(Pham, 2019) – – – – – – 88.28
(Liu, Liu et al., 2019) – – – – – – 88.15
(Yang, Niven et al., 2019) – – – – – – 88.06
Proposed model 0.96 0.99 0.98 0.95 0.79 0.86 96.11
FNC-1 dataset
(Attardi & Slovikovskaya, 2020) – – 0.53 – – 0.31 93.19
(Chaudhry et al., 2017) 0.89 0.89 0.89 0.89 0.78 0.83 95.3
Proposed model 0.93 1.00 0.97 1.00 0.80 0.89 94.73
Covid-Stance dataset
(Mutlu et al., 2020) – – – – – – 76.83
Proposed model 0.95 0.96 0.95 0.93 0.92 0.93 94.14
LIAR-PLUS dataset
(Alhindi et al., 2018) – – – – – – 70
Proposed model 0.68 0.93 0.79 0.93 0.66 0.77 78.3
We have empirically tested and validated the role of emotion and novelty in the sharing of fake news. Second, these findings have
uncovered new facets of fake-news sharing and detection, proposing a deep learning and machine learning based framework. These
revelations contribute to the theoretical knowledge in the domain.
We have shown the practical implementation of a deep learning and machine learning based fake news detection framework in
this study. Since only a single literature (Qin et al., 2016) exists that utilizes novelty information. Very few pieces of literature exist
that use emotion information for fake news detection. This can go a long way in mitigating the spread and detection of fake news,
especially related to politics, pandemics, healthcare, religion, or ethnicity, which has become a grave concern globally.
Second, our results show the co-relation between novelty and fake news, emotion and fake news, novelty, and emotion. This
indicates that news with high novelty and emotional appeal instincts pushes people to forward news items instantaneously. This
supports the need for regulatory control to some extent, particularly during a crisis such as the COVID-19 global pandemic, wherein
fake news spread can induce fear and panic among the public. This poses an enormous challenge, mostly because fake news items
may resemble credible journalism, causing people to believe them and act accordingly.
Finally, our experimental results suggest that social media users share news and information because of novel information and
emotional appeal. Earlier studies have also discussed the role of emotions associated with a news post. Regulators, researchers,
scientists, journalists, and organizations could benefit from these findings and utilize them to prevent and detect the spread of fake
news.
7. Conclusion and future work
In this paper, we have proposed deep learning and machine learning based framework that takes news headline body pairs as
input and detects whether this news is fake or real. It leverages novelty and emotion-aware representations of the news pair for fake
news detection. To extract the novelty and emotion-aware representations, we use pre-trained models. All the representations are
added along with the scaffold labels and passed to the logistic regression model for the final classification task. We have conducted
extensive experiments on four publicly available datasets. Experimental results show that the proposed model performs better for
detecting fake news compared to the existing models.
There are many possibilities to investigate and extend this work in the future. Here, we list two possible directions for our future
research: (a) The fake news detection task can be performed using a multitasking approach where novelty and emotion prediction
tasks will be the supporting task, and fake news detection will be the main task; (b) Nowadays, many news posts contain images,
audios and videos. Hence, a possible extension would be to include these modalities for fake news detection.
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared
to influence the work reported in this paper.
Acknowledgments
Asif Ekbal acknowledges the Young Faculty Research Fellowship (YFRF), supported by Visvesvaraya Ph.D. scheme for Electronics
and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India
Corporation (formerly Media Lab Asia).
16
References
Abdul-Mageed, M., & Ungar, L. (2017). Emonet: Fine-grained emotion detection with gated recurrent neural networks. In Proceedings of the 55th annual meeting
of the association for computational linguistics (volume 1: long papers) (pp. 718–728).
Ajit Rajasekharan, A. L. (2020). What are the main differences between the word embeddings of ELMo, BERT, Word2vec, and GloVe? - Quora. https:
//www.quora.com/What-are-the-main-differences-between-the-word-embeddings-of-ELMo-BERT-Word2vec-and-GloVe (Accessed on 19 February 2021).
Akhtar, M. S., Ekbal, A., Narayan, S., Singh, V., & Cambria, E. (2018). No, that never happened!! investigating rumors on Twitter. IEEE Intelligent Systems, 33(5),
8–15.
Alhindi, T., Petridis, S., & Muresan, S. (2018). Where is your evidence: Improving fact-checking by justification modeling. In: Proceedings of the first workshop on
fact extraction and verification, (pp. 85–90).
Alkhodair, S. A., Ding, S. H., Fung, B. C., & Liu, J. (2020). Detecting breaking news rumors of emerging topics in social media. Information Processing &
Management, 57(2), Article 102018.
Amplayo, R. K., Hong, S., & Song, M. (2018). Network-based approach to detect novelty of scholarly literature. Information Sciences, 422, 542–557.
An, X., Huang, J. X., & Wang, Y. (2020). Diversity and novelty in biomedical information retrieval. In Biomedical information technology (pp. 369–396). Elsevier.
Attardi, G., & Slovikovskaya, V. (2020). Transfer learning from transformers to fake news challenge stance detection (FNC-1) task. In Language resources and
evaluation. ELRA.
Barr, R. A. (2019). Fake news grabs our attention, produces false memories and appeals to our emotions. https://theconversation.com/fake-news-grabs-our-
attention-produces-false-memories-and-appeals-to-our-emotions-124842 (Accessed on 09 September 2020).
Becker, K., Moreira, V. P., & dos Santos, A. G. (2017). Multilingual emotion classification using supervised learning: Comparative experiments. Information
Processing & Management, 53(3), 684–704.
Brady, W. J., Wills, J. A., Jost, J. T., Tucker, J. A., & Van Bavel, J. J. (2017). Emotion shapes the diffusion of moralized content in social networks. Proceedings
of the National Academy of Sciences, 114(28), 7313–7318.
Breja, M. (2015). A novel approach for novelty detection of web documents. International Journal of Computer Science and Information Technologies.
Brennen, J. S., Simon, F., Howard, P. N., & Nielsen, R. K. (2020). Types, sources, and claims of Covid-19 misinformation. Reuters Institute, 7, 3–1.
Bro, R., & Smilde, A. K. (2014). Principal component analysis. Analytical Methods, 6(9), 2812–2831.
Chaudhry, A. K., Baker, D., & Thun-Hohenstein, P. (2017). Stance detection for the fake news challenge: identifying textual relationships with deep neural nets.
CS224n: Natural Language Processing with Deep Learning.
Chen, Q., Zhu, X., Ling, Z.-H., Inkpen, D., & Wei, S. (2018). Neural natural language inference models enhanced with external knowledge. In Proceedings of the
56th annual meeting of the association for computational linguistics (volume 1: long papers) (pp. 2406–2417).
Claire Wardle, H. D. (2017). One year on, we’re still not recognizing the complexity of information disorder online. https://firstdraftnews.org/articles/coe_
infodisorder/ (Accessed on 06 September 2021).
Cuan-Baltazar, J. Y., Muñoz Perez, M. J., Robledo-Vega, C., Pérez-Zepeda, M. F., & Soto-Vega, E. (2020). Misinformation of COVID-19 on the internet:
infodemiology study. JMIR Public Health and Surveillance, 6(2), Article e18444.
Da San Martino, G., Yu, S., Barrón-Cedeno, A., Petrov, R., & Nakov, P. (2019). Fine-grained analysis of propaganda in news article. In Proceedings of the 2019
conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP) (pp.
5640–5650).
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (1).
Ekman, P. (1992). An argument for basic emotions. Cognition and Emotion, 6(3–4), 169–200.
Ghanem, B., Ponzetto, S. P., Rosso, P., & Rangel, F. (2021). FakeFlow: Fake news detection by modeling the flow of affective information. In Proceedings of the
16th conference of the european chapter of the association for computational linguistics: main volume (pp. 679–689).
Ghanem, B., Rosso, P., & Rangel, F. (2020). An emotional analysis of false information in social media and news articles. ACM Transactions on Internet Technology
(TOIT), 20(2), 1–18.
Ghosal, T., Edithal, V., Ekbal, A., Bhattacharyya, P., Chivukula, S., & Tsatsaronis, G. (2020). Is your document novel? Let attention guide you. An attention-based
model for document-level novelty detection. Natural Language Engineering, 1–28. http://dx.doi.org/10.1017/S1351324920000194.
Giachanou, A., Rosso, P., & Crestani, F. (2019). Leveraging emotional signals for credibility detection. In Proceedings of the 42nd international ACM SIGIR conference
on research and development in information retrieval, (pp. 877–880).
Giachanou, A., Rosso, P., & Crestani, F. (2021). The impact of emotional signals on credibility assessment. Journal of the Association for Information Science and
Technology.
Guo, C., Cao, J., Zhang, X., Shu, K., & Yu, M. (2019). Exploiting emotions for fake news detection on social media. arXiv preprint arXiv:1903.01728.
Hanselowski, A., Avinesh, P., Schiller, B., Caspelherr, F., Chaudhuri, D., Meyer, C. M., et al. (2018). A retrospective analysis of the fake news challenge
stance-detection task. In Proceedings of the 27th international conference on computational linguistics (pp. 1859–1874).
He, X., Meng, X., Wu, Y., Chan, C. S., & Pang, T. (2020). Semantic matching efficiency of supply and demand texts on online technology trading platforms:
Taking the electronic information of three platforms as an example. Information Processing & Management, 57(5), Article 102258.
Hsu, C.-C., Ajorlou, A., & Jadbabaie, A. (2019). A theory of misinformation spread on social networks. Available At SSRN 3391585.
Imtiaz, Z., Umer, M., Ahmad, M., Ullah, S., Choi, G. S., & Mehmood, A. (2020). Duplicate questions pair detection using siamese malstm. IEEE Access, 8,
21932–21942.
Jain, G., Sharma, M., & Agarwal, B. (2019). Spam detection in social media using convolutional and long short term memory neural network. Annals of Mathematics
and Artificial Intelligence, 85(1), 21–44.
Jose, S.-D. J., Yosimar, J.-C. A., & Alejandro, E.-O. D. (2020). Smart monitoring based on novelty detection and artificial intelligence applied to the condition
assessment of rotating machinery in the industry 4.0. In New trends in the use of artificial intelligence for the industry 4.0. IntechOpen.
Kerner, H. R., Wellington, D. F., Wagstaff, K. L., Bell, J. F., Kwan, C., & Amor, H. B. (2019). Novelty detection for multispectral images with application to
planetary exploration. In Proceedings of the AAAI conference on artificial intelligence, vol. 33 (pp. 9484–9491).
Klinger, R., et al. (2018). An analysis of annotated corpora for emotion classification in text. In Proceedings of the 27th international conference on computational
linguistics (pp. 2104–2119).
Kouzy, R., Abi Jaoude, J., Kraitem, A., El Alam, M. B., Karam, B., Adib, E., et al. (2020). Coronavirus goes viral: quantifying the COVID-19 misinformation
epidemic on Twitter. Cureus, 12(3).
Kumar, S., & Bhatia, K. K. (2020). Semantic similarity and text summarization based novelty detection. SN Applied Sciences, 2(3), 332.
Kumari, R., Ashok, N., Ghosal, T., & Ekbal, A. (2021). Misinformation detection using multitask learning with mutual learning for novelty detection and emotion
recognition. Information Processing & Management, 58(5), 102631.
Kumari, R., Ashok, N., Ghosal, T., & Ekbal, A. (2021). A multitask learning approach for fake news detection: novelty, emotion, and sentiment lend a helping
hand. In 2021 International Joint Conference on Neural Networks (IJCNN) (pp. 1–8). IEEE.
Lee, S. (2015). Online sentence novelty scoring for topical document streams. In Proceedings of the 2015 conference on empirical methods in natural language
processing (pp. 567–572).
Liew, J. S. Y., & Turtle, H. R. (2016). Exploring fine-grained emotion detection in tweets. In Proceedings of the NAACL student research workshop (pp. 73–80).
17
Liu, Y., Jin, X., & Shen, H. (2019). Towards early identification of online rumors based on long short-term memory networks. Information Processing & Management,
56(4), 1457–1467.
Liu, S., Liu, S., & Ren, L. (2019). Trust or suspect? An empirical ensemble framework for fake news classification. In Proceedings of the 12th ACM international
conference on web search and data mining (pp. 11–15).
Liu, Z., Shabani, S., Balet, N. G., & Sokhn, M. (2019). Detection of satiric news on social media: analysis of the phenomenon with a french dataset. In 2019
28th international conference on computer communication and networks (pp. 1–6). IEEE.
Lu, Y.-J., & Li, C.-T. (2020). GCAN: Graph-aware co-attention networks for explainable fake news detection on social media. In Proceedings of the 58th annual
meeting of the association for computational linguistics (pp. 505–514).
Mutlu, E. C., Oghaz, T., Jasser, J., Tutunculer, E., Rajabi, A., Tayebi, A., et al. (2020). A stance data set on polarized conversations on Twitter about the efficacy
of hydroxychloroquine as a treatment for COVID-19. Data in Brief, Article 106401.
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G., & Rand, D. G. (2020). Fighting COVID-19 misinformation on social media: Experimental evidence for a
scalable accuracy-nudge intervention. Psychological Science, Article 0956797620939054.
Pennycook, G., & Rand, D. G. (2019). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning.
Cognition, 188, 39–50.
Pham, L. (2019). Transferring, transforming, ensembling: the novel formula of identifying fake news. In Proceedings of the 12th ACM international conference on
web search and data mining (pp. 11–15).
Pisner, D. A., & Schnyer, D. M. (2020). Support vector machine. In Machine learning (pp. 101–121). Elsevier.
Preslav, N. (2020). Can we spot the ‘‘fake news" before it was even written? arXiv preprint arXiv:2008.04374.
Qin, Y., Wurzer, D., Lavrenko, V., & Tang, C. (2016). Spotting rumors via novelty detection. arXiv preprint arXiv:1611.06322.
Ren, G., & Hong, T. (2019). Examining the relationship between specific negative emotions and the perceived helpfulness of online reviews. Information Processing
& Management, 56(4), 1425–1438.
Rohit, W., Deshmukh, S., & Jagdale, R. (2018). Novelty detection in BBC sports news streams. International Journal of Scientific Research in Computer Science
Applications and Management Studies.
Saikh, T., Ghosal, T., Ekbal, A., & Bhattacharyya, P. (2017). Document level novelty detection: textual entailment lends a helping hand. In Proceedings of the
14th international conference on natural language processing, (pp. 131–140).
Scheufele, D. A., & Krause, N. M. (2019). Science audiences, misinformation, and fake news. Proceedings of the National Academy of Sciences, 116(16), 7662–7669.
Shu, K., Cui, L., Wang, S., Lee, D., & Liu, H. (2019). Defend: Explainable fake news detection. In 25th ACM SIGKDD international conference on knowledge discovery
and data mining, KDD 2019 (pp. 395–405). Association for Computing Machinery.
Shu, K., Wang, S., & Liu, H. (2019). Beyond news contents: The role of social context for fake news detection. In 12th ACM international conference on web search
and data mining, WSDM 2019 (pp. 312–320). Association for Computing Machinery, Inc.
Slovikovskaya, V., & Attardi, G. (2020). Transfer learning from transformers to fake news challenge stance detection (FNC-1) task. In Proceedings of the 12th
language resources and evaluation conference (pp. 1211–1218).
Sotthisopha, N., & Vateekul, P. (2018). Improving short text classification using fast semantic expansion on multichannel convolutional neural network. In 2018
19th IEEE/ACIS international conference on software engineering, artificial intelligence, networking and parallel/distributed computing (pp. 182–187). IEEE.
Tasnim, S., Hossain, M. M., Mazumder, H., et al. (2020). Impact of rumors or misinformation on coronavirus disease (COVID-19) in social media. Center for
Open Science.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., et al. (2017). Attention is all you need. In Advances in neural information processing
systems (pp. 5998–6008).
Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science, 359(6380), 1146–1151.
Wang, W. Y. (2017). ‘‘Liar, liar pants on fire’’: A new benchmark dataset for fake news detection. In Proceedings of the 55th annual meeting of the association for
computational linguistics (volume 2: short papers) (pp. 422–426).
Wang, Y., & Pal, A. (2015). Detecting emotions in social media: A constrained optimization approach. In Twenty-fourth international joint conference on artificial
intelligence.
Xiaoye, S. (2019). Analysis of ByteDance. (Ph.D. thesis), Swiss Federal Institute of Technology Zurich.
Xiong, X., Li, Y., Zhang, R., Bu, Z., Li, G., & Ju, S. (2020). Dgi: Recognition of textual entailment via dynamic gate matching. Knowledge-Based Systems, Article
105544.
Yang, K.-C., Niven, T., & Kao, H.-Y. (2019). Fake news detection as natural language inference. In 12th ACM international conference on web search and data
mining (WSDM-2019) (in Fake News Classification Challenge, WSDM Cup 2019).
Yang, R., Zhang, J., Gao, X., Ji, F., & Chen, H. (2019). Simple and effective text matching with richer alignment features. In Proceedings of the 57th annual
meeting of the association for computational linguistics (pp. 4699–4709).
Zarrabian, S., & Hassani-Abharian, P. (2020). Covid-19 pandemic and the importance of cognitive rehabilitation. Basic and Clinical Neuroscience, 189–190.
Zhou, D., Zhang, X., Zhou, Y., Zhao, Q., & Geng, X. (2016). Emotion distribution learning from texts. In Proceedings of the 2016 conference on empirical methods
in natural language processing (pp. 638–647).
Zubiaga, A., Kochkina, E., Liakata, M., Procter, R., Lukasik, M., Bontcheva, K., et al. (2018). Discourse-aware rumour stance classification in social media using
sequential classifiers. Information Processing & Management, 54(2), 273–290.
18

Fake Content Detection

Uploaded by

Copyright:

Available Formats

Fake Content Detection

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Fake Content Detection

Uploaded by

Copyright:

Available Formats

Information Processing and Management 59 (2022) 102740

Contents lists available at ScienceDirect

Information Processing and Management

What the fake? Probing misinformation detection standing on the

ARTICLE INFO ABSTRACT

2.1. Novelty detection

2.2. Emotion prediction

2.3. Fake or misinformation detection

4.1. Novelty detection model

ℎ𝐴𝑆 𝑝 = 𝑀𝑎𝑥𝑝𝑜𝑜𝑙(𝐶𝑜𝑛𝑣2𝑑(𝑆𝐴 (𝑝, 𝑝))) (1)

ℎ𝐴𝑆 = 𝑐𝑜𝑛𝑐𝑎𝑡(𝑚𝑎𝑥(ℎ𝐴𝑆 𝑝 )) (2)

ℎ𝐵𝑆 = 𝑐𝑜𝑛𝑐𝑎𝑡(𝑚𝑎𝑥(ℎ𝐵𝑆 𝑝 )) (3)

𝑑𝑎𝑏 = 𝑒𝑎𝑖 𝑒̇ 𝑏𝑖 (6)

𝐹𝑎 = 𝑚𝑎𝑥𝑝𝑜𝑜𝑙(𝐺([𝐹𝑎1 ; 𝐹𝑎2 ; 𝐹𝑎3 ])) (9)

𝐹𝑏 = 𝑚𝑎𝑥𝑝𝑜𝑜𝑙(𝐺([𝐹𝑏1 ; 𝐹𝑏2 ; 𝐹𝑏3 ])) (10)

𝐹 ′ = [𝐹𝑎 , 𝐹𝑏 , |𝐹𝑎 − 𝐹𝑏 |, 𝐹𝑎 ∗ 𝐹𝑏 ] (11)

4.2. Emotion prediction model

𝐸𝐹′ 𝑠 = 𝑡𝑎𝑛ℎ(𝑊𝐸𝐹 𝑠 ∗ 𝐸𝐹 𝑠 + 𝑏𝑠𝐸𝐹 𝑠 ) (17)

4.3. Fake news detection model

𝐹𝑁𝐸 = [𝑁𝐹 + 𝑒𝑎𝑏 + 𝐸𝐹 + 𝐹𝑙𝑎𝑏𝑒𝑙 ] (19)

Now the logistic function is,

5. Datasets and experiments

5.1. Pre-trained datasets for supporting tasks

5.2. Target datasets for main task

5.3. Experimental setup

5.3.1. Bytedance dataset

5.3.2. FNC-1 dataset

5.3.3. Covid-stance dataset

5.3.4. LIAR-PLUS dataset

5.4.1. Siamese LSTM network

5.4.2. Multi-channel CNN

5.5. Results and analysis

Fig. 4. ByteDance ROC curve.

5.6. Ablation study

5.7. Error analysis

Fig. 5. FNC ROC curve.

Fig. 6. Covid-Stance ROC curve.

5.7.1. Qualitative analysis

5.7.2. Quantitative analysis

Here, NLP-Novelty_Label_Predicted; NLE-Novelty_Label_Expected; ELP-Emotion_Label_Predicted; ELE-Emotion_Label_Expected.

5.8. Comparison to the SOTA models

6. Theoretical and practical implications

7. Conclusion and future work

Declaration of competing interest

You might also like