Working paper no. 2017.

12

Troops,TrollsandTroublemakers:AGlobal
InventoryofOrganizedSocialMediaManipulation

Samantha Bradshaw, University of Oxford

Philip N. Howard, University of Oxford

 Contents

Executivesummary....................................................................................................3
Socialmediaanddemocracy........................................................................................4
Methodology.............................................................................................................5
I.Strategies,toolsandtechniquesforsocialmediamanipulation...................................8
Commentingonsocialmediaposts.....................................................................................9
Individualtargeting..........................................................................................................10
Governmentsponsoredaccounts,webpagesorapplications...............................................10
Fakeaccountsandcomputationalpropaganda...................................................................11
Contentcreation..............................................................................................................12
II.Organizationalforms..............................................................................................14
Government....................................................................................................................15
Politiciansandparties.......................................................................................................15
Privatecontractors..........................................................................................................16
Volunteers......................................................................................................................16
Paidcitizens...................................................................................................................16
III.Organizationalbudget,behaviorandcapacity........................................................18
Budgetinformation.........................................................................................................19
Organizationalbehavior..................................................................................................19
Capacitybuilding.............................................................................................................20
Conclusion................................................................................................................22
References................................................................................................................24
Seriesacknowledgements..........................................................................................35
Authorbiographies...................................................................................................36

Table1:Strategies,toolsandtechniquesforsocialmediamanipulation.................................13
Table2:Organizationalforms.................................................................................................17
Table3:Organizationalbudget,behaviorandcapacity...........................................................21

Figure1:Organizationaldensityofcybertroops,2017...........................................................22

2
Executivesummary
Cyber troops are government, military or political party teams committed to manipulating
public opinion over social media. In this working paper, we report on specific organizations
created,oftenwithpublicmoney,tohelpdefineandmanagewhatisinthebestinterestofthe
public.Wecomparesuchorganizationsacross28countries,andinventorythemaccordingto
the kinds of messages, valences and communication strategies used. We catalogue their
organizationalformsandevaluatetheircapacitiesintermsofbudgetsandstaffing.Thisworking
papersummarizesthefindingsofthefirstcomprehensiveinventoryofthemajororganizations
behindsocialmediamanipulation.

Wefind thatcybertroops are a pervasive and globalphenomenon. Many different countries

employsignificantnumbersofpeopleandresourcestomanageandmanipulatepublicopinion
online,sometimestargetingdomesticaudiencesandsometimestargetingforeignpublics.

The earliest reports of organized social media manipulation emerged in 2010, and by
2017therearedetailsonsuchorganizationsin28countries.
Lookingacrossthe28countries,everyauthoritarianregimehassocialmediacampaigns
targeting their own populations, while only a few of them target foreign publics. In
contrast,almosteverydemocracyinthissamplehasorganizedsocialmediacampaigns
thattargetforeignpublics,whilepoliticalpartysupportedcampaignstargetdomestic
voters.
Authoritarian regimes are not the only or even the best at organized social media
manipulation.Theearliestreportsofgovernmentinvolvementinnudgingpublicopinion
involvedemocracies,andnewinnovationsinpoliticalcommunicationtechnologiesoften
comefrompoliticalpartiesandariseduringhighprofileelections.
Over time, the primary mode for organizing cyber troops has gone from involving
military units that experiment with manipulating public opinion over social media
networkstostrategiccommunicationfirmsthattakecontractsfromgovernmentsfor
socialmediacampaigns.

3
Socialmediaanddemocracy
Socialmediahasbecomeavaluableplatformforpubliclife.Itistheprimarymediumoverwhich
youngpeople,aroundtheworld,developtheirpoliticalidentitiesandconsumenews.However,
socialmediaplatformslikeFacebookandTwitterhavealsobecometoolsforsocialcontrol.
Many governments now spend significant resources and employ large numbers of people to
generatecontent,directopinionandengagewithbothforeignanddomesticaudiences.This
workingpaperlaysthegroundworkforunderstandingtheglobaltrendsintheorganizedand
coordinateduseofsocialmediaformanipulatingpublicopinion.

Inthispaperwedefinecybertroopsasgovernment,militaryorpoliticalpartyteamscommitted
tomanipulatingpublicopinionoversocialmedia.Giventhatlittleisknownaboutthedifferences
incapacity,toolsandtechniquesofthesepracticesindifferentcountries,weconductedacross
nationalandcomparativestudyofglobalcybertroops.Examiningsocialmediaoperationsin25
countries, we have undertaken an inventory of budget expenditures, staffing, organizational
behaviorandcommunicationstrategiestoanalysesthesize,scaleandextenttowhichdifferent
kindsofpoliticalregimesdeploycybertroopstoinfluenceandmanipulatethepubliconline.

InJanuary2015,theBritishArmyannouncedthatits77thBrigadewouldfocusonnonlethal
psychologicaloperationsusingsocialnetworkslikeFacebookandTwittertofightenemiesby
gainingcontrolofthenarrativeintheinformationage(Solon,2015).Theprimarytaskofthis
unitistoshapepublicbehaviorthroughtheuseofdynamicnarrativestocombatthepolitical
propaganda disseminated by terrorist organizations. The United Kingdom is not alone in
allocatingtroopsandfundingforinfluencingonlinepoliticaldiscourse.Instead,thisispartofa
largerphenomenonwherebygovernmentsareturningtoInternetplatformstoexertinfluence
overinformationflowsandcommunicationchannelstoshapepublicopinion.Wecompareand
summarize this phenomenon in the following 28 countries: Argentina, Azerbaijan, Australia,
Bahrain,Brazil,China,theCzechRepublic,Ecuador,Germany,India,Iran,Israel,Mexico,North
Korea,thePhilippines,Poland,Russia,SaudiArabia,Serbia,SouthKorea,Syria,Taiwan,Turkey,
Ukraine,theUnitedKingdom,theUnitedStates,VenezuelaandVietnam.

Intermsofscope,thereareseveralthingswedonotinvestigate.First,althoughcybertroops
willoftenapplytraditionaloffensivecybertactics,suchashackingorsurveillance,totargetusers
for trolling or harassment campaigns, this is not a working paper about hackers or other
cybersecurity professionals who work in a governmental capacity. An important distinction
between cyber troops and other statebased actors operating in cyberspace is their role in
actively shaping public opinion. Second, there are many countries that have no domestic
organizationsforsocialmediamanipulation,butparticipateinmultilateralmutualdefensepacts
with programs for doing so. For example, NATO has an accredited international military

4
organization called the NATO Strategic Communication Center of Excellence with a list of
sponsoringnations,notallofwhichareintheinventorywepresenthere.Informalcivilsociety
organizations that use social media in a coordinated way are not included in this analytical
frame,norareprivatefirmsandindustrialassociationswithorganizedcampaignstomanipulate
publicopinion.

Methodology
We conducted the research for this working paper in three stages. First, we conducted a
systematic content analysis of news media articles. Second, we supplemented the content
analysis with other sources from think tanks, government agencies, civil society groups,
universitiesandothersourcesofcredibleresearch.Finally,weconsultedwithcountryexpertsto
checkfacts,findadditionalsourcesinmultiplelanguagesandassistinevaluatingthequalityof
sources. This methodology allowed us to purposefully select the cases for comparison, draw
widely from existing research and engage with country and region experts for points of
clarification.

Content analysis is an established research method in communication and media studies

(Herring,2009).IthasbeenusedtohelpunderstandhowtheInternetandsocialmediainteract
withpoliticalaction,regimetransformationanddigitalcontrol(Strangeetal.,2013;Joyceetal.,
2013; Edwards, 2013; Woolley, 2016). This qualitative content analysis was conducted to
understandtherangeofstateactorswhoactivelyusesocialmediatomanipulatepublicopinion,
aswellastheircapacity,strategiesandresources.WemodelledouranalysisafterJoyceetal.
(2013),Edwardsetal.(2013)andWoolley(2016),whoconductedaqualitativecontentanalysis
usingpurposivesamplingtobuildacodedspreadsheetofspecificvariablesthatappearinnews
articles.Ourcodedspreadsheetincludesfieldssuchasthesizeofthegovernmentteams,their
organizationalstructureandplacewithingovernment,strategiesandtools,skillsandtraining,
andcapacityandresources.Wepurposivelyselectedthefollowingkeywordsandusedthemin
combination for our search: astroturf*; bot; Facebook; fake; fake account; government;
information warfare; intelligent agent; military; persona management; progovernment;
propaganda;psychologicaloperations;psyops;socialmedia;sockpuppet*;troll*;Twitter.

Media bias is a significant concern when conducting a content analysis that uses purposive
sampling (Earl, 2004; Joyce et al., 2013). To mitigate any biases in the preliminary content
analysis,weusedLexisNexisandthetopthreesearchengineprovidersGoogle,Yahoo!and
Bingwhichprovidedhitstoavarietyofprofessionalandamateurnewssources.Atotalof104
newsstorieswereidentified.Wethenrankedthearticlesbasedontheircredibilityusingasimilar
rankingsystemtotheoneemployedbyJoyceetal.(2013)andWoolley(2016).Thearticleswere
scored on a threepoint scale, with three being the most credible and one being the least

5
credible.Articlesrankedatthreecamefrommajor,professionallybrandednewsorganizations,
including:ABCNews,BBCNews,Reuters,TheEconomist,TheGuardian,TheIndependent,The
Mirror,TheNewYorkTimes,TheTelegraph,TheWallStreetJournal,TheWashingtonPostand
WiredMagazine.Articlesrankedattwocamefromsmallerprofessionalnewsorganizations,or
commentaryorientedwebsitesorexpertblogs.Theseincludedwebsitessuchas:AlMonitor,
Buzzfeed,FreedomHouse,HumanRightsWatch,Medium,TheNewRepublic,TheNewStatesman,
TheObserver,Quartz,TheRegister,TheAtlantic,TheDailyDot,TheHill,TheIntercept,andThe
Verge.Articlesrankedatonecamefromcontentfarms,socialmediapostsorpersonalorhyper
partisanblogs.Thesearticleswereremovedfromthesample.

Atotalof83newsarticlesmadeupthefinalsample,andfromthesewewereabletoextract
severaldifferentkindsofvariables.Moreimportantly,wedefinedthreedomainsofcomparative
analysisthatallowedustosetindividualcountryprogramsintoaglobalcontext:(1)strategies,
tools and techniques of social media manipulation; (2) organizational form; and (3)
organizationalbudget,behaviorandcapacity.

Assemblingtheexistingcorpusofpublicnewsreportingontheuseofcybertroopsaroundthe
worldallowedustoestablishcasesoforganizedsocialmediamanipulationin23countries.We
thenmovedtothecorpusofmorespecializedworkingpapersthathavecomeoutofthinktanks,
governmentagencies,civilsocietygroups,universitiesandothersourcesofcredibleresearch.
These reports yielded additional details on the known country comparison set, and provided
additionalevidenceoncybertrooporganizationintwoadditionalcountries.

Onelimitationtoourmethodologywasthatweonlyaccessednewsmediaarticlesandthink
tank reports in the English language. In order to address this limitation, we made additional
queries with cybersecurity experts or people familiar with the political system in particular
countrieswhereneeded.Thisfinalstageofconsultationinvolveddoublecheckingnewsreports,
rather than adding new information off the record. We did not include any additional
observationsbycountryexpertsthatcouldnotbeverifiedinpublicationelsewhere.

We undertook additional research on additional countries where there is known trolling and
automated political communication activity. If we found evidence of suspicious activity, but
wereunabletotraceclearsignsoforganizationbehindthepoliticalcommunicationcampaign,
thecasesweredroppedfromtheanalysis.Inotherwords,inthisanalysiswefocusexclusively
on organized social media campaigns that have the clear support of political parties and
governments. Readers interested in those other countries where there is evidence of largely
unorganized attempts at social media manipulation should consult some of our projects
countryspecificreports,forexampleonCanada(McKelveyandDubois2017).

6
Finally, there are almost certainly cyber troop operations that have not been publicly
documented,anditislikelythatthecaselistwillgrowovertime.Butforthemomentitissafe
toconcludethattherearesignificantsocialmediamanipulationprogramsinthe28countries
weanalyzehere.Therearesimilaritiesontherelativestrategiesandorganizationalbehaviorof
thesecybertroops.

7
I.Strategies,toolsandtechniquesforsocialmedia
manipulation

8
Cyber troops use a variety of strategies, toolsand techniques for social media manipulation.
Generallyspeaking,teamshaveanoverarchingcommunicationsstrategythatinvolvescreating
official government applications, websites or platforms for disseminating content; using
accountseitherreal,fakeorautomatedtointeractwithusersonsocialmedia;orcreating
substantivecontentsuchasimages,videosorblogposts.Teamsalsodifferinthevalenceof
theirmessagesandinteractionswithusersonline.Valenceisatermthatisusedtodefinethe
attractiveness(goodness)oraverseness(badness)ofamessage,eventorthing.Someteams
useprogovernment,positiveornationalisticlanguagewhenengagingwiththepubliconline.
Otherteamswillharass,trollorthreatenuserswhoexpressdissentingpositions.Thefollowing
section outlines in more detail the strategies, tools and techniques used for social media
manipulation,andTable1summarizesthepointsofcomparisonacrossthecountrycases.

Commenting on social media posts

Cybertroopsinalmosteverycountryinoursampleactivelyengagewithusersbycommenting
onpoststhataresharedonsocialmediaplatforms.Thevalenceoftheseengagementsdiffers
acrossoursample.Somecybertroopsfocusonpositivemessagesthatreinforceorsupportthe
governmentspositionorpoliticalideology.Israel,forexample,hasastrictpolicyofengagingin
positiveinteractionswithindividualswhoholdpositionsthatarecriticalthegovernment(Stern
Hoffman,2013).Negativeinteractionsinvolveverbalabuse,harassmentandsocalledtrolling
againstsocialmediauserswhoexpresscriticismofthegovernment.Inmanycountries,cyber
troops engage in these negative interactions with political dissidents. In connection with the
government,AzerbaijansIRELIYouthhavebeenknowntopostabusivecommentsonsocial
media(Geybulla,2016).AndinMexico,journalistsarefrequentlytargetedandharassedover
socialmediabygovernmentsponsoredcybertroops(OCarrol,2017).

However, the valence of comments is not always clearly positive or negative. Instead, some
cybertroopswillpostneutralcomments,designedtodistractordivertattentionfromtheissue
beingdiscussed.SaudiArabia,forexample,engagesinhashtagpoisoning,wherecybertroops
spamtrendinghashtagstodisruptcriticismorotherunwantedconversationsthroughafloodof
unrelated tweets (Freedom House, 2013). Other countries, such as the Czech Republic, post
comments that are neither positive nor negative, but rather factcheck information (Faiola,
2017).Forthemostpart,thevalenceofcommentingstrategiesdoesnotoccurinisolation:cyber
troopswilloftenuseamixofpositive,negativeandneutralpostswhenengagingwithuserson
socialmedia.Thisisbestarticulatedbyamemberofthesocalled50CentParty,socalled
because of a rumor that governmentsponsored Internet commentators were paid 50 cents
everytimetheypostedmessagesonline.Theinformantnotedthatacommonstrategyisto
post emotive comments online in order to generate directed citizen rage towards the

9
commentator;therebydivertingcriticismawayfromthegovernmentorpoliticalissueoriginally
beingdiscussed(Weiwei,2012).

Individual targeting
Individualtargetingisacybertroopstrategythatinvolvesselectinganindividualorgroupto
influence on social media. In Poland, for example, opinion leaders, including prominent
bloggers,journalistsandactivists,arecarefullyselectedandtargetedwithmessagesinorderto
convincethemthattheirfollowersholdcertainbeliefsandvalues(Gorwa2017).Other,more
popularformsofindividualtargetinginvolveharassment.Harassmentgenerallyinvolvesverbal
abuse,hatespeech,discriminationand/ortrollingagainstthevalues,beliefsoridentityofauser
oragroupofusersonline.Individualtargetingisdifferentfromnegativevalencepostsonsocial
media,astheharassmentusuallyspansalongduration.Sometimes,theharassmenttakesplace
duringimportantpoliticalevents,suchaselections.Forexample,inSouthKorea,employees
from the National Intelligence Service launched a series of smear campaigns against South
Koreanoppositionpartiesintheleaduptothe2012presidentialelection(TheKoreanHerald,
2013).Moreoften,individualtargetingisapersistentaspectoftheInternetecosystemthatis
usedtosilencepoliticaldissentonline.Itisalsooneofthemostdangerousformsofcybertroop
activity,asindividualsoftenreceivereallifethreatsandsufferreputationaldamage.InRussia,
cybertroopshavebeenknowntotargetjournalistsandpoliticaldissidents.

FollowinganinvestigationintoarisingnumberofabusiveproRussianpostsontheInternet,
FinnishJournalistJessicaAroreceivedaseriesofabusiveemails,wasvilifiedasadrugdealer
on social media, and mocked as a delusional bimbo in a music video posted to YouTube
(Higgins,2016).InAzerbaijan,individualsarefrequentlytargetedonTwitterandothersocial
media platforms if they criticize the government (Geybulla, 2016). The trolling activities of
Azerbaijans IRELI Youth have even been shown to dissuade regular Internet users from
supporting political protest and engaging in political discussions online (Pearce & Kendzior,
2012).Somecybertroopteamshaveahighlycoordinatedsystemforidentifyingandtargeting
individuals. In Turkey, ringleaders will post a screenshot of an oppositional account so that
otherscanlaunchasmearcampaignagainstthatindividual(Sozeri,2015).InEcuador,individual
targeting is coordinated through the government using the webbased platform Somos +
(Morla,2015a).AndinRussia,leadersoftheKremlinalignedNashiYouthMovementhavesent
aroundalistofhumanrightsactivists,declaringthemthemostvileofenemies(Elder,2012).

Government-sponsored accounts, web pages or applications

Some countries run their own governmentsponsored accounts, websites and applications
designed to spread political propaganda. These accounts and the content that comes out of
themareclearlymarkedasgovernmentoperated.IntheUnitedKingdom,forexample,the77th

10
Brigade maintains a small presence on Facebook and Twitter under its own name (Corfield,
2017).Other countriesaremuch more active in an officialcapacity. Israel hasmorethan350
official government social media accounts, covering the full range of online platforms, from
Twitter to Instagram, and operating in three languages: Hebrew, Arabic and English
(Benedictus,2016).

Butitisnotjustsocialmediaplatformswherecybertroopsareactive.Inaddition,therearea
wide range of online platforms and applications that governments make use of to spread
politicalpropagandaorsilencepoliticaldissent,includingblogs,mobileapplicationsandofficial
government web pages. Sometimes these online resources help volunteers or other citizens
retweet,shareandlikegovernmentsponsoredcontent.UkrainesiArmy,alsoknownasthe
armyoftruth,operatesawebsitewherecitizensandvolunteerscanaccessandsharetruthful
informationonsocialmedia(Benedictus,2016).Inothercases,governmentsponsoredonline
resources can be used to galvanize progovernment supporters. In Ecuador, the government
launchedawebsitecalledSomos+toinvestigateandrespondtosocialmediauserswhocriticize
thegovernment.Thewebsitesendsupdatestosubscriberswhenasocialmediausercriticizes
thegovernment,allowingprogovernmentsupporterstocollectivelytargetpoliticaldissidents
(Morla,2015a).

Fake accounts and computational propaganda

Inadditiontoofficialgovernmentaccounts,manycybertroopteamsrunfakeaccountstomask
theiridentityandinterests.Thisphenomenonhassometimesbeenreferredtoasastroturfing,
whereby the identity of a sponsor or organization is made to appear as grassroots activism
(Howard, 2003). In many cases, these fake accounts are botsor bits of code designed to
interactwithandmimichumanusers.Accordingtomediareports,botshavebeendeployedby
governmentactors inArgentina(Rueda,2012), Azerbaijan (Geybulla, 2016), Iran (BBC News,
2016),Mexico(OCarrol,2017),thePhilippines(WilliamsS,2017),Russia(Duncan,2016),Saudi
Arabia (Freedom House, 2013), South Korea (SangHun, 2013), Syria (York, 2011), Turkey
(Shearlaw,2016)andVenezuela(VOANews,2015).Thesebotsareoftenusedtofloodsocial
medianetworkswithspamandfakenews.Theycanalsoamplifymarginalvoicesandideasby
inflatingthenumberoflikes,sharesandretweetstheyreceive,creatinganartificialsenseof
popularity,momentumorrelevance.Notallgovernmentsmakeuseofthisformofautomation.

InSerbia,forexample,ahandfulofdedicatedemployeesrunfakeaccountstobringattention
tothegovernmentsagenda(Rujevic,2017).Similarly,inVietnam,progovernmentbloggersare
responsibleforspreadingthepartyline(Pham,2013).Somecommentatorshavesuggestedthat
theuseofhumanrunaccountscouldbeduetoalackoftechnicalsophistication(Rujevic,2017).
Butasbotsbecomeincreasinglypolitical,socialmediaplatformshavebecomestricterintheir

11
takedown policies. As a result, many people have gone back to operating the accounts
themselves, rather than automating them. For example, in Mexico, when many of the
governmentsponsored spambots that were used to target journalists and spread
disinformation on social media were blocked, human agents went back to operating the
accountsthemselves(OCarrol,2017).Increasingly,cybertroopsareusingablendofautomation
and human interaction. These socalled cyborgs are deployed to help avoid detection and
makeinteractionsfeelmoregenuine.Finally,itisimportanttonotethatnotallcybertroopsuse
fakeaccounts.NorthKoreaisaninterestingcase,wherestolenSouthKoreanaccountsas
opposedtofakeidentitiesareusedtospreadpoliticalpropaganda(Benedictus,2016).

Content creation
Somecybertroopteamscreatesubstantivecontenttospreadpoliticalmessages.Thiscontent
creation amounts to more than just a comment on a blog or social media feed, but instead
includesthecreationofcontentsuchasblogposts,YouTubevideos,fakenewsstories,pictures
ormemesthathelppromotethegovernmentspoliticalagenda.IntheUnitedKingdom,cyber
troops have been known to create and upload YouTube videos that contain persuasive
messagesunderonlinealiases(Benedictus,2016).Someofthesepsychologicaloperations,
or psyops, have been framed as antiradicalization campaigns designed to deter British
Muslims from going to Syria (Williams, 2015). In Russia, some cyber troops create appealing
onlinepersonasandrunblogsonwebsitessuchasLiveJournal.AccordingtoChens(2015)story,
oneRussiancybertrooperranafortunetellingblogthatprovidedinsightintorelationships,
weightloss,FengShuiand,occasionally,geopolitics,withthegoalofweavingpropaganda
seamlesslyintowhatappearedtobethenonpoliticalmusingsofaneverydayperson.

12
 Table1:Strategies,toolsandtechniquesforsocialmediamanipulation
Country Messagingandvalence Communicationstrategy

Socialmedia Individualtargeting Fakeaccounts Government Contentcreation

comments websites,accounts
orapplications
Argentina +/ Evidencefound Automated .. ..
Australia +/ .. Automated .. ..
Azerbaijan +//n Evidencefound Automated .. ..
Bahrain EvidenceFound Automated,Human .. ..
Brazil +/n Evidencefound Automated,Human, .. Evidencefound
Cyborg
China +//n .. Human .. Evidencefound
CzechRepublic n .. .. .. ..
Ecuador +/ Evidencefound Automated,Human Evidencefound ..
Germany +/ Evidencefound Automated Evidencefound Evidencefound
India +/ .. .. .. Evidencefound
Iran +/n .. Automated .. Evidencefound
Israel + .. .. Evidencefound Evidencefound
Mexico +/ Evidencefound Automated,Human, .. Evidencefound
Cyborg
NorthKorea +/ .. Human .. ..
Poland EvidenceFound Human .. ..
Philippines +/ Evidencefound Automated .. ..
Russia +//n Evidencefound Automated,Human .. Evidencefound
SaudiArabia +/n .. Automated .. ..
Serbia +/ .. Human .. ..
SouthKorea +/ Evidencefound Automated,Human .. ..
Syria + Evidencefound Automated .. ..
Taiwan +//n Evidencefound Cyborg,Human Evidencefound Evidencefound
Turkey +/ Evidencefound Automated,Human Evidencefound
UnitedKingdom Evidencefound Human Evidencefound Evidencefound
Ukraine +/ .. Human Evidencefound
UnitedStates +//n .. Automated,Human, .. Evidencefound
Cyborg
Venezuela + .. Automated,Human Evidencefound ..
Vietnam + .. Human .. Evidencefound
Source:Authorsevaluationsbasedondatacollected20102017.
Note:Thistablereportsonautomatedandtrollingpoliticalactivity,evenifnotclearlyassociatedwithasponsoring
organization.Forsocialmediacomments:+=progovernmentornationalisticcomments,=harassment,trolling
ornegativeinteractionswithusers,n=distractingorchangingthetopicofdiscussion,orfactcheckinginformation.
Noinformationnotedwith...

13
II.Organizationalforms

14
Cyber troops are often made up of an assortment of different actors. In some cases,
governments have their own inhouse teams that are employed as public servants. In other
cases,talentisoutsourcedtoprivatecontractorsorvolunteers.SeeTable2forasummaryof
thefindingsreportedinthissection.

Government
Governmentbased cyber troops are public servants tasked with influencing public opinion.
Theseindividualsaredirectlyemployedbythestateascivilservants,andoftenformasmallpart
ofalargergovernmentadministration.Withinthegovernment,cybertroopscanworkwithina
government ministry, such as in Vietnam, in Hanoi Propaganda and Education Department
(Pham,2013),orinVenezuela,intheCommunicationMinistry(VOANews,2016).IntheUnited
Kingdom,cybertroopscanbefoundacrossavarietyofgovernmentministriesandfunctions,
includingthemilitary(77thBrigade)andelectroniccommunications(GCHQ)(Greenwald,2014c;
MacAskill, 2015). And in China, the public administration behind cyber troop activities is
incredibly vast. There are many local offices that coordinate with their regional and national
counterparts to create and disseminate a common narrative of events across the country
(Weiwei,2012).Othercybertroopsareemployedundertheexecutivebranchofgovernment.
Forexample,inArgentinaandEcuador,cybertroopactivitieshavebeenlinkedtotheofficeof
thePresident(Rueda,2012;Morla,2015a,2015b).

Politicians and parties

Politicalpartiesorcandidatesoftenusesocialmediaaspartofabroadercampaignstrategy.
Here we are interested in political parties or candidates that use social media to manipulate
publicopinionduringacampaign,eitherbypurposefullyspreadingfakenewsordisinformation,
orbytrollingortargetinganysupportfortheoppositionparty.Thisisdifferenttotraditional
digitalcampaignstrategies,whichhavegenerallyfocusedonspreadinginformationaboutthe
partyorcandidatesplatform,orsentadvertisementsouttovoters.

Social media is used by political parties to manipulate the public is to use fake accounts to
artificially inflate the number of followers, likes, shares or retweets a candidate receives,
creatingafalsesenseofpopularity.ThiswasatechniquethattheAustralianCoalitionpartyused
duringitscampaignin2013(Peel,2013).Sometimes,whenpoliticalpartiesorcandidatesuse
socialmediamanipulationaspartoftheircampaignstrategy,thesetacticsarecontinuedwhen
they assume power. For example, in the Philippines, many of the socalled keyboard trolls
hiredtospreadpropagandaforpresidentialcandidateDuterteduringtheelectioncontinueto
spreadandamplifymessagesinsupportofhispoliciesnowhesinpower(Williams,2017).

15
Private contractors
Insomecases,cybertroopsareprivatecontractorshiredbythegovernment.Privatecontractors
areusuallytemporary,andareassignedtohelpwithaparticularmissionorcause.Forexample,
theUnitedStatesgovernmenthiredapublicrelationsfirmtodevelopapersonamanagement
tool to develop and manage fake profiles on social media (Monbiot, 2011). Of course, the
boundary between a private contractor and the state is not always very clear. In Russia, the
InternetResearch Agency,aprivatecompany,is known to coordinate some oftheKremlins
socialmediacampaigns(Chen,2015;Benedictus,2016).

Volunteers
Somecybertroopsarevolunteergroupsthatactivelyworktospreadpoliticalmessagesonsocial
media. They are not just people who believe in the message and share their ideals on social
media.Instead,volunteersareindividualswhoactivelycollaboratewithgovernmentpartnersto
spread political ideology or progovernment messages. In many cases, volunteer groups are
madeupsolelyofyouthadvocacyorganizations,suchasIRELIinAzerbaijan(Geybulla,2016)or
NashiinRussia(Elder,2012).InIsrael,thegovernmentactivelyworkswithstudentvolunteers
fromJewishorganizationsorotherproIsraelgroupsaroundtheworld(SternHoffman,2013).
Thesecybertroopsareconsideredvolunteersbecausetheyarenotonaformalpayroll,asa
publicservantorprivatecontractorwouldbe.Inmanycases,however,volunteersreceiveother
rewards for their time. For example, in Israel, the topperforming students are awarded
scholarshipsfortheirwork(SternHoffman,2013),andinAzerbaijan,volunteerworkwithIRELI
isconsideredasteppingstonetomoreseniorrolesinpublicadministration(Geybulla,2016).

Paid citizens
Somecybertroopsarecitizenswhoareactivelyrecruitedbythegovernmentandarepaidor
remuneratedinsomewayfortheirwork.Theyarenotofficialgovernmentemployeesworking
inpublicservice,noraretheyemployeesofacompanycontractedtoworkonasocialmedia
strategy. They are also not volunteers, because they are paid for their time and efforts in
supportingacybertroopcampaign.Normally,thesepaidcitizensarerecruitedbecausethey
hold a prominent position in society or online. In India, for example, citizens are actively
recruited by cyber troop teams in order to help propagate political ideologies and messages
(Kohlil,2013).Sincethesecitizensarenotofficiallyaffiliatedwiththegovernmentorapolitical
party,theirindependentvoicecanbeusedtohelpdisseminatemessagesfromaseemingly
neutralperspective.

16
 Table2:Organizationalforms
Country Government PoliticiansandParties CivilSociety Citizens Private Number
Contractor ofForms
Argentina MinistryofCommunication RepublicanProposalParty .. .. .. 2
PresidentsOffice
Australia .. TheCoalition .. .. .. 1
Azerbaijan .. .. IRELI,theIT .. .. 1
Academy
Bahrain NationalCyberCrimeUnit .. .. .. .. 1
Brazil .. BrazilianSocialDemocracyParty .. Evidence AgenciaPepper 3
(PSDB),WorkersParty(PT) Found /no.bot
China StateInternetInformationOffice,Ministryof .. .. Evidence .. 2
IndustryandInformationTechnology, Found
MinistryofPublicSecurity,CommunistParty
Czech CentreAgainstTerrorismandHybrid .. .. .. .. 1
Republic Threats
Germany CyberKommandoderBundeswehr AlternativeforGermany(AFD) .. .. .. 2
Ecuador MinistryofStrategicSectors .. .. .. Ribeney, 2
PresidentsOffice Perceraand
XimahDigital
India .. BharatiyaJanataParty(BJP) .. Evidence .. 2
Found
Iran RevolutionaryGuard,SupremeCouncilof .. .. .. .. 1
Cyberspace
Israel IsraelDefenceForce .. IsraelUnderFire .. .. 2
PrimeMinistersOffice
Mexico .. InstitutionalRevolutionaryParty .. Evidence Andreas 3
(PRI) Found Sepulveda
North UnitedFrontDepartmentand .. .. .. .. 1
Korea ReconnaissanceGeneralBureau
Poland .. EvidenceFound .. EvidenceFound 1
Philippines .. ThePartidoDemokratikoPilipino EvidenceFound Evidence NicGabunada 4
LakasngBayan Found
Russia GRU .. Nashi Evidence Internet 4
TheKremlin Found Research
Agency
Saudi MinistryofDefenseTheSaudiideological .. SaudiElectronic Evidence Qorvis 4
Arabia WarfareCenter Army,Salmani Found
Army
Serbia PrimeMinistersOffice SerbianProgressiveParty .. .. .. 2
South NationalIntelligenceService .. .. .. .. 1
Korea
Syria SyrianElectronicArmy .. .. .. EGHNA 2
Taiwan .. DemocraticProgressiveParty .. .. .. 1
(DPP),NationalistParty(KMT)
Turkey .. JusticeandDevelopmentParty .. Evidence .. 2
(AKP) Found
United 77thBrigade,GCHQ .. .. .. .. 1
Kingdom
Ukraine InformationPolicyMinistry .. EvidenceFound .. .. 2
United DARPA,USCyberCommand,USAgencyfor DemocraticParty EvidenceFound Evidence Centcom,HB 5
States InternationalDevelopment,AirForce, RepublicanParty Found Gary
Pentagon
Venezuela CommunicationMinistry .. EvidenceFound .. .. 2
Vietnam HanoiPropagandaandEducation .. .. .. .. 1
Department
Source:Authorsevaluationsbasedondatacollected20102017.Note:Noinformationnotedwith...

17
III.Organizationalbudget,behaviorandcapacity

18
Cybertroopteamsdifferintheirbudgets,behaviorsandcapacity.Ourstudyhasfoundthatteam
sizesrangefromasmallteamoflessthan20(e.g.intheCzechRepublic)toavastnetworkof
two million individuals working to promote the party line (e.g. in China). Table 3 presents
comparativedataongovernmentcapacityandestimatedbudgets.Thebudgetcolumnincludes
the best estimate of resources and how that money is allocated. The management column
describes the organizational practices of the offices tasked with social media manipulation.
Thesecategoriesaredescribedinfurtherdetailbelow.

Budget information
Cyber troops spend various amounts of funds on their operations. The amount of publicly
availableinformationonbudgetsandspendingisrelativelylimited.Nevertheless,weareableto
reportonafewnumbers.Mostofthebudgetaryinformationhighlightedinthissectionrefersto
contractualamountsforoneoperation,asopposedtoanoverallannualexpenditureforstaffing,
technical equipment or other resources required. For example, Ecuador, which contracts out
cyber troop activity to private firms, spends, on average, USD200,000 per contract (Morla,
2015).EGHNA,whichcontractsoutworkfortheSyriangovernment,notesthattheusualproject
cost is about USD4,000 (EGHNA, 2017). In a few cases, such as in Russia, there have been
suggestions that military expenditures for social media manipulation operations have been
increasingovertheyears(Sindelar,2014).

Organizational behavior
Wehaveidentifiedseveralorganizationalpracticesofcybertroopteams:(1)aclearhierarchy
and reporting structure; (2) content review by superiors; and (3) strong coordination across
agenciesorteam;(4)weakcoordinationacrossagenciesorteams;(5)liminalteams.Insome
cases,teamsarehighlystructuredwithclearlyassigneddutiesandareportinghierarchy,much
like the management of a company or typical government bureaucracy. Tasks are often
delegatedonadailybasis.InRussiaandChina,forexample,cybertroopsareoftengivenalist
ofopinionsortopicsthataresupposedtobediscussedonadailybasis.Thesetopicsusually
relatetoaparticularpoliticalissuethatistakingplace(Cook,2011;Chen,2015).Aspartofthe
reportingstructure,managersorsuperiorswilloftenreviewtheworkoftheteam.

In Serbia, for example, cyber troops and their work are closely monitored and reviewed by
managers and leaders (Rujevic, 2017). Sometimes there is more than one agency or team
workingonpropagandacampaigns,suchasinChina,wherepropagandaofficesexistatthelocal
levelsofgovernment.Here,eachoftheseofficesfocusesonlocalissues,butalsocoordinates
broadermessagesacrossthecountrydependingonthedomesticpoliticalissuesbeingdiscussed
at the time (Weiwei, 2012; Lam, 2013). In other cases, teams are less organized, structured,
supervised,and coordinated. For example, the Saudi Electronic Army and the Salmani Army

19
have several members conducting campaigns on social media. These teams are often less
coordinatedandlessformalthanothercybertroopteams,butnonethelesshaveeffectsonthe
socialmediaenvironment(Hussein2017).

Capacity building
Cybertroopswilloftenengageincapacitybuildingactivities.Theseinclude:(1)trainingstaffto
improve skills and abilities associated with producing and disseminating propaganda; (2)
providingrewardsorincentivesforhighperformingindividuals;and(3)investinginresearchand
developmentprojects.Whenitcomestotrainingstaff,governmentswillofferclasses,tutorials
orevensummercampstohelppreparecybertroopsforengagingwithusersonsocialmedia.In
Russia,Englishteachersarehiredtoteachpropergrammarforwhentheycommunicatewith
Westernaudiences(Seddon,2014).Othertrainingmeasuresfocusonpolitology,whichaims
tooutlinetheRussianperspectiveoncurrentevents(Chen,2015).InAzerbaijan,youngpeople
areprovidedwithbloggingandsocialmediatrainingtohelpmaketheirmicrobloggingwebsites
more effective at reaching desired audiences. Reward systems are sometimes developed to
encouragecybertroopstodisseminatemoremessages.Forexample,inIsrael,thegovernment
providesstudentswithscholarshipsfortheirworkonproIsraelsocialmediacampaigns(Stern
Hoffman,2013).Itisimportanttonotethattrainingandrewardprogramsoftenoccurtogether.
InNorthKorea,forexample,youngcomputerexpertsaretrainedbythegovernment,andtop
performersareselectedtojointhemilitaryuniversity(Firn,2013).Finally,somecybertroopsin
somedemocraciesareinvestinginresearchanddevelopmentinareassuchasnetworkeffects
and how messages can spread and amplify across social media. For example, in the United
States,in2010,DARPAfundedaUSD8.9millionstudytoseehowsocialmediacouldbeusedto
influencepeoplesbehaviorbytrackinghowtheyrespondedtocontentonline(QuinnandBall,
2014).

20
 Table3:Organizationalbudget,behaviorandcapacity
Country Yearof Budgetinformation Organizationalbehavior Staff Capacitybuilding
earliest (USD) capacity
report
Argentina 2012 .. .. 3540 ..
Australia 2013 .. .. ..
Azerbaijan 2011 .. Clearhierarchyandreporting 50,000 Trainingis
structure,coordinationacross provided
agencies
Bahrain 2013 .. .. ..
Brazil 2010 3m Clearhierarchyandreporting .. Extendeduse,
structure,coordinationacross beyondelection
agencies,integratedwith day
campaignandpartyorganization
China 2011 .. Clearhierarchyandreporting 2,000,000 Trainingis
structure,coordinationacross provided,reward
agencies system
CzechRepublic 2017 .. Coordinationacrossagencies 20 ..
Ecuador 2014 Multiplecontractsto .. .. ..
privatecompanies,
estimatedat200,000
Germany 2016 .. .. <300 ..
India 2013 .. .. .. ..
Iran 2012 .. .. 20,000 ..
Israel 2013 .. .. 400 Rewardsystem
Mexico 2017 600,000 Informal,liminalteams .. limited
NorthKorea 2013 .. .. 200 Trainingis
provided,reward
system
Poland 2015 .. Somecoordinationacrossteams .. Trainingis
provided
Philippines 2016 200,000 Liminalmembership,butsome 400500 ..
coordinationacrossteams
Russia 2012 10m Clearhierarchyandreporting 400 Trainingis
structure,contentisreviewedby provided
superiors,coordinationacross
agencies
SaudiArabia 2013 .. Liminalmembership,less .. ..
coordinatedacrossteams.
Serbia 2017 .. Clearhierarchyandreporting .. ..
structure,coordinationacross
agencies
SouthKorea 2013 .. .. <20 ..
Syria 2011 4,000percontractwith Liminalmembership .. ..
EGHNA
Taiwan 2010 .. .. ..
Turkey 2013 Multipleprograms,one Highlycoordinatedteams 6,000 Trainingis
valuedat209,000 provided
UnitedKingdom 2014 .. .. 1,500 ..
Ukraine 2015 .. .. 20,000 ..
UnitedStates 2011 Multipleprograms,valued .. .. Investsin
at2.7m,42mand8.9m Researchand
Development
Venezuela 2015 .. .. .. ..
Vietnam 2013 .. .. 1,000 ..
Source:Authorsevaluationsbasedondatacollected20102017.Note:AllcurrencyvaluesinUSDfromyearof
report.Noinformationnotedwith...

21
Conclusion
Thereisnodoubtthatindividualsocialmediauserscanspreadhatespeech,trollotherusers,or
setupautomatedpoliticalcommunicationcampaigns.Unfortunately,thisisalsoanorganized
phenomenon, with major governments and political parties dedicating significant resources
towardstheuseofsocialmediaforpublicopinionmanipulation.

Figure 1 is a country heat map of cyber troop capacity, defined by the number of different
organizationaltypesinvolved.Inmanycountries,politicalactorshavenoreportedabilitytofield
socialmediacampaigns.Insomecountries,oneortwoknownpoliticalactorsoccasionallyuse
socialmediaforpoliticalmessaging,andinafewothercountriestherearemultiplegovernment
agencies,politicalparties,orcivilsocietygroupsorganizingtrollingandfakenewscampaigns.

Figure1:Organizationaldensityofcybertroops,2017

Inthisfigure,countrieswithmanykindsoforganizations(governments,politicalparties,civil
societygroups,organizedcitizens,orindependentcontractors)areindarkershadesofred.Data
is taken from the far right column of Table 2, and this figure reveals which countries have
multiplekindsofactors,allusingorganizedsocialmediacampaigns,tobattleforpublicopinion.

Organized social media manipulation occurs in many countries around the world. In
authoritarianregimesittendsto bethegovernmentthatfundsandcoordinatespropaganda
campaigns on social media. In democracies, it tends to be the political parties that are the
primaryorganizersofsocialmediamanipulation.

22

Inmanycountries,cybertroopshavemultipleaffiliations,funders,orclients.Sowhilethe
primaryorganizersofsocialmediamanipulationmaybegovernmentagenciesorpolitical
parties,itisalsoimportanttodistinguishthosecountrieswheremanykindsofactorsmakeuse
ofcybertroops.Nodoubttheorganizationofcybertroopswillcontinuetoevolve.Itwilllikelyremain,
however,aglobalphenomenon.

23
References
Arnaudo,D.(2017).ComputationalPropagandainBrazil:SocialBotsDuringElections.
ComputationalPropagandaProjectWorkingPaperSeries,2017(8).Retrievedfrom
http://comprop.oii.ox.ac.uk/2017/06/19/computationalpropagandainbrazilsocialbots
duringelections/
Baldor,L.C.(2016,February26).USmilitarylaunchescampaignofcyberattacksagainst
IslamicState.USNews&WorldReport.Retrievedfrom
https://www.usnews.com/news/politics/articles/20160226/apnewsbreakdodlaunches
aggressivecyberwaragainstis
Ball,A.(2014,July25).NoTweetsforYou:CorreasOpponentsCensored.PanAmPost.
Retrievedfromhttps://panampost.com/annekeball/2014/07/25/notweetsforyou
correasopponentscensored/
Ball,J.(2014,July14).GCHQhastoolstomanipulateonlineinformation,leakeddocuments
show.TheGuardian.Retrievedfromhttps://www.theguardian.com/uk
news/2014/jul/14/gchqtoolsmanipulateonlineinformationleak
BBCTrending.(2016a,March16).WhosatthecontrolsofIransbotarmy?BBCNews.
Retrievedfromhttp://www.bbc.co.uk/news/blogstrending35778645
BBCTrending.(2016b,December7).Trollsandtriumph:adigitalbattleinthePhilippines.BBC
News.Retrievedfromhttp://www.bbc.co.uk/news/blogstrending38173842
Bender,V.J.,&Oppong,M.(2017,July2).FraukePetryunddieBots.FranffurterAllgemeine.
Retrievedfromhttp://www.faz.net/aktuell/politik/digitalerwahlkampffraukepetryund
diebots14863763.html
Benedictus,L.(2016,November6).Invasionofthetrollarmies:fromRussianTrump
supporterstoTurkishstatestooges.TheGuardian.Retrievedfrom
https://www.theguardian.com/media/2016/nov/06/trollarmiessocialmediatrump
russian
Bloom,A.J.(2012,February23).DARPAAndTheirRobotArmyAreComingForYour
Memes.TheAmericanConservative.Retrievedfrom
http://www.theamericanconservative.com/2012/02/23/darpaandtheirrobotarmyare
comingforyourmemes/
Brooking,E.T.,&Singer,P.W.(2016,November).WarGoesViral.TheAtlantic.Retrievedfrom
https://www.theatlantic.com/magazine/archive/2016/11/wargoesviral/501125/
Bugorkova,O.(2015,March19).Ukraineconflict:InsideRussiasKremlintrollarmy.BBC
News.Retrievedfromhttp://www.bbc.co.uk/news/worldeurope31962644

24
Butler,D.,Gillum,J.,&Arce,A.(2014,April4).WhiteHousedefendsCubanTwittertostir
unrest.YahooNews.Retrievedfromhttp://news.yahoo.com/whitehousedefends
cubantwitterstirunrest222510641.html
Carroll,R.(2012,August9).FakeTwitteraccountsmaybedrivingupMittRomneysfollower
number.TheGuardian.Retrievedfrom
https://www.theguardian.com/world/2012/aug/09/faketwitteraccountsmittromney
Chen,A.(2015,June2).TheAgency.TheNewYorkTimes.Retrievedfrom
https://www.nytimes.com/2015/06/07/magazine/theagency.html
Cook,S.(2011,October11).Chinasgrowingarmyofpaidinternetcommentators.Freedom
House.Retrievedfromhttps://freedomhouse.org/blog/china%E2%80%99sgrowing
armypaidinternetcommentators#.VRMVvTSsUrM
Corfield,G.(2017,January3).Armysocialmediapsyopsbodsstrugglingtoattractfreshblood.
TheRegister.Retrievedfrom
https://www.theregister.co.uk/2017/01/03/77_brigade_struggling_recruit_40_pc_below_
establishment/
CzechNewsAgency.(2017,May22).DisinformationwebstobeactivebeforeCzechelections,
expertsays.PragueMonitor.Retrievedfrom
http://praguemonitor.com/2017/05/22/disinformationwebsbeactiveczechelections
expertsays
DailySabah.(2015,May11).AKPartyfoundedNewTurkeyDigitalOfficeforthegeneral
electionsonJune7.DailySabah.Retrievedfrom
http://www.dailysabah.com/elections/2015/05/11/akpartyfoundednewturkeydigital
officeforthegeneralelectionsonjune7
DefenseNews.(2016,April1).GermanyoutlinesplantocreateBundeswehrcybercommand.
Retrievedfromhttps://defensenewsalert.blogspot.co.il/2016/04/germanyoutlinesplan
tocreate.html
Duncan,J.(2016,October16).Russialaunchestrollfactorytofloodinternetwithliesabout
UK.MailOnline.Retrievedfromhttp://www.dailymail.co.uk/~/article3840816/index.html
Earl,J.,Martin,A.,McCarthy,J.D.,&Soule,S.A.(2004).TheUseofNewspaperDatainthe
StudyofCollectiveAction.AnnualReviewofSociology,30(1),6580.
EGHNA.(2012).EGHNAMarketplace.RetrievedJune26,2017,from
https://www.drupal.org/eghna
Elder,M.(2012,February7).EmailsgiveinsightintoKremlinyouthgroupspriorities,means
andconcerns.TheGuardian.Retrievedfrom

25
 https://www.theguardian.com/world/2012/feb/07/nashiemailsinsightkremlingroups
priorities
Elliot,C.(2014,May4).ThereaderseditoronproRussiatrollingbelowthelineonUkraine
stories.TheGuardian.Retrievedfrom
https://www.theguardian.com/commentisfree/2014/may/04/prorussiatrollsukraine
guardianonline
Faiola,A.(2017,January22).AsColdWarturnstoInformationWar,anewfakenewspolice
combatsdisinformation.WashingtonPost.Retrievedfrom
https://www.washingtonpost.com/world/europe/ascoldwarturnstoinformationwar
anewfakenewspolice/2017/01/18/9bf49ff6d80e11e6a0e6d502d6751bc8_story.html
Fassihi,F.(2012,March16).IransCensorsTightenGrip.WallStreetJournal.Retrievedfrom
https://www.wsj.com/articles/SB10001424052702303717304577279381130395906
Fiedler,V.M.(2016,December16).AfDstelltAppfrverunsicherteBrgervor.Der
Tagesspiegel.Retrievedfromhttp://www.tagesspiegel.de/politik/nrwwahlkampfafd
stelltappfuerverunsichertebuergervor/14991488.html
Fielding,N.,&Cobain,I.(2011,March17).Revealed:USspyoperationthatmanipulatessocial
media.TheGuardian.Retrievedfrom
https://www.theguardian.com/technology/2011/mar/17/usspyoperationsocial
networks
Finley,K.(2015,August).ProGovernmentTwitterBotsTrytoHushMexicanActivists.
RetrievedJuly14,2017,fromhttps://www.wired.com/2015/08/progovernmenttwitter
botstryhushmexicanactivists/
Firn,M.(2013,August13).NorthKoreabuildsonlinetrollarmyof3,000.TheTelegraph.
Retrievedfrom
http://www.telegraph.co.uk/news/worldnews/asia/northkorea/10239283/NorthKorea
buildsonlinetrollarmyof3000.html
FreedomHouse.(2013).SaudiArabia.RetrievedApril10,2017,from
https://freedomhouse.org/report/freedomnet/2013/saudiarabia
Gallagher,P.(2015,March27).TheOrwelliantrollfactoriesspoutingproKremlin
propaganda.TheIndependent.Retrievedfrom
http://www.independent.co.uk/news/world/europe/revealedputinsarmyofpro
kremlinbloggers10138893.html
Gavilan,J.(2016,June4).DutertesP10Msocialmediacampaign:Organic,volunteerdriven.
Rappler.Retrievedfromhttp://www.rappler.com/newsbreak/richmedia/134979rodrigo
dutertesocialmediacampaignnicgabunada

26
Geybulla,A.(2016,November21).InthecrosshairsofAzerbaijanspatriotictrolls.Open
Democracy.Retrievedfromhttps://www.opendemocracy.net/odrussia/arzu
geybulla/azerbaijanpatriotictrolls
Glowacki,W.(2015,September28).PrawoiSprawiedliwokrlujewpolskiminternecie.
Pomagawtymzdyscyplinowanaarmiatrolli.GazetaKrakowska.Retrievedfrom
http://www.gazetakrakowska.pl/artykul/8866523,prawoisprawiedliwosckrolujew
polskiminterneciepomagawtymzdyscyplinowanaarmiatrolli,3,id,t,sa.html
Gorwa,R.(2017).ComputationalPropagandainPoland:FalseAmplifiersandtheDigitalPublic
Sphere.ComputationalPropagandaProjectWorkingPaperSeries.Retrievedfrom
http://comprop.oii.ox.ac.uk/wpcontent/uploads/sites/89/2017/06/CompropPoland.pdf
Greenwald,G.(2014a,February24).HowCovertAgentsInfiltratetheInternettoManipulate,
Deceive,andDestroyReputations.TheIntercept.Retrievedfrom
https://theintercept.com/2014/02/24/jtrigmanipulation/
Greenwald,G.(2014b,April4).TheCubanTwitterScamIsaDropintheInternetPropaganda
Bucket.TheIntercept.Retrievedfromhttps://theintercept.com/2014/04/04/cuban
twitterscamsocialmediatooldisseminatinggovernmentpropaganda/
Greenwald,G.(2014c,July14).HackingOnlinePollsandOtherWaysBritishSpiesSeekto
ControltheInternet.TheIntercept.Retrievedfrom
https://theintercept.com/2014/07/14/manipulatingonlinepollswaysbritishspiesseek
controlinternet/
Greenwald, G. (2015, June 22). Controversial GCHQ Unit Engaged in Domestic Law
Enforcement, Online Propaganda, Psychology Research. The Intercept. Retrieved from
https://theintercept.com/2015/06/22/controversialgchqunitdomesticlaw
enforcementpropaganda/
Hazazi,Hussein.2017.150.Okaz.June
14.
Herring, S. C. (2009). Web Content Analysis: Expanding the Paradigm. In J. Hunsinger, L.
Klastrup, & M. Allen (Eds.), International Handbook of Internet Research (pp. 233249).
Springer Netherlands. Retrieved from http://link.springer.com/chapter/10.1007/9781
402097898_14
Higgins,A.(2016,May30).EfforttoExposeRussiasTrollArmyDrawsViciousRetaliation.The
New York Times. Retrieved from
https://www.nytimes.com/2016/05/31/world/europe/russiafinlandnatotrolls.html
Ho, Catherine. 2016. Saudi Government Has Vast Network of PR, Lobby Firms in U.S.
WashingtonPost,April20.

27
 https://www.washingtonpost.com/news/powerpost/wp/2016/04/20/saudigovernment
hasvastnetworkofprlobbyfirmsinus/.
Howard,P.N.(2003).DigitizingtheSocialContract:ProducingAmericanPoliticalCulturein
theAgeofNewMedia.TheCommunicationReview,6(3),213245.
https://doi.org/10.1080/10714420390226270
Hunter,I.(2015,June6).Turkishrulingpartyssocialmediacampaignersdenybeingatroll.The
Independent.Retrievedfromhttp://www.independent.co.uk/news/world/europe/turkish
presidentssocialmediacampaignersdenybeingatrollarmy10301599.html
HrriyetDailyNews.(2015,June15).ErdoanattendsAktrollwedding,chatswithwell
knownsuspect.HrriyetDailyNews.Retrievedfrom
http://www.hurriyetdailynews.com/erdoganattendsaktrollweddingchatswithwell
knownsuspect.aspx?PageID=238&NID=84013&NewsCatID=338
Jacobs,A.(2014,July21).ItsAnotherPerfectDayinTibet!TheNewYorkTimes.Retrieved
fromhttps://www.nytimes.com/2014/07/22/world/asia/trendingattractivepeople
sharingupbeatnewsabouttibet.html
Jones,M.(2017,June7).Hacking,botsandinformationwarsintheQatarspat.Washington
Post.Retrievedfromhttps://www.washingtonpost.com/news/monkey
cage/wp/2017/06/07/hackingbotsandinformationwarsintheqatarspat/
Jones,M.O.(2016,June22).Around51%ofTweetson#BahrainHashtagCreatedby
AutomatedSectarianBots.RetrievedJuly14,2017,from
https://marcowenjones.wordpress.com/2016/06/22/around51oftweetsonbahrain
hashtagbyautomatedsectarianbots/
Joyce,M.,Antonio,R.,&Howard,P.N.(2013).GlobalDigitalActivismDataSet.ICPSR.
Retrievedfromhttp://www.icpsr.umich.edu/icpsrweb/ICPSR/studies/34625/version/2
Kaiman,J.(2014,July22).FreeTibetexposesfakeTwitteraccountsbyChinapropagandists.
TheGuardian.Retrievedfromhttps://www.theguardian.com/world/2014/jul/22/free
tibetfaketwitteraccountschinapropagandists
Kililkaya,E.(2015,September4).AreTurkeysAKPTwittertrollsheadingforunemployment?
AlMonitor.Retrievedfromhttp://www.almonitor.com/pulse/originals/2015/09/turkey
electionsakpmullsdumpingsocialmediatrolls.html
Kizilkaya,E.(2013,November15).AKPssocialmediawars.AlMonitor.Retrievedfrom
http://www.almonitor.com/pulse/originals/2013/11/akpsocialmediatwitter
facebook.html

28
Kohlil,K.(2013,October11).CongressvsBJP:Thecuriouscaseoftrollsandpolitics.TheTimes
ofIndia.Retrievedfromhttp://timesofindia.indiatimes.com/india/CongressvsBJPThe
curiouscaseoftrollsandpolitics/articleshow/23970818.cms
Lam,O.(2013,October17).ChinaBeefsUp50CentArmyofPaidInternetPropagandists.
GlobalVoicesAdvocacy.Retrievedfromhttps://advox.globalvoices.org/2013/10/17/china
beefsup50centarmyofpaidinternetpropagandists/
MacAskill,E.(2015,January31).BritisharmycreatesteamofFacebookwarriors.TheGuardian.
Retrievedfromhttps://www.theguardian.com/uknews/2015/jan/31/britisharmy
facebookwarriors77thbrigade
Marczak,B.(2013,July31).BahrainGovtusingfakeTwitteraccountstotrackonlinecritics.
RetrievedJuly14,2017,fromhttps://bahrainwatch.org/blog/2013/07/31/bahraingovt
usingfaketwitteraccountstotrackonlinecritics/
McKelvey,F.,&Dubois,E.(2017).ComputationalPropagandainCanada:TheUseofPolitical
Bots.TheComputationalPropagandaProjectWorkingPaperSeries.Retrievedfrom
http://comprop.oii.ox.ac.uk/2017/06/19/computationalpropagandaincanadatheuse
ofpoliticalbots/
Monbiot,G.(2011,February23).Theneedtoprotecttheinternetfromastroturfinggrows
evermoreurgent.TheGuardian.Retrievedfrom
https://www.theguardian.com/environment/georgemonbiot/2011/feb/23/needto
protectinternetfromastroturfing
Morla,R.(2015a,January26).EcuadorsCorreaRecruitsLegionofSocialMediaTrolls.
Retrievedfromhttps://panampost.com/rebecamorla/2015/01/26/ecuadorscorrea
recruitslegionofsocialmediatrolls/
Morla,R.(2015b,March25).CorreasSocialMediaTrollCenterExposedinQuito.PanAmPost.
Retrievedfromhttps://panampost.com/rebecamorla/2015/03/25/correassocialmedia
trollcenterexposedinquito/
Mukherji,A.(2015,February10).HasArvindKejriwalmurderedBJPsTwittertrolls?Timesof
IndiaBlog.Retrievedfromhttp://blogs.timesofindia.indiatimes.com/pointand
shoot/hasarvindkejriwalmurderedbjpstwittertrolls/
NewsAZ.(2011,June8).IreliyouthunionfocusingonIT.NewsAZ.Retrievedfrom
http://news.az/articles/society/38037
OCarroll,T.(2017,January24).Mexicosmisinformationwars.Medium.Retrievedfrom
https://medium.com/amnestyinsights/mexicosmisinformationwars
cb748ecb32e9#.n8pi52hot

29
Orcutt,M.(2012,June21).TwitterMischiefPlaguesMexicosElection.MITTechnologyReview.
Retrievedfromhttps://www.technologyreview.com/s/428286/twittermischiefplagues
mexicoselection/
Parfitt,T.(2015,June24).MylifeasaproPutinpropagandistinRussiassecrettrollfactory.
TheTelegraph.Retrievedfrom
http://www.telegraph.co.uk/news/worldnews/europe/russia/11656043/Mylifeasapro
PutinpropagandistinRussiassecrettrollfactory.html
Pearce,K.E.,&Kendzior,S.(2012).NetworkedAuthoritarianismandSocialMediain
Azerbaijan.JournalofCommunication,62(2),283298.https://doi.org/10.1111/j.1460
2466.2012.01633.x
Peel,T.(2013,August26).TheCoalitionsTwitterfraudanddeception.IndependentAustralia.
Retrievedfromhttps://independentaustralia.net/politics/politicsdisplay/thecoalitions
twitterfraudanddeception
Pham,N.(2013,January12).Vietnamadmitsdeployingbloggerstosupportgovernment.BBC
News.Retrievedfromhttp://www.bbc.co.uk/news/worldasia20982985
Phillips,T.(2016,May20).Chineseofficialscreate488mbogussocialmediapostsayear.The
Guardian.Retrievedfromhttps://www.theguardian.com/world/2016/may/20/chinese
officialscreate488msocialmediapostsayearstudyfinds
Quinn,B.,&Ball,J.(2014,July8).USmilitarystudiedhowtoinfluenceTwitterusersinDarpa
fundedresearch.TheGuardian.Retrievedfrom
https://www.theguardian.com/world/2014/jul/08/darpasocialnetworksresearch
twitterinfluencestudies
Rawnsley,A.(2011,July15).PentagonWantsaSocialMediaPropagandaMachine.WIRED.
Retrievedfromhttps://www.wired.com/2011/07/darpawantssocialmediasensorfor
propagandaops/
Rebello,A.(2016).Guerrapelovoto.UOLEleicoes.Retrievedfrom
https://www.uol/eleicoes/especiais/acampanhaportrasdastimelines.htm#umcaso
depolicia
Ressa,M.A.(2016,October3).Propagandawar:Weaponizingtheinternet.Rappler.Retrieved
fromhttp://www.rappler.com/nation/148007propagandawarweaponizinginternet
Reyes,R.R.,&Mallari,M.R.(2016,November27).MoneyandcredulitydriveDutertes
keyboardarmy.BusinessMirror.Retrievedfrom
http://www.businessmirror.com.ph/moneyandcredulitydriveduterteskeyboardarmy/
Rockefeller,H.(2011,February17).UPDATED:TheHBGaryEmailThatShouldConcernUsAll.
RetrievedJuly14,2017,fromhttps://www.dailykos.com/story/2011/2/16/945768/

30
Rosenkranz,B.(2017,May31).WasMargotKmannwirklichberdieAfDgesagthat.
bermedien.Retrievedfromhttp://uebermedien.de/16231/wasmargotkaessmann
wirklichueberdieafdgesagthat/
Rueda,M.(2012,December27).2012sBiggestSocialMediaBlundersinLatAmPolitics.ABC
News.Retrievedfromhttp://abcnews.go.com/ABC_Univision/ABC_Univision/2012s
biggestsocialmediablunderslatinamericanpolitics/story?id=18063022
Rujevic,N.(2017,January5).Serbiangovernmenttrollsinthebattlefortheinternet.Deutsche
Welle.Retrievedfromhttp://www.dw.com/en/serbiangovernmenttrollsinthebattle
fortheinternet/a37026533
Sanghun,C.(2013a,June14).SouthKoreanIntelligenceAgentsAccusedofTarring
OppositionOnlineBeforeElection.TheNewYorkTimes.Retrievedfrom
http://www.nytimes.com/2013/06/15/world/asia/southkoreanagentsaccusedof
tarringoppositionbeforeelection.html
Sanghun,C.(2013b,November21).ProsecutorsDetailAttempttoSwaySouthKorean
Election.TheNewYorkTimes.Retrievedfrom
http://www.nytimes.com/2013/11/22/world/asia/prosecutorsdetailbidtoswaysouth
koreanelection.html
Schimmeck,V.T.(2017,March31).DasneueCyberKommandoderBundeswehrMilitrsmit
ComputermausundLaptop.Deutschlandfunk.Retrievedfrom
http://www.deutschlandfunk.de/dasneuecyberkommandoderbundeswehrmilitaers
mit.724.de.html?dram:article_id=382767
Seddon,M.(2014,June2).DocumentsShowHowRussiasTrollArmyHitAmerica.BuzzFeed.
Retrievedfromhttp://www.buzzfeed.com/maxseddon/documentsshowhowrussias
trollarmyhitamerica
Shearlaw,M.(2015,April2).FromBritaintoBeijing:howgovernmentsmanipulatethe
internet.TheGuardian.Retrievedfrom
https://www.theguardian.com/world/2015/apr/02/russiatrollfactorykremlincyber
armycomparisons
Shearlaw,M.(2016,November1).Turkishjournalistsfaceabuseandthreatsonlineastrolls
stepupattacks.TheGuardian.Retrievedfrom
https://www.theguardian.com/world/2016/nov/01/turkishjournalistsfaceabusethreats
onlinetrollsattacks
Sindelar,D.(2014,August12).TheKremlinsTrollArmy.TheAtlantic.Retrievedfrom
https://www.theatlantic.com/international/archive/2014/08/thekremlinstroll
army/375932/

31
Slovyansk,B.P.(2014,April27).Putins300whipupUkrainianturmoil.TheTimes.Retrieved
fromhttps://www.thetimes.co.uk/article/putins300whipupukrainianturmoil
53h79xl77jx
Smith,M.(2017,January5).RussiausedcyberpropagandatoinfluenceopinioninEurope,US
Senatetold.TheMirror.Retrievedfromhttp://www.mirror.co.uk/news/uknews/russia
usedcyberpropagandainfluence9569457
Solon,O.(2015,January31).Twitter,the@BritishArmyneedsYOU!TheMirror.Retrieved
fromhttp://www.mirror.co.uk/news/technologyscience/technology/cyberwarfare
armyseekstwitter5076931
Sonnad,N.(2014,December18).HackedemailsrevealChinaselaborateandabsurdinternet
propagandamachine.Quartz.Retrievedfromhttps://qz.com/311832/hackedemails
revealchinaselaborateandabsurdinternetpropagandamachine/
Sozeri,E.K.(2015,October22).MappingTurkeysTwittertrolls.TheDailyDot.Retrievedfrom
https://www.dailydot.com/layer8/turkeytwittertrolls/
Sozeri,E.K.(2016,September30).RedHackLeaksRevealtheRiseofTurkeysPro
GovernmentTwitterTrolls.TheDailyDot.Retrievedfrom
https://www.dailydot.com/layer8/redhackturkeyalbayrakcensorship/
SternHoffman,G.(2013,August14).Governmenttousecitizensasarmyinsocialmediawar.
TheJerusalemPost.Retrievedfromhttp://www.jpost.com/Diplomacyand
Politics/Governmenttousecitizensasarmyinsocialmediawar322972
Strange,A.,Parks,B.C.,Tierney,M.J.,Dreher,A.,&Ramachandran,V.(2013).Chinas
DevelopmentFinancetoAfrica:AMediaBasedApproachtoDataCollection(Working
PaperNo.323).Retrievedfromhttps://www.cgdev.org/publication/chinasdevelopment
financeafricamediabasedapproachdatacollection
Telesur.(2016,July17).Argentina:MacriHiresArmyofTrollstoBlastOnlineCritics.
Retrievedfromhttp://www.telesurtv.net/english/news/ArgentinaMacriHiresArmyof
TrollstoBlastOnlineCritics201607170010.html
TheChosunilbo.(2013,August13).NorthKoreasVastCyberWarfareArmy.TheChosunilbo.
Retrievedfrom
http://english.chosun.com/site/data/html_dir/2013/08/13/2013081300891.html
TheDailyMail.(2016,May20).ChinesegovernmentbackedsocialmediausersfloodWeb.
MailOnline.Retrievedfromhttp://www.dailymail.co.uk/wires/ap/article
3600250/ChinesegovernmentbackedsocialmediausersfloodWeb.html

32
TheEconomist.(2015,March12).Battleofthememes.TheEconomist.Retrievedfrom
http://www.economist.com/news/europe/21646280russiahasshownitsmastery
propagandawarukrainestrugglingcatchupbattleweb
TheGuardian.(2015,February2).Ecuadorspresidentwagessocialmediacounterattackaimed
atdefamers.TheGuardian.Retrievedfrom
https://www.theguardian.com/world/2015/feb/02/ecuadorpresidentsocialmedia
counterattack
TheKoreanHerald.(2013,December19).11cyberwarfareagentsfaceindictment.TheKorean
Herald.Retrievedfromhttp://www.koreaherald.com/view.php?ud=20131219000660
TheTelegraph.(2011,July21).Pentagonlookstosocialmediaasnewbattlefield.The
Telegraph.Retrievedfromhttp://www.telegraph.co.uk/technology/social
media/8651284/Pentagonlookstosocialmediaasnewbattlefield.html
Toor,A.(2014,July22).ChinareportedlyusesfakeTwitteraccountstospreadTibet
propaganda.TheVerge.Retrievedfrom
http://www.theverge.com/2014/7/22/5925703/chinareportedlyusesfaketwitter
accountstospreadtibetpropaganda
Vivanco,J.M.(2014,December15).CensorshipinEcuadorhasmadeittotheInternet.Human
RightsWatch.Retrievedfromhttps://www.hrw.org/news/2014/12/15/censorship
ecuadorhasmadeitinternet
VOANews.(2015,August4).VenezuelaRulingPartyGamesTwitterforPoliticalGain.VOA.
Retrievedfromhttp://www.voanews.com/a/venezuelarulingpartygamestwitterfor
politicalgain/2902007.html
Wator,J.(2017,June23).Trollewpolskiminternecie.Najczciejfaszywkiwypuszczaprawica.
Wyborcza.Retrievedfromhttp://wyborcza.pl/7,156282,22001141,trollewpolskim
internecienajczesciejfalszywkiwypuszcza.html?disableRedirects=true
Weiwei,A.(2012,October17).ChinasPaidTrolls:Meetthe50CentParty.NewStatesman.
Retrievedfrom
http://www.newstatesman.com/politics/politics/2012/10/china%E2%80%99spaidtrolls
meet50centparty
Whitaker,B.(2016,July28).HowTwitterrobotsspamcriticsofSaudiArabia.AlBab.Retrieved
fromhttp://albab.com/blog/2016/07/howtwitterrobotsspamcriticssaudiarabia
Williams,C.(2015,August4).MilitarymarketerstargetIsilwithantiradicalisationcampaign.
TheTelegraph.Retrievedfrom
http://www.telegraph.co.uk/finance/newsbysector/mediatechnologyandtelecoms/digital
media/11782880/MilitarymarketerstargetIsilwithantiradicalisationcampaign.html

33
Williams,K.B.(2016,February26).USlaunchesaggressivecyberwarcampaignagainstISIS.
TheHill.Retrievedfromhttp://thehill.com/policy/cybersecurity/270889uslaunches
aggressivecyberwarcampaignagainstisis
Williams,S.(2017,January4).RodrigoDutertesArmyofOnlineTrolls.NewRepublic.
Retrievedfromhttps://newrepublic.com/article/138952/rodrigodutertesarmyonline
trolls
Woolley,S.C.(2015,August4).#HackingTeamLeaks:EcuadorisSpendingMillionson
Malware,ProGovernmentTrolls.GlobalVoicesAdvocacy.Retrievedfrom
https://advox.globalvoices.org/2015/08/04/hackingteamleaksecuadorisspending
millionsonmalwareprogovernmenttrolls/
Woolley,S.C.(2016).Automatingpower:Socialbotinterferenceinglobalpolitics.First
Monday,21(4).Retrievedfromhttp://firstmonday.org/ojs/index.php/fm/article/view/6161
Yinanc,B.(2017,April27).AKtrollsweredetrimentaltotheYescamp.HrriyetDailyNews.
Retrievedfromhttp://www.hurriyetdailynews.com/aktrollsweredetrimentaltothe
yescamp.aspx?pageID=449&nID=112472&NewsCatID=412
York,J.C.(2011,April21).SyriasTwitterspambots.TheGuardian.Retrievedfrom
https://www.theguardian.com/commentisfree/2011/apr/21/syriatwitterspambotspro
revolution

34
Seriesacknowledgements
The authors gratefully acknowledge the support of the European Research Council,
Computational Propaganda: Investigating the Impact of Algorithms and Bots on Political
Discourse in Europe, Proposal 648311, 20152020, Philip N Howard, Principal Investigator.
AdditionalsupporthasbeenprovidedbytheFordFoundation.Projectactivitieswereapproved
bytheUniversityofOxfordsResearchEthicsCommittee(CURECOIIC1A15044).Anyopinions,
findingsandconclusionsorrecommendationsexpressedinthismaterialarethoseoftheauthors
anddonotnecessarilyreflecttheviewsofthefundersortheUniversity.

Fortheirassistanceandadviceonthisresearch,wearegratefultoGillianBolsover,RobGorwa,
MarcOwenJones,LisaMariaNeudert,FadiSalem,AkinUnver,andSamuelWoolley.

35
Authorbiographies
Samantha Bradshaw is a DPhil. candidate at the Oxford Internet Institute and works on the
ComputationalPropagandaprojectasaresearchassistant.PriortojoiningtheCOMPROPteam,
sheworkedattheCentreforInternationalGovernanceInnovationinWaterloo,Canada,where
she was a key member of a small team facilitating the Global Commission on Internet
Governance.SheholdsanMAinglobalgovernancefromtheBalsillieSchoolofInternational
Affairs, and a joint honors BA in political science and legal studies from the University of
Waterloo.Samanthatweetsfrom@sbradshaww.

PhilipN.HowardisastatutoryProfessorofInternetStudiesattheOxfordInternetInstituteand
aprofessorialfellowatBalliolCollegeattheUniversityofOxford.Hehaspublishedeightbooks
and over 120 academic articles and public essays on information technology, international
affairs and public life. Howards books include The Managed Citizen (Cambridge, 2006), the
DigitalOriginsofDictatorship andDemocracy (Oxford, 2010) and most recentlyPaxTechnica:
How the Internet of Things May Set Us Free or Lock Us Up (Yale, 2015). He blogs at
www.philhoward.organdtweetsfrom@pnhoward.

36


ThisworkislicensedunderaCreativeCommonsAttributionNonCommercial
ShareAlike4.0InternationalLicense

37