Defining Hate Speech PDF

Research Publication No. 2016-20

December 2016
Defining Hate Speech
Andrew F. Sellars
This paper can be downloaded without charge at:

https://cyber.harvard.edu/publications/2016/DefiningHateSpeech
The Berkman Klein Center for Internet & Society Research Publication Series:
The Social Science Research Network Electronic Paper Collection:
https://ssrn.com/abstract=2882244
23 Everett Street • Second Floor • Cambridge, Massachusetts 02138

+1 617.495.7547 • +1 617.495.7641 (fax) • http://cyber.law.harvard.edu/ •
[email protected]

Electronic copy available at: https://ssrn.com/abstract=2882244
December 2016
Defining
Hate Speech
Andrew F. Sellars
Electronic copy available at: https://ssrn.com/abstract=2882244

DEFINING HATE SPEECH
Andrew F. Sellars*
Suggested Citation: Sellars, Andrew F. Defining Hate Speech (December 8,

2016). Berkman Klein Center Research Publication No. 2016-20. Available
at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2882244
Harmful Speech Online Project

23 Everett Street | Second floor | Cambridge, Massachusetts 02138
+1 617.495.7547 | +1 617.495.7641 (fax)
http://cyber.harvard.edu
[email protected]
* Formerly the Corydon B. Dunham First Amendment Fellow, Berkman Klein Center for Internet & Society at Harvard
University. Currently the Director of the BU/MIT Technology and Cyberlaw Clinic at Boston University School of Law.
Thanks to Susan Benesch for many thoughtful inputs for this essay, and to my Berkman Klein colleagues Amar Ashar,
Rob Faris, Amy Johnson, J. Nathan Matias, and Niousha Roshani, for the thought-provoking discussions that helped
push and shape my thoughts here. Thanks also to Berkman Klein interns Prashanth Bhat, Daisy Joo, and Ofra Klein
for their comments and edits.
Table of Contents
4 Introduction
5 The Theoretical Context of Hate Speech

America’s Brief Turn to Hate Speech Proscription 5
Speech Theory 9
“The Marketplace of Ideas” 9
Democratic Self-Governance 11
“The Tolerant Society” 12
Other Theories and Procedural Values 13
14 Attempts to Define Hate Speech

Can We Know It When We See It? 14
Academic Attempts 15
Legal Attempts 18
Attempts by Online Platforms 20
24 Emerging Themes and Continuing Questions

Common Traits in Defining Hate Speech 24
1 - Targeting of a Group, or Individual as a Member of a Group 25
2 - Content in the Message that Expresses Hatred 25
3 - The Speech Causes a Harm 26
4 - The Speaker Intends Harm or Bad Activity 27
5 - The Speech Incites Bad Actions Beyond the Speech Itself 28
6 - The Speech is Either Public or Directed at a Member of the Group 29
7- The Context Makes Violent Response Possible 29
8 - The Speech Has No Redeeming Purpose 30
Continuing Questions 31
32 Conclusion
Andrew F. Sellars
Introduction with theories of free expression, surprisingly lit-

tle work appears to have been done to define
Few pairs of words evoke such a diverse range the term “hate speech” itself. Without a clear
of feelings, perspectives, and reactions as “hate definition, how will scholars, analysts, and reg-
speech.” Calls are made for it to be embraced, 1
ulators know what speech should be targeted?
tolerated, ridiculed, targeted for counter-
2 3
In other areas of social science there has been
speech, blocked on websites, actionable in a
4 5
great work done to unpack the complexity of
civil lawsuit, made criminally illegal, or the ba-
6 7
content analysis, and the inherent context and
sis of war crimes prosecution, with no shortage
8
biases that must be addressed in such analy-
of shading in between. If one can find a single sis. But when talking of hate speech, a shock-
10
point of agreement, it is perhaps that the top- ing degree of the discussion — be it academic 11
ic is ripe for rigorous study. And with a massive or in public discourse — looks solely to finding
12
amount of today’s speech happening through specific words or phrases that the observer be-
the Internet, the trend is toward studying hate lieves signal the presence of hate speech. Is that
speech online. 9
a sound strategy? 13
But just what is hate speech, and how will we This essay attempts to draw out a framework
know it when we see it online? For the great by which the concept of “hate speech” can be
depth of discussion about the harms of it, how identified and understood from a mixture of
it is spread, the appropriate public and private sources. It also, I hope, begins a comparison of
responses to it, and how it should be reconciled two different theoretical fields — free speech
theory and critical race theory — both of which
seem to address hate speech, but without much
1 Jonathan Rauch, The Case for Hate Speech, The
Atlantic (Nov. 2013), http://www.theatlantic.
discourse between each. Ultimately, I aim to
com/magazine/archive/2013/11/the-case-for-hate- identify several traits of hate speech that, when
speech/309524/. combined and applied to a text, can be used
2 Lee C. Bollinger, The Tolerant Society (1986). to isolate “hate speech” worthy of that highly
3 Julien Mailland, The Blues Brothers and the American loaded definition.
Constitutional Protection for Hate Speech: Teaching
the Meaning of the First Amendment to Foreign Audi- In writing this I do not aim to summarize the
ences, 21 Mich. St. Int’l L. Rev. 443, 459–60 (2013). body of literature on hate speech. I do not seek
4 Iginio Gagliardone et al., UNESCO, Countering to develop a theory for what hate speech does
Online Hate Speech (2015), available at http:// to a society; answer whether and how it should
unesdoc.unesco.org/images/0023/002332/233231e. be responded to through social, political, legal,
pdf. or technological channels; or summarize the
5 Anita Bernstein, Abuse and Harassment Diminish role of digital communication technologies in
Free Speech, 35 Pace L. Rev. 1, 25–27 (2014).
6 Richard Delgado, Words That Wound: A Tort Action 10 See, e.g., Klaus Krippendorff, Content Analysis:
for Racial Insults, Epithets, and Name-Calling, 17 An Introduction to its Methodology 22–25
Harv. C.R.–C.L. L. Rev. 133 (1982). (2004).
7 Mari J. Matsuda, Public Response to Racist Speech: 11 See Claire Hardaker, Misogyny, Machines, and
Considering the Victim’s Story, 87 Mich. L. Rev. 2320 the Media, or: How Science Should Not Be Re-
(1989). ported (May 27, 2016), http://wp.lancs.ac.uk/
8 Gregory S. Gordon, Formulating a New Atrocity drclaireh/2016/05/27/misogyny-machines-and-the-
Speech Offense: Incitement to Commit War Crimes, media-or-how-science-should-not-be-reported/ (re-
43 Loyal U. Chi. L.J. 281 (2012). viewing bad syntax analysis on online hate speech
9 See, e.g., Danielle Keats Citron, Hate Crimes in by a professional research organization).
Cyberspace (2014); Arthur Gaus, Trolling Attacks 12 See Lori Tharps, “Reprint Reporting” and Race,
and the Need for New Approaches to Privacy Torts, Colum. Journalism Rev. (Sept. 25, 2013), http://
47 U.S.F. L. Rev. 353 (2012); but see Lynn Edelman & www.cjr.org/minority_reports/reprint_reporting_
Jon Deitrich, Extremist Speech and the Internet: The and_race.php (criticizing the trend toward searching
Continuing Importance of Brandenburg, 4 Harv. L. & for and publishing Twitter posts with slurs as the
Pol’y Rev. 361, 362–63 (2010) (demonstrating some heart of a news story).
skepticism towards the exceptionalism of the Inter- 13 Perhaps unsurprisingly, my view is no. See notes
net). 115–28 infra and accompanying text.
4
Andrew F. Sellars
changing hate speech. I hope only to help clar- Quite famously, the United States does not
ify what it is that scholars seek to study, and make hate speech per se illegal under any defi-
explain why the definitions used in such study nition, while many other nations do. These du- 17
must be approached with nuance. eling perspectives are often shorthanded to the
American approach versus the European ap-
The essay begins with a review of regulation of proach, though this is far too simplistic a gen-
hate speech in the United States, and how such eralization. There was a brief moment, however,
18
speech was defined in the brief period when when the United States Supreme Court did up-
such speech was considered punishable consis- hold a facial hate speech proscription against a
tent with the First Amendment. As will be seen, First Amendment challenge. The story of Ameri-
19
such regulation presented serious tensions with ca’s turn toward, and then away from, law-driv-
free speech principles. Those theories of free en hate speech regulation serves as an effec-
speech are next introduced to draw out why tive illustration of the tensions at play between
such punishment was so fraught with difficulty. speech and the harm it causes. 20
Part II of the essay looks to existing attempts to

define hate speech in academia, law, and the As James Whitman notes in his comparative re-
regulation of online platforms. Part III explains view of dignity-related laws in France, Germa-
why any solution or methodology that purports ny, and the United States, differing approaches
to present an easy answer to what hate speech to the common goal of egalitarianism have led
is and how it can be dealt with is simply not a to the absence of a protectable interest in one’s
product of careful thinking. But that said, there honor in the United States. But as he notes, the
21
are a series of criteria that may be used to help related concept of one’s reputation is protect-
guide a researcher to identify a discrete corpus ed in America through defamation law. Indeed, 22
of hate speech for further study.

17 See, e.g., James Q. Whitman, Enforcing Civility
and Respect: Three Societies, 109 Yale L. Rev. 1279,
The Theoretical Context of 1281 (2000).
18 C. Edwin Baker, Hate Speech, in The Content and
Hate Speech Context of Hate Speech, supra note 16, at 57, 59.
Milkos Haraszti refers to this in a more nuanced way
as the difference between “two very different min-
America’s Brief Turn to Hate Speech imums,” a “minimum regulation” approach versus
a “minimum of mutual respect” approach. Milkos
Proscription Haraszti, Foreward: Hate Speech and the Coming
There cannot be serious doubt that expressed Death of the International Standard before It Was
hatred based on a person’s immutable charac- Born (Complaints of a Watchdog) in The Content
teristics, ethnic background, or religious identity and Context of Hate Speech, supra note 16, at xiii.
causes a harm. But every principle of freedom
14
19 Or, more formally, as a Fourteenth Amendment chal-
of expression recognizes times when govern- lenge as a violation of his substantive due process
ments should protect speech in spite of the harm rights to free speech. See Beauharnais v. Illinois, 343
that it causes. There is also always an upper
15 U.S. 250, 251–52 (1952).
limit the protection of speech: every implement- 20 I seek only to illustrate the tensions here; I leave to
ed system of freedom of expression finds some others whether this turn was normatively correct.
point where speech has such a strong nexus See, e.g., Waldron, supra note 14, at 34–64.
to harm that it can be prohibited. Those three
16 21 Whitman draws a distinction between the French
and German approach of “leveling up” the dignity
simple boundaries aside, all opinions as to the
of all citizens to that of the classic aristocracy to the
proper balance fracture. American approach of “leveling down” all its citizens,
so that they “all stand together on the lowest rung of
the social ladder.” Whitman, supra note 17, at 1282.
14 See, e.g., Jeremy Waldron, The Harm in Hate 22 I am using the concepts of honor and reputation
Speech (2012); Delgado, supra note 6, at 136–49. somewhat interchangeably here. It is worth noting,
15 See infra notes 59–100 and accompanying text. as Whitman does, that the United States blends the
16 Michael Herz & Peter Molnar, Introduction, in The concepts of “honor” and “reputation” in a way that
Content and Context of Hate Speech at 1, 4 (Mi- European governments do not. Whitman, supra note
chael Herz & Peter Molnar eds., 2012). 17, at 1292.
5
Andrew F. Sellars
American judges speak of the rights in defama- evidence that when an African-American family
tion law with the same rhetoric often used to call “moves into a district or block, real estate val-
for hate speech proscription, defending “[t]he ues immediately go down,” and that “in white
right of a man to the protection of his own repu- districts in Chicago the number of offenses re-
tation” as essential to “our basic concept of the ported were much smaller in number than those
essential dignity and worth of every human be- reported” in districts with black residents. 30
ing — a concept at the root of any decent sys-

tem of ordered liberty.” It is therefore not sur-
23
Of course, his proffered evidence is illogical.
prising to see many attempts to proscribe hate Beauharnais’s real estate factoid, even if true
speech in the United Sates under the framing of and causally connected, would only show that
defamation law, through a “group libel” claim. 24
he was not alone in his racism. As for his crime
statistics, a long body of criminology schol-
The historical high point for this argument was arship demonstrates why higher crime rates
the 1952 United States Supreme Court decision among African Americans are better attributed
Beauharnais v. Illinois, a case concerning an
25
to the racial biases found throughout the crim-
Illinois statute that prohibited the publication inal justice system, which repeat themselves in
of any piece of media that “portrays deprav- reports of crime. A jury hearing such evidence
31
ity, criminality, unchastity, or lack of virtue of may have found his conclusion to be false. But 32
a class of citizens, of any race, color, creed or the courts did not even allow such evidence to
religion,” if the publication exposed that class reach a jury. Instead, the Illinois Supreme Court
of citizens to “contempt, derision, or obloquy held that any evidence of truth was irrelevant,
which is productive of breach of the peace or because Beauharnais lacked any “good mo-
riots.” The government of Illinois portrayed this
26
tives” or “justifiable ends.” The United States
33
as a “form of criminal libel” law. 27

Supreme Court allowed that judgment to stand. 34
Joseph Beauharnais was charged under this It is easy to agree with the courts in Beauhar-
statute after he published a racist screed that nais at a normative level, but as a methodology
advocated against black families moving into for determining whether speech is actionable as
predominantly white neighborhoods in Chica- hate speech it is uncomfortable and confusing.
go, specifically because of the social ills that he Under the approach in Beauharnais, it is a judge
alleged would follow. Following the adjudica-
28
alone who decides which motives are “good,”
tory framework of libel law, Beauharnais chal-
lenged his prosecution on free speech grounds has done away with the “truth plus motives” defense
— specifically, that his statements were true, in favor of more speech-protective defenses, but it
and, as the Illinois Constitution provides, true still creeps into defamation cases from time to time.
statements are a defense to libel so long as they See Thomas Edward Powell II, The Truth Will Not Set
You Free in Nebraska: Actual Malice and Nebraska’s
are published “with good motives and for justi-
“Truth Plus Motives” Defense, 72 Neb. L. Rev. 1236
fiable ends.” Beauharnais sought to introduce
29
(1993).
30 Beauharnais, 97 N.E.2d at 346.
23 Rosenblatt v. Baer, 383 U.S. 75, 92 (1966) (Stewart, J., 31 For a modern review of this scholarship, see Michael
concurring). See Robert C. Post, The Social Founda- Rocque, Racial Disparities in the Criminal Justice
tions of Defamation Law: Reputation and the Consti- System, 2011 Race & Justice 292 (2011).
tution, 74 Cal L. Rev. 691, 707 (1986).
32 At common law, a defamation defendant would be
24 See, e.g., Whitman, supra note 17, at 1292–93; strictly liable for publishing a falsehood, except in
Charles Lawrence, If He Hollers Let Him Go: Regulat- a few particular areas. This, too, changed with the
ing Racist Speech on Campus, 1990 Duke L.J. 431, overlay of constitutional principles of defamation
463–64 (1990). law beginning in the 1960s. Hon. Robert D. Sack,
25 343 U.S. 250 (1952). Sack on Defamation § 5.1 (4th ed. 2010). Beau-
26 Id. at 251. harnais today could argue that this is a protected
27 People v. Beauharnais, 97 N.E.2d 343, 346 (Ill. 1951). opinion based on disclosed facts. See, e.g., Standing
28 See id. Comm. on Discipline v. Yagman, 55 F.3d 1430 (9th
29 See Ill. Const. art. II § 4 (1870), as amended, Ill. Cir. 1995).
Const. art. I § 4 (1970). The modern limitation on def- 33 Beauharnais, 97 N.E.2d at 347.
amation doctrine imposed by the First Amendment 34 Beauharnais, 343 U.S. at 266.
6
Andrew F. Sellars
and once a judge makes such a determination, opted to frame their hate speech law as one
all consideration of truth or merit is beside the akin to defamation. But because it was a defa-
point. This approach does avoid putting racial
35
mation case, its future application was subject
questions before a jury, a famously perilous to the United States Supreme Court’s series of
path for justice, but what of the racist judge?
36
decisions limiting that doctrine under the First
What guides, limits, or informs the judge’s deter- Amendment. The Beauharnais decision is now
41
mination? Could there ever be speech that de- generally considered dead letter. Under mod-
42
means a group but nevertheless has “good mo- ern First Amendment scholarship, legal sanction
tives”? Would Beauharnais have been allowed of hate speech is limited to a few specific con-
to offer evidence if he had published the same texts, such as speech that directly incites crimi-
notion in a dispassionate academic setting? 37
nal activity or specific threats of violence.
43 44
What if he had targeted an individual instead of

a group? How is this different than adopting an
38
But this was not an easy truce. America reopened
“official truth,” which scholars have long con- the debate on hate speech following the “Skok-
sidered an anathema to the American approach ie controversy” of 1977, where a neo-Nazi group
to speech? This whole line of questioning feels
39
sought permission to hold a march through
peculiar, because the true harm in Beauhar- Skokie, Illinois, a predominantly Jewish suburb
nais’s words was not his purported facts at all, of Chicago. While subject to robust discussion
45
but his opinion and its expression. Opinions are and debate at the time, the enduring memory
46
not so easily subjected to the true/false analy- of the case is the symbolic stance that America
sis of libel law. 40
took towards this public expression of hatred 47
— allowing the (very small) group of individuals

Subsequent cases could have worked to resolve spouting hateful rhetoric to speak, while also
these tensions, had the Illinois legislature not allowing the speakers to be surrounded by (of-
ten much larger) groups of individuals heckling,
35 Contemporaneous scholarship suggests that the gawking at, or shouting down the hateful speak-
“good motives” provision in many state defamation
ers, with law enforcement perilously guarding
laws was both confusing, and presented interesting
procedural challenges as to whether and in what
order litigants had to offer evidence of motive and 41 This included the “good motives” limitation to the de-
truth. See generally B. Sidler, The Requirement of fense of truth in defamation actions. See Garrison v.
“Good Motives and Justifiable Ends,” 43 Chi.-Kent L. Louisiana, 379 U.S. 64, 70–73 (1964); New York Times
Rev. 92 (1966). Co. v. Sullivan, 376 U.S. 254 (1964).
36 See generally Brian J. Seer & Mark Many, Racism, 42 See Nuxoll v. Indian Prairie Sch. Dist., 523 F.3d 668,
Preemptory Challenges, and the Democratic Jury: 672 (7th Cir. 2008).
The Jurisprudence of a Delicate Balance, 79 J.L. & 43 Brandenburg v. Ohio, 395 U.S. 444 (1969); see also
Criminology 1 (1988). Edelman & Deitrich, supra note 9, at 370.
37 Matsuda, supra note 7, at 2364–65 (questioning how 44 Watts v. United States, 394 U.S. 705 (1969); see also
hate speech law should impact the “dead-wrong Elonis v. United States, 135 S. Ct. 2001 (interpreting
social scientist”). the scope of the federal threats statute).
38 See Beauharnais, 343 U.S. at 300–01 (Jackson, J., 45 The situation played out over a series of cases in
dissenting) (exploring this question). both state and federal court. See Collin v. Smith, 447
39 See Am. Communication Ass’n v. Douds, 339 U.S. F. Supp. 676 (N.D. Ill.), aff’d 578 F.2d 1197 (7th Cir.)
382, 442–43 (1950) (Jackson, J., dissenting in part) cert. denied 439 U.S. 916 (1978); Village of Skokie v.
(“The danger that citizens will think wrongly is seri- National Socialist Party, 366 N.E.2d 347 (Ill. App.
ous, but less dangerous than atrophy from not think- 1977), modified 373 N.E.2d 21 (Ill. 1978).
ing at all.… The priceless heritage of our society is 46 Bollinger, supra note 2, at 14–15 (“Few legal dis-
the unrestricted constitutional right of each member putes in the last decades caught the public eye with
to think as he will. Thought control is a copyright of such dramatic power as did that case. For well over
totalitarianism, and we have no claim to it.”). a year, as the case moved ponderously through the
40 See Robert Post, Racist Speech, Democracy, and the courts, it was seldom out of the news and often on
First Amendment, 32 William & Mary L. Rev. 267, 298 the front pages of newspapers when it was in the
(1991) (hereinafter Post, Racist Speech); see Milkov- news.”).
ich v. Lorain Journal Co., 497 U.S. 1 (1990) (discussing 47 “Symbolic” because despite earning the right to
differences between statements of facts and opin- march, the neo-Nazis never actually marched in
ions in defamation law). Skokie. Bollinger, supra note 2, at 27.
7
Andrew F. Sellars
the border between the two factions. 48

this approach is equally subjective and elusive.
It is the general public that decides, collectively
The trend starting at Beauharnais, running but subjectively, what is or is not hate speech.
through Skokie, and leading to today can be The same problems of bias and blindness arise,
viewed as an increasing division between legal but in a more social and structural manner.
and normative responses to hate speech. The 49
Court in Beauharnais saw the two regulatory The subjectivity of hate speech also emerges
forces in complete alignment — a defendant in the circumstances where American law does
50
in a libel case could only defend himself if he allow for its punishment. In fact, if prosecutors
could show a presence of “good motives,” and or law enforcement subjectively desire to pun-
the goodness of the motives were determined ish hate speech they often have the tools to do
by the presiding judicial figure. At the same so under the laws as they exist today. A critical
time, no clear definition of “hate speech” can review of cases will find examples where courts
emerge from this formulation, given its high lev- use the tremendous breadth of criminal and tort
el of subjectivity. The Skokie case presents a law to find a means for sanctioning the person,
near-complete division between law and norms; even when their speech is constitutionally pro-
even those who defended the legal right of the tected. And if the speaker committed a crime,
53
neo-Nazis to march condemned the ideas of Na- the prosecutor can in some cases apply hate
zism in the strongest normative terms: “heaping crimes escalations, and the First Amendment
scorn and ridicule on the group to an extent lim- doctrine tolerates such escalation. If the speech 54
ited only by shortcomings of imagination and el- happens in a workplace, an educational institu-
oquence.” Whether this is the better approach tion, or the speaker is a government employee,
51
for hate mitigation or not, the definition under discrimination laws provide an avenue for pun-
52
ishment of speech directly. Because these laws

55
48 The Skokie case itself was vividly depicted in this are not designed to target hate speech specif-
way in the 1980 comedy The Blues Brothers, where ically, their drafters have not labored to cast
the brothers confront the scene in their iconic car, clear definitions as to what speech qualifies,
leading Jake Blues to grumble “I hate Illinois Nazis”
but because they are so broad, a figure of au-
and drive at the group, forcing them to jump into
a river for safety, to the cheers of onlookers. Julien
thority can bring them to bear against speech
Mailland argues that this scene should be used by they subjectively find hateful.
jurists seeking to teach the differences between
approaches to hate speech in France and the United ing Racist Speech on Campus: A Modest Proposal,
States. Julien Mailland, The Blues Brothers and the 1990 Duke L.J. 484, 495–507 (1990). Critics often
American Constitutional Protection for Hate Speech: rebut by noting that the historical record suggests
Teaching the Meaning of the First Amendment to that allowing speech to prevent violent action is
Foreign Audiences, 21 Mich. St. Int’l L. Rev. 443, 451 problematic; escalating racist speech often accom-
(2013). panies escalating racist violence, especially in the
49 See, e.g., Bollinger, supra note 2, at 12. all-too-often case when the population does not call
50 This “forces” analysis is drawn from Lawrence Lessig. out the speech. See Matsuda, supra note 7, at 2352
Lawrence Lessig, Code Version 2.0 120–38 (2006). n.166.
51 Vincent Blasi, The Teaching Function of the First 53 See, e.g., Fisher v. Carrousel Motor Hotel, Inc., 424
Amendment, 87 Columbia L. Rev. 387, 389 (1987) S.W.2d 627, 630 (Tex. 1967) (a restaurant patron was
(hereinafter Blasi, Teaching Function). Indeed, look- allowed to recover damages because, in the process
ing purely at normative responses, the United States of having racial slurs hurled at him, a waitress forci-
is often more hostile to many forms of hate speech bly pulled a plate from his hands, thus committing a
than nations that make such speech illegal. Baker, physical battery).
supra note 18, at 60. 54 Wisconsin v. Mitchell, 508 U.S. 47 (1993).
52 Proponents of a normative approach to hate speech 55 See, e.g., Imperial Diner, Inc. v. State Human Rights
regulation note that open and public confrontation Appeal Bd., 417 N.E.2d 231 (N.Y. 1980); Harris v. Har-
can be more effective than a punishment scheme vey, 605 F.2d 330 (7th Cir. 1976). The constitutional-
that only triggers when the government can see the ity of these types of discrimination actions has not
speech in question. Baker, supra note 18, at 71, 73–75. been fully explored. See Eugene Volokh, Freedom of
A focus on speech may even distract law enforce- Speech vs. Workplace Harassment Law – A Growing
ment away from other, perhaps more harmful, forms Conflict, http://www2.law.ucla.edu/volokh/harass/
of hate-motivated activity. Nadine Strossen, Regulat- (last visited Nov. 27, 2016).
8
Andrew F. Sellars
This is not cause for a feeling of relief, for the ciple of freedom of expression that he identifies
converse is also true. When hatred is not seen as distinct from a “minimal principle of liberty,”
by those in a position of power they will often that is, the basic premise in a liberal democra-
refuse to use the legal tools at their disposal. To cy that citizens should be free to do what they
take one example, a staggering volume of harm- wish, and that the government should only be al-
ful speech online — particularly sexist and rac- lowed to act when the actions of a person harm
ist speech — occurs in a form that would almost the rights of another. If that alone provided the
60
certainly constitute actionable threats under basis for action, governments would have all the
federal law. But federal law enforcement ap-
56
reason they need to regulate hate speech, how-
pears devotionally uninterested in pursuing such ever it is defined. A theory of freedom of expres-
61
crimes, and local law enforcement usually lacks sion helps to explain why many governments,
the resources and knowledge to meaningfully online platforms, and scholars are reluctant to
investigate such activity. Mari Matsuda notes
57
take that step, in spite of the harms it causes.
numerous studies where overt and undoubted
incidents of hatred against minority groups are Greenawalt notes that there are several differ-
treated by government and law enforcement as ent theories proffered to explain the interest in
not important, mere “pranks,” or otherwise wor- freedom of expression, and while there have
thy of casual dismissal. 58
been some attempts to unify the theories into a
single, coherent whole, the prevalent thought
62
Much of the debate around hate speech seems is that there are a series of different, valid justi-
hopelessly tangled in this subjective analysis, fications, with the array being greater than the
but in order to come to more observable defi- sum of its parts. Greenawalt divides the field
63
nitions, it is worth exploring the countervalue into two general halves, “consequentialist” prin-
that is consciously or subconsciously informing ciples, which defend speech because of pro-
these subjective definitions: freedom of expres- ductive results of such protection, and “noncon-
sion. A brief review of the underlying theories of sequentialist” principles, which defend speech
freedom of expression, and how they inform the due to its inherent value. This section will review
64
procedural and structural approach American some of the dominant consequentialist themes
courts take to speech issues, can better frame that inform discussions of hate speech.
the definitions that follow.
“The Marketplace of Ideas”
Speech Theory The dominant proffered justification — and the
Kent Greenawalt’s Free Speech Justifications one most frequently expressed in the decisions
provides a leading exposition on the various of the United States Supreme Court — is the
65
theories of freedom of expression, their merits, “marketplace of ideas” theory, which posits that
and their detractors. Greenawalt draws a prin-
59
society will be better able to progress if govern-
ment is kept out of the business of adjudicating
56 See 18 U.S.C. § 875(c); Last Week Tonight with John
Oliver: Online Harassment (HBO television broadcast
what is true versus false, valid versus invalid, or
June 21, 2015) (detailing specific threats women acceptable versus abhorrent. Credit to this idea
have received online that clearly fit the legal defini- is usually given to John Stuart Mill, though its
tion of true threats).
57 Citron, supra note 9; see also Ann Merlan, The 60 Id. at 120–23.
Cops Don’t Care About Violent Online Threats. What 61 Frederick Schauer, Free Speech: A Philosophical
Do We Do Now?, Jezebel (Jan. 29, 2015), http:// Enquiry 10 (1982).
jezebel.com/the-cops-dont-care-about-violent-online- 62 See, e.g. Martin H. Redish, Freedom of Expres-
threats-what-d-1682577343. sion: A Critical Analysis (1984).
58 Matsuda, supra note 7, at 2327. 63 Greenawalt, Free Speech Justifications, supra note
59 Kent Greenawalt, Free Speech Justifications, 89 59, at 126–27.
Columbia L. Rev. 119 (1989) (hereinafter Greenawalt, 64 Id. at 127–30.
Free Speech Justifications). Greenawalt, it should be 65 See Ronald J. Krotoszynski, Jr., The First Amend-
noted, has also written about racial insults and epi- ment In Cross-Cultural Perspective 21–24 (2006)
thets, specifically. See Kent Greenawalt, Insults and (noting influences of both the marketplace theory
Epithets: Are They Protected Speech?, 42 Rutgers L. and the self-governance theory, though the former
Rev. 287 (1990). seems to dominate the outcomes of the Court).
9
Andrew F. Sellars
roots go back to John Milton’s Areopagitica in the victims of genocide. When countries con-
69
1644. Frederick Siebert said this of Milton’s the-

66
struct “official truths,” those truths can change
ory: as you move from country to country, present-
ing a perilous situation for those who wish to
Milton was confident that Truth was defi- speak on global platforms.
nite and demonstrable and that it had
unique powers of survival when permitted The marketplace theory has no shortage of crit-
to assert itself in a “free and open encoun- icism. Greenawalt identifies several critiques,
ter.” […] Let all with something to say be ranging from the deeply philosophical to the
free to express themselves. The true and more pragmatic. Chief among the latter are
70
sound will survive; the false and unsound concerns about the state and makeup of the
will be vanquished. Government should be “marketplace” today. This can be especially
71
kept out of the battle and not weigh the problematic in issues concerning the rights and
odds in favor of one side or the other. And dignities of marginalized groups, given the dom-
even though the false may gain a tempo- inance of majoritarian groups in mass media. 72
rary victory, that which is true, by draw- And while the Internet can help diversify the
ing to its defense additional forces, will possible spectrum of voices as a structural mat-
through the self-righting process ultimate- ter, there is still great concern that even with
73
ly survive. 67
those affordances power is still concentrated in
majoritarian groups. 74
Under this framework Joseph Beauharnais

would have been at liberty to make his argument More systemically, the emergence of social
about racially segregated neighborhoods in science concepts like bounded rationality and
Chicago, dubious evidence and all, and others cognitive bias raise more fundamental prob-
would have been allowed to respond to him by lems for the theory, and whether we can trust
critiquing his evidence and providing evidence individuals to act reasonably or rationally when
of their own. And in the end the correct view engaging in debates about what is true, valid,
would overtake Beauharnais, and maybe even or acceptable. The marketplace theory pres-
75
change his mind personally. These principles ents a framework where the sides engaged in
saved the Pittsburgh Courier from a libel lawsuit intellectual agon are able and willing to receive
from Beauharnais himself, after the newspaper arguments to the contrary and reconsider their
called him “a sinister character in Chicago who position, or that society can see the spectrum
is more dangerous than the nation’s worst gang- of debate and consider their position openly,
ster,” and one who “conducts a vicious and risky instead of finding the evidence and truth that
business — the promotion of racial hatred, with already supports their preexisting feelings on a
biased whites as his steady clients.” 68
Marketplace advocates in the hate speech de- 69 Haraszti, supra note 18, at xiii, xvii.
bate also point to counterexamples of idea 70 Greenawalt, Free Speech Justifications, supra note
regulation, such as the disparate treatment of 59, at 132–41.
speech concerning the massacre of Armenians 71 See, e.g., C. Edwin Baker, Media, Markets, and
in 1915. In Turkey, writers have been prosecuted Democracy (2002).
under hate speech laws for calling the action 72 Owen Fiss, The Irony of Free Speech 50–78 (1996).
genocide, whereas in Switzerland a politician 73 See generally Yochai Benkler, The Wealth of
was prosecuted for denying the Armenians were Networks: How Social Production Transforms
Markets and Freedom (2006).
66 See John Milton, Areopagitica (1644), available at 74 See, e.g., Albert-László Barabási & Réka Albert, Emer-
https://www.dartmouth.edu/~milton/reading_room/ gence of Scaling in Random Networks, 286 Science
areopagitica/text.shtml. 509 (1999). For a review of the scholarly back-and-
67 Frederick Siebert, The Libertarian Theory of the forth on this question, see Yochai Benkler et al., So-
Press, in Four Theories of the Press 39, 44–45 cial Mobilization and the Networked Public Sphere,
(Frederick Siebert et al. eds. 1956). 32 Pol. Comm. 594 (2015).
68 Beauharnais v. Pittsburgh Courier Publ’g Co., 243 75 Greenawalt, Free Speech Justifications, supra note
F.2d 705, 706 (7th Cir. 1957). 59, at 134–36.
10
Andrew F. Sellars
matter. Both social science and informal per-

76
Second in popularity among the consequential-
sonal experience call into question that prem- ist justifications of free speech is the “self-gov-
ise. Deeper examination and empirical study
77
ernance theory.” This viewpoint is associated
of the marketplace theory is sorely needed, and with Alexander Meiklejohn, who drew a theory of
thanks to the more observable nature of Inter- free speech by looking at both the First Amend-
net-mediated communication, is now under- ment and the Speech and Debate Clause of the
way. 78
Constitution to suggest that the true goal of the
constitutional protection for speech is to ensure
And even accepting the validity of this theory, that the public can freely and fairly debate all
one may find justification for restricting hate- political issues. He drew an ideological goal for
81
ful speech on the grounds that it excludes, co- public discourse as the town hall governance
erces, or frightens some would-be participators style in New England, where, as he put it, “what
because of their affiliation with the targeted is essential is not that everyone shall speak, but
group, thus denying them access to the mar- that everything worth saying shall be said.” To 82
ketplace (as audience or speakers or both). This borrow once again from the Beauharnais case,
line of argumentation is usually ascribed first under this theory Joseph Beauharnais should
to Catharine MacKinnon, who observed the si- be allowed to argue for the exclusion of African
lencing effect that pornography can have on Americans from Chicago neighborhoods as a
women, but has since been applied to the hate matter of policy, others could challenge his con-
speech context particularly. Those in favor of
79
clusions or epistemology, and the body politic
an affirmative role of the government in setting would then be free to accept or reject his argu-
that marketplace under the marketplace theo- ment. 83
ry can justify some regulation of hate speech

80
when its effect excludes speakers from the mar- Meiklejohn’s theory is relatively more tolerant of
ketplace, but any attempt to prohibit topics, state regulation of the press, though the regula-
positions, or values on their merit is typically a tion contemplated is more like the imposition of
nonstarter. Robert’s Rules of Order than any promotion or
suppression of any given ideology. Therefore,84
Democratic Self-Governance one can find reasons both to tolerate or to reg-

ulate hate speech under this framework, or tol-
76 For a review of some of the scholarship on these erate some forms of hateful speech, while block-
points, see Yochai Benkler et al., supra note 74. ing others because, as Meiklejohn puts it, they
77 See supra notes 71–74. This is not to mention that are not “worth saying.” On the one hand, the
85
little of the harmful speech observed online feels as concern of the silencing effect raised above has
though it has little-to-no relationship to the honest even stronger salience when considering who
offer of proof in a factual debate. Citron, supra has access to the political process, and who can
note 9, at 197–99. elect and persuade legislators to pass laws en-
78 See, e.g., Jennifer Brundidge, Encountering “Dif-
ference” in the Contemporary Public Sphere: The
Contribution of the Internet to the Heterogeneity of 81 Alexander Meiklejohn, Political Freedom: The
Political Discussion Networks, 60 J. of Comm. 680 Constitutional Powers of the People at 16–17
(2010). (1960).
79 See, e.g., Catharine MacKinnon, Difference and 82 Meiklejohn, supra note 81, at 26; see also John Doe
Dominance: On Sex Discrimination, in Feminism Un- No. 1 v. Reed, 561 U.S. 186, 223 (2010) (Scalia, J., con-
modified (1987); see also Post, Racist Speech, supra curring in judgment) (noting the influence of the New
note 40 at 275 (noting the harm of racist speech to England town meeting in open governance law).
the marketplace of ideas); Fiss, supra note 72, at 16 83 Post, Racist Speech, supra note 40.
(“In this context, the classic remedy of more speech 84 For what it’s worth, Robert’s Rules of Order would
rings hollow. Those who are supposed to respond likely sanction many forms of hateful speech. The
cannot.”). For a recent article applying this logic to rules stress that debate decorum not resort to per-
the Internet, see Anita Bernstein, Abuse and Harass- sonal attack. “The measure, not the member, is the
ment Diminish Free Speech, 35 Pace L. Rev. 1 (2014). subject of debate.” Robert’s Rules of Order 392
80 See, e.g., Jerome A. Barron, Access to the Press: A (11th ed. 2011).
New First Amendment Right, 80 Harv. L. Rev. 1641 85 My thanks to Susan Benesch for this framing obser-
(1967). vation. See Meiklejohn, supra note 81, at 26.
11
Andrew F. Sellars
suring greater tolerance. One may justify hate

86
can cause. Under this framework, Beauharnais
93
speech regulation on the ground that it ensures would be legally allowed to publish his screed,
true inclusion in self-governance. On the other but only so that we all to confront the fact that
hand, protecting speech under this theory helps people still feel this way in our society, and we94
to ensure that counter-majoritarian positions would be free (indeed, encouraged) to call out
will be heard, and that citizens feel an “obliga- this racism for what it is, and commit ourselves
tion to test our thinking” about what is assumed to the ongoing struggle for equality and toler-
as true. Indeed, other scholars have shown how
87
ance.
an ill-defined law against hate speech can ac-
tually be used against marginalized groups by Responses to Bollinger have been mixed. Vincent
majoritarian powers, to further exclude them Blasi noted that Bollinger’s theory in some ways
from the political process. 88
“is based on an expansive conception of the role
of government in shaping the attitudes of the
“The Tolerant Society” citizenry,” placing it in tension with other major
A less dominant justification, but one with spe- speech principles. Matsuda focuses on harm
95
cial salience in the context of harmful speech, is experienced in the process of Bollinger’s ap-
the “tolerance theory” most famously espoused proach, noting that “[t]olerance of hate speech
by Lee Bollinger. Bollinger’s The Tolerant Soci-
89 is not a tolerance borne by the community at
ety takes on several key questions related to ex- large. Rather, it is a psychic tax imposed on
tremist and hateful speech, including the ques- those least able to pay.” This concern is aug-
96
tion raised above as to why we are so reluctant mented considering law enforcement’s present
to punish it under law, while we see no problem reluctance to enforce existing laws that address
with normatively shunning speech, often quite threats and discrimination. Moran notes that
97
aggressively. Bollinger notes that the advan-

90 the paradigm case Bollinger adopts for his work,
tage of this “nonlinear structure” can allow us the Skokie case, may not resemble more recent
to better identify and respond to the underlying examples of hate speech, as in that case nearly
hatred that is expressed in hate speech. “Free all who were involved disagreed with the view-
speech provides a discrete and limited context points of the Nazi speakers, and in more recent
in which a general problem manifests itself and cases (such as those concerning racist speech
in which that problem can usefully be singled against African Americans) the intolerance can
out for attention.” And against a natural incli-
91 often be imputed to society more generally. 98
nation of intolerance of other ideas or people,

tolerated expression of other ideas “demon- Bollinger has responded to some of these ob-
strates powerfully, more powerfully than a gen- jections in later scholarship. Beyond his own
99
eral injunction to be appropriately tolerant in responses, there may be some small evidence
all circumstances ever would, to ourselves and of Bollinger’s theory in action. It is worth noting
others, a commitment to exercise moderation that the United States, despite famously avoid-
throughout social intercourse.” 92
93 Id. at 129.
Allowing for extremist speech forces society to 94 Which, sixty years later, is still true. See Daniel
confront, and thus not forget, that intolerance Denvir, It’s Mostly White People Who Prefer to Live
exists, it can rest within all of us, and we will al- in Segregated Neighborhoods, CityLab (June 25,
ways have work to do in order to address the 2015), http://www.citylab.com/housing/2015/06/its-
spectrum of harms that societal intolerance mostly-white-people-who-prefer-to-live-in-segregat-
ed-neighborhoods/396887/.
95 Blasi, Teaching Function, supra note 51, at 413.
96 Matsuda, supra note 7, at 2323.
86 See supra note 79 and accompanying text.
97 See supra notes 57; Matsuda, supra note 7, at 2338.
87 Meiklejohn, supra note 81, at 73.
98 Mayo Moran, Talking About Hate Speech: A Rhetori-
88 See Herz & Molnar, supra note 16, at 3. cal Analysis of American and Canadian Approaches
89 See Bollinger, supra note 2. to the Regulation of Hate Speech, 1994 Wisc. L. Rev.
90 Id. at 35–36, 109. 1425, 1452 n.113 (1994).
91 Id. at 121. 99 Lee C. Bollinger, The Tolerant Society: A Response to
92 Id. at 123. Critics, 90 Columbia L. Rev. 979 (1990).
12
Andrew F. Sellars
ing legal punishment for hate speech, has been bated, concerns over prior restraint of speech
104
the source of what seems to be the deepest are undoubtedly a hallmark of free speech juris-
and most voluminous literature addressing hate prudence in the United States. Courts are ex-
105
speech as a topic of public concern. The con-

100
tremely reluctant to endorse regimes of speech
tours of acceptable behavior are debated open- enforcement that remove speech before it is ac-
ly and vigorously every day, and the recent in- tually adjudicated to be illegal or actionable,
crease of attention to issues misogyny, racism, even when the speech falls into a proscribable
and other intolerance online could well be seen category. A series of cases around obscenity
106
as exposing a state of affairs that always exist- prosecution in the United States have ensured
ed, but was never discussed. Under a theory like that judgments about what speech is unlawful
Bollinger’s, this could be our system working it- cannot be made by law enforcement agents in
self to a more tolerant place. The human cost the field, that governments cannot interfere
107
of this process, however, is real, and its depth with the dissemination of allegedly unlawful
remains to be seen. speech without judicial review, and that seizure
108
of a book or film to obtain evidence of illegal

Other Theories and Procedural Values speech cannot effectively remove the speech
These are just three of several theories Green- from circulation. This procedural protection
109
awalt examines in his sweeping review of the ensures against over-censorship (accidental or
justifications for freedom of expression. Others deliberate), but also safeguards free speech
include the value in having a society overtly principles against the exigencies of any given
accommodate a plurality of interests and per- period, when governments may be strongly in-
spectives, the importance of speech as a check
101 clined to censor speech for precisely the wrong
on abuses of governmental power, and the 102 reasons. 110
value exposure to a variety of opinions can be

for autonomous development. Some of these
103 Similar motivations guide the general approach
militate against suppression of hate speech on 104 See generally Leonard Levy, Emergence of a Free
their own, and some instead serve as warnings Press (1985).
against how laws against hate speech could be 105 Thomas I. Emerson, The Doctrine of Prior Restraint,
abused when in the wrong hands. Many of the 20 Law & Contemp. Problems 648 (1955).
same concerns and responses addressed to the 106 See, e.g., Ctr. for Democracy & Tech. v. Pappert,
theories above can be brought to these justifi- 337 F. Supp. 2d 606, 657 (E.D. Pa. 2004); Dawn C.
cations as well. Nunziato, How (Not) to Censor: Procedural First
Amendment Values and Internet Censorship World-
Moving from theory to praxis, an underappreci- wide, 42 Georgetown J. of Int’l Law 1123, 1128–29
ated aspect of the American approach to free- (2011) (reviewing procedural safeguards under First
Amendment jurisprudence).
dom of expression is the process by which courts
107 Marcus v. Search Warrants of Property, 367 U.S. 717,
address free speech issues, and the procedures
731–32 (1961).
that are implemented to safeguard speech.
108 Bantam Books, Inc. v. Sullivan, 372 U.S. 58, 69–71
Though the importance of this concept to the (1963).
drafting of the First Amendment is frequently de- 109 Ft. Wayne Books, Inc. v. Indiana, 489 U.S. 46, 62–65
(1989); Roaden v. Kentucky, 413 U.S. 496, 503–04
(1973).
100 Friedrich Kübler, How Much Freedom for Racist 110 For just a few of many articles on this concept, see
Speech?: Transnational Aspects of a Conflict of Hu- Nunziato, supra note 106; Jack Balkin, Old School/
man Rights, 27 Hofstra L. Rev. 335, 347 (1998). New School Speech Regulation, 127 Harv. L. Rev. 1
101 See Greenawalt, Free Speech Justifications, supra (2014); Vincent Blasi, The Pathological Perspective
note 59, at 141–42. and the First Amendment, 85 Colum. L. Rev. 449
102 This view is most commonly cited to Vincent Blasi, (1985); Henry P. Monaghan, First Amendment “Due
The Checking Value in First Amendment Theory, 1977 Process”, 83 Harv. L. Rev. 518, 537–38 (1970). I
Am. Bar Found. Research J. 523 (1977); see also have previously written about this in the context of
Greenawalt, Free Speech Justifications, supra note copyright infringement actions. See Andrew Sellars,
59, at 142–43. Seized Sites: The In Rem Forfeiture of Copyright
103 See Greenawalt, Free Speech Justifications, supra Infringing Domain Names (2011), http://papers.ssrn.
note 59, at 143–45. com/sol3/papers.cfm? abstract_id=1835604.
13
Andrew F. Sellars
that the Supreme Court takes to new forms of when I see it.” What makes hate speech even
115
content regulation by the legislative and execu- more difficult than obscenity is that for hate
tive branches, often called the “categorical ap- speech Justice Stewart’s statement is less likely
proach.” The approach is distinguished from
111
to be true.
a general balancing test, where a court would
assess any given speech against the harms it In fact, the consensus view appears to be that
causes. This categorical approach further
112
a wide array of different forms of speech could
guards against improper use of speech laws, as or could not fit a definition of “hate speech,”
it avoids the ideological predispositions or para- depending on the speech’s particular context,
noia that can permeate a balancing test. 113
which rarely makes it into the definition itself. 116
Looking to content of speech, epithets and in-

sults may be easy to define and identify, but
Attempts to Define Hate an epithet devoid of context may lead a schol-
Speech ar to see hate speech where the speaker, re-
cipient, and subject of discussion may not. 117
Coded speech can be especially corrosive to a

This is a difficult background from which to draw group’s dignity, but can be hard to see unless
an objective definition of hate speech that gets one knows to look for it. As Henry Louis Gates,
118
at the “bad” forms of speech while leaving out Jr. puts it, it is wrong “to spend more time worry-
the “good.” The wide variety of approaches to ing about speech codes than coded speech.” 119
the term are daunting to summarize, but by re-

114 The rhetoric of hatred employed can be quite
viewing them and the context in which they sit, varied, even within one type of hate speech. To 120
scholars may be better equipped to appreciate make matters worse, certain online hate groups
the complexity of what they endeavor to study. have resorted to using steganography in their
This section reviews existing attempts to define online communications, putting symbolic mark-
such speech, and draws out some key similar ers on online speech in order to identify a hate
themes. speech target to other members of the group, 121
Can We Know It When We See It? 115 Jacobellis v. Ohio, 378 U.S. 184, 197 (1964) (Stewart,
J., concurring). Catharine MacKinnon has noted
Before turning to definition, there is one last
that the phrase is “more interesting than it is usually
piece of complication that helps to explain taken to be,” as it is both a reflection of how Justice
why defining the most important or worst hate Stewart reaches the question, and a reflection of
speech can be so difficult. Before the Supreme his position of power – “his seeing determines what
Court settled on a formal definition for obscen- obscenity is in terms of what he sees it to be.” Cath-
ity, Justice Stewart famously summarized his arine MacKinnon, Pornography, Civil Rights, and
feelings on identifying obscenity as “I know it Speech, 20 Harv. C.R.–C.L. L. Rev. 1, 3, (1985).
116 See, e.g., Bhikhu Parekh, Is There a Case for Banning
Hate Speech?, in The Content and Context of
111 See Gregory P. Magarian, The Marrow of Tradition: Hate Speech, supra note 16, at 37, 40.
The Roberts Court and Categorical First Amend- 117 See Delgado, supra note 6, at 179–80.
ment Speech Exclusions, 56 Wm. & Mary L. Rev. 1339 118 See Parekh, supra note 116, at 41.
(2015). 119 Henry Louis Gates, Jr., War of Words: Critical Race
112 See Leslie Kendrick, Content Discrimination Revisit- Theory and the First Amendment, in Speaking of
ed, 98 Va. L. Rev. 231 (2012). Race, Speaking of Sex: Hate Speech, Civil Rights,
113 John Hart Ely, Flag Desecration: A Case Study On and Civil Liberties 17, 47 (Henry Louis Gates, Jr. et
the Roles of Categorization and Balancing in First al. eds. 1995).
Amendment Analysis, 88 Harv. L. Rev. 1482, 1501 120 See Priscilla Marie Meddaugh & Jack Kay, Hate
(1975). Speech or “Reasonable Racism?” The Other in
114 As Robert Post put it when addressing “racist Stormfront, 24 J. Mass Media Ethics 251 (2009)
speech,” the term “probably has as many differ- (looking at the various rhetorical tactics with one
ent definitions as there are commentators, and it white supremacist group).
would be pointless to pursue its endlessly variegated 121 See Cooper Fleishman & Anthony Smith,
shades of meaning.” Robert Post, Racist Speech, (((Echoes))), Exposed: The Secret Symbol Neo-Nazis
supra note 40, at 271. Use to Target Jews Online, Tech.mic (June 1, 2016),
14
Andrew F. Sellars
or adopting benign terms as code words for ra- longer narrative of hatred? What of the news
cial slurs. Once identified, online tools can sur-
122
coverage that extensively debated the film and,
face such speech quickly, but one would have in so doing, showed clips of the film again and
to know to look for it first. again? What of the litany of scholars who re-
viewed and discussed the film when a curious
Identifying which types of hateful expression turn of events found the film being examined
are the most harmful can be especially elusive, in federal court in a copyright dispute? All of 125
for the nexus between any given hateful speech these people repeated the hateful message in a
and a harmful consequence can be so hard to modified way. Whose dissemination or modifi-
identify ex ante. The situation with the Innocence cation should be the focus? 126
of Muslims video in 2012 is a powerful example.

The video was created under false pretenses by Given these difficulties, it is no surprise that
Nakoula Basely Nakoula, casting and produc- most scholarship on hate speech starts not with
ing a film called Desert Warrior before taking a definition, but with examples. This is useful 127
the film to another group to add in vitriolic an- as an empathetic framing — researchers must
ti-Islam content over the original footage. The not forget that there are people directly affect-
123
video was so poorly produced and overdubbed ed by the definitional choices that are made
that I suspect most viewers (or certainly most in this space — but the lack of definitions in
128
American viewers) would never take it serious- scholarship translates to uncertain definitions
ly, and the film lingered in obscurity online for in law and social science research, and even
months, before another figure, notorious anti-Is- more uncertain application of principles in on-
lamic figure Terry Jones, drew attention to it for line spaces. To the extent these groups have en-
an event on the anniversary of the September deavored to identify hate speech, they are re-
11th attacks. Islam opponent Morris Sadek saw it viewed below.
through Jones, dubbed the film into Arabic (and
thus removed the obvious overdubs in the origi- Academic Attempts
nal), and then shared it with colleagues in Egypt. Academics define hate speech for a variety of
Once it was in Egypt, al Nas Television personal- different ends, and their particular motivations
ity Khaled Abdullah used the film to promote his often drive the approach they take to a defini-
longstanding narrative that the United States is tion. Some do not overtly call for legal sanction
at war with Islam, and riots ensued. 124
for such speech and seek merely to understand
the phenomenon; some do seek to make the
129
Whose speech along the chain is worthy of iden- speech illegal, and are trying to guide legisla-
tification as “hate speech?” Nakoula for creat- tors and courts to effective statutory language; 130
ing the film? Jones for amplifying its message some are in between. No doubt all definers here
131
with intent to incite hatred? Sadek for mislead-

ing Arabic audiences into thinking it was any- 125 Garcia, 786 F.3d 733; Rebecca Tushnet, Performance
thing more than a hokey overdub of a pathetic Anxiety: Copyright Embodied and Disembodied, 60
movie? Abdullah for using the film as part of a J. Copyright Soc’y 209 (2013).
126 This concern was central to a European Court of
https://mic.com/articles/144228/echoes-exposed- Human Rights case, Jersild v. Denmark, where a
the-secret-symbol-neo-nazis-use-to-target-jews-on- journalist who interviewed members of a racist group
line#.KANtlJ0Pk. in Denmark was charged as an aider and abettor of
122 Nikhil Sonnad, Alt-Right Trolls are Using These Code the hate speech made by the interviewees. A divided
Words for Racial Slurs Online, Quartz (Oct. 1, 2016), ECHR reversed the conviction. 298 Eur. Ct. H.R. (ser.
http://qz.com/798305/alt-right-trolls-are-using-goo- A) (1994).
gles-yahoos-skittles-and-skypes-as-code-words-for- 127 See, e.g., Waldron, supra note 14, at 2–3; Matsuda,
racial-slurs-on-twitter/. supra note 7, at 2320–21.
123 Garcia v. Google, Inc., 786 F.3d 733 (9th Cir. 2015) 128 Matsuda, supra note 7, at 2327.
(en banc). 129 See, e.g. Calvin R. Massey, Hate Speech, Cultural
124 See Rebecca MacKinnon & Ethan Zuckerman, Don’t Diversity, and the Foundational Paradigms of Free
Feed the Trolls, Index on Censorship (Dec. 3, 2012), Expression, 40 UCLA L. Rev. 103 (1992).
http://www.indexoncensorship.org/2012/ 12/dont- 130 See, e.g., Matsuda, supra note 7.
feed-the-trolls-muslims/. 131 See Parekh, supra note 116, at 46 (defining hate
15
Andrew F. Sellars
feel as though some hate speech will fall outside inequality which she calls a study of “outsider
their definition, but one assumes each intended jurisprudence,” or a critical examination of
137
to cover the most prevalent or egregious exam- how law can be “both a product and a promot-
ples.132
er of racism.” Matsuda also engages direct-
138
ly with critics who raised objections under the

Many assessments of the academic landscape First Amendment, and crafts an approach that
begin with Richard Delgado’s highly influential she believed satisfies those objections. Her ap-
article Words that Wound. Delgado begins
133
proach requires racist speech to be actionable
with a review of a tension between cases like if: (1) the message is “of racial inferiority;” (2)
the Skokie case, where racist speech is award- the message is “directed against a historically
ed full protection, and cases that have allowed oppressed group;” and (3) the message is “pros-
for compensation for victims of racial harass- ecutorial, hateful, and degrading,” which Mat-
ment in the workplace and elsewhere. Delga- 134
suda later clarifies has an intent-element within
do focuses on racism in particular, and makes it. Compared to Delgado, Matsuda appears to
139
a detailed argument for why law should sanc- adopt a more overt content formulation in the
tion racist speech. In crafting a proposed tort first element of this definition. To be actionable
for racist speech, Delgado proposed a defini- the speech must “den[y] the personhood of
tion that would require the plaintiff to prove: (1) target group members,” and treat all members
that “[l]anguage was addressed to him or her of the targeted group as “alike and inferior.” 140
by the defendant that was intended to demean Matsuda also limits her definition to speech tar-
through reference to race;” (2) “that the plain- geting historically oppressed or subordinated
tiff understood as intended to demean through groups. Finally, the speech is actionable if it “is,
reference to race; and” (3) “that a reasonable and intended as” speech that is harmful. 141
person would recognize as a racial insult.” Un- 135
packing Delgado’s definition, it is notably elu- Calvin Massey engaged directly with Matsuda’s
sive on criteria for content, and instead focuses definition as part of a longer assessment of how
primarily on intent, impact, and objective per- hate speech fits into a theory of freedom of ex-
ception: the intent of the speaker to “demean pression as a theoretical matter. Massey notes
142
through reference to race,” the impact that the that “most definitions tend to prejudice the
tort victim understood the speech as it was in- discussion, by defining the term in a way that
tended, and that a “reasonable person” (a pop- shapes, if not predetermines, the outcome, or
ular, if controversial, abstract figure in tort and by using terms laden with subjectivity.” He set- 143
criminal law ) could identify the speech as a

136
tles on a definition for study that “hate speech
racial insult, one assumes making that assess- is any form of speech that produces the harms
ment in the context in which it is spoken. which advocates for suppression ascribe to hate
speech: loss of self-esteem, economic and so-
Mari J. Matsuda builds upon Delgado’s work, cial subordination, physical and mental stress,
and looks at hate speech as a criminal matter, silencing of the victim, and effective exclusion
placing the issues of hate speech in the con- from the political arena.” He notes that his ap-
144
text of greater structural analysis of law and proach “treat[s] all racists the same; polite, civil
and unconscious racists are considered here to
speech that should be subject to sanction, but argu- be no less malignant than the vulgar, nasty, and
ing that “law must be our last resort”).
132 This review also no doubt overlooks many other
efforts to define hate speech, but proceeds in the
hope that the examples presented here can sketch 137 Matsuda, supra note 7, at 2323.
the appropriate contours. 138 Id. at 2325 (citing Derrick Bell, And We Are Not
133 Delgado, supra note 6. Saved (1987)).
134 Id. at 133 (citing Contreras v. Crown Zellerbach, Inc., 139 Id. at 2357–58.
565 P.2d 1173 (Wash. 1977)). 140 Id. at 2358.
135 Delgado, supra note 6, at 179. 141 Id.
136 See, e.g., Victoria Nouse, After the Reasonable Man: 142 Massey, supra note 129.
Getting Over the Subjectivity/Objectivity Question, 143 Id. at 105 n.2.
11 New Crim. L. Rev. 33 (2008). 144 Id.
16
Andrew F. Sellars
brutal ones.” This suits his framing as a the- put forth by Matsuda and Moran, covering both
145
oretical study, instead of an effort to create a speech that is directly hateful and speech that
workable framework for punishment. incites hatred in others. There is also a degree153
of magnitude in Ward’s definition, noting that

Mayo Moran also engaged with Matsuda’s defi- a speaker should be seen as employing hate
nition, and looked to both public debate of hate speech if “their attacks are so virulent that an
speech and regulation of such speech in the observer would have great difficulty separating
United States and Canada. Moran was skepti- the message delivered from the attack against
146
cal that one could present an effective descrip- the victim.” This is in some ways a reflection of
154
tion of the problem, noting that “no pre-theoret- a “no redeeming purpose” element, discussed
ical description exists for anything, much less further below.” 155
for complex social and cultural phenomena. So,

describing the problem already involves choices Susan Benesch has looked to a particular sub-
and commitments that favor a certain way of set of hate speech that is more directly linked to
seeing the world, that makes some arguments the incitement of mass violence, which she calls
more persuasive than others, some cases more “dangerous speech.” Rather than aim for an 156
relevant, and some facts easier to ‘find.’” Nev- elements-based definition, Benesch looks to five
147
ertheless, she sketches “certain elements” that “variables” that are relevant for determining the
occur in definitions in other literature and in ar- dangerousness of the speech, whether (1) there
eas of law, and settles on a definition for study is a “powerful speaker with a high degree of in-
148
as “speech that is intended to promote hatred fluence;” (2) there is a receptive audience with
against traditionally disadvantaged groups.” “grievances and fear that the speaker can cul-
149
Moran does not adopt a content element in her tivate;” (3) a speech act “that is clearly under-
definition, and is distinct from Matsuda’s on in- stood as a call to violence;” (4) a social or his-
tent, in that it looks to speech that “promotes torical context that is “propitious for violence,
hatred” instead of speech that is itself hateful. for any of a variety of reasons;” and (5) an “in-
This could therefore extend study to speech that fluential means of dissemination.” Benesch’s 157
is more subtle and coded, so long as it is still formulation is noteworthy in that it spends con-
intended to “promote hatred.” The use of the siderably more energy on the context in which
150
word “promote” instead of “incite” is also note- speech is happening, and what sorts of soci-
worthy, in that it gives the impression of the less etal or environmental descriptors or qualities
imminent standard than “incitement” normally should be of most concern. Her formulation is 158
carries. specifically targeted toward incidents of mass

violence and the speech that causes such vio-
Kenneth Ward engaged in further analysis a lence, but similar contextual descriptors could
few years later, and specifically looked to times likely be developed for the specific harms other
where judges uphold speech restrictions with scholars seek to study.
the goal of promoting more speech overall. He 151
defined hate speech as “any form of expression 153 See id.

though which speakers primarily intend to vili- 154 Id. at 766.
fy, humiliate, or incite hatred against their tar- 155 See infra notes 312–19.
gets.” This can be seen as a blend of the intents 156 Susan Benesch, Proposed Guidelines for Dangerous
152
145 Id. Speech, Dangerous Speech Project (Feb. 23,

2013), http://dangerousspeech.org/guidelines/.
146 Moran, supra note 98.
157 Id. An earlier version of Benesch’s formulation rear-
147 Id. at 1428.
ranged some of these elements, and looked also to
148 Id. at 1429. whether the “marketplace of ideas” is still function-
149 Id. at 1430. ing. See Susan Benesch, Vile Crime of Inalienable
150 See id. Right: Defining Incitement to Genocide, 48 Va. J.
151 Kenneth D. Ward, Free Speech and the Development Int’l L. 485, 519–25 (2008).
of Liberal Virtues: An Examination of the Controver- 158 See generally Jonathan Leader Maynard & Susan
sies Involving Flag Burning and Hate Speech, 52 U. Benesch, Dangerous Speech and Dangerous Ideolo-
Miami L. Rev. 733 (1998). gy: An Integrated Model for Monitoring and Preven-
152 Ward, supra note 151, at 765. tion, 9 Genocide Studies & Prevention 70 (2016).
17
Andrew F. Sellars
Bhikhu Parekh engaged directly with the ques- a harms-based element. On the content side,
163
tion of whether one could develop a workable they look specifically to the use of symbolism
definition of hate speech for the purposes of to convey a hateful message, and the Supreme
regulation. He drafted a list of eleven examples Court’s assessment of such symbolism in R.A.V.
of speech that may or may not be hate speech v. City of St. Paul. On the intent element, they
164
and then developing a framework for triaging look specifically toward a subjective intent
those examples. He began by noting the diver-
159
“only to promote hatred, violence or resentment
sity in the examples he created: against” a marginalized group or member of
a marginalized group, “merely because of the
Some of them express or advocate views status of the minority.” On the harms formu-
165
but do not call for action. Some are abu- lation, Marwick and Miller largely draw from
sive or insulting but not threatening. Some Massey, and add on a requirement that the re-
express dislike of a group but not hatred, cipient must subjectively experience the harm. 166
and some of those that do are so subtle as Perhaps most interestingly, Marwick and Miller’s
not to be obviously abusive or insulting. definition supposes a “no redeeming purpose”
Some take a demeaning or denigrating element similar to Ward’s above, noting sever-
view of a group but wish it no harm and al times that their definition should extend to
even take a [patronizingly] indulgent atti- speech that is “intended only to promote ha-
tude toward it. 160
tred,” targeting of a person “merely because of
the status of the minority,” and done “without
He went from there to distill “three essential communicating a legitimate message.” 167
features” to form a definition of hate speech:

(1) “it is directed against a specified or easily Legal Attempts
identifiable individual or, more commonly, a A growing number of countries seek to regulate
group of individuals based on an arbitrary or speech directly, and in so doing have endeav-
normatively irrelevant feature;” (2) the speech ored to define such speech in their criminal
“stigmatizes the target group by implicitly or ex- codes. The similarities and differences between
168
plicitly ascribing to it qualities widely regarded these definitions bear an interesting reflection
as undesirable;” and (3) “because of its nega- of how each society values different aspects
tive qualities, the target group is viewed as an of hate speech regulation. Jeremy Waldron,
169
undesirable presence and a legitimate object of author of one of the current leading books to
hostility.” Interestingly, Parekh’s definition ap-
161
defend hate speech regulation, largely looks to
pears to lack any requirement that the speaker
intend harm, and thereby sweeps in forms of
expression that would raise serious issues un-
163 Alice Marwick & Ross Miller, Online Harassment,
der free speech theories, such as the criminolo- Defamation, and Hateful Speech: A Primer of
gy scholarship about disparate reported crime the Legal Landscape (2014), available at http://
rates between different races discussed above, ir.lawnet.fordham.edu/cgi/viewcontent.cgi?arti-
which discusses differences between races in a cle=1002&context=clip.
dry, scientific manner. 162
164 505 U.S. 377 (1992); see also Charlotte Taylor, Hate
Speech and Government Speech, 12 U. Pa. J. Const.
Finally, in 2014, Alice Marwick and Ross Miller L. 1115 (2010).
took an extensive review of existing definitions, 165 Marwick & Miller, supra note 163, at 17.
including those by Massey and Ward, and 166 Id. at 17.
came up with three general elements that are 167 Id.
used to define hate speech: (1) a content-based 168 Haraszti, supra note 18, at xiv.
element, (2) an intent-based element, and (3) 169 I select British and formerly-British-colonial govern-
ments for this section, as the relatively similarity in
the systems to the United States can better lend to
easy comparison. Of course, such a quick review
159 Parekh, supra note 116, at 38–39. does not fully explain the similarities and differenc-
160 Id. at 40. es between these approaches and how they are
161 Id. at 40–41. enforced, but I hope the review can surface some in-
162 See supra note 31. teresting priorities for approaching these definitions.
18
Andrew F. Sellars
these definitions to frame his argument. 170

er hate speech “for the purpose of removal.” 175
Canada also separately prohibits those who

To begin with the now-defunct American exam- “communicat[e] statements in a public place”
ple, the Illinois statute that was initially upheld speech that “incite[] hatred against any iden-
in the Beauharnais decision stated that it would tifiable group,” but only when such incitement
be unlawful for any person to make a public 171
“is likely to lead to a breach of the peace.” 176
dissemination or presentation “which publi- Canada also specifically punishes anyone

cation or exhibition portrays depravity, crim- “who advocates or promotes genocide,” with
inality, unchastity, or lack of virtue of a class “genocide” defined as acts “committed with in-
of citizens, of any race, color, creed or religion, tent to destroy in whole or part any identifiable
which said publication or exhibition exposes the group.” Targeted groups for all three actions
177
citizens of any race, color, creed or religion to can include groups identified by color, race, re-
contempt, derision, or obloquy or which is pro- ligion, national or ethnic origin, age, sex, sexual
ductive of breach of the peace or riots.” There 172
orientation, or mental or physical disability. 178
appear to be two independent ways one can

be liable under this statute: (1) disseminating The United Kingdom has had a specific prohibi-
speech that portrays a group as having a bad tion against hate speech since the Race Rela-
virtue and thereby expose a member of a group tions Act of 1965, which prohibited incitement to
to experience “contempt, derision, or obloquy;” discrimination or incitement to racial hatred. 179
or (2) disseminating similar speech that caus- Notably, this law originally required a showing
es a breach of the peace. This all should be un- that the defendant intended to incite hatred,
derstood within the greater common law libel but this element was later removed. Under the 180
framework, which usually included within it an current law, the Public Order Act of 1986 (as
intent element. 173
amended), prohibits a person who disseminates
or displays any speech that is “threatening,
Canada has provisions which prohibit hate abusive or insulting,” if “he intends thereby to
speech in a way similar to the former American stir up racial hatred,” or if “having regard to all
law. The Criminal Code of Canada punishes any- the circumstances racial hatred is likely to be
one who “willfully promotes hatred against any stirred up thereby.” This formulation accounts
181
identifiable group,” but specifically excludes for deliberate harm as well as negligent harm, a
from this definition statements that are proven more objective measurement that does not re-
by the defendant to be true, statements that
174
quire interrogation into the defendant’s motives.
are offered “in good faith,” when expressing “an
opinion on a religious subject,” statements that Australia has a blend of federal and state laws
are “relevant to the public interest, the discus- that address hate speech. Federally, the Racial
sion of which was for the public benefit,” or if Discrimination Act of 1975 prohibits actions that
“in good faith,” the person was pointing out oth- are “reasonably likely, in all the circumstances,
to offend, insult, humiliate or intimidate anoth-
170 Waldron, supra note 14 at 8 (“By ‘hate speech reg-
ulation,’ I mean the regulation of the sort that can be
er person or group of people” when the act is
found in Canada, Denmark, Germany, New Zealand, “done because of the race, [color] or national
and the United Kingdom….”) or ethnic origin of the other person or of some
171 Interestingly, the statute only applied if the dissem- or all of the people in the group.” As with Can-
182
ination was public. Private dissemination of hate ada, the statements must be made in public in
speech was not included. See People v. Simcox, 40
N.E.2d 525, 526 (Ill. 1942).
172 Beauharnais, 343 U.S. at 251. 175 Canada Criminal Code § 319(3).
173 See Note, Defamation, 69 Harv. l. Rev. 875, 902–03 176 Id. § 319(1).
(1956).
177 Id. § 318.
174 This is an interesting switch in burdens compared to
178 Id. § 318(4).
how the United States approaches burdens of truth
and falsity in defamation cases, at least when the 179 See Race Relations Act of 1965, ch. 73 § 6.
speech is on a matter of public concern. See Phila- 180 Matsuda, supra note 7, at 2346–47.
delphia Newspapers, Inc. v. Hepps, 475 U.S. 767, 776 181 Public Order Act 1986 § 18(1).
(1986). 182 Racial Discrimination Act 1975 § 18C(1).
19
Andrew F. Sellars
order to be actionable. Interestingly, the State

183
of the audience; and (6) the likelihood of effec-
of Victoria, Australia, does allow for punishment tively inciting harm. 188
for private communication, if done to “incite[]

hatred against, serious contempt for, or revul- Finally, the European Union’s “Framework De-
sion or severe ridicule of” a person or class of cision on combating certain forms and expres-
persons, but specifically excludes artistic per-
184
sions of racism and xenophobia by means of
formances, statements made for “any genuine criminal law” has become newly relevant, as it
189
academic, artistic, religious, or scientific pur- serves as the basis of the definition for the new
pose,” “any purpose that is in the public inter- cooperative agreement reached between pri-
est,” or “publishing a fair and accurate report of vate online platforms and the European Com-
any even or matter of public interest.” 185
mission to police hate speech on the websites’
platforms. In the European Union regulatory
190
There are also efforts to adopt international fabric a “framework decision” is not itself a
standards. Most significant is the International binding statement of law, but is a strong indi-
Covenant on Civil and Political Rights (ICCPR), cation of how relevant EU and EU nation laws
which (in keeping with the theme of this essay) should be interpreted on this point, and now
also has within it conflicting rules. On the one may well become the basis of hate speech reg-
hand, Article 19 of the ICCPR makes clear that ulation for a large section of the popular Inter-
“[e]veryone shall have the right to hold opinions net. The framework defines hate speech as one
191
without interference” and that “[e]veryone shall of three things: (1) “Public incitement to violence
have the right to freedom of expression.” On the or hatred directed against a group of persons or
other, Article 20 of the ICCPR states that “[a]ny a member of such group defined on the basis of
advocacy of national, racial or religious hatred race, [color], descent, religion or belief, or na-
that constitutes incitement to discrimination, tional or ethnic origin;” (2) the same, when done
hostility or violence shall be prohibited by law.” 186
through “public dissemination or distribution
Toby Mendel at the Centre for Law and Democ- of tracts, pictures, or other material;” and (3)
racy argues that the two can be reconciled “publicly condoning, denying or grossly trivial-
against each other, but admits that courts have izing crimes of genocide, crimes against human-
not done a good job doing so. One notable ef-
187
ity, and war crimes [as defined in EU law], when
fort to harmonize the two is “the Rabat Plan of the conduct is carried out in a manner likely to
Action,” a multistakeholder processes convened incite violence or hatred against such group or
by the U.N. Office of the High Commissioner of a member of such group.” This “genocide de-
192
Human Rights. The Rabat Plan sought to devel- nial” approach to hate speech regulation is re-
op a six part test for assessing when speech is flected in the laws of several countries. 193
severe enough to warrant punishment under Ar-

ticle 20, which looks to: (1) the social and po- Attempts by Online Platforms
litical context in which the statement is made; While governments have a large role to play in
(2) the position or status of the speaker in so- defining hate speech, especially outside of the
ciety; (3) the specific intent to cause harm; (4) United States, perhaps the most active space in
the degree to which the content of the speech adjudicating definitions of hate speech comes
was “provocative and direct,” and the “nature from no government at all. Private online plat-
of the arguments deployed in the speech”; (5) forms, like Facebook, Twitter, YouTube, and oth-
the extent and reach of the speech and the size ers, routinely draft definitions of hate speech
183 Id. § 18C(2). 188 Rabat Plan of Action ¶ 22.
184 Racial and Religious Tolerance Act 2001 § 8(1) (Victo- 189 Framework Decision 2008/913/JHA (Nov. 28, 2008),
ria, Aus.). available at http://eur-lex.europa.eu/legal-content/
185 Id. § 11 (Victoria, Aus.). EN/TXT/?uri=uriserv%3Al33178 (hereinafter “EU
186 International Covenant on Civil and Political Rights, Framework Decision on Racism and Xenophobia”).
available at http://www.ohchr.org/en/professionalin- 190 See infra note 208 and accompanying text.
terest/pages/ccpr.aspx. 191 See EU Framework Decision on Racism and Xenopho-
187 Toby Mendel, Does International Law Provide for bia, supra note 189.
Consistent Rules on Hate Speech?, in The Content 192 See id.
and Context of Hate Speech, supra note 16, at 417. 193 See, e.g., Parekh, supra note 116, at 41.
20
Andrew F. Sellars
for use in moderating their online platforms, the right to delete or modify content or remove
and these definitions can also provide insight users at a platform’s sole discretion. It is there- 199
for scholars. fore unnecessary — and perhaps unwise — for

a platform to neatly delineate the precise scope
The objectives of private online platforms, and of unacceptable behavior on their platform.
how they regulate, are worth exploring before Platforms want the flexibility to respond at will.
turning to the definitions themselves. It is now
194
unremarkable to observe that the regulation of This approach, however, puts the regulation of
online platforms has some key structural differ- speech by online platforms precisely at odds
ences than the regulation of governments. In with the thoughtful development of structur-
195
the United States, platforms have profound lib- al and procedural safeguards for speech dis-
erty to set rules around speech as they choose. cussed above, which help to ensure against ar-
They are not bound to protect the speech of their bitrary enforcement, or overreacting to speech
users under First Amendment, as they are not that touches on the pathological hysteria of the
state actors, and indeed are immunized from a era. Scholars have noted that unclear speech
200
wide array of liability for speech generated by rules on online platforms can lead to many
their uses under specific federal laws. What bad results. And because the decision-mak-
196 201
seems to motivate inclusion of hate speech reg- ing around these questions can happen be-
ulation is a desire not to engender controversy hind closed doors — in fact, often through use
on their platforms, be that for liability reasons of large teams of underpaid laborers in other
or because litigating to enforce their legal safe countries — there are really two sets of relevant
harbors is often expensive. 197
standards one should study: the public declara-
tion of the rule, which has a vague or ceremo-
It is also important to consider the platforms’ nial role, and the actual operational document
greater options and subtler motivations in ar- hidden from public view. The impression of a 202
ticulating rules for hate speech. Technology af-

fords many possible responses to hate speech, education_and_video_games_the_league_of_leg-
ends_tribunal/ (describing how Riot Games formerly
which can extend to deleting content, modify-
adopted a mixture of technological and social
ing content, blocking users, making content in- responses, including a user-staffed tribunal, on their
visible to some but not all users, and even more platform to cut bullying and offensive behavior).
creative experimentation such as temporary 199 See, e.g., Terms of Use, Instagram, https://help.ins-
bans or internal quasi-judicial resolution among tagram.com/478745558852511 (last updated Jan. 19,
users. Most, if not all, platforms always reserve
198
2013), (“We may, but have no obligation to, remove,
edit, block, and/or monitor Content or accounts
194 Though as others have noted, the relationship containing Content that we determine in our sole
between platforms and governments is nuanced, discretion violates these Terms of Use.”); Statement
fluid, and still provides a good deal of regulation of Rights and Responsibilities, Facebook (last updat-
capabilities for state actors. Jack Goldsmith & Tim ed Jan. 30, 2015), https://www.facebook.com/terms
Wu, Who Controls the Internet? Illusions of a (hereinafter “Facebook Statement of Rights”) (“We
Borderless World (2006). can remove any content or information you post on
195 See generally Lawrence Lessig, Code 2.0 (2006). Facebook if we believe that it violates this Statement
196 See generally Adam Holland et al., Intermediary of our policies.”).
Liability in the United States, Global Network 200 See supra notes 104–13 and accompanying text.
of Internet & Society Research Centers (Feb. 201 See, e.g., Erica Nowland et al., Account Deactiva-
18, 2015), https://publixphere.net/i/noc/page/ tion and Content Removal: Guiding Principles
OI_Case_Study_Intermediary_Liability_in_the_Unit- and Practices for Companies and Users (Sept.
ed_States. 2011), https://cdt.org/files/pdfs/Report_on_Ac-
197 See id.; see also David S. Ardia, Free Speech Sav- count_Deactivation_and_ Content_Removal.pdf.;
ior or Shield for Scoundrels: An Empirical Study of Jillian York, Guns and Breasts: Cultural Imperial-
Intermediary Immunity Under Section 230 of the ism and the Regulation of Speech on Corporate
Communications Decency Act, 43 Loyola L.A. L. Rev. Platforms (March 17, 2016), http://jilliancyork.
373 (2010). com/2016/03/17/guns-and-breasts-cultural-imperial-
198 See Justin Reich, Civic Education and Video Games: ism-and-the-regulation-of-speech-on-corporate-plat-
The League of Legends Tribunal (May 23, 2013), forms/.
http://www.edtechresearcher.com/2013/05/civic_ 202 Jeffrey Rosen, The Delete Squad, The New Re-
21
Andrew F. Sellars
user is that the adjudication is seamless and solely on the subjective interest of the relevant
mechanized, but in fact there are many humans actors in the Executive Branch.
on the other side of the machine. 203
Turning to specific platforms, the social video

The platforms are also subject to informal pres- website YouTube has both a “Terms of Service”
sure from governments, who would no doubt and a set of “Community Guidelines,” the lat-
love to have this freedom and flexibility in their ter of which is incorporated by reference into
own regulation, however contrary it would be the formal agreement between YouTube and its
to the public’s interest. In the United States, users. All discussion of what constitutes hate
the pressures placed by the Executive Branch speech is confined to the Community Guide-
around the Wikileaks “Cablegate” memos and lines, while the Terms of Service mention “offen-
204
the platforms hosting the Innocence of Muslims sive content” when it makes clear that the plat-
video have required courts to develop new the- form is not liable for the offensive content of its
205
ories of First Amendment doctrine to address users (which is both in the contract and gen-
209
the soft power coercion of governments on erally true under United States law ). The Com- 210
platforms and intermediaries. In Europe, the munity Guidelines begin by first insisting that a
206
coerced cooperation manifests in the Europe- user “respect the YouTube community,” explain-
an “code of conduct,” to address hate speech, ing “[w]e’re not asking for the kind of respect
which obligates companies to prohibit hate reserved for nuns, the elderly, and brain sur-
speech on their platform, respond quickly to re- geons. Just don’t abuse the site.” This is later 211
ports of hate speech, provide regular updates explained in detail in a later section specifically
to the member countries about enforcement addressing “hateful content,” which immedi- 212
statistics, and to promote counterspeech on the ately opens with a statement articulating the
platforms targeted against hate speech. In tensions between free speech and hate speech
207
the European example at least, some attempt is regulation: “Our products are platforms for free
made to define when this soft pressure should expression But we don’t support content that
trigger. The United States pressure is based promotes or condones violence against individ-
208
uals or groups based on race or ethnic origin, re-

public (April 29, 2013), https://newrepublic.com/ ligion, disability, gender, age, nationality, veter-
article/113045/free-speech-internet-silicon-val- an status, or sexual orientation/gender identity,
ley-making-rules; Adrian Chen, The Laborers Who or whose primary purpose is inciting hatred on
Keep Dick Pics and Beheadings Out of Your Face- the basis of these core characteristics.” The list 213
book Feed, Wired (Oct. 23, 2014), http://www.wired. of protected characteristics has many elements
com/2014/10/content-moderation/. similar to the laws described above. The nexus 214
203 Mary Gray, The Paradox of Automation’s “Last Mile,” to violence is in the definition, but it extends to
Social Media Collective Research Blog (Nov. 12,
2015), https://socialmediacollective.org/2015/11/12/ 209 See Terms of Service, YouTube, https://www.you-
the-paradox-of-automations-last-mile/. tube.com/ static?template=terms § 5(D) (last updat-
204 See Michael Lambert, The State as Soft Power – The ed June 9, 2010) (the user must “acknowledge that
Intermediaries Around Wikileaks (Feb. 18, 2015), [they] will be exposed to [c]ontent that is inaccurate,
https://mlonml.com/2015/02/18/the-state-as-soft- offensive, indecent, or objectionable” agree to waive
power-the-intermediaries-around-wikileaks/. any claims against YouTube for the content); id. § 10
205 Michelle Quinn, Google Decides to Leave Video (the user must “specifically acknowledge that You-
on YouTube, Politico (Sept. 14, 2012), http://www. Tube shall not be liable for content or the defamato-
politico.com/story/2012/09/google-decides-to-leave- ry, offensive, or illegal conduct of any third party”).
video-on-youtube-081245. 210 See generally Holland, supra note 196.
206 Backpage.com, LLC v. Dart, 807 F.3d 229 (7th Cir. 211 Community Guidelines, YouTube, https://www.
2015). youtube.com/yt/ policyandsafety/communityguide-
207 Code of Conduct on Countering Illegal Hate Speech lines.html (last viewed Nov. 27, 2016).
Online, European Commission (May 31, 2016), 212 Id.
http://ec.europa.eu/justice/fundamental-rights/ 213 Id.
files/hate_speech_code_of_conduct_en.pdf. 214 The one atypical inclusion may be discrimination
208 The Code of Conduct adopts the EU Framework De- based on veteran status, though this is likely a re-
cision identified above. See supra notes 189–93 and flection of existing U.S. law on this point. See, e.g., 38
accompanying text. U.S.C. § 4212.
22
Andrew F. Sellars
either “promot[ing]” or “condon[ing]” such vio- hate speech subject to removal from the plat-
lence, thus extending beyond incitement as the form as “content that directly attacks people
concept is understood under First Amendment based on their race; ethnicity; national origin;
doctrine. YouTube also introduces intent, but
215
religious affiliation; sexual orientation; sex, gen-
makes it an optional requirement. der, or gender identity; or serious disabilities
or diseases.” Beyond this, Facebook express-
222
Much like YouTube, the social media platform ly bans “[o]rganizations and people dedicated
Twitter adopts a bifurcated structure to its reg- to promoting hatred against these protected
ulation, with a Terms of Service that disclaims groups.” These rules reflect a definition that
223
liability for offensive content on the platform, includes a protected group and a specific harm
and refers users a set of “Twitter Rules” to dis- related to the speech. Intent to cause harm is not
cuss platform norms. Within a section of those
216
directly part of Facebook’s main definition, but
rules entitled “abusive behavior,” Twitter spe- is part of the critical definition of the types of
cifically prohibits “hateful conduct,” defined hate groups that may be prohibited. Facebook
as “promot[ing] violence against or directly at- also expressly considers “innocent” use of some
tack[ing] or threaten[ing] other people on the forms of hate speech — specifically, “shar[ing]
basis of race, ethnicity, national origin, sexual content containing someone else’s hate speech
orientation, gender, gender identity, religious for the purpose of raising awareness or educat-
affiliation, age, disability, or disease.” Twitter 217
ing others about that hate speech.” 224
also makes clear that it does not allow accounts

“whose primary purpose is inciting harm to- But behind the curtain, a leaked copy of Face-
wards others on the basis of these categories.” book’s internal guidelines for its outsourced
218
content moderation team shows a much more

The social media platform Facebook adopts a granular and content-oriented definition frame-
similar structure to its regulation of hate speech. work for hate speech. Facebook’s 2012 “Abuse
225
Its “Statement of Rights and Responsibilities” Standards” operation manual asks content 226
mirrors a standard terms of service, and also moderators to flag nine different forms of “hate
219
references a set of “Community Standards” content,” including (1) slurs or racial comments
that are not directly incorporated to their terms of any kind; (2) attacking based on a protected
as a condition of the agreement, but are part category; (3) hate symbols, either out of context
of a suite of other documents “[y]ou may also or in the context of hate phrases or support of
want to review.” Again, the terms only disclaim hate groups; (4) showing support for organiza-
220
liability, and require that the user not “use Face- tions and people primarily known for violence;
book to do anything … discriminatory.” On (5) depicting symbols primarily known for
221
its Community Standards, Facebook identifies hate and violence, unless comments are clear-
ly against them; (6) “versus photos” […] com-
215 See generally Brandenburg, 395 U.S. 444. paring two people (or an animal and a person
216 See Terms of Service, Twitter, https://twitter.com/
that resembles that animal) side by side; and
tos?lang=en (last updated Jan. 27, 2016).
217 The Twitter Rules, Twitter, https://support.twitter. 222 Facebook Community Standards, supra note 220.
com/articles/18311 (last viewed June 2, 2016). 223 Id.
218 Id. 224 Id.
219 Statement of Rights and Responsibilities, Facebook 225 See Adrian Chen, Inside Facebook’s Outsourced
(last updated Jan. 30, 2015), https://www.facebook. Anti-Porn and Gore Brigade, Where “Camel Toes”
com/terms (hereinafter “Facebook Statement of are More Offensive than “Crushed Heads,” Gawker
Rights”). Facebook does make clear that it reserves (Feb. 16, 2012), http://gawker.com/5885714/inside-
the right to delete any content that, in its view, vio- facebooks-outsourced-anti-porn-and-gore-brigade-
lates any of its policies. See id. § 5(2). where-camel-toes-are-more-offensive-than-crushed-
220 See id.; Community Standards, Facebook, https:// heads.
www.facebook.com/ communitystandards (last visit- 226 See oDesk, Abuse Standards 6.2 (2012), avail-
ed May 29, 2016) (hereinafter “Facebook Community able at https://www.scribd.com/doc/81877124/
Standards”). Abuse-Standards-6-2-Operation-Manual. Facebook
221 Facebook Statement of Rights, supra note 219, at no doubt has updated its procedures since the time
§ 15(2). of this publication.
23
Andrew F. Sellars
(7) photo-shopped images showing the subject drawn. The following part reviews some of the
in a negative light. In one of the few attempts congenerous concepts that appear throughout
to engage in some form of definitional balanc- the definitions above.
ing here, moderators are instructed that “humor
overrules hate speech UNLESS slur words are A question naturally arises as to what to do with
present or the humor is not evident.” 227
these character traits. The traits identified be-
low do not form a single definition, but could
Later slides articulate the categories along be used as a scoring system to help improve
which hate-based content can be subject for a researcher’s confidence that the speech in
action, including race, ethnicity, national ori- question is the type of speech that is likely to
gin, religion, sex, gender identity, sexual orien- cause the harms that hate speech causes, is
tation, disability, or any serious disease. There unlikely to have the redeeming cultural values
appears also to be toleration built into Face- of consequentialist justifications of freedom of
book’s enforcement model for political speech: expression, and thus is worthy of identification
content that includes hate symbols should not as “hate speech.” Confidence scoring is a tech-
be escalated when it concerns a public figure or nique used in other areas of language study,
head of state, but should be escalated when it and is a useful analytic tool, in that it embraces
includes an ordinary person or law enforcement the fact that speech is inherently a complicat-
officer.228
ed, multitudinous, and highly contextual human
behavior, and that all categorization of speech
Facebook’s enforcement suite perhaps is the can only be done by matters of degrees. More 230
best example out of all the definitions as to how qualitatively speaking, looking solely at one or
definitions of hate speech should be balanced two of these factors will likely result in a vastly
with other speech considerations. Carve-outs for overbroad corpus, with a lot of speech included
humor, discussion of public figures, and speech that could not be fairly called “hate speech.”
that opposes hatred, with specific caveats for Looking at several factors will probably drive a
when the speech uses slurs, shows at least some researcher closer to what most consider “hate
attempt to accommodate speech values in the speech,” but may still identify speech for which
definition of what should be actionable, even if there is good reason to keep it outside the defi-
such accommodation would be unconstitution- nition. Speech that hits all of these criteria is
al if done by a state actor in the United States. likely to be speech that most countries and on-
This particular definition suite is also notable for line platforms would define as “hate speech,”
the level of proxies to other identifiers of hate and may even be actionable speech in the Unit-
speech, including symbols, organizations and
229
ed States, as a true threat or incitement to im-
231
people clearly oriented toward “violence,” and minent lawless action. 232
overt forms of dehumanizing content.

Finally, as I believe the criteria below make clear,
the specific environment and context around
Emerging Themes and any incident of hateful speech can be quite
Continuing Questions relevant when considering how society should
230 See, e.g., Christine Pao et al., Confidence Scoring

Common Traits in Defining Hate for Speech Understanding Systems (1998), avail-
able at https://groups.csail.mit.edu/sls/publica-
Speech tions/1998/icslp98-confidence.pdf.
Much as I would like to, I offer no single defini- 231 See United States v. Elonis, 135 S. Ct. 2001 (2015).
tion to govern all of the above-cited examples. 232 Brandenburg, 395 U.S. 444. It may be that only look-
From these myriad definitions, however, I being for speech that qualifies under every category
lieve some common themes and insights can be seeks to prove too much. Some of these elements,
like the requirement that the speech cause a sec-
227 See id. ond-level harm, could be too limiting. A consideration
228 See id. at 4. Interestingly, obesity is expressly carved of all but one or two factors, with a reason as to why
out. some were omitted, could still produce a valid work-
229 Also identified by Marwick & Miller, supra note 163. ing framework.
24
Andrew F. Sellars
respond. I do not believe that one should take how they will handle the subjective–objective
these criteria and develop a methodological ap- line in whichever groups they seek to study.
proach that claims to capture and analyze all
hate speech online, or even all hate speech on 2 - Content in the Message that
a specific platform or about a certain group. Expresses Hatred
These traits may help surface specific case As a basis for legal punishment, content-based
studies, but those studies should examine their definitions have a long climb under American
unique context. law. The Supreme Court has made the “cate-
gorical approach,” and its hostility toward con-
1 - Targeting of a Group, or Individual tent-based restriction of speech, a key ingredi-
as a Member of a Group ent to First Amendment doctrine. Only a small
239
This factor may be the only threshold factor, as list of types of speech are proscribable absent
it is the one that separates “hate speech” from an extraordinary state interest, and the Su-
any other form of harmful speech, such as bul- preme Court has made clear that they are not
lying or threats. To meet the definition of hate inclined to expand that list. If the goal is to cre-
240
speech, the speech should target a group or an ate a workable definition under law, as opposed
individual as they relate to a group. to merely for study, most will be well-advised to
avoid definitions that speak to message or con-
Which “groups” count has some variance. Some tent.
definitions use descriptors, such as “historically
oppressed,” “traditionally disadvantaged,”
233 234
Ward expressly rejects a content limitation;
or “minority.” Others prefer to list actionable
235
“any form of expression” can be hate speech,
groups. Parekh’s definition is notable in that
236
if made with the intent to incite hatred. Moran 241
it does not look for a defined group, but looks similarly looks solely toward speech intended to
to see whether the speaker targets someone “promote hatred,” however formulated. Ben- 242
based on “an arbitrary or normatively irrelevant esch looks more toward the likely effect that the
feature.” 237
speech has on the relevant audience.” Delga- 243
do’s definition seems to avoid a content defini-

By any of these frameworks, which groups make tion, though he does look to speech that makes
the list is a fascinating reflection of the values a “reference to race,” and suggests that one
244
of the particular organization or context. Race, should be able to objectively identify the speech
ethnicity, and religion appear most frequently; as an insult, though his definition does not spec-
gender, sexual orientation, and gender identity ify whether that identification should be done
appear somewhat frequently; veteran status, based on content, context, or some combina-
physical ability, and suffering from serious dis- tion of the two. (Though Delgado’s and Mo-
245
eases only occasionally appear to make the list. ran’s definitions are more general, I believe they
While legal definitions frequently use such defi- would still be considered content-based defini-
nitional lists, they are not without nuance. tions under American law. ) 246
Finally, any attempt to classify individuals will

239 See supra notes 111–13 and accompanying text.
be an attempt to objectify what can be often
240 United States v. Stevens, 559 U.S. 460, 469 (2010).
complicated and at-times-subjective questions
241 Ward, supra note 151, at 763.
about identity. Those who seek to study hate
238
242 See Moran, supra note 98, at 1430.

speech online must determine for themselves
243 Benesch, supra note 156.
244 See Delgado, supra note 6, at 179.
233 Matsuda, supra note 7, at 2357–58.
245 Id.
234 Moran, supra note 98, at 1430.
246 The definitions still require a judge or law enforce-
235 Marwick & Miller, supra note 163, at 17. ment to examine the content of the message to see
236 See, e.g., Canada Criminal Code § 318(4). if it meets the criteria, and are drafted to respond to
237 Parekh, supra note 116, at 40–41. the communicative impact of the speech in question.
238 See generally Martha Minow, The Supreme Court See Reed v. Town of Gilbert, 135 S. Ct. 2218, 2228
1986 Term – Foreword: Justice Engendered, 101 Harv. (2015); FCC v. League of Women Voters of Cal., 468
L. Rev. 10 (1987). U.S. 364, 383 (1984).
25
Andrew F. Sellars
Some scholars do propose a content-based famous for hatred. Symbols in this context are
256
definition. Matsuda looks for speech that pro- in many senses similar to an epithet, where one
motes “racial inferiority,” or “denies the person- can assume a degree of intent in many situa-
hood of target group members,” and treats all tions, with some key exceptions. Of course, the
257
members of the targeted group as “alike and ability to scour the Internet for instances of a
inferior.” Parekh looks to speech that “stig-
247
visual symbol is a notoriously difficult computer
matizes the target group by … ascribing to it science problem. 258
qualities widely regarded as undesirable.” 248
This approach is also popular in foreign law; the 3 - The Speech Causes a Harm
United Kingdom looks to speech that is “threat- This essay has largely taken harm as a giv-
ening, abusive, or insulting.” Australia looks to
249
en with hate speech. As a look to the harms of
actions (one assumes chiefly speech acts) that speech often permeate definitions, however, I
are likely to “offend, insult, humiliate, or intim- will turn to these harms now.
idate others.” The Rabat Plan looks to see if
250
the speech is “provocative and direct,” and asks Some definitions look specifically to extrinsic
EU member states to look to the “nature of the harms beyond the speech itself, most often
arguments employed.” This could be either a
251
physical violence. The European Union frame-
content characterization or an analysis of the work decision, Twitter’s Terms of Service, and
speech’s tone. The statute at issue in Beauhar- Benesch’s study of dangerous speech all look
nais sought to punish speech that “portrays de- to speech that causes a physical harm. First 259
pravity, criminality, unchastity or lack of virtue” Amendment scholarship looks to violence spe-
in a group. These can each give a framework
252
cifically in its incitement and true threats frame-
for study, but to look solely at what is said with- works, and often grapples with violent responses
out its context could lead many online schol- to otherwise-lawful speech, sometimes referred
ars to make the same errors discussed above to as the “heckler’s veto” problem. 260
concerning coded speech, re-appropriation of

slurs, and other ways seeming hate speech can Others, like Delgado, look instead to the myriad
be serving a very different purpose. In the on-253
ways in which speech can cause harm, includ-
line space especially, context and “context col- ing deep structural considerations. Delgado 261
lapse” as speech moves outside its intended au- notes that targeting hatred due to immutable
dience are very important considerations. 254
characteristics causes a more salient harm in
part because the victim will also despair that
As an interesting alternative approach, Marwick they cannot change the attribute that gives rise
& Miller look specifically to use of symbolism in to the hatred. They can permeate and impact
262
hate speech, a place where the Supreme Court the victim’s relationship with others, especial-
has provided an interesting distinction between ly across racial, religious, gender, or ethnic di-
what is and is not proscribable. Facebook’s in-
255
vides. Performance in work, relationships, and
263
ternal content guidelines also put a strong em- social and personal life will no doubt suffer due
phasis on symbols, or references to key figures to the pervasive and withering damage that
consistent denial of one’s self-worth causes. 264
247 See Matsuda, supra note 7, at 2358. 256 See supra notes 225–29 and accompanying text.
248 Parekh, supra note 116, at 40–41. 257 See generally Delgado, supra note 6.
249 Public Order Act 1986 § 18(1). 258 See Tasks, XKCD, http://xkcd.com/1425/ (last visited
250 Racial Discrimination Act 1975 § 18C(1). Nov. 27, 2016).
251 Rabat Plan of Action ¶ 22. 259 See supra notes 156, 191, and 217, and accompanying
252 Beauharnais, 343 U.S. at 251. text.
253 See supra notes 116–28 and accompanying text. 260 See, e.g., Terminiello v. City of Chicago, 337 U.S. 1
254 See Alice Marwick & danah boyd, I Tweet Honestly, I (1949); Feiner v. New York, 340 U.S. 315 (1951).
Tweet Passionately: Twitter Users, Context Collapse, 261 Matsuda, supra note 7, at 2338; see Post, Racist
and the Imagined Audience, 13 Data & Society 114 Speech, supra note 40, at 271–77.
(2010). 262 Delgado, supra note 6, at 136–37.
255 Compare R.A.V., 505 U.S. 377 to Virginia v. Black, 538 263 Id. at 137.
U.S. 343 (2003). 264 Id. at 139–40.
26
Andrew F. Sellars
Growing scholarship and attention is placed on 4 - The Speaker Intends Harm or Bad
the structural harms of such hateful treatment, Activity
and how feelings of diminished expectation or Many definitions look to whether a speaker of
self-worth can pass through generations from hate speech intends some harm or other bad
parents to children. Indeed, the harms of racist
265
activity, but what exactly the speaker should
speech, sexist speech, anti-Semitic speech, and intend to constitute “hate speech” is subject to
other forms of harmful speech may be difficult dispute.
to analyze as a universal matter, given the spe-
cific contexts of each. I doubt that any sincere
266
Some definitions use a non-physical framework.
scholar would argue that these harms do not ex- Under Delgado the speaker should have an in-
ist, though they may well argue that some or all tent to demean, with Ward this was an intent
270
of these harms are necessary in order to protect to “vilify, humiliate, or incite hatred,” with 271
the free speech principles at stake. Moran this is an intent to “promote hatred.” 272
Matsuda does not include intent as part of her

Massey looks almost exclusively to harm as the definition, but explains later that speech should
basis of his assessment of hate speech. This is 267
be actionable if it “is, and is intended as, perse-
effective for Massey’s purposes of discussion, cutorial, hateful, and degrading.” The statute
273
but it does not elucidate clear standards for in Beauharnais, coming from defamation law,
identification without a great deal of consider- likely had a tacit intent component. Canada’s 274
ation and context, and is very ill-suited as a defi- law, similarly, looks to speech that willfully “pro-
nition to be used for legal sanction, as it has no motes hatred.” The Rabat Plan of Action iden-
275
consideration of culpability. Most instead con- tifies an intent to cause harm as a key trait for
sider harms along with other factors. For exam- Article 20 of the ICCPR. 276
ple, the statute in Beauharnais punished those

who, among other things, published speech that With others the intended harm needs to be more
exposed a member of a group to “contempt, de- physical. Twitter targets conduct that “pro-
rision, or obloquy.” 268
motes violence” or “directly attacks” a group,
suggesting an underlying intent. Marwick and
277
The more a definition relies solely on harms, the Miller make look to an intent to promote “hatred,
more it risks sweeping in cases that would likely violence or resentment.” Others have intent as
278
give pause to academics, lawyers, and online an optional component. YouTube makes intent a
platform moderators alike. Many forms of high- component of their assessment, but not a man-
ly important speech for self-governance can datory one. The United Kingdom does as well
279
cause discomfort at best and harms at worst. — the speaker can either intend to stir up racial
A politician expressing indignation at governing hatred, or the circumstances can be such that it
principles like diversity or equality is import- is likely to be stirred up. Facebook looks to in-
280
ant to hear and to know, and yet can certainly tent when determining whether to block groups
cause harm to marginalized groups. Many of dedicated to hate speech, but intent is not part
269
the current debates on college campuses about of their general definition.

diversity, tolerance, and how universities should
respond to confrontational topics and ideas are Intent seems to be the element that many point
highly important discussions that may cause
harm to the speakers and listeners involved. Be- 270 Delgado, supra note 6, at 179.
fore one would consider such discussion “hate 271 Ward, supra note 151, at 765.
speech,” I would hope that one would look to 272 Moran, supra note 98, at 1430.
other aggravating factors. 273 Matsuda, supra note 7, at 2358.
274 See supra notes 21–30 and accompanying text.
275 Canada Criminal Code § 319(3).
265 Id. at 147. 276 Rabat Plan of Action ¶ 22.
266 Matsuda, supra note 7, at 2331. 277 See The Twitter Rules, supra note 217,
267 Massey, supra note 129, at 105 n.2. 278 Marwick & Miller, supra note 163, at 17.
268 Beauharnais, 343 U.S. at 251. 279 See supra notes 209–15 and accompanying text.
269 Matsuda, supra note 7, at 2334. 280 Public Order Act 1986 § 18(1) (U.K.).
27
Andrew F. Sellars
to when trying to work a definition that avoids ada both look to speech that incited a “breach
the harder questions under speech theory. How of the peace or riots.” Some, like the EU Frame-
287
can one truly say they are trying to move the work Decision on racism and xenophobia, look
needle in the marketplace of ideas, or submit to both. And some, like the Rabat Plan of Ac-
288
genuine proposals for self-governance, if they tion, contemplate simply inciting further harm
intend only to hurt someone or some group? An 281
as a relevant factor, suggesting by implication
intent to harm does not answer the free speech that the harm of speaking is not enough. 289
objections raised by Bollinger, but intent is a

282
piece of many existing definitions of actionable “Incitement” is a term of art under Ameri-
speech in the United States, so one assumes
283
can speech law, drawn from the famous hate
that it can help in this environment as well. Nev- speech case Brandenburg v. Ohio, and tends to
ertheless, discerning intent will be extremely dif- only apply when the incitement is imminent, or
ficult to do well when studying online speech. almost inevitable. It is not at all clear that the
290
The speaker’s intent may be obscured, denied, definitions here seek to put such a strong quali-
or simply not disclosed. Speakers may lie about fier on the term. Courts looking at online speech
their intent to others; they may even by lying have already struggled to decide how concepts
about their intent to themselves. That this fac- like imminence in incitement doctrine should be
tor can be so crucial to balancing speech prin- understood with regard to online speech. 291
ciples and yet so hard to identify may be one

of the more frustrating aspects of studying hate Online researchers should keep in mind that it is
speech online. not always apparent what speech does or does
not incite, especially when considering concepts
5 - The Speech Incites Bad Actions like inciting hatred, which may be undetectable
Beyond the Speech Itself or preconditioned in the audience. Even when
Perhaps also a reflection of a tendency towards the definition looks to violence, the Innocence
avoiding the free speech conflicts with identi- of Muslims example teaches, even when one can
fication of “hate speech,” many definitions re- establish a nexus from speech to violence, there
quire that the speaker incite some other conse- may be many intervening steps and intervening
quence as a result of the speech. Here again, speakers along the way. When examining a
292
however, what specifically one should be incit- particular incident, like the riots in response to
ing is subject to debate. 284 the Innocence of Muslims video, careful forensic
work should be done to track how the speech is
Some definitions look for incitement of other altered, recontextualized, and appropriated as
non-physical reactions. Several definitions look it goes from the initial speaker to the intended
to incitement of “hatred,” which could be seen
285 audience. 293
as “harm” in element 3, or could be seen as an

independent bad reaction. Other definitions 287 Beauharnais, 343 U.S. at 251; Canada Criminal Code
look to violence. Benesch focuses on murder, eth- § 319(1). It should be noted that Supreme Court de-
nic cleansing, and other atrocities, and looks to cisions subsequent to Beauharnais also severely lim-
speech that incites that specifically. The stat- ited punishment of speech on such vague grounds
286
ute at issue in Beauharnais and the law in Can- as speech that may cause a “breach of the peace.”
Cox v. Louisiana, 379 U.S. 536, 551–52 (1965).
281 See, e.g., Citron, supra note 9. 288 See EU Framework Decision on Racism and Xenopho-
282 See supra notes 89–92 and accompanying text. bia, supra note 189.
283 See Brandenburg, 395 U.S. at 447–48 (incitement 289 Rabat Plan of Action ¶ 22.
must be “directed to” producing imminent lawless 290 Brandenburg, 395 U.S. 447–48.
action); Virginia v. Black, 538 U.S. 343; Elonis, 135 S. 291 See Planned Parenthood v. Am. Coalition of Life
Ct. at 2013 (2015) (as a statutory matter, threats are Activists, 290 F.3d 1058 (9th Cir. 2002).
only actionable under federal law if the defendant 292 See supra notes 123–25 and accompanying text.
intends to threaten). 293 See also Susan Benesch, Charlie the Freethinker:
284 See Herz & Molnar, supra note 16, at 1, 4. Religion, Blasphemy, and Decent Controversy, 10
285 See, e.g., Ward, supra note 151, at 765; YouTube Religion & Hum. Rights 244, 252 (2015) (noting that
Community Guidelines, supra note 211. in two famous cases where religiously provocative
286 Benesch, supra note 156. speech lead to physical violence, “ill-intentioned
28
Andrew F. Sellars
6 - The Speech is Either Public or lating speech “at the home” is a step too far for
Directed at a Member of the Group many regulators of speech, as it feels more like
Several definitions look to audience, though the regulating freedom of thought itself. 300
audience in question often depends whether the

defining party seeks to examine public spread But for those that are hoping to study the larger
of hatred, or the effect of an insult hurled at an effects of hate speech beyond what can be reg-
individual. Some, like Benesch, look for public ulated, it may be wise to leave in private speech
declarations of hatred and the social harms that is directed to those outside of the targeted
that flow therefrom, and some, like Delgado’s, group. First, if one is looking for where to inter-
294
target the personal attack. Canada’s and Aus- vene in hate speech to mitigate its lasting ef-
295
tralia’s hate speech laws make clear that only fect, the speech that happens privately may be
“public” statements are subject to punishment, the best target. Ingrained racism spread within
though it would seem that public statements to the privacy of one’s home or social circle may
a single individual of the protected group would be the most calcified and hardest sentiment to
be encompassed under these definitions. Mar- dislodge. Second, the concepts of “public” vs.
296
wick and Miller also seem to cover both spaces, “private” speech, if they were clear before the
by requiring that the subject “experience” the Internet, are even more confused today. Some
harm of the speaker’s message, which could be “private” conversations happen on public plat-
either direct or indirect. On the other hand, forms such as Twitter, where other users can
297
Matsuda evades this somewhat; she requires not only see what was said, but amplify those
that the message be “directed against a his- conversations to others, despite not being a
torically oppressed group,” but it is not clear if party to what was initially discussed, just as a
that is a reference toward its content or its au- researcher may be accidentally lead through
dience. 298 “context collapse” to see a racial slur where
neither the sender nor recipient perceived it as
Parekh contemplates a public audience for his such, a speaker may find that what was meant
definition for a hate speech crime, and specif- to be only a private conversation can be brought
ically that the speech cause the public to view public through the architecture of the speaker’s
the target group “as an undesirable presence online platform. 301
and a legitimate object of hostility.” Audience

299
reactions such as these may be easier to ob- 7- The Context Makes Violent
serve and detect online, but it is a bit of a puzzle Response Possible
as to why a criminal defendant under Parekh’s Of the various academic definitions offered,
law would be not guilty if he or she delivered Benesch’s is the one that most directly address-
remarks to an unmoved audience. es the question of context at length. Benesch
looks to factors such as the power of the speak-
The missing group from this trait is speech that er, the receptiveness of the audience, and the
is neither public nor targeted at the member of history of violence in the area where the speech
the group. The absence of this type of speech takes place. If the focus is on speech that is
302
from the definitions above may be a reflection actually likely to catalyze physical harm, this
of where many see the limits of governance in contextual analysis seems critical.
a liberal democracy. With few exceptions, regu-
figures relentlessly publicized the mocking content, Others reference this indirectly. Delgado’s use
in ways that they knew were likely to [catalyze] of a “reasonable person” standard for identify-
violence, and among audiences they knew were ing the remark as a racial insult is a reference
likely to react with violence”). to a standard concept in tort and criminal law,
294 See Benesch, supra note 156.
295 See Delgado, supra note 6, at 179. 300 See, e.g., Stanley v. Georgia, 394 U.S. 557 (1969) (ob-
296 Canada Criminal Code § 319; Racial Discrimination scenity possession in the home is not punishable);
Act 1975 § 18C(2). but see Osborne v. Ohio, 495 U.S. 103 (1990) (refus-
297 Marwick & Miller, supra note 163, at 17. ing to extend Stanley to child pornography).
298 Matsuda, supra note 7, at 2358. 301 See Marwick & boyd, supra note 255.
299 Parekh, supra note 116, at 40–41. 302 Benesch, supra note 196.
29
Andrew F. Sellars
where the liability or guilt of a person is made community most likely to be offended by the
in reference to what a reasonable person would message.” The same speech principles that
310
do or understand in those circumstances. In the limited obscenity to the places where it is ac-
analogous space of workplace harassment, tually viewed as obscene can inform some of
the precise role of social context for a “reason- the speech-oriented limitations to hate speech
able person” test is still debated. The United
303
observation. As noted a few times above, it is im-
Kingdom looks to the “circumstances” around portant to bear in mind the limits of what one
speech to see if “racial hatred is likely to be can see without the benefit of local context,
stirred up thereby.” The Rabat Plan of Action
304
though it is also important to note how speech
advises that EU member states look to “the can leave its original context and transform into
social and political context,” the status of the something else that may in fact be worthy of
speaker, and the size of the audience. 305
study, especially on a global communications
platform.
If a scholar seeks to look at a legal attempt to
codify context in law, they may be well-served 8 - The Speech Has No Redeeming
by looking to the example set by the United Purpose
States approach to obscenity. American ob- It would seem as though every definition is
scenity law famously has a contextual element aware of its own limitations, and several choose
to it, and how it has been applied in online en- to address this by trying to excise out the “good”
vironments presents an interesting analogy for speech that may have fallen into the original
those who seek to study or regulate hate speech definition. Marwick and Miller state their gen-
online. Starting with the Supreme Court’s deci- eral definition of hate speech as “speech that
sion in Roth v. California, and cemented with
306
carries no meaning other than hatred towards
the Court’s Miller v. California, the test for deter- a particular minority” A version of this can also
311
mining whether a given work is obscene requires be seen in Ward’s definition. This lack of “good”
312
proof that, among other things, a person ap- purposes is a repeat theme in speech laws, in-
plying “contemporary community standards” cluding with obscenity and many definitions
313
would view the work as prurient. The court in

307
of actionable harassment. The difficulties in
314
Miller directly avoided a nationwide standard using a “no legitimate purpose” test for speech
because, as they put it “our Nation is simply too in the abstract, however, are well documented,
big and too diverse for this Court to reasonably as they involve deep subjective assessments on
expect that such standards could be articulated the part of the adjudicator, and the complex na-
for all 50 States in a single formulation.” With 308
ture of most speech would require adjudicators
the advent of the Internet, the Supreme Court to either overlook this nuance punish speech
was again asked to adopt a nationwide stan- with some redeeming purpose, or embrace the
dard, because, as advocates put it at the time, complexity and thus exclude nearly all speech
“[o]nce a provider posts its content on the Inter- from the definition. Twitter tries to engage with
315
net, it cannot prevent that content from entering

any community.” The Court again refused, as
309
310 Id. at 877–78.
they felt the harm to speech too great if “any 311 Marwick & Miller, supra note 163, at 16; see also id.
communication available to a nation wide au- at 17 (definitions should look to whether a speaker
dience will be judged by the standards of the intends “only to promote hatred, violence, or resent-
ment” (emphasis added).
303 See Melissa K. Hughes, Through the Looking Glass: 312 See Ward, supra note 151, at 766 (noting that speech
Racial Jokes, Social Context, and the Reasonable qualifying as hate speech should be “so virulent”
Person in Hostile Work Environment Analysis, 76 S. that an observer would have difficulty separating
Cal. L. Rev. 1437 (2003). the attack from the message).
304 Public Order Act 1986 § 18(1) (U.K.). 313 Miller, 413 U.S. 15, 24 (1973).
305 Rabat Plan of Action ¶ 22. 314 See, e.g., Del. Code Ann. tit. 11, § 1311(a)(1); Fla. Stat.
306 354 U.S. 476 (1957). § 784.048(1)(d).
307 413 U.S. 15 (1973). 315 See Eugene Volokh, One-To-One Speech vs. One-To-
308 Id. at 30. Many Speech Criminal Harassment Laws, and “Cy-
309 Reno, 521 U.S. at 853 (quoting ACLU v. Reno, 929 F. berstalking,” 107 Northwestern L. Rev. 731, 776–80
Supp. 824, 844 (E.D. Pa. 1996)). (2013).
30
Andrew F. Sellars
this complexity of language by targeting for re- activists” platforms, feels different in kind. The 321
moval groups whose “primary purpose” is incit- Southern Poverty Law Center has gone as far as
ing harm, though this determination — like most to identify certain “men’s rights” platforms as
in online platform regulation — is a subjective hate sites. 322
one made by Twitter alone. 316
Matsuda notes the tensions presented by this

Canada tries to solve this puzzle by specifical- contradiction, but ultimately finds that angry or
ly exempting certain types of speech from its hateful speech by a “subjugated group” should
overall definition, including speech expressing be seen as “a victim’s struggle for self-identity
“good faith” opinions on a religious subject, in response to racism,” and therefore, one pre-
speech that is true, or made in the public in- sumes, not hate speech in many cases. That
terest. The State of Victoria, Australia does a
317
said, these power considerations could be flu-
similar balancing act. This compromise may
318
id — Matsuda also notes that should the victim
split the baby. Proponents of hate speech regu- group change power and be placed “in a dom-
lation may be legitimately concerned that hate inant or equalized position,” the special protec-
speech will simply disguise itself as scientif- tion for such speech against sanction should be
ic or religious analysis and thus avoid regula- lost. That these standards can evolve will force
323
tion. Opponents may worry that qualifications scholars to stay on their toes as they observe
like “good faith” will wind up playing the same communities.
role as “good motives” did in Beauharnais — a
judge will make a subjective judgment of good- I also worry that I have not fully addressed how
ness, and society will be left without a princi- to handle harmful uses of another’s speech, both
pled standard. Triaging through observation of when done deliberately and when the dissemi-
speech online to identify “good faith” offerings nator subconsciously uses another’s speech to
will also have many of the same concerns as at- substantiate a point that the original speech
tempts to discern intent above. does not support. For an example of the lat-
ter, an Islamophobe may subconsciously gloss
Continuing Questions over dozens of news stories about hate crimes
Any assessment or classification inherently has against Muslims, find the one story where a
within it value choices, prioritizations, and omit- Muslim person was the aggressor, and post that
ted elements. At the end of her landmark article story online as justification of their bigotry. I
319
on defining hate speech, Matsuda noted several do not believe punishing that individual would
types of more nuanced or troubling speech that change much of anything. I would hope that
evaded her definition, and I see similar limita- counterspeech and discourse can solve these
320
tions with my own framework. sorts of problems, but I worry for the efficacy
of the marketplace in such irrational spaces, for
One of the hard remaining questions from the the reasons critics have noted above. 324
definition framework above, or at least how

such framework can be put to practice, is how to And finally, I worry that in trying to deal with
deal with the relative power of different groups the hard tensions between speech theory and
in different online spaces. The definitions above harmful speech, we may find ourselves repeat-
largely don’t account for the natural feeling of ing many of the same biases and errors that
most scholars that, for example, speech that have led many to openly question the effica-
promotes the power of women online should not
be thought of as a form of hate speech, but the 321 I would suspect that only select content from these
same empowerment message directed toward platforms would be a form of “hate speech” subject
to the criteria above, but I also suspect that scholars
men, at least as it is seen today in “men’s rights
would more easily identify that speech when it is a
man speaking of a woman, instead of vice-versa.
316 See The Twitter Rules, supra note 217. 322 Misogyny: The Sites, S. Poverty L. Ctr. (March 1,
317 Canada Criminal Code § 319(3). 2012), https://www.splcenter.org/fighting-hate/intel-
318 See Racial and Religious Tolerance Act 2001 § 11. ligence-report/2012/misogyny-sites.
319 Moran, supra note 98, at 1428. 323 Matsuda, supra note 7, at 2361–62.
320 See generally Matsuda, supra note 7, at 2361–70. 324 See supra notes 70–80 and accompanying text.
31
Andrew F. Sellars
cy of free speech theory itself. Regulators are

incredibly anxious to respond to hate speech,
and they will look to scholarship to justify their
decisions. As scholars, we owe it to those regu-
lators to give a fair assessment. By now I hope it
is clear that hate speech is highly context-sen-
sitive, and many studies who look to how these
issues play out on “the Internet,” forget that
it is not one grand corpus, where everyone is
speaking to everyone. No one person sees the
Internet; we only see what our framework and
perspective leads us to see. The technological
325
affordances of the Internet may mean that con-

tent can slip between communities and spread
the way no speech has ever had before, but that
does not mean that each new audience will be
looking at these issues in the same way.
Conclusion
In the above sections I have labored through
many definitions and examined their theoretical
shortcomings. All of this may feel as though I am
trying to work scholars and regulators to such a
confused place that they refuse to study hate
speech altogether. This is not my intent at all.
I only seek to illustrate that this is difficult, it is
difficult for good reasons in light of the compet-
ing interests at play, and we should approach
these questions in an intelligent manner. A per-
son who says they have an easy solution to the
problem of hate speech, or even how to observe
and document hate speech, is simply not think-
ing hard enough. This issue is a critical one, and
we must approach it critically. The definition of
hate speech in a study or regulatory environ-
ment may be the most important part of the
project’s design. Looking through the eight traits
identified above, and using several of them in
designing an identification system will help en-
sure that the research targets what most would
agree is the type of speech that is causing us
so much concern, and the speech that deserves
the most rigorous study.
325 See, e.g., Ethan Zuckerman, Rewire: Digital Cos-

mopolitans in the Age of Connection (2013).
32

Defining Hate Speech PDF

Uploaded by

Copyright:

Available Formats

Defining Hate Speech PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Defining Hate Speech PDF

Uploaded by

Copyright:

Available Formats

Research Publication No. 2016-20

Defining Hate Speech

This paper can be downloaded without charge at:

23 Everett Street • Second Floor • Cambridge, Massachusetts 02138

Electronic copy available at: https://ssrn.com/abstract=2882244

Suggested Citation: Sellars, Andrew F. Defining Hate Speech (December 8,

Harmful Speech Online Project

5 The Theoretical Context of Hate Speech

14 Attempts to Define Hate Speech

24 Emerging Themes and Continuing Questions

Introduction with theories of free expression, surprisingly lit-

Part II of the essay looks to existing attempts to

of hate speech for further study.

ing — a concept at the root of any decent sys-

as a “form of criminal libel” law. 27

What if he had targeted an individual instead of

— allowing the (very small) group of individuals

the border between the two factions. 48

ishment of speech directly. Because these laws

1644. Frederick Siebert said this of Milton’s the-

Under this framework Joseph Beauharnais

matter. Both social science and informal per-

ry can justify some regulation of hate speech

Democratic Self-Governance one can find reasons both to tolerate or to reg-

suring greater tolerance. One may justify hate

aggressively. Bollinger notes that the advan-

nation of intolerance of other ideas or people,

speech as a topic of public concern. The con-

of a book or film to obtain evidence of illegal

value exposure to a variety of opinions can be

Looking to content of speech, epithets and in-

Coded speech can be especially corrosive to a

the term are daunting to summarize, but by re-

of Muslims video in 2012 is a powerful example.

with intent to incite hatred? Sadek for mislead-

ly with critics who raised objections under the

person would recognize as a racial insult.” Un- 135

criminal law ) could identify the speech as a

of magnitude in Ward’s definition, noting that

for complex social and cultural phenomena. So,

carries. specifically targeted toward incidents of mass

defined hate speech as “any form of expression 153 See id.

145 Id. Speech, Dangerous Speech Project (Feb. 23,

features” to form a definition of hate speech:

these definitions to frame his argument. 170

Canada also separately prohibits those who

dissemination or presentation “which publi- Canada also specifically punishes anyone

appear to be two independent ways one can

order to be actionable. Interestingly, the State

for private communication, if done to “incite[]

severe enough to warrant punishment under Ar-

for scholars. fore unnecessary — and perhaps unwise — for

ticulating rules for hate speech. Technology af-

Turning to specific platforms, the social video

uals or groups based on race or ethnic origin, re-

also makes clear that it does not allow accounts

content moderation team shows a much more

overt forms of dehumanizing content.

230 See, e.g., Christine Pao et al., Confidence Scoring