Talk:Community health initiative/User reporting system consultation 2019

Thoughts on the consultation structure

I personally think that the process will lead to a well-rounded corpus of feedback and information that will help our team design and build the best-possible reporting system for Wikimedia wikis! I'm especially looking forward to the focus groups who will be able to dive deep into certain topics. — Trevor Bolliger, WMF Product Manager 🗨 21:56, 19 February 2019 (UTC)

The issue with enforcing existing rules

Hi, I just wanted to leave a short note to (again) thank you for looking into the issue of harassment. From my point of view, the main issue in my community (German-language Wikipedia) seems to be that that existing rules against harassment are simply not enforced because (1) the admins fear backlash from toxic users, and because (2) admins tend to be users who have a very 'laissez-faire' attitude towards harassment in the first place. I am aware that this project may not address these issues directly, but maybe there's room to do so further down the road. Kind regards, --Gnom (talk) Let's make Wikipedia green! 16:55, 10 March 2019 (UTC)

Hello Gnom, thank you for sharing your views. My perspective (which is informed by my own observations, reading the comments from other contributors for over a decade, and the research collected by the WMF Community health initiative over the past 3 or 4 years) the situations as you describe it is made worse by not having a well functioning "user reporting system."

In the current system, it take a whole of effort and insider knowledge to effectively present a case. The lack of tracking and the way that cases are archived means that a report is often viewed as one off event or a new situation instead of a continuing pattern of problematic conduct. A system with:

1) a better routing system

2) a user friendly form that walks you through how to make a high quality report

3) clear paths for escalation

4) a method for tracking

would go a long way towards putting the a good report in the appropriate channel for action to be taken.

Every day many volunteer administrators, stewards, other functionaries and experienced users protect the wikis from a steady steam of abuse. The system works well in many cases and that needs to be recognized and retained. But improved tools and workflows from a better user reporting system will make it easier for a target to file a report. And it can open the door for more people to help moderate issues, and perhaps bring fresh opinions and approaches, too.

In the next week, I will be working with WMF Anti-Harassment Tools team staff to bring some preliminary ideas for products into the discussion. The success of this project depends on hearing the voices of users from many wikis who are looking at it from a variety of of perspectives. I hope that you will stay around to comment and invite other Wikimedians to join the discussion. Cheers, SPoore (WMF) Strategist, Community health initiative (talk) 15:35, 3 April 2019 (UTC)

Formal rules and harassment

I wonder if this will spiral into a dispute whether harassment should be about following the rules, and some very strict reporting system, or it should be about what users feel is harassment. Strict rules could be nice for admins, but they tend to be interpreted by the letter and will not give any real guidance for cases where users methodically push limits over long time.

One thing I have seen in some cases are conflicts between users where one of the users has backing from a group of users. In those cases the user from the larger group tend to win out, even if the case as such should be rather straight forward. It does not happen very often, but it does happen and it is very detrimental for the community. Usually conflicts in the community are handled by admins, but I wonder if this is wrong. Admins claim they are objective, but my opinion is that they are highly biased. Perhaps we could use an anonymous and random jury, where each member just vote on the outcome. The random jury should be picked from users that has no tendency to appear together with the users involved in the dispute.

A technical solution could be to wrap a conflict/dispute in special tags, which appends a short statement about not making any followup, and a judgement from the jury about the current verdict. Such tag could be added by anyone, but then only removed by admins, unless the tags are empty. When they are added the content should only be removed by the persons involved, and only by undoing their own contributions. When a conflict is marked it should not be possible to add new content inside the tags. That makes it possible to backtrack out of the conflict for the involved persons, and stop further escalation. Note that voting on the outcome must either be on content as it is when the tags were inserted, or for each revision intersecting with the content. I believe the former would be the correct one. — Jeblad 18:47, 2 April 2019 (UTC)

The proposed technical solution will break down in a number of cases, not sure if there are any easy workarounds.

The core problem is that harassment can be ongoing for some time, and it can be interspersed with other discussions, and even good dialog with the harassers. So there are a number of possibly bad interactions, which may go either way. A case could then comprise a lot of contributions. — Jeblad 00:48, 3 April 2019 (UTC)

Hello Jeblad, I agree that this consultation will need to consider who handles the different types of reports of harassment. Last year during a session at Wikimania, the Wikimedia Foundation Anti-Harassment Tools team lead a round table discussion called 'Building a better harassment reporting system' to determine the various ways that reports about harassment and abuse are taken by volunteers acting in different roles. While to my knowledge no one mentioned a random jury system, there does seem to be a wide variety of ways that "cases" get reported and actioned. I'm interested in learning from more people on more wikis about which methods of handling might be more open for abuse such as you described? SPoore (WMF) Strategist, Community health initiative (talk) 00:54, 3 April 2019 (UTC)

As I replied to my own post (you get really high quality discussions when you are replying to your own posts) the proposed solution will break down. The problem is in how to mark the troublesome discussion, not a random jury system. A random jury system sounds kind of cool but are nothing more than a random selection from known users. It is possible to pick users from a more coherent group, but that has a risk of getting a biased group.

What you probably want is a system where it is easy to report harassment, but also very easy to verify whether the report has any merit. It is extremely common to find users claiming harassment, when they are in fact being asked to follow established rules. That said it is also very easy to find admins that oversteps rules when someone points out obvious mistakes. You can't take any of them at face value, there must be some way to verify whether the claims have any merit. — Jeblad 01:26, 3 April 2019 (UTC)

As a minor note, I have a vague recollection that a random jury system was proposed at a session in Montréal, perhaps [this one]. It was not an in-depth discussion, but it stuck in my mind because I thought it was something worth considering. Off the top of my head, a truly random jury composed of all potential editors won't work—it needs to be a subset of all editors with some form of vetting if nothing more than minimum contributions. One obvious challenge with a jury approach is the potential need for confidentiality.--Sphilbrick (talk) 21:52, 17 June 2019 (UTC)

Thank-like reporting

We have an open system of contributions, with users that are highly opinionated, and how do we stop harassment without starting a blame game? We have a highly volatile community, and somehow we must avoid escalating conflicts.

What if we could report harassment the same way we thank a user for a contribution, but without the public thank, and without the reported person seeing who is the reportee. As long as only one person reports the user nothing more happen, but at some level a warning goes off and a special user group gets a notification. The same could happen if a user gets several reports over a given time or number of contributions. The reports will still not be visible to ordinary users, only to the specific group assigned the task of handling reports, and possibly to the reportee.

Because the reported users will be warned they will probably behave more responsible, and because they don't know who reported them they must behave responsible towards all. This will probably lead to individual self justice. If only logged in autoconfirmed users can report users the spam/troll problem should be fairly small. If the log entries can be deleted, then cleanup can be done if someone goes ballistic.

There are probably several types of reports, like harassment, threats, improper behavior, copyvio, and even overly use of rollback in edit wars. Perhaps they can be lumped together in a single type of "report", like "thank" is now a single type. — Jeblad 19:35, 3 April 2019 (UTC)

I think this approach has some merit and is worth exploring. I can see some potential for abuse which might be addressed with a throttle but it's worth considering.--Sphilbrick (talk) 21:57, 17 June 2019 (UTC)

Approach to decision making

As noted on the project page, there may be difficult decisions to make about about which software features to prioritize and how to allocate resources toward other aspects of a user reporting system. When it comes time to make a decision about which features to build, options will be weighed by the following criteria:

• Which option(s) most aligns with Wikimedia movement values?
• Which option(s) is most in alignment with Strategic Direction of Knowledge Equity?
• Which option(s) most aligns with the goal "to build a new harassment reporting system that produces higher quality reports that can be successfully processed and does not further alienate victims of harassment."
• Which option will result in more accessible user experience, for anyone on any device?
• Which option will result in a more sustainable product that will be resilient to changing technologies, evolving use cases, and user expectations?
• Which option(s) do not introduce undue risk for achieving our project goals?

Will these criteria lead to the best decisions about which products to prioritize? Is the meaning of them clear and spelled out in a way that is understandable? SPoore (WMF) Strategist, Community health initiative (talk) 00:22, 3 April 2019 (UTC)

The apparent conflation of "reporting systems" and harassment

I read over the summary and also looked closely at the flow chart. The first thing that struck me is that the overwhelming majority of the reporting systems identified, and the overwhelming majority of reports made by both the formal and informal systems, have no relation to harassment in even its loosest definition. It is very unclear to me why they are even included in the "harassment" rubric (except possibly to say that they are obviously not designed for nor intended to address harassment). There is also a pretty apparent conflation of any type of inappropriate user behaviour with harassment.

I find this just plain wrong.

I do not in any way deny that harassment has occurred on and in relation to participation in Wikimedia projects; indeed, I've been at the receiving end of various types of harassment and discriminatory behaviour on a number of occasions, as recently as last month. Given the various roles I do hold or have held on the project, I have seen some undoubtedly harassing and/or discriminatory behaviour on more occasions than I can count; I've also done my best to help users who have unintentionally opened themselves up to potential harassment (most commonly from off-wiki parties). I *do* understand the problem. I just don't think that conflating every type of problem and every type of problem resolution system with this particular, real, serious problem will lead to useful, community supported outcomes. Comparing what happens at the 3-revert noticeboard with harassment management just doesn't cut it, and it's kind of embarrassing that you paid a lot of money for that report. Risker (talk) 21:33, 4 April 2019 (UTC)

Hello Risker,

Thank you for reading over the materials and leaving feedback. I’m sorry to hear that you have recently experienced a serious episode of harassment. Let me know if there are any additional actions that the Foundation’s Trust & Safety team can take to assist you will dealing with the situation.

As well, I appreciate the work that you do to help less experienced contributors mitigate harassment. The Wikimedia movement depends on volunteers like you to help our communities deal with harassment. The User reporting system project will need to draw on the experience of functionaries and and members of community governance groups – including stewards, admins, checkusers, oversighters, and Arbitration Committees – in order to make well informed product decisions. As a current and former member of several of these groups, I value your insight.

As you know, Wikimedia wikis differ from most websites because most user dispute reports are handled by volunteers instead of being channeled directly to an in-house team of employees as is the more common way. The Community health initiative and Trust and Safety team have undertaken several internal and external research projects to learn how Trust and Safety type issues are managed on the Foundation platform (wikis) and to identify potential areas for improvement. As you note, the results are available for review on the User reporting system consultation page. We’ve made them available to the Wikimedia movement to inform decision making by all stakeholders for the user reporting system project and beyond. WMF Researcher Claudia Lo is the key contact for this work.

From the current research and preliminary community consultations I anticipate that routing and escalation paths for the reporting system will be key considerations when defining the scope of the features to build. Related to this, I understand you to be saying that combining different types/levels of user conduct reports in the user reporting system could diminish the effectiveness of the system. (I hope that is not too much of an oversimplification.) This is an important consideration and will need to evaluated along with 1) ease of access to the system for new users, 2) the potential to overwhelm volunteers with frivolous or abusive reports, 3) better tracking and archiving of report to more effectively identify long term abuse.

In the next few weeks, after notifications of the consultation have happen in the broad global movement, preliminary ideas for products will be added to the consultation. I'll ping you then and hope that you will return to offer your thought about the more concrete ideas. SPoore (WMF) Strategist, Community health initiative (talk) 21:52, 5 April 2019 (UTC)

ArbCom as a private reporting system

On enwiki, currently, the Arbitration Committee is the only body that can take reports related to off-wiki information or other things too private to be posted on-wiki. We routinely do take those reports, and in many cases, act upon them. Given the call for more private reporting, it was odd this wasn't a focus of the reporting system summary. ~ Rob₁₃^Talk 15:58, 7 April 2019 (UTC)

Hi Rob₁₃, thank you for pointing out the significant role that the Arbitration Committee on English Wikipedia has in managing cases of off wiki harassment or other sensitive information. Having a dedicated body like Arbitration Committee designated to receive the information might be enhanced if there was a clearer routing to them. This is something we'll want to discuss further when the essential components of the reporting system are discussed in a few weeks. SPoore (WMF) Strategist, Community health initiative (talk) 16:21, 8 April 2019 (UTC)

"Formal private reporting systems" and ArbCom

I have two questions about on the enwiki reporting system report, from my perspective as a member of enwiki's Arbitration Committee (ArbCom):

1. How was it decided that emails to ArbCom are a "very severe" and "hard to find" reporting system? Users are advised to contact ArbCom on several relevant help pages, such as en:Wikipedia:Harassment#Dealing with harassment and en:Wikipedia:Dispute resolution#Sensitive issues and functionary actions.
2. The major recommendation of the report appears to be the creation of "new formal private reporting systems". How will these be made compatible with the longstanding consensus on enwiki that "matters unsuitable for public discussion" should be referred to ArbCom?

Thanks. @SPoore (WMF) and CLo (WMF): Joe Roe (talk) 16:23, 7 April 2019 (UTC)

Hello Joe Roe, thank you for your questions. I want to assure you that no decisions have been made that are incompatible with pre-existing governance on local wikis. Syncing with Arbitration Committee's on English Wikipedia and other Foundation wikis is essential to the success of any user reporting system. We'll continue to reach out to ArbCom's as the consultation progresses to ensure that they participate at crucial times in discussions. @CLo (WMF): starting a project to better understand the expectations of new(er) users when they want to make a report. She can elaborate more it and also answer in more detail about the characterization of emails to ArbCom. SPoore (WMF) Strategist, Community health initiative (talk) 16:40, 8 April 2019 (UTC)

Thanks SPoore (WMF). I hope CLo (WMF) will be able to answer my questions more directly. It is disappointing to see that the development of a new "private reporting system" was announced to the New York Times before ArbCom, the body responsible for private reports on the English Wikipedia, was consulted about it. I can't help but feel this undermines your assurance that the new system will respect the existing community consensus on enwiki. Joe Roe (talk) 19:59, 8 April 2019 (UTC)

Hello Joe Roe, I'll go over your two points in order:

1. From my own study of policy pages, looking at answers on the Village Pump as well as the Teahouse, and other places where we could reasonably expect a newer editor to go in search of help, I found that ArbCom is relatively hard to find. Additionally, there seems to be a fair bit of social weight to opening up an ArbCom case, and it's not presented as something to take lightly. On top of all of this, since reports for other common disputes such as vandalism are almost entirely public processes, switching over to a very closed reporting method such as ArbCom represents a significant shift in how one might be expected to make a report. We occasionally see things like users not realizing that certain requests for administrator action, such as asking for the removal of sensitive information, should be done privately via email despite this being stated at the top of the same noticeboards they're posting to, and I did not think that we should expect this to necessarily be different for harassment cases.
2. The definition of "formal private reporting system" I was working with for that particular report doesn't mean a new closed system of reports. Rather, we know from talking to administrators that there's already an informal way to report disagreements privately, and that's emailing or PMing administrators directly. Of course, for this to happen, the user making these informal reports has to already be familiar with the administrators and be comfortable enough with the community to message them individually, which is a significant barrier for newcomers. A "formal" private reporting system–that is, one clearly accessible, visible, and purpose-made to receive and direct reports to the appropriate parties, could be a useful tool for tackling issues of harassment and user misconduct.

I hope that this addresses your concerns. Thank you for your questions!—User:CLo (WMF) (talk) 21:34, 8 April 2019 (UTC)

I have to credit that this is one of the more remarkable pieces of magic I've seen. Wikipedia used to have an autonomous community with WMF watching out for legal threats we couldn't handle ourselves. Now "autonomous" has been replaced by "informal" -- we now have an informal community, waiting to be upgraded and replaced by formal administration! Not since some bankers invented the concept of "identity theft" to transform their losses to fraudsters into a supposed failing of whoever had his name written on the phony form, who should ever after be vigilant lest some banker be defrauded, have I seen a magic to rival this. Wnt (talk) 10:47, 17 June 2019 (UTC)

Dealing with harassment leads to further harassment

I think that the problem in the headline needs to be addressed in any new reporting system we are working on. This happens on two levels:

• A user publicly complains that they are harassed. Other community members accuse them of being harmful and accuse them of willing to revenge for harassment. This goes along the lines: "This user published my personal data to harass me [evidence]. Please block them" - "You are not a nice person. You should not make requests like that just because you are looking for revenge. Go back to writing articles instead of trying to block useful editors"
• An administrator reacts to a harassment report by warning or blocking a user. Other community members accuse them of being too strict, as community will lose some contributions because of that block. This gives something like that: "User A has again violated [rule] and harassed user B despite multiple warnings, they are blocked for a long period" - "You admins hate user A because they are telling truth about you admins and your friends. You are again harassing him with all these groundless blocks. Just leave them alone and let them contribute as you admins write bad article while this guy writes good ones."

Both of these patterns mean that (a) users are less willing to report harassment, as they are afraid of being harassed for reporting legitimate cases of harassment, (b) admins are less willing to react to harassment reports as they risk being harassed themselves. On the other side, reporting cannot be completely private either for transparency reasons: our communities have public block logs and warning on talk pages.

I can think of some kind of ticket-like system like Phabricator, with tickets used to report harassment that are both public (i.e. visible to anyone, or at least to registered users) and restricted (i.e. cannot be edited by anyone except administrators and ticket author). This is an early idea, and I do not insist on any specific setup — NickK (talk) 03:07, 8 April 2019 (UTC)

Hello NickK, that you for proving this input with good examples. And also a suggested solution. Please continue to think about this and share more details as they come to you. SPoore (WMF) Strategist, Community health initiative (talk) 20:17, 8 April 2019 (UTC)

This appears to be true in the Fram case under the WMF-run system, more than in the usual harassment cases on Wikipedia I think. I don't actually know that the specific person harassed was the one who filed the winning complaint. Bear in mind also that the dialog above sounds forced because it is -- usually there is some underlying issue, a disagreement of philosophy that leads to many editors honestly believing that the accused did not commit any kind of wikicrime. The harassment policy and enforcement on Wikipedia is very vague, which is what fuels this kind of problem, but having these things entirely secret is guaranteed to make it worse. Wnt (talk) 12:12, 17 June 2019 (UTC)

Tools are only a portion of the solution

Hi all, I am glad to see this focus on harassment is happening. I am encouraged to see that it is supported by research.

I am, however, concerned we are failing to address culture in these plans. Let me explain. First, I am concerned the solutions developed are addressing harassment once it has occurred. I do not see a focus on prevention. Second, many people are posting about cultural and behavioral concerns, but I do not feel the response is one of open listening. Third, while I love what you all are doing, we need solutions for the people who are experiencing these situations. I know you all mention “better tools” was a response from the Community Engagement survey, but the WMF staff made “better tools” an optional answer to the survey questions. The community response wasn’t an unbiased response. It was an optional answer to a question, both designed by the WMF. For example, if people had to rank items, and that was on the list of limited options, yes, it will get chosen. Perhaps if you had “Improve policies and culture around bad behavior” that would have received a lot of response too. I think reflecting on the bias and the balance of the information used to support this project, as well as the feedback on this discussion page, will help guide further advances from the Anti-Harassment team.

Again, I am delighted you are developing software solutions, but I am wondering how you are addressing the culture? Others on this discussion page have brought this up before me, but I feel the response was not one of listening, but of defense of why this project is a valid solution, and not addressing the gaps this project does not fill. This is why I am reframing the question about culture again here, so hopefully it can be addressed. Thank you for your response to this. Best, Jackiekoerner (talk) 15:48, 8 April 2019 (UTC)

Hello Jackiekoerner, I'm glad that you brought this issue to my attention. It is easy to get tunnel vision when working on a project and look at a topic too narrowly. It is true that I was replying almost entirely in a way that redirected the comments back toward the reporting system. I definitely could have done a better job responding to questions and comments about the culture of the Wikimedia movement and the Foundation's past and future work to make a safer environment.

I want to assure you that the Community health initiative and Trust & Safety are focused on the preventative and supportive measure, too. Past work includes Training modules about Online Harassment, Friendly Space Policy enforcement at Events, educational pamphlets for event organizers about safety at events, and supporting local wiki as they create policy about user conduct.

Going forward the user reporting system project is just one aspect of the Wikimedia Foundation's plans to grow a thriving community. The Foundation's Medium-term plan 2019 lists as a priority -- Thriving Movement In particular, Outcome 6 and Metric 7 are indicative of the focus on addressing cultural issues such as an universal code of conduct.

While I agree that it is important to not lose sight of the broader issues, it is also important to focus on the work at hand. As the User reporting system consultation moves forward and most of the discussion on the page will be with the Product Manager about software features, I can direct the ideas and comments about prevention and support to other pages where we will be discussing policy, support, and training.

But I hope that contributors with a broad range of experience, including targets of harassment, will stay engaged with the User reporting system project because it is essential to learn from them and their allies in order to build a product that meets their needs. Again, thank you for the nudge towards discussing the broader topic and the work that you are doing with the Working Group. SPoore (WMF) Strategist, Community health initiative (talk) 22:57, 8 April 2019 (UTC)

global vs. local

Hi. Please include into your research the fact that reports can affect users which are either just active on one wiki or acting on a couple of wikis. Reports should reach the right bunch of people who have the tools to deal with this. Best, —DerHexer (Talk) 14:52, 16 April 2019 (UTC)

Thank you, DerHexer, that is indeed an important factor. There are several aspects to this point that need to be delineated. I'm going to add several that I think of immediately, and would appreciate you and others improving them or adding in others. SPoore (WMF) Strategist, Community health initiative (talk) 16:04, 17 April 2019 (UTC)

1. There are cross wiki workflows done by people who have specialized tools that work globally (on all wikis.) These people have access to particular user rights (tech) and belong to a particular group (social). To have action taken through these workflows, a group needs to see the report or otherwise get a request for action from another person or group with the authority to act.
2. There are workflows that happen on local wikis that might or might not be the same on other local wikis. The tools on local wikis are generally the same as other wikis but some local types of customization might be in place.
3. There might be different workflows on different local wikis.
4. There might be different policies on different local wikis. These policies might identify specific people or groups who have the responsibility to take reports. These same people or groups may or may not have the authority, responsibility or access to tools to take action on reports.

@CLo (WMF):, you did some work related to crosswiki workflows for the stewards. It would be good to learn how that work influences our thinking about the workflows for the User reporting system as it connects to the larger wikimedia ecosystem. SPoore (WMF) Strategist, Community health initiative (talk)

Hello DerHexer, that's definitely an issue we're thinking about. We don't have a complete report (of the kind that Sydney has linked on the main page) on these cross-wiki workflows, but we are aware that they exist and are critical to the way projects are run. For example, we're aware that steward workflows and the work of the small wiki monitoring team are highly dependent on what I'd call cross-wiki reports. To elaborate, I interpret this as the process where one user (or several) notices suspicious behaviour on a specific wiki, raises the issue with local admins, and this process is repeated over time across several wikis; those admins or users then escalate to stewards or another appropriate cross-wiki group to handle the issue. It's a complicated topic, to put it mildly. I think that the prompts that have been raised are pretty much in line with what I am considering, but just to throw a few more questions into the ring:

• How do we handle different permissions across wikis when deciding on the chain of routing for a report?
• How do asynchronous communications attached to these hypothetical reports need to account for this complex "handing-off"?

...and I am certain there are more discussions to be had on this subject. —User:CLo (WMF) (talk) 22:27, 17 April 2019 (UTC)

Need for change of attitude towards those reporting harassment

I've just read the blog post about this consultation, and I'm concerned by the attitudes to people reporting harassment and the lack of understanding of this kind of situation that it reveals. First of all, people being harassed are told they should confront the harasser directly on their talk page and ask them to stop. This is extremely unlikely to be productive; harassment is not, in general, accidental behaviour which harassers will stop when asked to do so. Engaging with a harasser is more likely to lead to increased harassment, and should not be the recommended first step.

Second, people reporting harassment are told they should be further engaging with their harassers by notifying them on their talk page if they start a report on the ANI. Again, this should not be a part of the process. If the situation is such that the alleged harasser is deemed to have a right of reply, this should be sought by admins, not by the person who has reported them. Cases where an editor is clearly targeting another editor, whether by leaving offensive comments on their talk page or through strategically reverting their edits for spurious reasons/flagging their pages for deletion/etc, should not be adjudicated via a public discussion which anyone, including the harasser, can join in. There is clearly a much greater need for easy, private ways of reporting, which can then be taken public if there is a need for further discussion or to publish the results of the admins' investigation. Regarding harassment situations as parallel to innocuous disputes between editors demonstrates a fundamental misunderstanding of how harassers operate, particularly how they use community rules to create plausible deniability that their actions are 'really' harassment. Whatever solution(s) are developed, they need to prioritise protecting victims of harassment from further harassment and retaliation.

Eritha (talk) 15:53, 17 May 2019 (UTC)Eritha

Update

Hello,

A quick update to note that there are adjustments being made to the timeline for designing and developing the User reporting system. The plans will be updated in the next few weeks. SPoore (WMF) Strategist, Community health initiative (talk) 18:39, 13 June 2019 (UTC)

How do you stop systems from being abused?

There is the potential for anti-harrassment systems being abused by raising misleading or false reports - particularly if they are dealt with secretly without input from the accused or the community. Considering how many attempts are mode to use existing editor behavior systems such as ANI or Arbcom to win content disputes this is a real danger.Nigel Ish (talk) 16:57, 15 June 2019 (UTC)

• The answer, of course, is they don't. Due process doesn't seem to be a concern of any importance here. The whole process is a farce; WMF will do whatever they want to do, regardless of community response. CoffeeCrumbs (talk) 09:58, 17 June 2019 (UTC)

How to handle privacy vs "fair trial"

@SPoore (WMF): Will this be an "absolute privacy" set-up?

While T&S can investigate any claims, it's impossible for them to know some information unless they ask the accused editor. However, in the major recent example, the WMF has indicated that they can't ask about a specific incident because it would either directly or indirectly violate the accuser's privacy.

This makes absolute privacy conflict with a full fair trial (because even neutral participants risk not having all the facts) - the former isn't more important than the latter, so how is the system going to handle this?

Presumably it will need to notify users of it that some information may have to be provided to the accused in order to enable full consideration, though this would be minimised as far as was practical? Nosebagbear (talk) 17:06, 15 June 2019 (UTC)

Simple, people don't get a fair trial. Or at least what I am gathering from WMF proposed policy. The WMF doesn't care about the community members who built the project. Afootpluto (talk) 18:07, 15 June 2019 (UTC)

WMF and Local Interplay

A few thoughts/questions I have:

• Projects have widely varying levels of capacity to handle issues of harassment. Is there one answer for a tool the project builds or do there need to be a few, depending on the level of local capacity on this issue? How can this system be used to build/increase local capacity in this area over time?
• On English Wiki the barriers to reporting of harassment are high - the easiest places to find are public, the private places to find are private and still might direct back to a public forum. The disadvantages to this seem apparent and creating troubling situations. Are there any advantages to this higher barrier?
• In lowering the barrier to user reporting, is there capacity from the combination of volunteer and foundation time in this area to handle an increase in reports? Does this vary based on the project in question? If not, how can that capacity be generated?
• How can this initiative make clearer patterns of harassment where nothing on its own would be troubling but reoccurring low to medium harassment by one editor of multiple other editors is happening?
• This document makes clear that the foundation will make final decisions in this initiative. What processes/procedures will it implement to get local buy-in when an attempt for consensus seems to not be on the table?

I know I'm supposed to be answering questions but I am hoping that the formulation of these questions in and of itself provides some helpful feedback. Best, Barkeep49 (talk) 17:45, 15 June 2019 (UTC)

Hi Barkeep49, thank you for your questions! I hope some of my answers to your more research-oriented questions will provide a little clarity – they're certainly questions that are on the team's mind as we think about potential prototypes.

• With regards to different projects' capacities for handling issues of harassment, a related design issue we are aware of is the fact that differently-sized projects have different pathways and workflows to handle harassment. For example, a very small project might actually be better off, currently, handling harassment via informal channels (by which I mean, channels that were not purpose-built for handling harassment cases), such as email, or the use of general non-specialized noticeboards. On very small projects, the additional administrative overhead of building a formal (that is, custom built and designed to handle harassment) system might not be worth the benefits it brings. However a larger project might benefit from a more formal system, with the advantages of better documentation, and a structured pathway, since they may have the administrators necessary to handle the increased bureaucracy that such a system brings. It may well be that we must acknowledge that any formal, purpose-built pathway for harassment cannot handle every possible case.
• Barriers to reporting are definitely something I've been looking at! One of the motivations behind our push for this tool is the knowledge that private reporting, which is sometimes necessary for safety reasons, is extremely difficult to access for newcomers. As it stands, this means that only users who are already familiar with the community are in a position to make private reports, via the medium of emails, IRC PMs, or other pathways. However, through interviews with current and former admins, we are also aware that this high barrier can be beneficial in that it is a barrier against abuse of the system. That is to say, if a bad-faith actor wanted to use private reports as a vector for harassing administrators, the difficulty of finding the current private reporting channels means that it deters those who aren't as motivated for whatever reason. One of our challenges is to come up with a system that makes it easier for good-faith actors to make reports, while providing solutions for the volunteers receiving these reports to deal with malicious uses of the reporting system. We've heard some suggestions, such as batched tickets, or some way to filter or tag users, but settling on a specific technical solution feels premature at this stage.
• Questions of administrator training, support, and retention are definitely on our minds, though I believe they're a little outside the scope of this project :) It is a topic of great interest to me, and is something that we do aim to cover with regards to research projects in the future.
• Based on interviews with administrators, my review of old AN/I cases, as well as some work I've been doing looking into past SPI cases, it seems that the final judgement call on what is considered harassing behaviour is the result of noticing those patterns. Oftentimes the clearest indicator that harassment is occurring is the very fact that there is a recurrent pattern of uncivil, threatening, or otherwise aggressive behaviour that constantly just falls short of immediately-sanctionable actions. One of the potential advantages of creating a purpose-built harassment reporting system could be a better way for administrators to track such occurrences over time. Even if that single diff or incident is not actionable on its own, being tied to a history of similar boundary-pushing behaviour might be. My research indicates that this kind of communication and record-keeping already happens, just that right now it tends to exist purely in the memories of long-time administrators and is not always written down, which means it's knowledge susceptible to being lost to admin burnout or attrition.

As for your final question on buy-in, I am probably not the right person to answer this. But I would like to reiterate that we are aware of the importance of gaining the community's trust for whatever structure we build. Speaking for myself, my hope is that the reporting system ends up being used to supplement existing community processes.

P.S. please forgive me for the indenting not working on bullet points. Still trying to figure that part out. —User:CLo (WMF) (talk) 22:10, 24 June 2019 (UTC)

Thanks for your thoughtful and detailed response. To indent bullets you can do ":*Comment" or as many : as you need to properly indent. Best, Barkeep49 (talk) 01:10, 25 June 2019 (UTC)

Minimum design requirements

• The system must respect the privacy of the harassed.
• The complainant must substantiate their complaint with evidence.
• The accused must be able to reply to the complaint (suitably anonymized).
• Investigators must be able to ask the reporter and respondant questions.
• The process must be accountable to the community.
• The decision must be appealable to the community (e.g. Arbcom). All materials provided must be available for inspection by the body hearing the appeal.
• Reporters must be sanctionable for filing harassing or vexatious complaints, forum shopping or other conduct. Such sanctions must include the withdrawal of access to the reporting system.

Anything that does not satisfy all of these points is not fit for purpose. It should be possible to post a suitably anonymized summary of the case on a public board (and yes, that involves the ability to name the complainant publicly if they are sanctioned).

Other questions:

• What about email harassment? Per point #2, evidence is required. We cannot verify a case of email harassment without copies of the emails concerned.
• Likewise, what about mute lists? We want to see evidence that the accused has gone out of their way to harass the complainant. Part of that is circumventing mute lists.
• Low volume but quality reports are necessary to make a difference. Volunteers manning the reporting system don't want whinges that can be taken care of ANI, whinges by editors who lost a content dispute, complaints from those who cannot deal with legitimate criticism of their edits, unblock requests, etc.
• What about IPs? Secondly, there should be a barrier to entry to deter abuse or denial of service. MER-C (talk) 17:56, 15 June 2019 (UTC)

Endorse the standpoint given by MER-C. Starship.paint (talk) 09:09, 17 June 2019 (UTC)

I think that's not workable -- how do you "anonymize" a complaint of harassment when the moment the underlying issue is hinted at, the identity is obvious? Also, "mute lists" are bullshit. Actions against "vexatious complaints" are a paved road to Hell. And your proposals focus a bit too much on the rights of the person accused rather than the rights of the community. Bear in mind that, as in ordinary law, the principal right we are trying to uphold is our right as citizens to know we are not being pulled out and prosecuted unreasonably one by one.

I would say that better design considerations are that--

• Harassment must be very explicitly defined. Issue by issue, in detail. A lot of this is common sense -- we know that pinging an adversary in an edit war three times can happen, but pinging him 300 times is something else, and if he says "stop pinging me" you shouldn't do it without some good reason. So if it's common sense -- write it down. Assure people the same rules and standards are being used on everyone, and at least then we'll be less convinced it's because of personal relationships and dislikes.
• Jurors should be unconnected to the persons and issues at hand. This is to prevent the sort of COI being alleged by some community members responding to the Fram case.
• Sanctions must be justified in regard to particular actions. Saying "we've looked at a long term pattern of editing and the totality is..." aren't going to cut it. No one can look at the totality for any prolific editor - it's too much! Investigators look and they find stuff and they miss stuff. If you don't sit between the seraphim, the only thing you can do is judge specific edits/messages or groups of them.
• Public edits that bring sanctions, whether individually or as a pattern, must be listed and published to the community. There should be a database for us to look at and say "don't be like that" if we are not to be like that in the future.
• Most importantly, this stuff should be left to local administrators and general community members. If you actually need to override their process -- a dubious proposition -- then you need to tell them what they are doing that is so desperately deficient so they can get some reform proposals underway immediately. I mean, I know there are deficiencies - my top two points there. But it is up to helpful editors to decide when editing becomes unhelpful. Wnt (talk) 11:17, 17 June 2019 (UTC)

• Harassment can't be defined precisely. What you can do once and say "sorry, I made a mistake" you may not be able to do hundreds of times. And while pinging a single user 10 times when it's relevant is clearly appropriate, doing it 5 inappropriate times could be harassment.
• An other critical issue is to allow reporting without risk of it triggering further harassment. You report certain users publicly, or even you report them privately and the handling committee publicizes it, you run the risk of further harassment. 37.26.149.129 15:48, 24 June 2019 (UTC)

Accountability

I am strongly opposed to the proposed reporting system to the extent it imposes civility standards without local community involvement, review, or approval, or results in secret trials by secret and unaccountable judges without the right of representation, defense or appeal, on secret evidence submitted by secret accusers.

I object to the Foundation imposing any non-legally necessary sanctions within the purview of established local conduct policy and community processes.

T&S shouldn't be imposing temporary or local sanctions or modifying advanced permissions. They should have only the one tool that they have ever needed for their legitimate purpose of ToU enforcement of serious, legally necessary sanctions: the global permanent ban.

Finally, I agree with Jimbo that all bans are appealable to him in his capacity as an individual founder,[1] as long as he maintains engagement with the movement as a whole and outsiders. (His level of engagement, by the way, sets a bar to which rest of the Foundation should aspire, but has as yet not ever come close.) I ask the Community Health team to explicitly acknowledge this avenue of appeals. EllenCT (talk) 18:23, 15 June 2019 (UTC)

I concur with all EllenCT has written above about the accountability of the report system, and strongly oppose this proposal, in particular while there is the possibility of it resulting in secret trials by secret and unaccountable judges without the right of representation, defense or appeal, on secret evidence submitted by secret accusers. A system like this, specially if it gets to be entirely managed by WMF, without proper accountability, seems very prone to abuse, e.g., for vendettas, pet revenge, and imposing an editorial line/POV approved by the ones deciding over the complaints. It looks like a free card to corruption and abuse, with a great potential of increasing harassment and bringing it into a whole new level, not reducing it.--- Darwin ^Ahoy! 18:35, 15 June 2019 (UTC)

I'd like to quote from Anti-Harassment Tools Team Design Researcher Claudia Lo's November 2018 "Reporting systems on English Wikipedia" written for the Community Health Initiative:

the Wikimedia community highly prizes transparency. For reporting systems, this is interpreted as publicly-viewable processes, outcomes, and the identities of the involved users. Transparency in this case is not just a design consideration put into place to achieve a certain kind of efficiency or mode of operation, but a value to be strived for in the way the entire system operates.... whatever changes we recommend, it must adhere to these values even as we change key features, otherwise it will not be trustworthy.

EllenCT (talk) 19:40, 15 June 2019 (UTC)

• Excellent quote - properly notes the issue and really the underlying truth that's caused the problems. Until very recently for en-wiki, and slightly longer for de-wiki, T&S actions have always been taken to be correct. As such, they were granted more leeway in their privacy and decision-making so long as we believed it was reasonable. The loss of that (heavily due to the lack of transparency) makes every single step to change 10x harder. Without transparency, no trust. Without trust, no change. Nosebagbear (talk) 19:47, 15 June 2019 (UTC)

Ellen is making a helluva lot of sense to me as well. Shearonink (talk) 01:04, 17 June 2019 (UTC)

Endorsing all prior comments in this section. Starship.paint (talk) 09:10, 17 June 2019 (UTC)

I, too, endorse all prior comments in this section. T&S needs to recognize that without trust of them by the community, trust that office actions will be fair and will not be a takeover of community dispute resolution procedures, these kinds of plans will fail. --Tryptofish (talk) 21:06, 17 June 2019 (UTC)

I agree with several sentiments above. But more specifically, I get the impression that there has been a presumption that private reporting of harassment where sexuality is an issue will get a better outcome. After all, many of us remember the utter debacle of the ArbCom case against Fae, where an organized off-site campaign with significant, obvious anti-gay sentiments was able to organize a bullying campaign that led to Fae being sanctioned for a tiny technical offense of trying to preserve privacy and, worse, for a few brief complaints about anti-gay harassment. However -- how can anyone know the dictator of T&S won't make the same kind of bogus decision in the future? Can WMF promise us that no managers will ever be hired for the position who are of Pakistani or Kenyan nationality or any other country where homosexuality remains illegal? That no one will be hired who subscribes to fundamentalistic belief systems? I understand, of course, that such workers would likely be required to uphold anti-discriminatory policy whether they believe in it or not ... but how can we, or anyone, check that will actually happen? Community discussion is a place where a lot of things never get resolved, but there are merits to that. Wnt (talk) 10:41, 17 June 2019 (UTC)

• Accountability, or the lack thereof, is the primary reason I oppose WMF getting more involved. Dennis Brown (talk) 14:16, 25 June 2019 (UTC)

Project update and timeline

Hello all,

Thank you for your thoughts and ideas so far. Me and other members of the Anti-Harassment Tools team will answer your specific questions and comments.

A few general comments:

• This project was put on hold about a month ago because a) the Anti-Harassment Tool teams Project Manager left the Foundation b) the Foundation is undergoing internal reorganization in ways that might effect the timeline and staff working on this project c) the Foundation's new Medium Term Plan had to be completed in order to know if URS fit into the scope of work in the next annual plan. These are related to the internal workings of the Foundation and I would not normally mention them to the community because it usually isn't relevant or of any interest. But in this instance I want to let you know the reason that the timeline for this consultation changed and that that you will have plenty of time to comment about the User reporting system.
• Normally, the Anti-Harassment Tools team starts a project page on English and German Wikipedia and other any other local wikis where we are doing a pilot project or direct communication with the local wiki seems important. I haven't began the local project pages yet because we are doing a phased consultation and that part comes in the next phase. Thank you to a few communities that already made a local page. :-) People are welcome to comment on Meta or their local page.

I want to assure you that the Foundation understands that the User reporting system can't achieve its goals without in depth consultation with local communities. We know that this can not be a one size fits all project. That is the reason we have been doing research for over a year about what is working and what is not working in our current system. I encourage you to look at the research and stay engaged with us.

Again thank you for participating in this discussion. I'm sorry if the change to the timeline caused any worry or confusion. SPoore (WMF) Strategist, Community health initiative (talk) 22:43, 17 June 2019 (UTC)

Thanks for the update, and for your work on this. – Ajraddatz (talk) 20:29, 21 June 2019 (UTC)

@SPoore (WMF) Since you mentioned the German Wikipedia: The Anti-Harrasment Tools team should consider to present their project and asked for local feedback in the Kurier. Kurier and its discussion page are the German equivalent of both, Signpost and Village Pump. If you want to address the community it is the way to go. Other pages which may sound appropriate like WP:Projektdiskussion are backwaters with very few contributors. ---<(kmk)>- (talk) 00:09, 25 June 2019 (UTC)

Encouragement

As I tried to explain during the initial drafting of the 2030 Strategy session back in 2017, I believe that both harassment and false claims of harassment are major problems for en.wp. (My own experience of being prosecuted repeatedly by a banned user and their supporters is worth study. I would be happy to discuss this further -- privately -- with T&S researchers. I have looked at extensive evidence from ArbCom cases, AE findings & RfC/U in order to try to understand how I was wrongly prosecuted in 2016-2017 and wrongly labeled as a harasser/hound). I would appreciate that you add my name to the list of those being periodically informed about requests for participation in these consultations.

Looking around today, I remembered that you had surveyed a small sample of users (admins only?) concerning AN/I, but have not yet conducted inquiry into WP:AE (arbitration enforcement). I would be happy to help provide data concerning how this latter noticeboard has, in some cases, been used as a streamlined procedure for suppressing legitimate inquiry.

Initial inquiry into harassment needs to be conducted with protection for the reporter, as otherwise it is too easy for any worthwhile inquiry to be derailed by established editors simply smearing the reporter. I have recent examples of this as well as examples dating back to 2013. The recent vociferous resistance to change on en.wp should probably be understood as being potentially motivated both by:

• "good faith concerns" about potential abuse of this protection of the victim (concerns with opacity)
• "less savory concerns" about the challenge posed to a system allowing deeply embedded actors to nip healthy inquiry into systemic problems in the bud

Thank you for following up on the widely-recognized community health concerns and attempting to address them. SashiRolls (talk) 00:39, 18 June 2019 (UTC)

I recommend delay

I recommend that the Foundation delay implementing this user reporting system until there is a clearer understanding of what matters will prompt Foundation intervention as compared to being referred to a project's own disciplinary procedures, such as Requests for arbitration on the English Wikipedia. There is a good deal of confusion as to this issue in regard to a recent ban implemented by the Foundation of a then-administrator on the English Wikipedia. --Metropolitan90 (talk) 05:58, 19 June 2019 (UTC)

Ya think? A wee bit o' drama there? I'd also like to see "contact local arbcom first and only escalate to T&S if they aren't willing/able to handle it, except in extreme cases (pedophiles, real world stalkers, that sort of thing) Beeblebrox (talk) 23:37, 23 June 2019 (UTC)

Don't put this on large wikis with community processes like ArbCom that can handle this themselves (azwiki yes, enwiki no)

Fram. Enough said. — pythoncoder  (talk | contribs) 01:34, 25 June 2019 (UTC)

I don't think that's all that needs to be said. There's probably room (and intention) to integrate this with existing local wiki reporting systems, completely separate from the T&S ban system. – Ajraddatz (talk) 02:06, 25 June 2019 (UTC)

@Pythoncoder: az.wiki would argue that their processes are just as valid as en.wiki. To a certain extent, that's true. –MJL ‐Talk‐^☖ 03:29, 25 June 2019 (UTC)

Yeah, even as an enwiki member I am not sure about an exception (to say nothing that we'd need a consensus anyhow). For example, who gets to decide who is exempted and who is not? If a project that has gotten an exemption develops a governance issue to the point that local processes can no longer handle problems, what then? Jo-Jo Eumerus (talk, contributions) 07:21, 25 June 2019 (UTC)

• The Fram situation demonstrates exactly why the local Arbcom should be ones to handle local issues that aren't legal in nature, don't involve minors, or do not require emergency action. The lack of accountability to the greater community is always going to cause problems and drama, just as this has. Dennis Brown (talk) 14:14, 25 June 2019 (UTC)