User:Cool3/Admin Study

This page presents the essential conclusions and methods of my study of desysopped administrators, exemplary administrators, and a control group of randomly selected administrators. The goal of the study is to draw conclusions about the RfA process by looking at the RfAs of administrators from these groups. The raw data for this study can be found at User:Cool3/Desysop

Thus far, the essential conclusions of the study are:

• There is no significant correlation between multiple RfAs and poor behavior in the future. 19.4% of the desysopped admins had more than one RfA while 16.7% of the exemplary administrators had more than one.
• There is no significant correlation between experience (beyond the generally accepted baseline) and admin performance.
• The overall support percentage in an RfA is a very poor predictor of how successful the administrator will be.
• There is good reason to be concerned about self-nominations. 32% of desysopped admins had self-nominated vs. 11% of exemplary admins and 15% in the control group.

Sample selection

• Desysopped admins: Desysopped administrators were chosen as a group representing those who failed to properly perform administrator duties. With no offense to any particular user listed in this group, these are meant to represent poor administrators. The desysopped administrators were taken from Wikipedia:Requests for de-adminship. Administrators desysopped due to compromised accounts were excluded from the sample as they are not assumed to have misused the tools.

• Exemplary admins: I have done my best to find an objective way of identifying exemplary administrators. There is no process in place for formally recognizing the best (beyond barnstars, etc. which are highly subjective). In general, though, those users who receive additional tools (oversight, checkuser, bureaucratship) tend to be among the very best admins and are selected by broad community consensus. I do not mean to suggest that these users exclusively represent our best administrators, but I do suggest that they are certainly among our best.

• Average admins: A sample of 20 randomly selected administrators were chosen to represent the "average" admin. This is not to suggest that any one of them is "average", but I feel that as a group they represent a fairly average administrator. Why did I pick 20? It was a round number and about the same as the number of exemplary admins (i.e., no real reason).

Phase 1

Phase 1 was the quantitative, number driven portion of the study, based on 8 measurable factors: Number of RfAs, Months Active, Edit Count, Support !Votes, Oppose !Votes, Neutral !Votes, Support %, and whether or not a user self-nominated. All data was collected from the candidate's successful RfA. For those who have been sysopped, RfAs after the desysopping were ignored in the RfA count (as they are not relevant to the purpose at hand). The goal of this phase was to answer simple questions about some common assumptions in the RfA process, presented below:

Multiple RfAs

Many editors are suspicious of candidates who have had multiple RfAs. Are editors with lots of RfAs really more likely to end up as problem admins?

The evidence here is somewhat mixed, but on the whole I'm not really seeing any strong correlation between lots of RfAs and a bad admin, although there is some sort of connection. Of the 31 desysopped admins, 6 (19.4%) had multiple RfAs. In the exemplary group, 3 out of 18 admins had multiple RfAs (16.7%). Out of the control group, though, only 2 (10%) had multiple RfAs. I'll leave you to form your own conclusions based on this data, as there is room for interpretation in many directions. On the whole, though this data certainly suggests that reflexively opposing multiple RfA candidates is clearly unjustified.

Experience

How long does it take to become good admin material? Are inexperienced admins more likely to end up being desysopped?

Because our samples span the entire history of RfA, and standards have changed, straightforward averages are unlikely to be useful in answering this question. A simple examination of the data, just eyeballing it, though seems to show no link between experience and later quality. Just about all of the admins here, in all categories, had what was considered "enough" experience for the time when they were promoted. Candidates with very high levels of experience ended up desysopped (e.g., Rama's Arrow and Bedford) while others with low levels (Cool Hand Luke and Cecropia) ended up in the exemplary sample. There is undoubtedly some minimum level of experience required for adminship, but no evidence here shows that either barely reaching that level or amply exceeding it results in a better administrator.

The Discretion Zone

Are candidates whose RfAs pass in the "discretion zone" (<80% support) more likely to end up as poor administrators?

The data here is really insufficient to render a verdict. Only 4 of the admins studied here passed in the discretion zone. 3 of them came from the desysop group (9.6% of those desysopped) while 1 came from the exemplary group (5.6% of exemplary admins). So yes, there is some suggestion here that "discretion zone" admins are more likely to be desysopped, but it's really hard to form a conclusion on such limited data.

Overall support percentage

Beyond the discretion zone, does the overall support percentage matter? Are those who receive more support more likely to turn out as good admins?

The answer here is again complicated. 8 of the 31 desysopped admins passed through RfA with unanimous (100%) support. So clearly, it was hard for the RfA !voters to see their future coming.

On average, the desysopped admins received 92.96% support in their RfAs. The exemplary group received 94.64% support in their RfAs. The control group admins received 96.62% support in their RfAs. So for all of the groups, the percentage was in the 90s. The desysopped group did have slightly less support, but hardly to a significant level. Whatever the link between support percentages and outcomes, though, I think it is fairly clear that raising requirements wouldn't do much good. In order to eliminate 50% of the desysopped admins, we would have to raise the requirement in excess of 95% which would also eliminate 39% of the exemplary sample.

Self-noms

Many editors regard self nominations with a certain suspicion. Is there reason to believe that editors who self-nominate will make poor admins?

It seems that there may some reason for this concern. In the desysop group, nearly 1/3 of editors self-nominated (32%) while in the exemplary group only 11% self-nommed and in the control group 15% self-nommed. This is the most statistically significant finding in this study so far, but we are still dealing with a fairly small set of data.