Behind the shroud: a survey of editors in ecology and evolution

RESEARCH COMMUNICATIONS RESEARCH COMMUNICATIONS Behind the shroud: a survey of editors in ecology and evolution Olyana N Grod1*, Christopher J Lortie1, and Amber E Budden2 An online survey of ecology and evolution editors was conducted to assess the characteristics of journal editors and describe manuscript-handling practices. A total of 450 respondents – representing 155 ecology and evolution journals – participated. The following patterns were detected: (1) there are more male than female editors; (2) the greater the number of manuscripts handled per year by editors, the lower the proportion are rejected without review; and (3) previous review time, scientific status, and seniority of reviewers are factors that editors consider when selecting a reviewer. This research highlights the potential importance of editor characteristics in the peer-review process; we suggest that increased transparency promotes both recognition of the editor population and more equitable reviewing practices. Front Ecol Environ 2010; 8(4): 187–192, doi:10.1890/090048 (published online 27 Oct 2009) S cientific journals rely on editors to direct the peerreview process and thus to help in the dissemination of knowledge (Freda and Kearney 2005). Editors are responsible for monitoring and coordinating the publication process by selecting reviewers, communicating with authors and reviewers, and, ultimately, selecting or rejecting manuscripts on the basis of merit (Kassirer 2001). In order for this system to be effective, editors must be objective, trustworthy, and ethical, and should always have the best interests of the journal and its readership in mind (Kassirer 2001). Despite the editors’ pivotal role in the peer-review process, the day-to-day tasks they perform, the way in which they carry out their work, and how they reach decisions on manuscripts remain a mystery for many scientists (Freda and Kearney 2005). Quite simply, this is due to a lack of public, quantitative data on the responsibilities of editors at the discipline level. Research on peer review has focused largely on issues of “blinding” (Fisher et al. 1994; Budden et al. 2008), reviewer characteristics (Evans et al. 1993; Kliewer et al. 2005; Grod et al. 2008), and manuscript evaluations (Evans et al. 1993; Kliewer et al. 2005). As a result, most research discusses editors only in the context of how they can ensure that manuscript assessments are fair and are based on scientific merit (Lortie et al. 2007). In medicine, studies of editor characteristics have revealed that more males than females hold editorial positions (Wilkes and Kravitz 1995; Dickersin et al. 1998), although the number of female editors is increasing (Dickersin et al. 1998). Male editors were, on average, older than their female counterparts; female editors handled more manuscripts and rejected manuscripts at higher rates than males; and similar numbers of reviewers were used per manuscript by both genders (Gilbert et al. 1994). The average age of editors for US and 1 Department of Biology, York University, Toronto, Ontario, Canada *([email protected]); 2National Center for Ecological Analysis and Synthesis, Santa Barbara, CA © The Ecological Society of America Canadian medical journals was 61 and their average number of years as an editor was 7.6 (Wilkes and Kravitz 1995). That study also reported that 40% of editors make a decision in the event of reviewer disagreement while 43% of editors sent papers out for additional review. Importantly, 88% of editors chose reviewers for manuscripts who were already on file for the journal (Wilkes and Kravitz 1995). Such studies are critical for maintaining transparency in the review process and for ensuring that it functions smoothly and fairly. Our hope is that similar data for ecology and evolution journals will provide insights into potential improvements in our disciplines. The objectives of this study were to evaluate the characteristics of editors working on ecology and evolution journals and to quantitatively examine editorial practices. Manuscript and review handling (the number of manuscripts handled, handling time, and rejection rate) and reviewer selection were explored in relation to editor characteristics, specifically gender. This is the first broad study of editors in the fields of ecology and evolution.  Methods Survey An online survey of ecology and evolution journal editors was posted from 31 July to 1 November 2007. The survey was distributed via e-mail to 2434 editors of 250 ecology and evolution journals published by Blackwell, Elsevier, Springer, and the Ecological Society of America. Journals were selected on the basis of their subject category, containing either the word “ecology” or “evolution”. Editors with responsibility for influencing publication outcome were selected from journal websites and invited to participate. This did not include members of an editorial board, technical editors, or editorial assistants. If editor e-mail addresses were unlisted on the journal webpage, they were retrieved through institutional websites. Follow-up www.frontiersinecology.or g 187 Ecology editors in review Count (a) 200 180 160 140 120 100 80 60 40 20 0 Male Female Our questions were not of a sensitive nature and respondents were assured that the results would be published in a form that prevented individual respondents or journals from being identified. We recognize that the data used in this study are self-reported and thus subject to some degree of recall bias (Wilkes and Kravitz 1995). Statistical analyses Main (b) Count 188 ON Grod et al. Associate Subject Editor position 180 160 140 120 100 80 60 40 20 0 Handling Male Female North America Europe Region Other Figure 1. A summary of the respondents to an online survey of editors, according to (a) editor position and (b) region. In all instances, there is a significant gender difference between male and female respondents (all P < 0.001). Editor titles vary between journals and publishing houses; for the purposes of this paper, “subject” editors are considered subject-matter experts, whose responsibilities include overseeing the review process for manuscripts. These editors may also be known as editor, section editor, or regional editor. “Handling” editors are those editors responsible for processing manuscripts; alternate titles include, but are not limited to, managing editor, assistant editor, communicating editor, and reviewing editor. e-mails were sent only to those from whom we received an out-of-office response following the first contact. The survey consisted of 19 questions, involving a combination of multiple-choice, open-ended, and Likert-scale questions (WebPanel 1). Respondents were given the opportunity to avoid responding to some of the questions (indicated by an asterisk in WebPanel 1) and could exit the survey at any point. Respondents who indicated that they were not editors or were simply members of an editorial board were excluded from the analyses. Where the response to an open-ended question was inappropriate – either answered using incorrect units, or because it was an invalid entry – the response was voided. The lower value of range responses was used in all instances. Countries of host institutions were categorized to the following regions: North America, Europe, or “Other”, following Leimu and Koricheva (2005); regions of increasing scientific activity but with only small sample sizes were assigned to the category “Other”. Publishing experience was estimated by subtracting the editors’ reported year of first peer-reviewed publication from 2007, the year in which the survey was conducted (Cassey and Blackburn 2004). www.fr ontiersinecology.or g We conducted all statistical analyses using JMP version 5.1. Descriptive statistics were reported as percentages and means ± SE. Chi-square analyses, contingency tests, and one-way analysis of variance were used to describe interactions between editor characteristics – gender, region, editor position, years as editor, and years since an editor’s first publication in a scientific journal. Gender, region, and years as editor were included in generalized linear mixed models on the number of manuscripts handled and handling time. In the latter case, the number of manuscripts handled was also included as a factor. Chi-square analyses were conducted on editor responses for rate of rejection without review and reviewer selection criteria. These two responses were further explored by editor characteristics (other than editor position), number of manuscripts handled, and handling time, and were analyzed using ordinal logistic regressions.  Results Description of respondent population A total of 450 respondents – representing 155 ecology and evolution journals – completed the survey. The response rate for ecology journal and evolution journal editors was 18.5% and 62%, respectively. The distribution by editorial position was as follows: of all respondents, 21.8% (n = 98) were main editors (those with titles such as editor-in-chief, chief editor, senior editor, deciding editor, and associate- or co-editor-in-chief), whereas 51.6% (n = 232), 15.3% (n = 69), and 11.3% (n = 51) were associate editors, subject editors, and handling editors, respectively (␹23 = 160.63, P < 0.001). Main editors made the ultimate decision on publication of a manuscript (editor position, ␹23,9 = 70.43, P < 0.001). Representation of editors from “Other” regions was 17.1% (n = 77); for the remainder, 42.1% (n = 189) were from within Europe and 40.8% (n = 183) (␹22 = 59.45, P < 0.001) were from North America. There were significantly more male (82.9%) than female (17.1%) editors (␹21 = 211.95, P < 0.001, n = 450), and this was consistent for all editor designations (all P < 0.001; Figure 1a) and across all regions (all P < 0.001; Figure 1b). There was no gender bias between survey respondents and nonrespondents (␹21 = 0.27, P = 0.60). On average, respondents had been editors for 6.91 ± 0.32 years and had published peer-reviewed manuscripts of their own for 22.31 ± 0.46 years (mean ± 1 SE). Male © The Ecological Society of America ON Grod et al. editors had been publishing for longer than female editors (F1 = 12.54, P < 0.001, n = 449; Table 1), but there was no gender difference in the number of years as editors (F1 = 1.47, P = 0.226, n = 437; Table 1) nor between regions (F2 = 1.36, P = 0.257, n = 436; Table 1). Editorial practices The mean number of manuscripts handled by editors each year was 71.76 ± 7.18, and main editors handled more manuscripts than associate, handling, or subject editors (F3,9 = 48.48; post-hoc analyses F3,432 = 51.72, P < 0.001). No difference was found by gender or region. Although the relationship was weak, there was a significant difference between years as editor and the number of manuscripts handled (F1,9 = 4.54, P = 0.034, r21,430 = 0.28). The mean handling time of manuscripts was 12.34 ± 4.61 hours. There was no difference in handling time by gender, region, years as editor, and editor position, nor in relation to the number of manuscripts handled per individual editor (F10,382 = 0.74, P = 0.69). Overall, 61.1% of editors reported that they “rejected without review” less than 25% of manuscripts they individually handled (␹24 = 527.62, P < 0.001). Although there was variation between regions (region, ␹22,8 = 7.44, P < 0.024), the most frequent response from editors across all regions was that they rejected without review less than 25% of manuscripts (chi-square, all P < 0.001). The greater the number of manuscripts handled by an editor, the lower the proportion of manuscripts that were rejected without review (MS handled, ␹21,8 = 11.83, P < 0.001). Given that there was a significant relationship between the number of manuscripts handled and editor position, we tested rejection data by editor position and found that main editors were primarily responsible for the effect of manuscripts handled on rejection rate (␹21 = 5.52, P = 0.019) rather than associate, subject, or handling editors combined (␹21 = 2.35, P = 0.13). There was no effect of gender, years as editor, or handling time. In cases of split reviews (ie where there was one positive and one negative review), there was no variation in rejection rate (␹28 = 7.05, P = 0.53), with most editors indicating a rejection rate of either 25–50% (n = 171) or 50–75% (n = 164). What are editors looking for in a reviewer? Familiarity with research topic and quality of previous reviews were rated as the most important criteria that editors considered when selecting a reviewer (68.9%, ␹23 = 86.14, P < 0.001 and 86.2%, ␹23 = 244.42, P < 0.001, respectively; Tables 2 and 3). Speed of previous reviews and status of scientist were seen as less important (␹23 = 178.94 and ␹23 = 71.30, respectively, all P < 0.001; Tables 2 and 3). Whether the reviewer was recommended by the submitting author was deemed less important (␹23 = 274.52, P < 0.001; Tables 2 and 3), and the majority of editors (81.1%) considered the odds that a reviewer will © The Ecological Society of America Ecology editors in review Table 1. Description of journal editors’ involvement in the peer-review process and their corresponding workload Mean (± SE) Years since first publication Gender* Female Male Region North America Europe Other Years as editor Gender Female Male Region North America Europe Other 18.81 (1.02) 23.03 (0.50) 23.37 (0.67) 21.70 (0.71) 20.91 (1.18) 6.04 (0.66) 7.08 (0.36) 7.20 (0.44) 6.34 (0.45) 7.70 (1.07) Number of manuscripts handled each year Gender Female 68.00 (15.20) Male 72.51 (8.07) Region North America 71.21 (10.66) Europe 75.29 (12.17) Other 65.05 (15.36) Editor position* Main 214.62 (23.98) Associate 22.31 (2.15) Subject 36.63 (10.60) Handling 65.67 (22.25) Handling time (hours) Gender Female Male Region North America Europe Other Editor position Main Associate Subject Handling 8.01 (2.32) 13.19 (5.49) 5.57 (0.91) 17.39 (10.15) 15.70 (9.80) 32.80 (22.89) 7.41 (1.40) 7.72 (1.84) 6.25 (2.11) Notes: Data are presented as mean ± standard error (SE). (*) denotes a significant difference at P < 0.05. reject a manuscript as “not important” (␹23 = 770.73, P < 0.001; Tables 2 and 3). Editors who reviewed fewer manuscripts prefered shorter reviewer turnaround times (the time it takes a reviewer to submit their report on a manuscript; MS handled, ␹21,8 = 5.01, P = 0.025). The importance of reviewer status was rated differently according to editor gender (␹21,8 = 10.08, P = 0.002), with males rating status as “very important” relative to females (test of proportions, z = 2.01, P = 0.045). Seniority of a reviewer was also rated differently by editors from different regions, with North Americans viewing seniority as less important than those from Other regions (␹22,8 = 7.66, P = 0.022). In all three www.frontiersinecology.or g 189 Ecology editors in review 190 ON Grod et al. Table 2. Importance of particular criteria when selecting a reviewer Rate the importance of the following criteria when selecting a reviewer for a paper you are handling Not important % (n) Somewhat important % (n) Important % (n) Very important % (n) Familiarity 8.2 (37) 22.9 (103) 37.1 (167) 31.8 (143) Quality of previous reviews 2.4 (11) 11.3 (51) 41.3 (186) 44.9 (202) Speed of previous reviews (turnaround time) 7.6 (34) 36.4 (164) 44.4 (200) 11.6 (52) Odds she/he will reject manuscript 81.1 (365) 13.6 (61) 4.2 (19) 1.1 (5) Status of scientist 18.9 (85) 36.4 (164) 32.7 (147) 12.0 (54) Recommended by submitting authors 40.9 (184) 47.1 (212) 11.6 (52) 0.4 (2) Notes: All respondents, n = 450. Percentages based on total number of respondents. Bolded values indicate most commonly chosen response. All chi-square analyses were significant. regions, significantly more editors valued seniority as “somewhat important” in the event of a split review (individual chi-squares, all P < 0.001). When using a review, editors from North America valued manuscript comments while editors from Europe and Other regions valued both comments and a clear recommendation (␹22,7 = 6.32, P = 0.042; Table 4). When submitting their own research for publication, editors rated journal impact factor as most important and journal rejection rate as least important, which is consistent with author opinion (Aarssen et al. 2008; Table 3). In addition, editors reported being satisfied with the peer-review process as both an author and editor (Table 5).  Discussion Editors play a prominent role in the peer-review process, increasing the value of scientific studies by enabling publication of those deemed to be of merit. It is therefore critical that we understand the role of editors in the peer-review process to ensure that the process is transparent and fair. As in medicine, editorial positions in ecology and evolution are dominated by males (Wilkes and Kravitz 1995; Dickersin et al. 1998; Garrow et al. 1998). Importantly, female editors have less publishing experience than their male counterparts (see Table 1), although the former have held editorial appointments for similar lengths of time. The same trend was seen among reviewers in ecology and evolution (Grod et al. 2008), suggesting that the available pool of female scientists qualified for editor positions is smaller than that for males. In 2006, females comprised 29.5% of professors, associate professors, and assistant professors in the biological sciences in the US (NSF 2009), suggesting that females are underrepresented in editorial positions. Knowing the age of the editor respondents would have provided additional insight into the driving force behind the gender inequality. It is also unclear whether females are approached to take on editorial positions just as often as males and whether females decline these positions more often than males (Dalton 2006). Having a better understanding of the career trajectory for editors may also help to explain this Table 3. Importance of particular criteria when selecting a journal for submission of research Rate the importance of the following criteria when selecting a journal for submission of your research Not important % (n) Somewhat important % (n) Important % (n) Very important % (n) Speed of decision 3.1 (14) 24.1 (108) 53.9 (241) 18.8 (84) Speed of publication 2.9 (13) 33.9 (151) 48.9 (218) 14.3 (64) Impact factor or status 2.0 (9) 12.2 (55) 40.6 (182) 45.1 (202) Familiarity 6.1 (27) 26.2 (117) 52.9 (236) 14.8 (66) Accessibility to others 3.1 (14) 15.9 (71) 48.7 (218) 32.2 (144) Journal rejection rate 45.2 (202) 37.6 (168) 15.9 (71) 1.3 (6) Notes: A response was not required for this question; thus, respondent numbers vary. Percentages based on total number of respondents. Bolded values indicate most commonly chosen response.All chi-square analyses were significant. www.fr ontiersinecology.or g © The Ecological Society of America ON Grod et al. Ecology editors in review 191 Table 4. Components of a review that editors deem important Region A clear recommendation from reviewers to accept/reject % (n) Detailed and comprehensive comments on the manuscript % (n) Both equally important % (n) Other % (n) North America 1.1 (2) 56.3 (103) 41.0 (75) 1.6 (3) Europe 2.7 (5) 35.5 (67) 61.4 (116) 0.5 (1) Other 2.6 (2) 33.8 (26) 63.6 (49) 0 (0) Notes: All respondents, n = 450. Percentages based on total number of respondents. Bolded values indicate most commonly chosen response. gender difference and could be beneficial to academics interested in this career path (Freda and Kearney 2005). Editor designation is an important distinction in ecology and evolution journals, as main editors handled more manuscripts and rejected fewer manuscripts without review than did associate, subject, and handling editors. In medicine, main editors reject outright at least half the manuscripts they see, in view of the limited publishing space available (Dickersin et al. 2007). This may not be the case in our discipline, because main editors in ecology and evolution inspect all manuscripts submitted to their journal, but direct them to associate or subject editors. Hence, main ecology editors may restrict their initial decisions to reject only those manuscripts with content that is clearly not suited to their specific journal. Of course, the practices of main editors – as well as associate, subject, and handling editors – will vary from journal to journal and could be dependent on journal policy. Consequently, regional differences in rejection rates could be explained by variation in editorial and journal policies, if there were systematic differences across regions. However, no difference in editor position by region was found in this study (␹26,440 = 3.55, P = 0.74). Variation in the titles assigned to editorial positions was also not consistent, making it more difficult to group editors according to title and role. For example, the title of main editor could be used in one journal, whereas the equivalent position at another journal might be designated as editor-in-chief. Identification of these differences is necessary and a valuable source of information for potential authors. Increased transparency – journals reporting on how manuscripts are handled, while maintaining editor/reviewer anonymity, and the role of the various editors – could streamline the peer-review process in ecology and evolution, to the benefit of authors. When asked to rate the importance of various criteria when selecting a reviewer to evaluate a manuscript, reviewer turnaround time was of importance to editors handling fewer manuscripts. These editors may have less time because they are volunteer rather than salaried editors; our survey did not ask respondents to discriminate between the two. Time constraints are the leading reason why reviewers decline a request to review a paper (Tite and Schroter 2007). An expectation of, or emphasis on, short turnaround times would therefore probably further deter reviewers from engaging in the peer-review process. Clear information reviewing timelines by journals and editors could streamline the review process by reducing unnecessary communication and checks on the progress of a manuscript. Male editors believe reviewer status to be more important than do female editors. Since status was not defined for the respondents in this study, this could have been interpreted in several ways, including, but not limited to, the following attributes: experience, productivity, quality, and breadth of knowledge. Nevertheless, the fact that male editors believed that the status of a reviewer was important reinforces the possibility that an “old boy network” could still be lingering in the peer-review process (Lloyd 1990). The fact that female editors do not rate this criterion as important could suggest that they have had fewer positive experiences with senior scientists within the hierarchy of science, and thus value it less. Alternatively, younger and female editors may simply have a less well-developed network of connections. In cases of split reviews, North American editors are more likely to rely on reviewer seniority than are editors in other regions. This seems to suggest that either regions, or journals associated with different regions, have different formal or informal policies in place with respect to Table 5. Respondent satisfaction with the peer-review process as an author and as an editor Rate your satisfaction with the peer-review process Highly dissatisfied % (n) Dissatisfied % (n) Neither satisfied nor dissatisfied % (n) Satisfied % (n) Highly satisfied % (n) As an author 1.8 (8) 6.2 (28) 16.4 (74) 66.2 (298) 9.3 (42) As an editor 0.4 (2) 4.0 (18) 9.2 (41) 75.8 (338) 10.5 (47) Notes: Bolded values indicate most commonly chosen response. All chi-square analyses were significant. © The Ecological Society of America www.frontiersinecology.or g Ecology editors in review 192 ON Grod et al. split reviews. In the absence of consensus between reviewers, it is unclear whether editors make the ultimate decision themselves or whether they solicit additional reviews. It is probable that editors are trained or mentored differently in different regions. However, most editors of biomedical journals are not formally trained (Garrow et al. 1998) and are therefore basing their practice on the guidance of an experienced editor or their predecessor (Freda and Kearney 2005), whose approach in weighing reviews appears to vary between regions. Clearly, in ecology and evolution, most editors ranked the quality of previous reviews by reviewers as “important” and “very important” when considering them for future service, and that the likelihood of a given reviewer rejecting a manuscript was seen as unimportant. This shows that editors have the best interests of the journal and the authors in mind when making editorial decisions. Although there are no inherent differences in the way male and female editors manage and assess manuscripts, females are underrepresented in this role, and we recommend that women be encouraged to take on editorial positions. Representation of women in senior positions more accurately reflects the contribution of women to the discipline. In addition, reviewer selection criteria should be made consistent among editors, perhaps via increased formal training of editors.  Acknowledgements This work was conducted as part of the “Role of publication-related biases in ecology” Working Group supported by the National Center for Ecological Analysis and Synthesis, a Center funded by NSF (grant #DEB0072909). Thank you to ND and AA at Elsevier for assisting us with the contact of editors. Many thanks to all the individuals who participated in the survey.  References Aarssen LW, Tregenza T, Budden AE, et al. 2008. Bang for your buck: rejection rates and impact factors in ecological journals. Open Ecol 1: 14–19. Budden AE, Tregenza T, Aarssen LW, et al. 2007. Double-blind review favors increased representation of female authors. Trends Ecol Evol 23: 4–6. Cassey P and Blackburn TM. 2004. Publication and rejection among successful ecologists. BioScience 54: 234–39. Dalton R. 2006. Societies spurn women editors. Nature 440: 974–75. Dickersin K, Fredman L, Flegal KM, et al. 1998. Is there a sex bias in choosing editors? J Am Med Assoc 280: 260–64. Evans AT, McNutt RA, Fletcher SW, et al. 1993. The characteristics of peer reviewers who produce good-quality reviews. J Gen Intern Med 8: 422–28. Fisher M, Friedman SB, and Strauss B. 1994. The effects of blinding on acceptance of research papers by peer review. J Am Med Assoc 272: 143–46. Freda MC and Kearney M. 2005. An international survey of nurse editors’ roles and practices. J Nurs Scholarship 37: 87–94. Garrow J, Butterfield M, Marshall J, et al. 1998. The reported training and experience of editors in chief of specialist clinical medical journals. J Am Med Assoc 280: 286–87. Gilbert JR, Williams ES, and Lundberg GD. 1994. Is there gender bias in JAMA’s peer review process? J Am Med Assoc 272: 139–42. Grod ON, Budden AE, Tregenza T, et al. 2008. Systematic variation in reviewer practice according to country and gender in the field of ecology and evolution. PLoS One 3: e3202. doi:10.1371/journal.pone.0003202. Kassirer JP. 2001. Why be a medical editor? J Am Med Assoc 285: 2253–54. Kliewer MA, Freed KS, DeLong DM, et al. 2005. Reviewing the reviewers: comparison of review quality and reviewer characteristics at the American Journal of Roentgenology. Am J Roentgenol 184: 1731–35. Leimu R and Koricheva J. 2005. What determines the citation frequency of ecological papers? Trends Ecol Evol 20: 28–32. Lloyd ME. 1990. Gender factors in reviewer recommendations for manuscript publication. J Appl Behav Anal 23: 539–43. Lortie CJ, Aarssen LW, Budden AE, et al. 2007. Publication bias and merit in ecology. Oikos 116: 1247–53. NSF (National Science Foundation). 2009. Women, minorities, and persons with disabilities in science and engineering: 2009. Arlington, VA: NSF. www.nsf.gov/statistics/wmpd/. Viewed 24 Jul 2009. Tite L and Schroter S. 2007. Why do peer reviewers decline to review? A survey. J Epidemiol Commun H 61: 9–12. Wilkes MS and Kravitz RL. 1995. Policies, practices, and attitudes of North American medical journal editors. J Gen Intern Med 10: 443–50. esa Podcasts Have you visited the newly designed ESA podcast site? The Ecologist Goes to Washington showcases the stories of ecologists involved in public policy. Field Talk tells the tales behind the science published in Ecology, Ecological Applications, and Ecological Monographs. Beyond the Frontier features interviews with authors of recent articles in Frontiers in Ecology and the Environment. C h e c k o u t a l l t h e p o d c a s t s a t w w w. e s a . o rg / p o d c a s t www.fr ontiersinecology.or g © The Ecological Society of America