Journal Rankings and Representation in Mathematics Education

East Tennessee State University Digital Commons @ East Tennessee State University ETSU Faculty Works Faculty Works 1-26-2016 Journal Rankings and Representation in Mathematics Education Samuel Otten University of Missouri Ryan Andrew Nivens East Tennessee State University, [email protected] Follow this and additional works at: https://dc.etsu.edu/etsu-works Part of the Curriculum and Instruction Commons, and the Science and Mathematics Education Commons Citation Information Otten, Samuel; and Nivens, Ryan Andrew. 2016. Journal Rankings and Representation in Mathematics Education. Philosophy of Mathematics Education Journal. Vol.31 1-8. http://socialsciences.exeter.ac.uk/education/research/centres/stem/publications/pmej/ pome31/Nivens%20and%20Otten%20%20Journal%20Rankings%20and%20Representation.doc ISSN: 1465-2978 This Article is brought to you for free and open access by the Faculty Works at Digital Commons @ East Tennessee State University. It has been accepted for inclusion in ETSU Faculty Works by an authorized administrator of Digital Commons @ East Tennessee State University. For more information, please contact [email protected]. Journal Rankings and Representation in Mathematics Education Copyright Statement Copyright by authors. Permission is given to freely copy the journal contents on a not-for-profit basis, provided full credit is given to the author and the journal, and the integrity of papers is preserved. This document was originally published in the Philosphy of Mathematics Education Journal. This article is available at Digital Commons @ East Tennessee State University: https://dc.etsu.edu/etsu-works/288 JOURNAL RANKINGS AND REPRESENTATION IN MATHEMATICS EDUCATION Ryan A. Nivens & Samuel Otten East Tennessee State University, USA & University of Missouri – Columbia, USA nivens @ etsu.edu & ottensa @ missouri.edu Introduction Publish or perish has long been the mantra academics live by. For decades, this referred to a list of publications on a researcher’s Curriculum Vitae. One can assess the impact of a publication on the scholarly literature by looking at citation counts – that is, how many times a publication has been cited by other peer-reviewed publications (Garfield, 1963). In the modern age of technology and big data, however, new ways to measure scholarly output and impact have led to the creation of large databases. Moreover, the sources and targets of the citations can be identified and aggregated in various ways (Jiménez-Fanjul, Maz-Machado, & Bracho-López, 2013). University administrators are having discussions about how to use these technological capabilities to rate the scholarship of their professors and researchers (Howard, 2013; Shapiro, 2006). Looking at databases of scholarly journals, education and educational research is a relatively well-represented social science field. Within that community, there has frequently been talk of top-tier journals and what constitutes quality research (e.g., Hostetler, 2005). Mathematics education is typically treated as a sub-field of education and educational research. As such, it can be more difficult to find specific and comprehensive information on the relevant journals. Toerner and Arzarello (2012) presented the results of a survey completed by 75 experts in mathematics education who were asked to assign a letter grade to various journals with an international scope in the field. Their survey initially included 28 journals, but the results yielded rankings for only 17 because the respondents were unfamiliar with some of the journals. Additionally, Williams and Leatham (unpublished manuscript) conducted a similar survey around the same time. They produced a set of top-tier, mid-tier, and low-tier journals in mathematics education based on survey responses and citation counts. Although unpublished, their work has been shared with many scholars and used in tenure-and-promotion dossiers. Anecdotally, both studies found Educational Studies in Mathematics (ESM) and the Journal for Research in Mathematics Education (JRME) to be more highly regarded by mathematics education scholars than the other journals. However, these survey studies provide only a snapshot in time, and only from a subset of the researchers in the field. New databases, such as the Social Science Citation Index, keep a current list of journals that can be helpful for scholars to see the reputation of journals in their field. This can be helpful, since researchers usually know intimately only a subset, which might introduce regional bias or impartial comparisons based only on second-hand experience with some journals. Additionally, these databases are updated yearly. The balance of this article presents our research study in analyzing the current status of journal rankings in mathematics education, focused on the following question: To what extent do journal rankings and representation change over time, in particular the years 2010 – 2013? JOURNAL RANKING SYSTEMS While there are three major journal ranking systems currently in use: Web of Science, Scopus, and Google Scholar Metrics, we focus exclusively on Scopus for this study because it is freely available, and according to their webpage Scopus is the world’s “largest abstract and citation database of peer-reviewed literature: scientific journals, books and conference proceedings,” spanning “the fields of science, technology, medicine, social sciences, and arts and humanities.” Additionally, they list 5,000 publishers, 21,915 titles, and over 53 million records. The SCImago Journal Rank (SJR) is Elsevier’s indexing metric that uses the Scopus database. As such, we treat the SJR and Scopus database as one indexing source for our article and refer to it as Scopus throughout the paper. The SJR algorithm assigns journals prestige values and then begins adjusting the value with each citation from one journal to another, giving more value to citations from within the same field. As journals gain prestige, the citations from that journal begin to carry more weight. This algorithm continues until a relative steady-state is reached. At this point, the journal prestige measure is standardized by taking into account the number of publications contained in each journal over the relevant timespan (Butler, 2008). METHOD Looking through the Scopus database, we used the Social Science subject area, Education sub-category, and downloaded each dataset for years 2010 to 2013. To focus of our study on the field of mathematics education in particular, we searched for those journals that specifically target mathematics education, but we did not include general education research journals such as the Educational Researcher or Journal of Teacher Education. In our opinion, these journals are best compared to other general education journals rather than discipline-specific journals. Using these datasets, we identified the mathematics education journals as described in another study (see Nivens & Otten, in press). For each journal present in the database, we collected all journal statistics available. This resulted in the four datasets presented below. RESULTS Scopus Journal Rankings Across Time In answering our research question, we started with the Scopus 2010 dataset we compiled. It was the smallest of the four-year span. As you can see in Table 1, the representation of mathematics education grew over time. The Scopus database contains both journals and conference proceedings. We looked at the years 2010 through 2013 within the Social Sciences subject area and the education sub-category. The entire dataset is available to the public and can be found here. (link to: http://www.scimagojr.com/journalrank.php?area=3300&category=3304&country=all&ye ar=2012&order=sjr&min=0&min_type=cd). For our study, we note that within this subset that 2010 has 803 journals & proceedings listed, 2011 has 932, 2012 has 987, and 2013 has 1035. Table 1. Scopus Journals and Proceedings 2010-2013 Year Total number of journals & proceedings Total number of mathematics educationspecific journals & proceedings Number of Mathematics Education journals by Quartile Rank Q1 Q2 Q3 Q4 2010 2011 2012 2013 803 932 987 1035 16 19 22 23 2 2 4 5 5 7 2 3 4 5 12 8 5 5 4 7 Each of these lists were compiled in spreadsheet files from which we parsed out lists of mathematics education journals, keeping them in order of the SJR journal rank indicator. Table 2 shows the results of the 2013 dataset. The color indicates the quartile of the larger set the journal falls within. The “H index” is the Hirsch index as described in Hirsch (2005). “Total Docs 2013” is the number of articles in the journal that year, and “Total Docs 3 years” is the number of articles published in the three years 2011 – 2013. “Total Refs” is the number of references cited within all articles within the journal. “Total Cites 3 years” is the number of citations garnered from other sources for the articles contained within the journal over the three years 2011-2013. “Citable Docs 3 years” is the total number of articles available for citation counts. This column is slightly lower than the Total Docs 3 years column due to editorials and reviews being excluded. “Refs/Doc” is the average number of references per document published in a journal in the year. Table 2. 2013 Scopus data for journals in mathematics education. (Title abbreviations are used, see Appendix A for full names). Scopus Rank 32 116 185 189 209 321 354 390 461 469 470 473 497 523 524 680 694 704 711 712 721 756 789 Title JRME JMTE JMB IJSME MTL ZDM EJMST E MERJ RME CJSMT E JSE JMD IJMES T TMA FLM BMEB IJISME TS CMJ SERJ IEJME AMAP N IJTME ISSN 00218251 13864416 07323123 15710068 10986065 18639704 Quartile Q1 Q1 Q1 Q1 Q1 Q2 SJR 2.019 1.042 0.766 0.759 0.723 0.496 H index 44 14 24 15 7 14 Total Docs (2013) 34 38 52 131 15 84 Total Docs (3 years) 62 96 72 178 44 143 Total Refs 1,545 1,483 2,220 5,651 603 3,213 Total Cites (3 years) 91 81 71 174 50 93 Citable Docs (3 years) 57 77 72 173 42 132 Refs/ Doc 45.44 39.03 42.69 43.14 40.2 38.25 Country United States Netherlands United States Netherlands United Kingdom Germany 13058223 10332170 17540178 Q2 Q2 Q3 0.458 0.408 0.331 14 5 4 38 30 29 77 69 88 1,658 1,147 547 76 26 26 75 64 87 43.63 38.23 18.86 Turkey Netherlands United States 19424051 10691898 1735322 Q3 Q3 Q3 0.321 0.321 0.319 5 13 6 26 29 12 75 85 38 1,079 541 507 28 58 18 68 78 35 41.5 18.66 42.25 United States United States Germany 0020739X 14716976 02280671 19804415 22004270 14679639 07468342 15701824 13063030 Q3 Q3 Q3 Q3 Q4 Q4 Q4 Q4 Q4 0.295 0.276 0.276 0.188 0.178 0.174 0.166 0.163 0.158 11 6 3 1 3 2 3 2 2 135 22 0 65 28 38 59 9 6 313 57 47 150 39 51 181 22 24 2,056 377 0 1,437 929 333 384 317 237 82 16 16 12 10 11 21 5 8 308 57 38 137 36 39 168 18 19 15.23 17.14 0 22.11 33.18 8.76 6.51 35.22 39.5 United Kingdom United Kingdom Canada Brazil Australia United Kingdom United States New Zealand Turkey 8660182 17442710 Q4 Q4 0.146 0.137 8 1 10 19 82 19 159 328 15 3 80 15 15.9 17.26 Hungary United Kingdom The presence of journals indexed in Scopus in the field of mathematics education has grown over the years. The Scopus Education category grew nearly 29% from 2010 to 2013, but the number of mathematics education journals increased by nearly 44% in the same span. The representation by country also increased as shown in Table 3. In 2010, seven countries were represented yet by 2013 there are ten represented. All of the countries listed in 2010 increased in representation by 2013, and three new countries were added. Table 3. Representation by Country in Scopus in 2010 and 2013 2010 Country United States United Kingdom Netherlands Germany Turkey Australia Hungary 2013 Number of Journals 4 3 3 2 2 1 1 Country United States United Kingdom Netherlands Germany Turkey Australia Brazil Canada Hungary New Zealand Number of Journals 6 5 3 2 2 1 1 1 1 1 IMPLICATIONS AND DISCUSSION The Williams and Leatham (unpublished) survey contained all three NCTM practitioner journals, none of which were listed in the Toerner and Arzarello (2012) article, and which we did not find listed in the Scopus database, despite their importance to our field of research. Additionally, the Australian practitioner journals are also not found in any of these compilations. This indicates that Scopus is more specifically focused on research, which confirms the presence of research journals over practitioner journals. Another example, the NCSM Journal of Mathematics Education Leadership, which is intended for teacher leaders, was also not listed in Scopus across the years investigated. Another interesting outlet is Facebook. Through the use of Groups, researchers can join and discuss topics, share ideas, and even share titles of newly published books and articles. By sharing such things with audience members who are apt to cite your work, this sort of media becomes a news feed for researchers in the field. As an example, the group “Mathematics Education Researchers” on Facebook contains 1,374 members, “Mathematics Methods Discussion Group” contains 825 members, “Mathematics Education and Society” has 122 members, “Center for the Study of Mathematics Curriculum” has 47 members. You can also find many Facebook pages for the journals. Even more scholarly than Facebook groups are similar sites for researchers. In particular, ResearchGate is a site where researchers can follow one another, post articles, and easily share articles and ideas before or after publication. With sites such as ResearchGate, articles in conference proceedings that were not published for a year after a conference can now be made available immediately, even at the conference. The power of this resource is only in the early stages of use, and soon we will see just how well our field will use it. Jiménez-Fanjul, Maz-Machado, and Bracho-López (2013) conducted an in-depth study of four journals (JRME, ESM, BMEB, and RELIME) over a few decades. They looked at authorship and co-authorship patterns as well as language and institution. Our study provides a broader landscape of the journals in one of the largest databases available. Studies like theirs fill in more detail not focused exclusively on journals but about what is being produced and by whom. While their analysis tracked the quantity of articles produced over time and frequency of country representation, our study showed the increase of country representation over time. References Butler, D. (2008). Free journal-ranking tool enters citation market. Nature, 451, 7174, 6-6. doi:10.1038/451006a Garfield, E. (1963), New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14(3), 195-201. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569-16572. doi:10.1073/pnas.0507655102 Hostetler, K. (2005). What is “good” education research?. Educational Researcher, 34(6), 16-21. Howard, J. (2013). Rise of 'altmetrics' revives questions about how to measure impact of research. The Chronicle of Higher Education, Retrieved from https://login.ezproxy.etsu.edu:3443/login?url=http://search.proquest.com/docview /1370714491?accountid=10771 Jiménez-Fanjul, N., Maz-Machado, A., & Bracho-López, R. (2013). Bibliometric analysis of the mathematics education journals in the SSCI. International Journal of Research in Social Sciences, 2(3), 26-32. Accessed October 10, 2014 at http://www.ijsk.org/uploads/3/1/1/7/3117743/3_social_journals.pdf Nivens, R. A., & Otten, S. (in press). Assessing Journal Quality in Mathematics Education. Journal for Research in Mathematics Education. Shapiro, H. N. (2006). Promotion & tenure & the scholarship of teaching & learning. Change: The Magazine of Higher Learning 38(2), 38-43. Toerner, G., & Arzarello, F. (2012, December). Grading mathematics education research journals. Newsletter of the European Mathematical Society, 86, 52-54. Accessed October 10, 2014 at http://www.ems-ph.org/journals/newsletter/pdf/2012-1286.pdf Williams, S., & Leatham, K. (n.d.). Report on venue study. Provo, UT: Department of Mathematics Education, Brigham Young University Appendix A ACRONYM Journal URL Year founded # issues per year Avg # articles/issue AMAPN Acta Mathematica Academiae Paedagogicae Nyíregyháziensis http://www.emis.de/journals/AMAPN/index.html 1998 2 14 BMEB BOLEMA: Mathematics Education Bulletin http://www.scielo.br/scielo.php?script=sci_serial&pid=0103636X&lng=en&nrm=iso 1986 4 16 CJSMTE Canadian Journal of Science, Mathematics and Technology Education http://www.tandfonline.com/toc/ucjs20/current#.VDarwOe7nzU 2001 4 6 CMJ College Mathematics Journal http://www.maa.org/pubs/cmj.html 1970 5 10 EJMSTE EURASIA Journal of Mathematics, Science & Technology Education http://www.ejmste.com/ 2005 6 19 FLM For the Learning of Mathematics http://flm-journal.org/ 1981 3 9 IEJME International Electronic Journal of Mathematics Education http://www.iejme.com/ 2006 3 6 IJISME International Journal of Innovation in Science and Mathematics Education http://openjournals.library.usyd.edu.au/index.php/CAL 1997 6 7 IJMEST International Journal of Mathematical Education in Science and Technology http://www.tandfonline.com/toc/tmes20/current#.VDare-e7nzU 1970 8 8 IJSME International Journal of Science and Mathematics Education http://www.springer.com/education+%26+language/mathematics+ 2003 education/journal/10763 6 13 IJTME International Journal for Technology in Mathematics Education http://www.researchinformation.co.uk/time.php 2004 4 6 JMB The Journal of Mathematical Behavior http://www.journals.elsevier.com/the-journal-of-mathematicalbehavior/ 1980 4 10 JMD Journal fur Mathematik-Didaktik http://link.springer.com/journal/13138 1980 2 6 JMTE Journal of Mathematics Teacher Education http://www.springer.com/education+%26+language/mathematics+ 1998 education/journal/10857 6 5 JRME Journal for Research in Mathematics Education http://www.nctm.org/publications/toc.aspx?jrnl=jrme 1970 5 3 JSE Journal of Statistics Education http://www.amstat.org/publications/jse/ 1993 3 5 MERJ Mathematics Education Research Journal http://www.springer.com/education+%26+language/mathematics+ 1989 education/journal/13394 4 8 MTL Mathematical Thinking and Learning http://www.tandfonline.com/toc/hmtl20/current#.VDas_ue7nzU 1999 4 5 RME Research in Mathematics Education http://www.tandfonline.com/toc/rrme20/current#.VDauCee7nzU 1999 3 10 SERJ Statistics Education Research Journal http://iase-web.org/Publications.php?p=SERJ 2002 2 5 TMA Teaching Mathematics and its Applications http://teamat.oxfordjournals.org/ 1982 4 5 TS Teaching Statistics http://onlinelibrary.wiley.com/journal/10.1111/%28ISSN%291467 1979 -9639 3 5 ZDM ZDM – The International Journal on Mathematics Education http://link.springer.com/journal/11858 7 11 1997