Descriptive Content Analysis of Studies on 21st Century Skills

SDU International Journal of Educational Studies Descriptive Content Analysis of Studies on 21st Century Skills Neslihan Ültay1, Necla Dönmez Usta2, Eser Ültay3 1 Giresun University Giresun University 3 Giresun University 2 To cite this article: Ültay, N., Dönmez Usta, N., & Ültay, E. (2021). Descriptive Content Analysis of Studies on 21st Century Skills.. SDU International Journal of Educational Studies, 8(2), 85-101. Doi: 10.33710/sduijes.895160 Please click here to access the journal web site... SDU International Journal of Educational Studies (SDU IJES) is published biannual as an international scholarly, peer-reviewed online journal. In this journal, research articles which reflect the survey with the results and translations that can be considered as a high scientific quality, scientific observation and review articles are published. Teachers, students and scientists who conduct research to the field (e.g. articles on pure sciences or social sciences, mathematics and technology) and in relevant sections of field education (e.g. articles on science education, social science education, mathematics education and technology education) in the education faculties are target group. In this journal, the target group can benefit from qualified scientific studies are published. The publication languages are English and Turkish. Articles submitted the journal should not have been published anywhere else or submitted for publication. Authors have undertaken full responsibility of article's content and consequences. SDU International Journal of Educational Studies has all of the copyrights of articles submitted to be published. SDU International Journal of Educational Studies Volume 8, Number 2, November 2021, Page 85-101 e-ISSN: 2148-9068 Descriptive Content Analysis of Studies on 21st Century Skills Neslihan Ültay1, Necla Dönmez Usta2, Eser Ültay3 1 Giresun University Orcid ID: 0000-0002-9783-0486 2 Giresun University Orcid ID: 0000-0002-8075-7446 3 Giresun University Orcid ID: 0000-0001-6839-6361 Accepted: 03.06.2021 Received:11.03.2021 Abstract The aim of the study is to bring together national studies on 21st century skills under the headings of “purpose, method, skill types, sampling, data collection tools, results, and recommendations”. Descriptive content analysis is done for 84 national studies on 21st century skills. Within the scope of the research, the data of each study were analyzed with the examination form. Each study was recorded in the form and then the tables were formed. According to the results of the study, the purpose of the studies with the highest frequency was “investigation of the relationship between 21st century skills and various variables”, the method with the highest frequency was “survey”, and the skills that were explained as “P21 skills” had the highest frequencies. While “teacher candidates” and “teachers” were the most preferred sample groups, “scales” were the most used data collection tools. Conclusions of the studies showed that 21st century skills were existing in our lives, education systems, and also textbooks more or less because 21st century skills were the skills that not apart from the life itself. Therefore, it was suggested in the studies that it was the goal of education to raise individuals in line with the demands of 21st century. Key words: 21st century skills, Descriptive content analysis, Meta-analysis *Corresponding Author: Neslihan Ultay, Giresun University, [email protected] INTRODUCTION Education in the new world order, with the effect of Covid-19, which is declared as a pandemic today; requires individuals who are equipped to meet changing needs. In a world that has become digital with globalization, learning is not limited to classrooms and schools, but can only be achieved by integrating the concept of lifelong learning into their lives with enriched learning environments in all areas of life (National Ministry of Education, 2019). The rapid progress of technological developments requires learners to adopt the concept of lifelong learning by making personal development a focal point (Özgüzel, 2018). Learners of the 21st century must have high-level cognitive skills such as innovation, creative, solution-oriented, highly motivated, strong in communication, technology, and information literacy, problem solving, and critical thinking (Aydeniz, 2017). These skills that learners of this century should have are called 21st century skills (National Research Council, 2012). Rotherham and Willingham (2009) stated that these skills, which allow the solution of many problems such as problem solving and critical thinking, are as old as human history and it is wrong to call them 21st century skills. At this point, just because the skills are not new does not mean they are not important. However, this situation is considered important in terms of the relevant skills are gradually increasing. The fact that these skills are called 21st century SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 85 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) skills shows that they are blended in the information age and reinterpreted according to the age (Çolak, 2019). The term “21st century skills” refers to a broad range of knowledge, skills, work habits, and character traits that are thought to be critical to success in today's world. Although there are different opinions in the literature about what 21st century skills are, there is a general trend (Dicerbo, 2014; Lai & Viering, 2012). In the 21st Century Learning Framework (The Partnership for 21st Century Skills P21) prepared for 21st century skills, the P21 framework (partnership for 21st century learning (P21), 2007) includes basic topics; learning and innovation skills consist of three main skill areas known as knowledge, media and technology skills and life and career skills (Yalçın, 2018; Kylonen, 2012; Trilling & Fadel, 2009). Combined skills in these three main skill areas are shown in Figure 1 below (URL-1, 2019). Figure 1. 21st century skills framework (P21) According to Figure 1, Critical thinking within the main skill of Learning and Innovation; is the ability to analyze, synthesize, and evaluate by using inductive and/or deductive methods within the framework of logic, and to make judgments using high-level cognitive skills (Facione, Facione, & Giancarlo, 2000). It is possible for individuals having this skill to express problems clearly (Beyer, 1988) and transfer what they learn to their daily lives (Çalışkan, 2009). Problem solving is the realization of all solution steps in order to reach the determined goal or solve the problem (Haladyna, 1997). Individuals with this skill can produce solutions to the problem using existing or innovative methods (Yalçın, 2018). Communication skill is to use all verbal, non-verbal or written communication skills effectively, to communicate by keeping communication in maximum interaction and to be a good listener (Trilling & Fadel, 2009). Individuals having this skill can express themselves by using their communication skills effectively. Collaboration is the ability to work together within a common goal and to make effective contributions (Partnership for 21st Century Learning, 2015). Individuals having this skill can produce different ideas and develop solutions by working effectively with group members with different characteristics (Yalçın, 2018). Creativity is related to the methods developed by the learners to reach the solution of the problems are unique rather than mediocre (Karakuş, 2011). Individuals with this skill are open to innovations, self-confident and courageous to take risks (Özden, 2014). Information literacy included in the Information, Media, and Technology main skill is the production, use and evaluation of 21st century information on the basis of lifelong learning (Polat, 2005).Individuals who have this skill can learn independently. Media literacy is the analysis, SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 86 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) synthesis and evaluation of messages, and the production of new contents and sharing them in the media (Karaman, 2016). Individuals having this skill can make sense of messages and reach the suitable outcome by using the media effectively (Çolak, 2019). Information and technology literacy is the use of digital tools to access all forms of information, to reconstruct and evaluate information (Eryılmaz & Uluyol, 2015). Individuals with this skill can use digital tools for their intended purpose. Being within the scope of the Life and Career main skill, flexibility and adaptability are being able to approach a problem from different perspectives and adapt to this situation (Özden, 2014). Individuals with this skill are also prone to differentiation (Tomlinson, Brimijoin, & Narvaez, 2008). Initiative and self-direction, working independently in line with the gains, means clarifying the success criteria by distinguishing concrete and abstract goals. Individuals with this skill can turn lifelong learning into a process by showing their dependence on learning (Yalçın, 2018). Social and cross-cultural skills are the processes of initiating, maintaining, and ending interactions as well as including individuals and intercultural qualities (Zengin, Kırılmazkaya, & Zengin, 2012). People with this skill are accepted by their peers and they can adapt to the environment they live in becomes easier (Deniz, 2003). Productivity and accountability are the ability to undertake possible results in line with the determined gains and to create products as a result of ethical work (Eryılmaz & Uluyol, 2015). Individuals with this skill have the ability to work efficiently by managing time well in a work and to take responsibility for possible results. (Yalçın, 2018). Leadership and responsibility is the ability to adapt to innovations, to direct possible changes, to work efficiently in line with the determined gains and to guide them to use their potential to reach the target and to take responsibility (URL-2, 2020). Individuals with this skill feel responsible towards the group they lead and display an ethical behavior. As seen in Figure 1, these skills are directly related to assessment and evaluation, courses and curricula, professional training and learning environments. Having 21st century skills is directly related to the 21st century learners’ education they receive. At this point, the results of studies about 21st century skills are important for 21st century learners, teachers, and individuals in terms of shaping, organizing, or structuring education. Researchers and practitioners have been paying increasing attention to 21st century skills. For example, the current Hong Kong school curriculum emphasizes the importance of students' creativity development, and as a result, teachers are encouraged to develop or adopt innovative teaching methods to foster students' creativity in the classroom (Chan & Yuen, 2014). However, no model exists at this time for teaching the core 21st century skills in schools (Qian & Clark, 2016). On the other hand, Bybee (2009) suggested to use 5E model for 21st century skills development and stated that it is possible to draw some research-based inferences from the parallels between 21st century skills such as problem solving, self-motivation, communication, and system thinking and learning outcomes in science such as scientific reasoning, interest, and argumentation. School science programs that include activity-based teaching models have the potential to improve 21st century skills. Providing model curriculum materials that exemplify the objectives, changing teachers' perceptions of teaching explicitly to develop skills and abilities, and fostering fidelity to teaching models designed to help students are among the main challenges associated with this claim (Bybee, 2009). In educational settings, it may be useful to distinguish 21st century skills and to create some activities for the development of the specific skill. But in practice, it is difficult because 21st century skills are closely linked to each other. Real education is seeing connections between ideas, concepts, and disciplines in ways that help students understand relationships and how ideas relate to people and new situations. Therefore, while dealing with one 21st century skill, others engage in different skills. According to Germaine, Richards, Koeller and Schubert-Irastorza (2016), as the 21st century progresses, good teachers will seek ways to thoughtfully embrace the 4Cs and encourage more critical thinking, better communication, collaboration, and creativity in their professional lives as well as the quality of their students' learning experiences. SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 87 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) If we look at what has been done in the literature on a subject that has been so popular in recent years, it is seen that most of the studies are done to reveal the relationship of 21st century skills between different skills and different variables (Akman, 2019; Aydın, 2019; Elekoğlu & Demirdağ, 2020; Erdoğan & Cevat, 2020; Gökbulut, 2020; Kardeş, 2020; Kıyasoğlu & Ay, 2020; Korucu & Ünüvar, 2020). Altınpulluk and Yıldırım (2021) conducted a descriptive analysis of the studies about 21st century skills carried out in between 2010-2019 years searched in the Web of Science Social Sciences Citation Index (SSCI). In addition, it was observed that other descriptive content analysis, meta synthesis and meta-evaluation studies, which are synthesizing the studies on 21st century skills, were not conducted at national level (Eskici & Özsevgeç, 2019; McGrath & Fischetti, 2019) and not included the studies done in 2020. At this point, it is considered valuable to examine and present national studies on 21st century skills through descriptive content analysis. In this context, in this study, the purpose, method, sample, data collection tools, results and suggestions of the studies at national level about 21st century skills were gathered and discussed from variable points. The study can provide a general knowledge by reading this article instead of reading 84 studies one by one. In addition, thanks to this study, researchers can realize the gap in the literature and draw a road map for themselves. In this respect, the study will not only guide researchers, but also present studies on 21st century skills in our country in a descriptive framework, and will contribute to increasing the quality of their studies. In this context, the aim of the study is to bring together national studies on 21st century skills under the headings of “Purpose, Method, Skills, Sampling, Data Collection Tools, Results, and Recommendations”. For this purpose, the study was directed in the light of the following questions: • • • • • • • For what purposes were the studies on 21st century skills carried out? Which methods have been used in studies on 21st century skills? Which of the 21st century skills have been discussed? Which sample groups were used in studies on 21st century skills? Which data collection tools were used to collect data in studies on 21st century skills? What kinds of results have been obtained from studies on 21st century skills? What suggestions have been made in studies on 21st century skills? METHODOLOGY The Method of the Research In the broadest sense, content analysis is divided into three categories: Meta-analysis, meta-synthesis and descriptive content analysis (Çalık & Sözbilir, 2014). Meta-analysis is a quantitative application that includes statistical processes for combining, synthesizing, and interpreting the experimental findings of separate studies conducted on the same subject at different times and locations (Wolf, 1986). Meta-synthesis studies include qualitative research in a specific field, as well as a comparison of similarities and differences. Descriptive content analysis is the examination and organization of qualitative and quantitative studies on a specific subject, as well as the identification of broad trends in the field (Çalık, Ünal, Coştu & Karataş, 2008; Selçuk, Palancı, Kandemir & Dündar, 2014). In this study, it is aimed to describe how the trends of 21st century skills studies conducted at national level between 2010 and 2020, and to compare the results within the specified frameworks, and to interpret them in a more comprehensible manner as a whole (Çalık & Sözbilir, 2014; Ültay, Akyurt & Ültay, 2021; Ültay, Dönmez Usta & Durmuş, 2017), so descriptive content analysis is preferred. Data Collection and Data Analysis SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 88 SDU IJES (SDU International Journal of Educational Studies) Ultay, Dönmez-Usta & Ultay This research was conducted to examine the studies on 21st century skills that were conducted at national level between 2010 and 2020. In order to reach the sample of the study, databases of ERIC, Education Source, Science Citation Index, ULAKBİM, Education Source, Science Direct, Google Scholar, Giresun University Library and National Thesis Center were used and national studies were determined. Databases had been scanned with the keyword of “21st century skills”, “P21 skills”, and “4C skills”. In studies accessed from databases, the condition of being an article, a thesis, a book section, or a conference paper was sought. Studies that are not accessible or that have difficulty in reaching the full text are not included in the sample. After these stages, research was carried out on a total of 84 national studies, including 58 articles, 15 master's thesis, 2 doctoral dissertation, 6 conference papers, 2 book sections and a report. This research is limited to the studies conducted between in 2010-2020 on national 21st century skills and can be accessed from relevant databases. In order to analyze the studies more easily, each study examined was coded as S1, S2, …, S84 and these codes were used in the study. Within the scope of the research, the data of each study were analyzed with the "Examination Form" developed by the researchers. While the examination form was being developed, it was finalized within the scope of the necessary feedback by taking the opinions of the field experts. Since the aim of the study is to describe what about the studies on 21st century skills are and to interpret the existing situation, the final version of the related themes was created as indicated in Table 1. In addition, an example of the analysis of the studies in the examination form is presented in Table 1 below. Study Code S2 Year 2013 Skills Sample Table 1. Study examination form Author Purpose Method Gülen Investigation of the relationship Survey between 21st century skills and various variables Data Conclusions Recommendations Collection Tools Learning and Middle school Scale innovation, students Information, media, and technology There is/is not a significant It is possible to relationship between the students' determine how level of having 21st century skills 21st century skills and various variables studied (IT are expressed in skills, science self-efficacy educational beliefs, TIMMS science programs. achievement, etc.) Reliability and Validity The studies used in the research were examined in different times and places and the data obtained as a result of the examinations were recorded with the examination form. In order to ensure the reliability between two valuers, the Cohen's Kappa (Cohen's Kappa Coefficient) value between the two readers was calculated with the SPSS v.22 package program. Calculated Cohen's Kappa (κ) .857 was found and according to the classifications of Landis and Koch (1977), it was seen that almost perfect agreement was achieved. In this way, the reader reliability of the study was tried to be achieved. Apart from the reliability coefficient, two science education experts were included in the all phases of the study for ensuring the study’s validity. These experts were checked the studies’ suitability for 21st century skills topic and then controlled all the tables. SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 89 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) FINDINGS In this section, the findings of the descriptive content analysis of the studies about 21st century skills are summarized and shown in tables. The first research question was tried to be answered by Table 2 showing the purposes of the studies. Table 2. The purposes of the studies Purposes Study Code st Investigation of the relationship between 21 S2, S5-6, S9, S13-15, S19, S21, S26, S29, S32, S40, S46, S50, S52, S54, S59-61, S63century skills and various variables 65, S68-71, S73-76, S84 Determining the level of having 21st century S2, S7-8, S11-12, S16, S24, S26, S32, S37, S40, S43, S45, S47, S50, S52, S54-55, S57, skills or using these skills S60, S68-69, S73, S76-77 Investigation of the inclusion and applicability S1, S20, S23, S33-34, S36, S39, S42, S45, of S48, S51, S53, S56, S78 21st century skills in course contents or textbooks Researching how 21st century skills are defined S3-4, S16-18, S22, S27-28, S35, S41, S49, S66, S83 Examining the development of 21st century S25, S30-31, S38, S62, S67, S82 skills with experimental research A valid and reliable scale development study S10, S44, S58, S72, S80-81 for 21st century skills ƒ 32 25 14 13 7 6 According to Table 2, the purposes of the studies about 21st century skills are varied from “Investigation of the relationship between 21st century skills and various variables” with the highest frequency to “A valid and reliable scale development study for 21st century skills” with the lowest frequency. The second highest frequency belongs to the purpose of “Determining the level of having 21st century skills or using these skills”. Table 3 shows the distribution of methods of the studies about 21st century skills. Table 3. The methods of the studies Methods Study Code S2, S12-14, S19, S24, S26, S28, S37, S47, S50, Survey S56, S60, S63, S68, S84 S13, S19, S21, S29, S43, S46, S52, S59, S61, Correlational survey method S63, S65, S69, S71, S75, S77 Descriptive survey method S4, S8, S18, S25, S32-33, S54, S57, S70, S7374 Case study S3, S9, S11, S16-17, S30, S41, S55, S62, S83 Document analysis / Content analysis S20, S33-34, S36, S39, S42, S48, S51, S53, S78 Method not specified (studies conducted for S1, S5-6, S15, S22, S27, S35, S49, S66 giving information) Scale development study S10, S44, S58, S72, S80-81 Causal-comparative method S13, S19, S60, S70, S76 Quasi-experimental method S31, S38, S82 Weak experimental method S40 Descriptive mixed design (survey + S79 phenomenology) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 90 ƒ 16 16 11 10 10 9 6 5 3 1 1 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) As seen Table 3, the most used methods are “survey, correlational survey, and descriptive survey” methods. It can be said that the quantitative researches are predominantly preferred. Table 4 shows which 21st century skills are discussed in the studies. Table 4. The 21st century skills discussed in the studies Skill Type Study Code ƒ Learning and innovation S3-4, S9-11, S16-18, S20-25, S27-30, S3254 33, S35-42, S44-45, S47, S49, S51, S53, S5557, S62-63, S66-72, S74, S77-81, S83-84 S1, S3-6, S8-11, S14, S16, S18, S20-25, S28- 51 Information, media and technology skills 29, S32-35, S38-42, S44-45, S47, S49, S51, S53, S56, S63, S66-70, S72, S74, S77-81, S83-84 Life and career skills S3-5, S9-11, S16, S18, S20-21, S23-25, S27- 46 29, S32-33, S35, S37-42, S44-45, S47, S49, S51, S53, S56, S63, S66-68, S70-72, S74, S77-79, S81, S83-84 Teacher skills (Administrative skills, S13, S19, S26, S43, S50, S52, S64, S73, S75 9 technopedagogical skills, affirmative skills, flexible teaching skills and generative skills) Learner skills (Cognitive skills, autonomous S13, S19, S59-61, S73 6 skills, collaboration and flexibility,and innovativeness skills) Key subjects and 21st century themes S22, S27, S35, S66, S71 5 Active learning, problem solving, learning to S2, S12, S31, S54, S65 5 learn, collaboration and communication skills Cognitive, affective and sociocultural Knowledge, skill, character, meta-learning S7, S46, S48, S82 S15, S58, S76 4 3 According to Table 4, the skills that were explained as P21 skills (learning and innovation, information, media and technology skills, life and career skills) have the highest frequencies. Apart from P21 skills, teacher and learner skills are discussed in some studies. Knowledge, skill, character, and meta-learning skills has the lowest frequency. Table 5 shows the sample groups which were used in the studies. Sample Groups Teacher candidates Teachers Middle school students High school students Undergraduate students Education Faculty instructors Preschool students Business managers Table 5. The sample groups of the studies Study Code S3, S10-11, S13, S16, S19, S21, S24-25, S28-30, S32, S37, S45, S47, S55, S57, S59, S62-63, S67-68, S70, S79-80 S8, S13, S23, S26, S40-41,S43-44, S50, S52, S61, S64, S71, S73, S75, S83 S2, S7, S12, S31, S38, S46, S54, S65 S9, S60, S74, S81-82 S58, S69, S76, S84 f 26 16 8 5 4 S4, S56 2 S72 S77 1 1 When Table 5 is examined, it is seen that for 21st century skills studies, “teacher candidates” were mostly preferred as sample groups. The second highest frequency belongs to the “teachers” groups. SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 91 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) Preschool students and business managers have the lowest frequency as sample groups. Table 6 shows data collection tools of the studies. Table 6. Data collection tools of the studies Study Code S2, S4, S7-8, S10, S12-13, S19, S21, S23-26, S28-32, S37-38, S40, S43-47, S50, S52, S54, S56-61, S63-65, S67-77, S79-82, S84 Document analysis S3, S16, S18, S20, S33, S36, S39, S42, S48, S51, S53, S78 Interview S3, S7, S9, S23, S41, S45, S55, S62, S67-68, S79, S83 Literature review S14, S17, S22, S34-35 Open-ended questions S4, S11 Student product S11, S30 (Animation, presentation) Achievement test S46 Observation S55 Field notes S55 Mind maps S62 Diary S62 Data Collection Tools Scale f 54 12 12 5 2 2 1 1 1 1 1 As seen in Table 6, “scale” have the highest frequency and it means scales were mostly used in the studies as data collection tools. “Achievement test”, “observation”, “field notes”, “mind maps”, and “diary” have the lowest frequency. Table 7 shows the conclusions of the studies. Table 7. Conclusions of the studies Conclusions Study Code It was observed that the participants had adequate / S4, S7, S12-13, S24, S26, S32, S37, S40-41, S43, S45, S47, S50, S52, S54, inadequate knowledge about 21st century skills. S60-61, S64, S68-69, S73, S76-77, S8384 There is/is not a significant relationship between the S2, S6, S8, S21, S29, S46-47, S59-60, students' level of having 21st century skills and S63, S65, S68, S70-71, S73-76, S84 various variables studied (IT skills, science selfefficacy beliefs, TIMMS science achievement, etc.) It has been observed / thought that the application S9, S11, S14, S25, S30-31, S38, S55, has a positive / neutral effect on the development of S62, S67, S82 all / some of the 21st century skills. It was concluded that the curriculum / textbook S20, S23, S33, S36, S39, S42, S48, S51, examined reflected / did not reflect some of the 21st S53, S56, S78-79 century skills. Teacher and learner skills are determined according S3, S16-19, S22, S34, S37 to 21st century skills. Gaining a contemporary perspective and being S1, S6, S27, S34, S41, S53 successful depends on developing 21st century skills. Teachers / schools / parents should contribute more S5, S15, S27, S28, S49 for students to acquire 21st century skills. It has been observed that the measurement tool S10, S44, S58, S72, S80-81 developed to measure 21st century skills has sufficient psychometric properties. S35, S57 Measurement tools that can be used to measure 21st century skills have been introduced / prepared. SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 92 ƒ 26 19 11 12 8 6 5 6 2 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) It is understood that the targets that are planned to be S66 brought to the new generations are developed in line with market demands rather than an educational philosophy. 1 Conclusions on certain topics are given according to Table 7. The most repetitive effect is that the participants have adequate or inadequate knowledge of the skills of the 21st century. The least frequent effect is that the priorities identified for the new generations are determined not according to the philosophies of education, but according to the wishes of the corporations. Table 8 shows the recommendations of the studies. Recommendations Tablo 8. Recommendations of the studies Study Code f To keep up with the modern era, to ensure this, 21st century skills must be mastered or the appropriate teaching atmosphere must be organized. S1, S3-9, S11, S13-16, S18, S22-24, S27-29, 47 S32-33, S36, S38-41, S45-46, S49-50, S5255, S57, S60, S62, S68-70, S75, S78-80, S8384 The relationship of 21st century skills with different variables can be revealed. It is possible to determine how 21st century skills are expressed in educational programs To assess 21st century skills, various measurement instruments have been suggested. Pre-service and in-service training should be given to teachers and teacher candidates in order to teach 21st century skills. Studies should be conducted to address the different levels/degrees of skills of the 21st century. S3, S7, S19, S21, S25-26, S30-31, S34, S47, 16 S59, S63-64, S67, S73, S76 2, S17, S20, S23, S27-28, S32, S34, S36-37, 14 S39, S43, S51, S74 S10, S35, S42, S44, S58, S72, S81 7 S8, S29, S56-57, S65, S71, S75 7 S2, S12, S48, S81-82, S84 6 As shown in Table 8, the highest frequency is “to keep up with the modern era, to ensure that the skills of the 21st century must be mastered or that the appropriate teaching atmosphere must be organized”. “Studies should be conducted to address the different levels/degrees of skills of the 21st century” have the lowest frequency. DISCUSSION, CONCLUSIONS AND RECOMMENDATIONS In this research, descriptive content analysis has been carried out to bring together 84 national studies on 21st century skills published in between 2010-2020. These national studies are examined some key themes such as purposes, methods, skill types, data collection tools, conclusions, and recommendations. According to the results of this research, the purposes of the studies are highly concentrated on the investigation of the relationship between 21st century skills and various variables. The reason why the goals are concentrated around this goal may be that 21st century skills studies can be said as new for Turkey. From a more broad view, 21st century skills are not new for any country, these skills have been existed in programs, objectives, and textbooks for a long time. Critical thinking and problem solving, for example, have always been important parts of human progress, from the invention of basic tools to agricultural advancements, vaccine development, and land and sea exploration (Rotherham & Willingham, 2009). In fact, not for critical thinking but for creativity, in Turkish National Educational Ministry, to grow up creative people had been existed in the SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 93 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) educational goals (National Ministry of Education, 1973). 21st century skills are not new but gaining importance is new (Silva, 2009). Going back to the purposes of the studies, it is seen that some studies are tried to define what 21st century skills mean. There are a lot of classification styles for 21st century skills in the literature. But most of the classifications include critical thinking, creativity/innovation, problem solving, collaboration, and communication (Ekici, Abide, Canbolat & Öztürk, 2017) (S17). The task of educators should be to prepare today's students for a global and participatory world based on life, learning, study and advanced technology by gaining these skills (Lemke, 2010). When the methods of the studies are considered, it is seen that survey and some survey types are mostly preferred. This is not surprising, because a similar case is seen in some content analysis studies (i.e Ültay & Uludüz, 2018; Ültay, Durnacı & Ültay, 2019). Survey method studies are highly used in quantitative designs. It is not a valid point of view to say that a qualitative study is better than a quantitative one, and vice versa. But it can be said that a qualitative study requires more effort, time, and energy. Therefore, researchers may prefer using quantitative designs for their searches. Apart from that, formerly, it is said that 21st century skills are not new but gaining importance is new. This case can be clearly seen in the methods of the studies that some studies try to give information about 21st century skills. In the studies, as a data collection tool, scales are mostly used. It is compatible with doing a quantitative research. Aside from that, document analysis and interviews are used for data collection. The studies included in the content analysis show that “learning ad motivation,” “information, media and technology,” and “life and career” skills are mostly discussed. These 21st century skills are defined as P21 skills (Partnership for 21st century learning, 2007). While some studies call the 21st century skills with different names or use the different classification styles for 21st century skills, the common point is that most of them include some or all of P21 skills in themselves. For example, some studies use “teacher skills” and “learner skills” concepts but teacher and learner skills are still formed by P21 skills. Teacher and learner skills are evaluation of P21 skills from the perspectives of the teacher and the learner. While explaining teacher and learner skills, it is seen that Trilling and Fadel (2009) used P21 skills and do not present headings that are clearly different from each other, but that they put forward skill titles that explain and interact with each other. When the sample groups of the studies are taken into account, it is seen that in most of the studies teacher candidates and teachers are preferred. The reason for preferring teacher candidates and teachers as a sample may arise from them is easily accessible. In Turkey, academic searches are mostly done by faculty members such as research assistants, instructors, professors, and students of master of education or doctorate programs. For these researchers, teachers and teacher candidates are more accessible than the students in middle and high schools. In fact, this situation can be evaluated from a better perspective. The formation of scientific foundations in students depends on the teaching techniques used by teachers and of course teachers' candidates who are the teachers of future (Ünal & Akman, 2006). All in all, the majority of the purposes of the studies is the investigation of the relationships between 21st century skills and some variables. The other purpose is the determination of the levels of 21st century skills (see Table 2). When we see the methods of studies, it is seen that survey and some types of surveys are used (see Table 3). It is compatible with the purposes of the studies. The best way to reveal the levels of some characteristics is to use surveys. In addition, as we mentioned before teacher and teacher candidates who are the implementers or new skills or teaching methods/strategies are preffered (see Table 5). Therefore, it can be said that sample groups are also suitable with the purposes and methods. Apart from this, teacher and teacher candidates are the people who create a 21st classroom and they have to focus not to teach one more topic, but to teach in authentic contexts (Larson & Miller, 2011). In 21st century classrooms, students should have communicate and SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 94 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) collaborate online and offline environments. In order to do this, students communicate via social media networks such as Facebook, Instagram, Twitter, email, etc. According to Leu, Kinzer, Coiro and Cammack (2004), the accessibility to Internet does not decrease the importance of the teachers, just the roles of the teachers are changed. Teachers should be expert in all new technological techniques that can be used in the classrooms. According to the conclusions of the studies, it was found that, in most of the studies, whether the sample had 21st century skills at a sufficient level. Bozkurt and Çakır (2016) (S12) found that students has 21st century learning skills as active learning, problem solving, learning to learn, and collaboration and communication skills. Similarly, Önür ve Kozikoğlu (2019) (S54) concluded that the students' active learning skills were found to be at moderate level whereas it was concluded that learning to learn, problem solving, cooperation and communication skills are at high level. Different from that, the majority of studies found that 21st century skills affect information technology skills (Gülen, 2013; Eryılmaz & Uluyol, 2015) (S2 and S6). Lifelong learning also includes digital skills. In this sense, our teachers, who will educate our generation, should learn these skills, and apply them throughout their lives in order to keep up with our fast-paced age of information development and dissemination (Keskin & Yazar, 2015) (S8). Conclusions of the studies show that 21st century skills are existing in our lives, education systems, and also textbooks more or less because 21st century skills are the skills that not apart from the life itself. The age we are in is a period in which we can easily access information and there is a lot of information. During this period, most skills of 21st century skills are needed to access information (Karaşah Çakıcı & Yakışan, 2020) (S82). According to Colwill and Gallagher (2007), one of the main goals of education is to raise individuals in line with the demands of the 21st century. As a matter of fact, this is clearly seen in the recommendations (Zeybek , 2019; Özyurt, 2020) (S60, S80). Finally, this descriptive content analysis including of 2010-2020 years give information about national 21st century skills studies on what trend topics are studied about 21st century skills., what methods, samples and data collection tools are used. Additionally, which 21st century skills are discussed in these studies may give information to the researchers thinking about to study about 21st century skills. What conclusions are found can be seen easily and therefore, researchers have a quick idea about the positive and negative aspects of 21st century skills. What recommendations are suggested are also important for the researchers to draw a road map to do a study on 21st century skills. It is believed that the results of the research will provide an overview of the field of research and will therefore be particularly useful in terms of the recognition of the deficiencies. REFERENCES Akman, E. D. (2019). Sekizinci sınıf öğrencilerinin TIMSS fen sonuçlarının 21. yy beceri düzeyleri ve bazı değişkenler açısından incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Marmara Üniversitesi Eğitim Bilimleri Enstitüsü, İstanbul. *Alkış, M. (2020). Üniversite öğrencilerinin 21. yüzyıl becerilerinin bazı değişkenler açısından incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Necmettin Erbakan Üniversitesi, Eğitim Bilimleri Enstitüsü, Konya. (S76) Altınpulluk, H., & Yıldırım, Y. (2021). 2010-2019 yılları arasında yayınlanan 21. yüzyıl becerileri araştırmalarının incelenmesi. Anadolu Journal of Educational Sciences International, 11(1), 438-461. *Anagün, Ş. S., Atalay, N., Kılıç, Z., & Yaşar, S. (2016). Öğretmen adaylarına yönelik 21. yüzyıl becerileri yeterlilik algıları ölçeğinin geliştirilmesi: Geçerlik ve güvenirlik çalışması. Pamukkale Eğitim Fakültesi Dergisi, 40, 160-175. (S10) *Atalay, N., Anagün, Ş. S., & Kumtepe, E. G. (2016). Fen öğretiminde teknoloji entegrasyonunun 21. yüzyıl beceri boyutunda değerlendirilmesi: Yavaş geçişli animasyon uygulaması. Bartın Üniversitesi Eğitim Fakültesi Dergisi, 5(2), 405-424. (S11) *Atlı, K. (2019). Biyoloji dersi öğretim programının 21. yüzyıl becerilerinden yaratıcılık becerisi açısından değerlendirilmesi. Anadolu Öğretmen Dergisi, 3(1), 85-104. (S36) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 95 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) Aydeniz, M. (2017, Ekim). Eğitim sistemimiz ve 21. yüzyıl hayalimiz: Hedeflerine ilerlerken, Türkiye için STEM odaklı ekonomik bir yol haritası. University of Tennessee, Knoxville. *Aydın, A. (2019). İngilizce öğretmen adaylarının görüşleri çerçevesinde öğretmen eğitiminde 21. yüzyıl becerilerinin incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Hacettepe Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara. (S37) *Bal, M. (2018). Türkçe dersinin 21. yüzyıl becerileri açısından incelenmesi. Turkish Studies, 13(4), 49-64. (S20) *Bal, N. (2019). Temel robotik eğitiminin ortaokul öğrencilerinin 21. yüzyıl becerilerine ve bilgi işlemsel düşünme becerilerine etkisi. (Yayınlanmamış Yüksek Lisans Tezi). Hatay Mustafa Kemal Üniversitesi, Fen Bilimleri Enstitüsü, Hatay. (S38) *Başar, S. (2018). Fen bilimleri öğretmen adaylarının fende matematiğin kullanımına yönelik özyeterlik inançları, 21.yy becerileri ve aralarındaki ilişkinin incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Hacettepe Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara. (S21) *Belet Boyacı, Ş. D., & Güner, Özer, M. (2019). Öğrenmenin geleceği: 21. yüzyıl becerileri perspektifiyle Türkçe dersi öğretim programları. Anadolu Journal of Educational Sciences International, 9(2), 708738. (S39) Beyer, K. B. (1988). Developing a scope and sequence for thinking skills instruction. Educational Leadership. 45(7) 26-30. *Bozkurt, F. (2020). 21. yüzyıl becerileri açısından sosyal bilgiler öğretmenliği lisans programının değerlendirilmesi. Pamukkale Üniversitesi Eğitim Fakültesi Dergisi, 51, 34-64. (S79) *Bozkurt, Ş. B., & Çakır, H. (2016). Ortaokul öğrencilerinin 21. yüzyıl öğrenme beceri düzeylerinin cinsiyet ve sınıf seviyesine göre incelenmesi. PAU Eğitim Fakültesi Dergisi, 39, 69-82. (S12) Bybee, R. W. (2009). The BSCS 5E instructional model and 21st century skills. Colorado Springs, CO: BSCS *Cansoy, R. (2018). Uluslararası Çerçevelere Göre 21. Yüzyıl Becerileri ve Eğitim Sisteminde Kazandırılması 21st Century Skills According to International Frameworks and Building Them in the Education System. İnsan ve Toplum Bilimleri Araştırmaları Dergisi, 7(4), 3112-3134. (S22) *Cemaloğlu,N., Arslangilay, A.S., Üstündağ, M.T., & Bilasa, P. (2019). Meslek lisesi öğretmenlerinin 21. yüzyıl becerileri özyeterlik algıları. Ahi Evran Üniversitesi Kırşehir Eğitim Fakültesi Dergisi, 20, 845-874. (S40) *Ceylan, M. (2019). 21.yüzyıl becerileri bağlamında okul yöneticilerinin değişen rollerinin öğretmen görüşlerine göre incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Trakya Üniversitesi, Sosyal Bilimler Enstitüsü, Edirne. (S41) Chan, S., & Yuen, M. (2014). Personal and environmental factors affecting teachers’ creativity-fostering practices in Hong Kong. Thinking Skills and Creativity, 12, 69-77. *Ciğerci, F. M. (2020). Primary school teacher candidates and 21st century skills. International Journal of Progressive Education, 16(2), 157-174. (S67) Colwill, I., & Gallagher, C. (2007). Developing a curriculum for the twenty-first century: The experiences of England and Northern Ireland. Prospects. 37(4), 411-425. Çalık, M., & Sözbilir, M. (2014). İçerik analizinin parametreleri. Eğitim ve Bilim, 39(174), 33-38. Çalık, M., Ünal, S., Coştu, B. & Karataş, F.Ö. (2008). Trends in Turkish science education. Essays in Education, Special Edition, 23-45. Çalışkan, H. (2009). Sosyal bilgiler öğretiminde araştırmaya dayalı öğrenme yaklaşımının eleştirel düşünme becerisine etkisi. Kastamonu Eğitim Dergisi, 17(1), 57-70. *Çelebi, M., & Altuncu, N. (2019). 21. Yüzyıl Becerilerinin İngilizce Öğretim Programındaki Yeri. Educational Sciences Proceeding Book, 231-244. (S42) *Çelebi, M., & Sevinç, Ş. (2019). Öğretmenlerin 21. yüzyıl becerilerine ilişkin yeterlik algılarının ve bu becerileri kullanım düzeylerinin belirlenmesi. Educational Sciences Proceeding Book, 157-172. (S43) *Çoban, Ö., Bozkurt, S., & Kan, A. (2019). Eğitim yöneticisi 21. yy. becerileri ölçeğinin geliştirilmesi: Geçerlik ve güvenirlik çalışması. Kastamonu Eğitim Dergisi, 27(3), 1059-1071. (S44) *Çolak, M. (2019). Fen bilgisi öğretmen adaylarının 21.yüzyıl becerilerine yönelik algılarının çeşitli değişkenler açısından incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Niğde Ömer Halis Demir Üniversitesi Eğitim Bilimleri Enstitüsü, Niğde. (S45) *Çolak, M. (2018). Ortaokul fen bilimleri dersinin 21. yüzyıl becerilerini kazandırmadaki etkililiğine ilişkin öğretmen görüşleri. (Yayınlanmamış Yüksek Lisans Tezi). Erciyes Üniversitesi, Eğitim Bilimleri Enstitüsü, Kayseri. (S23) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 96 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) *Dağhan, G., Kibar, P. N., Çetin, N. M., Telli, E., & Akkoyunlu, B. (2017). Bilişim teknolojileri öğretmen adaylarının bakış açısından 21. yüzyıl öğrenen ve öğretmen özellikleri. Eğitim Teknolojisi Kuram ve Uygulama, 7(2), 215-235. (S16) Deniz, M. E. (2003). Üniversite öğrencilerinin sosyal beceri düzeylerinin bazı değişkenler açısından incelenmesi. Selçuk Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 9, 501-522. DiCerbo, K. (2014). Assessment and teaching of 21st century skills. Assessment in Education: Principles, Policy and Practice, 21(4), 502-505. doi: 10.1080/0969594X.2014.931836 *Diker Akman, E. (2019). Sekizinci sınıf öğrencilerinin TIMSS fen sonuçlarının 21.yy beceri düzeyleri ve bazı değişkenler açısından incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Marmara Üniversitesi, Eğitim Bilimleri Enstitüsü, İstanbul. (S46) *Dilekli, Y., & Karagöz, S. (2018). Genç öğretmen adaylarının 21. yüzyıl becerilerine sahip olma düzeylerinin incelenmesi (Aksaray Üniversitesi Eğitim Fakültesi örneği). III. In International Congress of Youth Researches Congress Book, Nahcivan. June. 1-17. (S24) *Donmuş Kaya, V., & Akpunar, B. (2018). An investigation of prospective teachers’ 21st century skills effect about digital storytelling events. MANAS Sosyal Araştırmalar Dergisi, 7(4), 1-10. (S25) *Durukan, A. Y. (2015). Dijital dünyada okul kütüphaneciliği ve 21.yüzyıl becerileri. Milli Eğitim, 45(208), 106-120. (S5) *Eğmir, E., & Çengelci, S. (2020). Öğretmenlerin 21. yüzyıl öğretim becerilerinin yansıtıcı düşünmeyi uygulama becerilerini yordama gücü. Tarih Okulu Dergisi, 13(45), 1045-1077. (S61) *Ekici, G., Abide, Ö.F., Canbolat, Y., & Öztürk, A. (2017). Analysis of resources on 21st century skills. Journal of Research in Education and Teaching, 6(1), 124-134. (S17) *Elekoğlu, F., & Demirdağ, S. (2020). Okul müdürlerinin 21. yüzyıl becerileri, iletişim becerileri ve liderlik stillerinin öğretmen algılarına göre incelenmesi. Karaelmas Eğitim Bilimleri Dergisi, 8(1), 101-117. (S71) *Erdoğan, D., & Eker, C. (2020). Türkçe öğretmen adaylarının 21. yy becerileri ile yaşam boyu öğrenme eğilimleri arasındaki ilişkinin incelenmesi. Karaelmas Eğitim Bilimleri Dergisi, 8(1), 118-147. (S70) *Erdoğan, Ö., Toy, M., & Kurt, M. (2020). Robotik uygulamaların fen bilgisi öğretmen adaylarının bazı 21. yüzyıl becerileri üzerindeki etkisinin incelenmesi. Avrasya Sosyal ve Ekonomi Araştırmaları Dergisi, 7(4), 117-137. (S62) *Erkılıç, M. (2020). 21. yüzyıl becerilerinin fizik başarısına etkisinin araştırılması. (Yayınlanmamış Yüksek Lisans Tezi). Necmettin Erbakan Üniversitesi, Eğitim Bilimleri Enstitüsü, Konya. (S84) *Erten, P. (2020). Öğretmen adaylarının 21. Yüzyıl becerileri yeterlilik algıları ve bu becerilerin kazandırılmasına yönelik görüşleri. Milli Eğitim Dergisi, 49(227), 33-64. (S68) *Eryılmaz, S. (2020). Öğrencilerin 21. yüzyıl öğrenme becerileri için veri toplama aracı: Geçerlik ve güvenirlik çalışması. Turkish Studies - Applied Sciences, 15(3), 301-323. (S81) *Eryılmaz, S., & Uluyol, Ç. (2015). 21. yüzyıl becerileri ışığında FATİH projesi değerlendirmesi. Gazi Üniversitesi Eğitim Fakültesi Dergisi GEFAD / GUJG, 2(35), 209-229. (S6) Eskici, G. Y., & Özsevgeç, T. (2019). Yaşam becerileri ile ilgili çalışmaların tematik içerik analizi: bir metasentez çalışması. International e-Journal of Educational Studies, 3(5), 1-15. Facione, P. A., Facione, N. C., & Giancarlo, C. A. (2000). The disposition toward critical thinking: Its character, measurement, and relationship to critical thinking skill. Informal Logic, 61-84. *Gelen, İ. (2017). P21-Program ve öğretimde 21. yüzyıl beceri çerçeveleri (ABD Uygulamaları). Disiplinlerarası Eğitim Araştırmaları Dergisi, 1(2), 15-29. (S18) Germaine, R., Richards, J., Koeller, M., & Schubert-Irastorza, C. (2016). Purposeful use of 21st century skills in higher education. Journal of Research in Innovative Teaching, 9(1), 19-29. *Gökbulut, B. (2020). Öğretmen adaylarının eğitim inançları ile 21. yüzyıl becerileri arasındaki ilişki. Turkish Studies, 15(1), 127-141. (S63) *Göktepe Yıldız, S. (2020). Lise öğrencilerinin 21. yüzyıl becerilerinin bazı demografik değişkenler açısından incelenmesi. Ulakbilge, 51, 884-897. (S74) *Gömleksiz, M. N., Sinan, A. T., & Doğan, F. D. (2019). Türkçe, Türk dili ve edebiyatı ile çağdaş Türk lehçeleri öğretmen adaylarının 21. yüzyıl becerileri yeterlik algıları. Avrasya Uluslararası Araştırmalar Dergisi, 7(19), 163-185. (S47) *Gülen, Ş. B. (2013). Ortaokul öğrencilerinin 21. yüzyıl öğrenme becerileri ve bilişim teknolojileri ile destekleme düzeylerinin cinsiyet ve sinif seviyesine göre incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Gazi Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara. (S2) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 97 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) *Gültekin, H. (2019). Türkçe dersi öğrenci ders kitaplarının 21. Yüzyıl becerileri açısından incelenmesi. (Yayınlanmamış Yüksek Lisans Tezi). Balıkesir Üniversitesi, Sosyal Bilimler Enstitüsü, Balıkesir. (S48) *Gümüş, A. (2019). Geleceğin eğitiminde yeni öğretmen becerileri. Analiz Raporu 4, İlke Yayın No:22 (S49) *Günüç, S., Odabaşı, H., & Kuzu, A. (2013). 21. yüzyıl öğrenci özelliklerinin öğretmen adayları tarafından tanımlanması: Bir Twitter uygulaması. Eğitimde Kuram ve Uygulama, 9(4), 436-455. (S2) *Gürültü, E., Aslan, M., & Alcı, B. (2018). İlköğretim öğretmenlerinin yeterliliklerinin 21. yüzyıl becerileri ışığında incelenmesi. Akademik Sosyal Araştırmalar Dergisi, 6(71), 543-560. (S26) *Gürültü, E., Aslan, M., & Alcı, B. (2019). Ortaöğretim öğretmenlerinin 21. yüzyıl becerileri kullanım yeterlikleri. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 35(4), 780-798. (S50) Haladyna, T. M. (1997). Writing test item to evaluate higher order thinking. USA: Allyn and Bacon. http://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf *Hamarat, E., & Arkan, A. (2018). 2023 Eğitim Vizyon Belgesi’nde gelecek becerileri. Seta Perspektif, 222, 17. (S27) *Hamlı, D., Hamlı, S., & Taneri, P. O. (2020). Temel eğitimde 21. yüzyıl becerilerinin sınıf öğretmenlerinin görüşlerine göre incelenmesi. Current Debates on Social Sciences 4, 94-107. (S83) *Karagöz, S., & Dilekli, Y. (2018). Gençlere göre 21. yüzyıl becerilerinin kazandırılmasında görev ve sorumluluk bilincinin bazı değişkenler açısından incelenmesi (aile, çevre, okul, bireysel özellikler, medya) III. In International Congress of Youth Researches Congress Book, Nahcivan. June. 18-25. (S28) *Karakaş, M. M. (2015). Ortaokul sekizinci sınıf öğrencilerinin fen bilimlerine yönelik 21.yüzyıl beceri düzeylerinin ölçülmesi. (Yayınlanmamış Yüksek Lisans Tezi). Eskişehir Osmangazi Üniversitesi, Eskişehir. (S7) Karakuş, M. (2011). Eğitim ve yaratıcılık. Eğitim ve Bilim Dergisi, 26(119), 3-7. Karaman, M. K. (2016). Öğretmen adaylarının medya okuryazarlık düzeyleri ve eleştirel düşünme eğilimleri üzerine bir araştırma. Gümüşhane Üniversitesi İletişim Fakültesi Elektronik Dergisi, 4(1), 326-350. *Karaşah Çakıcı, Ş., & Yakışan, M. (2020). Sorgulama temelli öğrenme yönteminin öğrencilerin 21. yüzyıl becerilerine ve yansıtıcı düşünme düzeylerine etkisi. Gazi Eğitim Bilimleri Dergisi, 6(3), 344-360. (S82) Kardeş, S. K. (2020). Okul öncesi eğitim programının 21. yüzyıl becerileri ve STEAM eğitimi bağlamında incelenmesi. Eğitimde Kuram ve Uygulama, 16(2), 109-119. *Kasap, M. (2020). 21. yüzyıl yönetici yeteneklerinin belirlenmesine yönelik bir çalışma; İstanbul ve Bursa örneği. Atatürk Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 24(3), 1037-1052. (S77) *Kayhan, E., Altun, S., & Gürol, M. (2019). Sekizinci sınıf Türkçe öğretim programı (2018)’nın 21. yüzyıl becerileri açısından değerlendirilmesi. Adnan Menderes Üniversitesi Eğitim Fakültesi Eğitim Bilimleri Dergisi, 10(2), 20-35. (S51) *Keskin, İ., & Yazar, T. (2015). Öğretmenlerin yirmi birinci yüzyıl becerileri ışığında ve yaşam boyu öğrenme bağlamında dijital yeterliliklerinin incelenmesi. International Journal of Human Sciences, 12(2), 16911711. (S8) *Kıyasoğlu, E., & Ay, Ş. Ç. (2020). Sınıf öğretmenlerinin 21. yüzyıl öğrenen ve öğreten becerilerinin incelenmesi. e-Kafkas Eğitim Araştırmaları Dergisi, 7(3), 240-261. (S73) *Koçak, Ö., & Göksu, İ. (2020). Examining 21st century skill levels of students and the relationship between skills. Inonu University Journal of the Faculty of Education, 21(2), 772-784. (S69) * Koçin, B., & Tuğluk, M. N. (2020). 2013 okul öncesi eğitim programının 21. yüzyıl becerileri açısından incelenmesi. Ulakbilge, 49, 621-649. (S78) *Korkmaz, Ç. (2019). Sınıf öğretmenlerinin yaşam boyu öğrenme eğilimleri ile yaşam ve 21. yüzyıl öğreten beceri düzeyleri arasındaki ilişki. (Yayınlanmamış Yüksek Lisans Tezi). Afyon Kocatepe Üniversitesi, Sosyal Bilimler Enstitüsü, Afyon. (S52) Korucu, A. T., & Ünüvar, M. (2020). Bilgisayar ve öğretim teknolojileri öğretmenliği bölümünde okuyan öğretmen adaylarının aldıkları eğitim ile bilişim liderliği vasfı ve 21. yüzyıl becerileri arasındaki ilişki. Bilgi ve İletişim Teknolojileri Dergisi, 2(1), 44-53. *Kotluk, N., & Kocakaya, S. (2015). 21. yüzyıl becerilerinin gelişiminde dijital öykülemeler: Ortaöğretim öğrencilerinin görüşlerinin incelenmesi. Eğitim ve Öğretim Araştırmaları Dergisi, 4(2), 354-363. (S9) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 98 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) *Kozikoğlu, İ., & Altunova, N. (2018). Öğretmen adaylarının 21. yüzyıl becerilerine ilişkin öz-yeterlik algılarının yaşam boyu öğrenme eğilimlerini yordama gücü. Yükseköğretim ve Bilim Dergisi/Journal of Higher Education and Science, 8(3), 522-531. (S29) *Kozikoğlu, İ., & Özcanlı, N. (2020). Öğretmenlerin 21. yüzyıl öğreten becerileri ile mesleğe adanmışlıkları arasındaki ilişki. Cumhuriyet Uluslararası Eğitim Dergisi, 9(1), 270-290. (S64) *Köğce, D., Özpınar, İ., Mandacı Şahin, S., & Aydoğan Yenmez, A. (2014). Öğretim elemanlarının 21. yüzyıl öğrenen standartları ve yaşam boyu öğrenmeye ilişkin görüşleri. Dicle Üniversitesi Ziya Gökalp Eğitim Fakültesi Dergisi, 22, 185–213. (S4) *Kurudayıoğlu, M., & Tüzel, A. G. M. S. (2010). 21. yüzyıl okuryazarlık türleri, değişen metin algısı ve Türkçe eğitimi. Türklük Bilimi Araştırmaları, 28, 283-298. (S1) *Kurudayıoğlu, M., & Soysal, T. (2019). 2018 Türkçe dersi öğretim programı kazanımlarının 21. yüzyıl becerileri açısından incelenmesi. Ahi Evran Üniversitesi Sosyal Bilimler Enstitüsü Dergisi, 5(2), 483496. (S53) *Kutlu Demir, Ö. (2018). 21st century learning: Integratıon of web 2. 0 tools in Turkish adult language classrooms. (Yayınlanmamış Doktora Tezi). Çağ Üniversitesi, Sosyal Bilimler Enstitüsü, Mersin. (S30) Kylonen, P. C. (2012). Measurement of 21st century skills within the common core state standards. Invitational Research Symposium on Technology Enhanced Assessments. Lai, E. R., & Viering, M. (2012). Assessing 21st century skills: Integrating research findings. Vancouver, B.C.: National Council on Measurement in Education. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174. Larson, L. C. & Miller, T. N. (2011). 21st century skills: Prepare students for the future. Kappa Delta Pi Record, 47(3), 121-123. Lemke, C. (2010). Innovation through technology. J. Bellanca, & R. Brandt, 21st century skills: Rethinking how students learn içinde (s. 243-274). Indiana: Solution Tree. Leu, D. J. Jr., C. K. Kinzer, J. Coiro, & D. W. Cammack. (2004). Toward a theory of new literacies emerging from the Internet and other information and communication technologies. In Theoretical models and processes of reading, 5th ed., ed. R. B. Ruddell and N. J. Unrau, 1570–613. Newark, DE: International Reading Association. McGrath, J., & Fischetti, J. (2019). What if compulsory schooling was a 21st century invention? Weak signals from a systematic review of the literature. International Journal of Educational Research, 95, 212-226. *Murat, M. (2018). Ters yüz sınıf modelinin beşinci sınıf öğrencilerinin 21. yüzyıl becerileri ve bilimsel epistemolojik inançlarına etkisi. (Yayınlanmamış Yüksek Lisans Tezi). Muğla Sıtkı Koçman Üniversitesi, Eğitim Bilimleri Enstitüsü, Muğla. (S31) National Ministry of Education, (2019). 2023 Eğitim Vizyonu Belgesi http://meb.gov.tr/: http://2023vizyonu.meb.gov.tr National Ministry of Education, (1973). Milli Eğitim Temel Kanunu. 12(14574). 5101-5113. National Research Council. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. Committee on Defining Deeper Learning and 21st Century Skills, James W. Pellegrino and Margaret L. Hilton, Editors. Board on Testing and Assessment and Board on Science Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. *Orhan Göksün, D. (2016). Öğretmen adaylarının 21.yy. öğrenen becerileri ve 21. yy. öğreten becerileri arasındaki ilişki. (Unpublished Doctoral Dissertation). Anadolu University, Institute of Educational Sciences, Eskişehir, Turkey. (S13) *Orhan Göksün, D., & Aşkım Kurt, A. (2017). Öğretmen adaylarının 21. yüzyıl. öğrenen becerileri kullanımları ve 21. yüzyıl. öğreten becerileri kullanımları arasındaki ilişki. Eğitim ve Bilim, 42(190), 107-130. (S19) *Önür, Z., & Kozikoğlu, İ. (2019). Ortaokul öğrencilerinin 21. yüzyıl öğrenme becerileri. Trakya Eğitim Dergisi, 9(3), 627-648. (S54) *Önür, Z., & Kozikoğlu, İ. (2020). The relationship between 21st century learning skills and educational technology competencies of secondary school students. Kuramsal Eğitimbilim Dergisi [Journal of Theoretical Educational Science], 13(1), 65-77. (S65) *Özdemir-Özden, D., Karakuş-Tayşi, E., Kılıç-Şahin, H., Demir-Kaya, S., & Bayram, F. Ö. (2018). Öğretmen adaylarının 21. yüzyıl becerilerine yönelik yeterlik algıları: Kütahya örneği. Electronic Turkish Studies, 13(27), 1163-1184. (S32) Özden, Y. (2014). Öğrenme ve öğretme (12 baskı). Ankara: Pegem Akademi. SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 99 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) Özgüzel, S. (2018). 21. yüzyılın akademik eğitim konsepti ve ünıversitelerın rolü. 21. Yüzyılda Eğitim ve Toplum (Education and Society In The 21st Century Cilt), 7(21), 951-964. *Özyurt, M. (2020). 21. yüzyıl becerileri öğretimi ölçeğinin Türk kültürüne uyarlanması: Geçerlik güvenirlik çalışması. OPUS Uluslararası Toplum Araştırmaları Dergisi, 16(30), 2568-2594. (S80) Partnership for 21st Century Learning (P21). (2007). Framework for 21st century. Qian, M., & Clark, K. R. (2016). Game-based Learning and 21st century skills: A review of recent research. Computers in human behavior, 63, 50-58. Rotherham, A. J., & Willingham, D. (2009). 21st century. Educational leadership, 67(1), 16-21. *Saçmalıoğlu, M. G. (2019). Beden eğitimi öğretmen adaylarının, mesleki öğrenme topluluğu aracılığıyla 21. yüzyıl becerilerini ortaya çıkarma süreçlerinin keşfedilmesi. (Yayınlanmamış Yüksek Lisans Tezi). Ankara Üniversitesi, Eğitim Bilimleri Enstitüsü, Ankara. (S55) *Sayın, Z., & Seferoğlu, S. S. (2016). Yeni bir 21. yüzyıl becerisi olarak kodlama eğitimi ve kodlamanın eğitim politikalarına etkisi. Aydın: XVIII. Akademik Bilişim Konferansı, 3-5. (S14) Selçuk, Z., Palancı, M., Kandemir, M. & Dündar, H. (2014). Eğitim ve bilim dergisinde yayınlanan araştırmaların eğilimleri: İçerik analizi. Eğitim ve Bilim, 39(173), 430-453. Silva, E. (2009). Measuring skills for 21st century learning. Phi Delta Kappa, 90(9), 630-634. *Tekerek, B., Karakaya, F., & Tekerek, M. (2018). An investigation on undergraduate programs of teacher training regarding 21th century skills: Example of elementary mathematics and science. Education Conference 2018 (EDUCCON2018), Türkiye. (S33) Tomlinson, C. A., Brimijoin, K., & Narvaez, L. (2008). The differentiated school: Making revolutionary changes in teaching and learning. ASCD. Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. Francisco: Jossey-Bass. *Tuğluk, M. N., & Özkan, B. (2019). MEB 2013 okul öncesi eğitim programının 21. yüzyıl becerileri açısından analizi. Temel Eğitim, 1(4), 29-38. (S56) *Uçak, S., & Erdem, H. H. (2020). Eğitimde yeni bir yön arayışı bağlamında “21. yüzyıl becerileri ve eğitim felsefesi”. Uşak Üniversitesi Eğitim Araştırmaları Dergisi, 6(1), 76-93. (S66) URL-1. (2019). Framework for 21st century learning. Retrieved from http://static.battelleforkids.org/documents/p21/P21_Framework_Brief.pdf. Access date: 22.02.2021. URL-2. (2020). Partnership for 21st Century Learning. P21 Framework Definitions. Retrieved from https://www.battelleforkids.org/networks/p21 Access date: 06.03.2021 Ültay, E., Akyurt, H., & Ültay, N. (2021). Descriptive content analysis in social sciences. IBAD Journal of Social Sciences, 6(10), 188-201. DOI: 10.21733/ibad.871703 Ültay, E., Dönmez Usta, N., & Durmuş, T. (2017). Eğitim alanında yapılan zihinsel model çalışmalarının betimsel içerik analizi. Yaşadıkça Eğitim, 31(1), 21-40. Ültay, E., Durnacı, Ü., & Ültay, N. (2019). Sınıf öğretmen adaylarının eleştirel düşünme eğilimlerini inceleyen çalışmalara ait içerik analizi. Akdeniz Eğitim Araştırmaları Dergisi, 13(29), 99-120. Ültay, E., & Uludüz, Ş. M. (2018). Sınıf öğretmeni adaylarının fen öğretimi öz yeterlik inançları üzerine yapılan çalışmaların incelenmesi. Ondokuz Mayıs Üniversitesi Eğitim Fakültesi Dergisi, 37(1), 129-143. Ünal, M., & Akman, B. (2006). Okul öncesi öğretmenlerinin fen eğitimine karşı gösterdikleri tutumlar. Hacettepe Üniversitesi Eğitim Fakültesi Dergisi, 30, 251-257. *Üzümcü, Ö., & Bay, E. (2018). Eğitimde yeni 21. yüzyıl becerisi: Bilgi işlemsel düşünme. Uluslararası Türk Kültür Coğrafyasında Sosyal Bilimler Dergisi, 3(2), 1-16. (S34) Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. London: Sage Publications. *Yalçın, S. (2018). 21. yüzyıl becerileri ve bu becerilerin ölçülmesinde kullanılan araçlar ve yaklaşımlar. Ankara Üniversitesi Eğitim Bilimleri Fakültesi Dergisi, 51(1), 183-201. (S35) *Yalçın, S. (2019). Öğretmen adaylarının 21. yy. becerilerini ölçmek için kullanabilecekleri araçlar hakkında farkındalıkları ve yeterlik algıları. Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 19(1), 383398. (S57) *Yalçın, V., Simsar, A., & Dinler, H. (2020). 5-6 yas çocuklari için 21. yy becerileri ölçeği (DAY-2): Geçerlik ve güvenirlik çalışması. Akdeniz Egitim Arastirmalari Dergisi, 14(32), 78-97. (S72) *Yalçın İncik, E. (2020). Öğretmenlerin yaşam boyu öğrenme eğilimleri ve 21. yüzyıl öğreten becerileri arasındaki ilişkinin incelenmesi. Bolu Abant İzzet Baysal Üniversitesi Eğitim Fakültesi Dergisi, 20(2), 1099-1112. (S75) *Yılmaz, E. (2016). 21. yüzyıl becerileri kapsamında dönüşen okul paradigması. İçinde, E. Yılmaz, M. Çalışkan & S. A. Sulak (Ed.), Eğitim Bilimlerinden Yansımalar, (ss.5-16). Konya: Çizgi Kitabevi. (S15) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 100 Ultay, Dönmez-Usta & Ultay SDU IJES (SDU International Journal of Educational Studies) *Yılmaz, E., & Alkış, M. (2019). 21. yüzyıl yeterlilikleri ölçeği’nin geliştirilmesi: Geçerlik ve güvenirlik çalışması. The Journal of International Lingual Social and Educational Sciences, 5(1), 125-154. (S58) *Yılmaz, Z., & Tanrıseven, I. (2019). Öğretmen adaylarının 21. yy öğrenen becerileriyle pedagojik bilgi ve becerileri arasındaki ilişkinin incelenmesi the investigation of the relationship between 21st century learner skills and pedagogical knowledge and skills. Proceeding Book, 73-90. (S59) *Zeybek, G. (2019). Lise öğrencilerinin 21. yüzyıl öğrenme becerileri kullanım düzeylerinin belirlenmesi. International Journal of Social Sciences and Education Research, 5(2), 142-156. (S60) Zengin, F., Kırılmazkaya, G., & Zengin, R. (2012). İlköğretim fen bilgisi ve sınıf öğretmen adaylarının sosyal becerilerinin karşılaştırılması. E-Journal of New World Sciences Academy, 7(2), 656-667. Note: The references used in the descriptive content analysis are shown with (*) SDU International Journal of Educational Studies, 8(2), 2021, Page 85-101 101