SOS KALO PAPER Gia LA in CSCL Two Groups Etc

Education and Information Technologies
https://doi.org/10.1007/s10639-023-11886-3
The impacts of the comprehensive learning analytics

approach on learning performance in online collaborative
learning
Lanqin Zheng1 · Kinshuk2 · Yunchao Fan1 · Miaolang Long1
Received: 10 October 2022 / Accepted: 5 May 2023

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature
2023
Abstract
Online collaborative learning has been an effective pedagogy in the field of educa-
tion. However, productive collaborative learning cannot occur spontaneously. Learn-
ers often have difficulties in collaborative knowledge building, group performance,
coregulated behaviors, learning engagement, and social interaction. To promote pro-
ductive collaborative learning, this study aims to propose and validate a compre-
hensive learning analytics approach in an online collaborative learning context. The
comprehensive learning analytics can automatically construct knowledge graphs,
analyze metacognitive learning engagement and social interaction and provide per-
sonalized feedback. A total of 90 college students participated in this study, and
they were assigned to the experimental group and control group. The students in the
experimental group conducted online collaborative learning with the comprehen-
sive learning analytics approach, while the students in the control group conducted
traditional online collaborative learning without any specific approach. The results
indicated that the comprehensive learning analytics approach significantly improved
collaborative knowledge building, group performance, coregulated behaviors, meta-
cognitive learning engagement, and social interaction compared with traditional
online collaborative learning. In this paper, the results of the study together with the
implications are discussed.
Keywords Learning analytics · Online collaborative learning · Collaborative

knowledge building · Coregulation · Learning engagement · Social interaction
* Lanqin Zheng
[email protected]
1
School of Educational Technology, Faculty of Education, Beijing Normal University, No. 19,
XinJieKouWai St., HaiDian District, Beijing 100875, People’s Republic of China
2
University of North Texas, Denton, TX 76207, USA
13
Vol.:(0123456789)
1 Introduction
As an effective pedagogy, online collaborative learning has been widely adopted

in the field of education. Online collaborative learning is defined as learners
who are geographically separated, learning together online to solve problems
(Reeves et al., 2004). However, learners often have difficulties in collaborative
knowledge building and coregulation (Zheng et al., 2021; Melzner et al., 2022).
Learners often engage less during traditional collaborative learning than technol-
ogy-enhanced collaborative learning (Labonté & Smith, 2022). Previous studies
have employed different strategies to promote collaborative knowledge building,
coregulation, and learning engagement. For example, Avcı (2020) combined sen-
tence openers and role assignment scaffolding to promote collaborative knowl-
edge building. Järvenoja et al. (2020) adopted the S-REG mobile application tool
to support coregulation in collaborative learning. Xie et al. (2021) developed
an online conversational agent to improve learning engagement in collaborative
learning.
Furthermore, previous studies revealed that learning analytics was an effec-
tive approach to facilitating collaborative learning (Hu et al., 2022; Wen & Song,
2021). However, very few studies have utilized the comprehensive learning ana-
lytics approach to facilitate collaborative knowledge building, group performance,
coregulated behaviors, metacognitive learning engagement, and social interaction
in the online collaborative learning context. The comprehensive learning analyt-
ics approach is defined as an approach to automatically analyzing collaborative
knowledge building, metacognitive learning engagement, and social interaction as
well as providing personalized feedback for each group. The traditional online col-
laborative learning approach, by contrast, refers to an online collaborative learning
approach without any learning analytics, feedback or other technology. The benefits
of the comprehensive learning analytics approach are twofold. One is to demon-
strate the learning analytics results about collaborative knowledge building, meta-
cognitive learning engagement, and social interaction in real-time for each group.
These learning analytics results are not separated but are demonstrated at the same
interface. Learners need not select, and they can browse all analytics results at the
same time. Another benefit is to provide immediate and personalized feedback for
each group based on comprehensive learning analytics results. Therefore, this study
aims to examine the impact of the comprehensive learning analytics approach on
collaborative knowledge building, group performance, coregulated behaviors,
metacognitive learning engagement, and social interaction in an online collabora-
tive learning context. The research questions are addressed as follows:
RQ1: What is the impact of the comprehensive learning analytics approach on

collaborative knowledge building?
group performance?
coregulated behavior patterns?
13

metacognitive learning engagement?
social interaction?
The next section contextualizes this study from the literature. Then, the method
and experimental design are explained in detail. Finally, the findings, implications
and conclusions are also presented and analyzed in depth.
2 Literature review
In this section, previous studies on online collaborative learning and learning ana-
lytics are systematically reviewed. In addition, the research gaps and purposes are
clearly presented.
2.1 Online collaborative learning
Online collaborative learning has attracted increasing attention in recent years. The
major concerns related to online collaborative learning include how to promote
high-quality collaborative knowledge building, group performance, coregulation,
deep learning engagement, and productive social interaction (Chan, 2012; Scarda-
malia & Bereiter, 2006; Stahl et al., 2014). However, learners often have difficulties
in achieving a high level of collaborative knowledge building, learning engagement,
and high-quality social interaction (Zheng et al., 2022). Learners often do not know
how to coregulate with peers during collaborative learning (Järvenoja et al., 2020).
Previous studies have explored some strategies to promote collaborative knowl-
edge building, group performance, coregulation, learning engagement, and produc-
tive social interaction. With regard to promoting collaborative knowledge building,
researchers have adopted different methods to facilitate a high level of collaborative
knowledge building. For example, Craig et al. (2021) employed a network visualiza-
tion tool to promote collaborative knowledge building in history classrooms. Zheng
et al. (2021) adopted personalized intervention to improve collaborative knowledge
building. Duvall et al. (2020) found that asynchronous online settings, sharing work,
discussion with peers, providing instructions or discussion criteria, and using appro-
priate digital tools could promote collaborative knowledge building. Lei and Chan
(2018) revealed that reflective meta-discourse contributed to collaborative knowl-
edge building. Shin et al. (2018) adopted a representation tool to support collabo-
rative knowledge building. However, few studies have adopted the comprehensive
learning analytics approach to promote collaborative knowledge building.
With respect to promoting group performance, different strategies have been
adopted in previous studies. For example, Chen et al. (2022) proposed a group
incentive mechanism to improve group performance in collaborative problem
solving. Shin et al. (2020) examined hard scaffolding, which refers to static sup-
port; peer scaffolding, which refers to scaffolding from knowledgeable peers; and
13
teacher scaffolding, which refers to scaffolding from teachers. They found that only
peer scaffolding could significantly predict group performance. Yilmaz et al. (2020)
examined the vertical versus shared e-leadership approach and revealed that both
leadership approaches could improve group performance. Nevertheless, few stud-
ies have adopted the comprehensive learning analytics approach to enhance group
performance.
Regarding coregulation, in the existing research, different methods to facilitate
coregulation in collaborative learning have been explored. For example, Järvenoja
et al., (2020) developed the S-REG tool to enhance the coregulation of groups’ emo-
tions and motivation in collaborative learning. Lim and Lim (2020) examined the
relationships between coregulation and motivation, and they found that mastery
goal orientation, which values improvements, learning new subjects, and using deep
learning strategies (Pintrich & De Groot, 1990), played a dominant role when learn-
ers coregulated with peers in collaborative learning. Nevertheless, few studies have
adopted the comprehensive learning analytics approach to enhance coregulation.
Concerning improving learning engagement in collaborative learning, previ-
ous studies have employed multiple strategies to enhance learning engagement.
For example, Unal and Cakir (2021) adopted a technology-supported collaborative
problem-solving method to promote learning engagement. Huang (2021) proposed a
smartphone-based collaborative vlog project to enhance learning engagement. How-
ever, very few studies have adopted a comprehensive learning analytics approach to
enhance metacognitive learning engagement.
In terms of facilitating productive social interaction, several effective strate-
gies have been used in previous studies. For example, Liu et al. (2021) developed a
mobile team-based competition environment to promote social interaction. Sun et al.
(2018) adopted a discussion forum to promote social interaction. Wang et al. (2018)
engaged learners in a 3D game‐based collaborative virtual learning environment
to enhance embodied social presence to further facilitate productive social interac-
tion. However, very few studies have adopted the comprehensive learning analytics
approach to enhance social interaction.
There are also learning analytics-based approaches that have been used in the lit-
erature. These approaches can improve learning engagement and learning perfor-
mance. Previous studies on learning analytics are reviewed in the following section.
2.2 Learning analytics
Learning analytics is defined as the measurement, collection, analysis, and report-

ing of data for understanding and optimizing learning and learning environments
(Siemens & Baker, 2012). Learning analytics is a new lens through which teach-
ers and practitioners can understand learning and improve learning (Clow, 2013).
Learning analytics has great potential for improving learner practice (Viberg
et al., 2018). Larrabee Sønderlund et al. (2019) found that the potential effec-
tiveness of learning analytics interventions lies in predicting student success and
retention. Previous studies have utilized learning analytics in different contexts
to provide feedback and interventions to improve learning. For example, Lu et al.
13
(2017) adopted learning analytics to improve learning engagement and outcomes

in a collaborative programming course and found that learning analytics indeed
improved learning outcomes and levels of learning engagement. Karaoglan
Yilmaz and Yilmaz (2022) employed learning analytics intervention to improve
learning engagement and revealed that learning analytics could improve students’
online learning engagement. Lim et al. (2021) provided learning analytics-based
feedback in four courses and found that learners held positive perceptions of
learning analytics-based feedback. Ochoa and Wise (2021) proposed that practi-
tioners should empower students’ agency in using learning analytic tools as part
of their learning process.
Furthermore, existing studies have adopted different approaches and methods
to conduct learning analytics. For example, Minović et al. (2015) employed a
visual and descriptive learning analytics method to track learning progress in a
serious game setting. Halimi and Seridi-Bouchelaghem (2021) adopted semantic
modeling techniques to conduct learning analytics and assess learners’ competen-
cies. Sun and Theussen (2022) utilized the social network analysis method as a
learning analytics tool to assess learners’ negotiation skills in a simulation game.
However, most studies only employed surface-level descriptive learning analytics
and called for integrating machine learning and other methods to enable predic-
tive and comprehensive analytics (Susnjak et al., 2022).
Comprehensive learning analytics are useful and necessary for all agents
involved in learning processes and educational decision-making (Agudo-Pere-
grina et al., 2014). Comprehensive learning analytics allow the optimization of
learning based on data (North, 2021). Comprehensive learning analytics con-
tribute to recognizing the strengths and weaknesses of all stakeholders (Ozdemir
et al., 2020). In previous studies, only a few studies adopted a comprehensive
learning analytics approach to facilitate learning. For example, Takii et al. (2021)
only proposed a comprehensive learning analytics-based plan for vocabulary
learning. Álvarez-Xochihua et al. (2017) conducted comprehensive multidomain
learning analytics to provide a global understanding of learning activity. How-
ever, there is a lack of studies on developing and employing the comprehen-
sive learning analytic approach in the online collaborative learning context. To
close the research gaps, this study aims to develop and examine a comprehensive
learning analytic approach to engaging learners in online collaborative learning
to promote collaborative knowledge building, group performance, coregulative
behaviors, learning engagement, and social interaction. The following sections
illustrate the proposed comprehensive learning analytics approach.
3 The comprehensive learning analytics approach
This section first illustrates how to design the comprehensive learning analytics
approach in detail. Then, the implementation of the comprehensive learning ana-
lytics approach in an online collaborative learning context is clearly described and
explained.
13
3.1 Design of the comprehensive learning analytics approach
In this study, a comprehensive learning analytics approach was proposed to promote

collaborative knowledge building, social interaction, learning engagement, group
performance, and coregulation in an online collaborative learning setting. The pro-
posed approach includes three phases, as shown in Fig. 1. The first phase is to col-
lect data through online collaborative learning.
The second phase is to conduct comprehensive learning analytics by analyzing col-
laborative knowledge building, metacognitive learning engagement, and social interac-
tion. The analysis of collaborative knowledge building is achieved through the auto-
matic construction of knowledge graphs based on online discussion transcripts in three
steps. The first step is to recognize knowledge entities through a deep neural network
model, namely, the BERT-BiLSTM-CRF model (bidirectional encoder representa-
tions from the transformers, bidirectional long short-term memory, conditional random
field). This model comprises BERT, BiLSTM, and CRF. BERT was proposed by Dev-
lin et al. (2019), who believed that BERT can pretrain deep bidirectional representa-
tions from unlabeled texts and capture the semantic relationships among labels. Long
short-term memory (LSTM) is a novel recurrent network architecture with a gradient-
based learning algorithm (Hochreiter & Schmidhuber, 1997). BiLSTM is a bidirec-
tional long short-term memory that can capture bidirectional semantic information in a
sentence, and BiLSTM is powerful for sequential labeling (Wang et al., 2015). CRF is
Phase 1:Collect data Collect data through online collaborative learning
Phase 2:
Conduct Analysis of collaborative
comprehensive knowledge building through
learning constructing knowledge graphs
Comprehensive
analytics learning analytics
Analysis of metacognitive
results
learning engagement
Analysis of social interaction
Phase 3: Feedback about collaborative knowledge building

Personalized
feedback and
recommendation Feedback about metacognitive learning engagement
Feedback about social interaction
Recommend learning resources
Fig. 1 The flowchart of the comprehensive learning analytics approach
13
a powerful framework for building probabilistic models to segment and label sequence
data (Lafferty et al., 2001). Furthermore, other competing models such as BERT and
BERT-LSTM-CRF were also compared with BERT-BiLSTM-CRF. In this study, the
accuracy of BERT-BiLSTM-CRF achieved 0.93, which is higher than those of BERT
(0.89) and BERT-LSTM-CRF (0.90). The second step is to extract relationships based
on the target knowledge graph that consists of target knowledge and correct relation-
ships. The third step is to demonstrate activated and unactivated knowledge graphs as
well as public knowledge graphs. In addition, the analysis of metacognitive learning
engagement is performed based on the same online discussion transcripts and achieved
through the deep neural network model BERT. Metacognitive learning engagement is
automatically classified into planning, monitoring, reflection and evaluation, as well as
off-topic information through BERT. This classification was adapted from Zheng et al.
(2021). A total of 17,113 online discussion transcripts collected by the first author were
employed to train the BERT model, and the accuracy achieved 0.91. In addition, social
interaction was automatically analyzed and presented through the interaction duration,
interaction number, and interaction relationships.
The third phase is to automatically provide personalized feedback and recommend
learning resources. Personalized feedback and recommendations were provided accord-
ing to the particular thresholds and predefined rules. The average of the metacogni-
tive learning engagement, the amount of activated knowledge, interaction duration and
number of the experimental group were used as the particular thresholds based on Lu
et al. (2017). Personalized feedback and recommendations are provided according to
whether a group’s metacognitive learning engagement, the amount of activated knowl-
edge, interaction duration and number are lower, higher, or equal to the thresholds.
3.2 Implementation of the comprehensive learning analytics approach
In this study, the implementation of the comprehensive learning analytics approach

included engaging participants in online collaborative learning through the developed
platform, conducting comprehensive learning analytics and presenting comprehensive
learning analytics results and feedback only for the participants of the experimental
group. For the control group, participants only conducted traditional online collabo-
rative learning without any learning analytics results or feedback. Figure 2 shows a
screenshot of the developed online collaborative learning platform. Figure 3 shows a
screenshot of comprehensive learning analytics. The next section addresses how the
comprehensive learning analytics approach was validated in this research through a
quasi-experimental study.
4 Methodology
In this section, the participants, experimental procedure, and data collection and
analysis methods are illustrated in detail. These subsections provide a clear picture
of how the present study was conducted.
13
Fig. 2 Screenshot of the online collaborative learning platform
Fig. 3 Screenshot of comprehensive learning analytics
4.1 Participants
The participants were 90 college students in a top-10 public university in China.

They registered for a mandatory course titled “Multimedia Technologies and Web-
page Making”. The average age of participants was 18 years old. They were enrolled
to major in literature as well as journalism and communication. All participants
were assigned to either the experimental group or the control group. There were 45
students in the experimental group and 45 students in the control group. Students
13
in both the experimental and control groups were further divided into collaborative
learning groups of three students each. Thus, there were 15 groups in the experi-
mental condition and 15 groups in the control condition. All participants agreed to
participate in this study, and they were informed that they could withdraw from the
study at any time.
4.2 Procedure
A quasi-experimental design was employed in this study to examine the impacts of

the comprehensive learning analytics approach on collaborative knowledge build-
ing, group performance, coregulated behavior patterns, metacognitive learning
engagement, and social interaction. Figure 4 shows the experimental procedure,
which included five phases. The first phase was to conduct a pretest to examine the
prior knowledge of participants. The second phase was to introduce how to use an
online collaborative learning platform and the comprehensive learning analytics
approach. The third phase was to conduct online collaborative learning through the
aforementioned online collaborative learning platform for one month to ensure that
the students had enough time to complete tasks. The online discussion forum was
the only means of communication and collaboration among participants. The col-
laborative learning task was to make posters using Photoshop software. More spe-
cifically, there were four subtasks, namely, discussing and selecting the theme of
Experimental group Control group

˄N = 45˅ ˄N = 45˅
Pre-test
Introduce how to use online collaborative learning

platform
Conduct online Conduct traditional

collaborative learning online collaborative
with the comprehensive learning
learning analytics approach
Submit group products
Interview
Fig. 4 The experimental procedure
13
a poster, discussing the techniques of making a poster, making a poster with group
members through Photoshop software, and evaluating and refining the poster. The
collaborative learning task, duration, and online discussion forum of the experimen-
tal groups were the same as those of the control groups. The only difference was
that the students in the experimental groups conducted online collaborative learning
with the comprehensive learning analytics approach (see Fig. 3), while the students
in the control groups conducted online collaborative learning without any specific
approach (see Fig. 2). Then, each group submitted the poster as the group product.
The poster of each group was evaluated based on the assessment criteria, as shown
in Table 1. The assessment criteria included five dimensions, namely, themes and
content of posters, image stitching, layers, channels as well as filters, paths, layout
and color matching. The full score of each dimension was 20, and the full score of
the five dimensions was 100. The last phase was to conduct semistructured online
interviews. The participants of the experimental groups were interviewed to obtain
their perceptions of using the comprehensive learning analytics approach.
4.3 Data collection and analysis methods
The datasets collected in this study included pretests, groups’ products, groups’
online discussion transcripts, and groups’ interview records. The pretest consisted of
eight multiple-choice questions with one correct answer, four multiple-choice ques-
tions with multiple correct answers, three fill-in-the-blanks questions, and two short
answer questions. The pretest questionnaire is provided in the appendix. The pretest
was developed by an experienced teacher who had taught multimedia technologies
and webpage making for more than 10 years. The full score of the pretest was 100.
The pretest was evaluated by two raters, and the interrater reliability calculated by
the kappa value was 0.89. In addition, the group product of each group was a poster
made using Photoshop software. The online discussion transcripts of each group
were automatically recorded through an online collaborative learning platform.
Interview records were automatically recorded through a real-time communication
tool.
The data analysis methods included the content analysis method, computer-
assisted knowledge graph analysis method, lag sequential analysis method, and sta-
tistical analysis method. The content analysis method was adopted to evaluate group
products based on the aforementioned assessment criteria. Two raters interdepend-
ently evaluated the group products of 30 groups, and the interrater reliability calcu-
lated by the kappa value was 0.85, implying high reliability. The computer-assisted
knowledge graph analysis method was employed to analyze online discussion tran-
scripts of 30 groups and calculate the collaborative knowledge building level. The
computer-assisted knowledge graph analysis method has been validated by previous
studies (Zheng et al., 2019, 2022), and it includes three steps. The first step is to
draw the target knowledge graph consisting of target knowledge and their relation-
ships. The second step is to segment online discussion transcripts based on prede-
fined rules. The two coders interdependently analyzed online discussion transcripts
of all groups, and the interrater reliability calculated by the kappa value reached
13
Table 1 The assessment criteria for posters
Dimensions 16–20 11–15 6–10 1–5
Themes of posters The theme and content is innova- The theme and content is some- The theme and content lacked The theme and content are copied
tive what innovative innovation from others
Image stitching The image-stitching is very The image-stitching is acceptable The image-stitching is unnatural The images are not stitched
natural and perfect
Layers, channels, and filters Appropriate use of diverse layers, Use of diverse layers and filters Use of layers without filters or Use of only a single layer without
channels, and filters without channels channels filters and channels
Paths Using different paths to create a Using different paths to create Only one type of path is used The poster is made without any
poster, and the effect is natural a poster, but the effect is path
unnatural
Layout and color matching The layout and color matching The layout and color matching is The layout is acceptable, but Both the layout and color matching
is perfect acceptable color matching is not appropri- are disordered and clashing
ate
13
0.90, implying high reliability. The third step is to automatically calculate the col-
laborative knowledge building level, which is equal to the activity quantities of all
knowledge nodes in a knowledge graph. In addition, the lag sequential analysis
method and content analysis method were utilized to analyze coregulated learning
behavioral patterns. Coregulated learning behaviors include orientating goals (OG),
making plans (MP), enacting strategies (ES), monitoring and controlling (MC), eval-
uating and reflecting (ER), and adapting metacognition (AM), which are adapted
from Saariaho et al. (2019) and Zheng et al. (2022), who proposed that these behav-
iors were coregulated learning behaviors. The two coders interdependently analyzed
the coregulated learning behaviors of all groups, and the interrater reliability calcu-
lated by the kappa value reached 0.9, indicating high reliability. GSEQ 5.1 software
developed by Quera et al. (2007) was adopted to analyze the coregulated behavior
transition. Furthermore, the semistructured interview records were independently
analyzed by two coders based on the thematic analysis method (Braun & Clarke,
2006). The interview records were categorized into five themes, including improv-
ing collaborative knowledge building, improving group performance, promoting
coregulated behaviors, promoting metacognitive learning engagement, and facilitat-
ing social interaction. The interrater reliability of the interview calculated by the
kappa value was 0.85, indicating high reliability.
5 Results
In this section, the impacts of the comprehensive learning analytics approach on col-
laborative knowledge building, group performance, coregulated behavior patterns,
metacognitive learning engagement, and social interaction are discussed in detail.
5.1 Analysis of collaborative knowledge building
To examine the impacts of the proposed approach on collaborative knowledge

building, one-way analysis of covariance (ANCOVA) was employed in this study.
Before performing ANCOVA, all datasets about collaborative knowledge building
were examined for normality distribution through the Kolmogorov–Smirnov test.
The findings revealed that all datasets had a normal distribution (p > 0.05). More-
over, the results indicated that there was no significant difference in prior knowl-
edge between the experimental group and the control group (t = 1.51, p = 0.13).
Furthermore, the results of the homogeneity test of the regression coefficient indi-
cated that the assumption of homogeneity of regression was not violated (F = 1.728,
p = 0.200). Hence, ANCOVA could be performed using the learning approach as the
independent variable, collaborative knowledge building as the dependent variable,
and the pretest as the covariant variable. Table 2 shows the ANCOVA results, and
the findings revealed that there was a significant difference in collaborative knowl-
edge building between the experimental and control groups (F = 16.057, p = 0.000).
The experimental group (adjusted mean = 504.07, std. error = 68.65) had higher
collaborative knowledge building than the control group (adjusted mean = 160.41,
13
Table 2 ANCOVA results of collaborative knowledge building

Group Number of Mean SD Adjusted mean SE F η2
groups
Experimental group 15 511.68 347.26 504.07 68.65 16.057*** 0.373

Control group 15 152.80 106.13 160.41 68.65
***
p < 0.001
Table 3 ANCOVA results of group performance

Group Number of Mean SD Adjusted mean SE F η2
group
Experimental group 15 92.40 3.04 92.35 0.45 21.93*** 0.20

Control group 15 89.27 3.03 89.31 0.46
***
p < 0.001
std. error = 68.65). Moreover, there was a large effect size (η2 = 0.383) according to
Cohen (1988). Furthermore, most of the interviewees (80%) believed that the com-
prehensive learning analytics approach contributed to linking prior knowledge with
new information to promote collaborative knowledge building. Therefore, both the
quantitative and qualitative analysis results revealed that the comprehensive learning
analytics approach had significant and positive impacts on collaborative knowledge
building.
5.2 Analysis of group performance
To examine the impacts of the proposed approach on group performance, one-way

analysis of covariance (ANCOVA) was employed in the present study. Before per-
forming ANCOVA, all datasets about group performance were examined for normal-
ity distribution through the Kolmogorov–Smirnov test. The findings revealed that all
datasets had a normal distribution (p > 0.05). Furthermore, the results of the homo-
geneity test of the regression coefficient indicated that the assumption of homo-
geneity of regression was not violated (F = 0.427, p = 0.515). Hence, ANCOVA
could be performed using the learning approach as the independent variable, group
performance as the dependent variable, and the pretest as the covariant variable.
Table 3 shows the ANCOVA results, and the findings revealed that there was a sig-
nificant difference between the group performance of the experimental and control
groups (F = 21.93, p = 0.000). The experimental group (adjusted mean = 92.35,
std. error = 0.45) had a higher group performance than the control group (adjusted
mean = 89.31, std. error = 0.46). Moreover, there was a large effect size (η2 = 0.20)
according to Cohen (1988). Furthermore, all interviewees (100%) believed that the
comprehensive learning analytics approach was beneficial to improving group prod-
ucts and acquiring new knowledge. Therefore, the comprehensive learning analytics
approach had significant and positive impacts on group performance.
13
Table 4 Adjusted residuals of the experimental group

Starting behaviour Subsequent behaviour
OG MP ES MC ER AM
Orientating goals (OG) 2.65* –0.92 –1.13 2.24 –1.69 –0.64

Making plans (MP) 3.17* 10.26* –4.76 0.63 –2.41 –0.30
Enacting strategies (ES) –2.05 –6.29 –4.26 6.98* 1.34 –0.92
Monitoring and controlling (MC) 0.75 2.69* 6.72* –6.72 –1.70 –1.18
Evaluating and reflecting (ER) –1.64 –1.85 0.45 –1.13 2.58* 1.35
Adapting metacognition (AM) 0.98 –0.95 –1.25 –0.86 1.33 7.23*
*
p < 0.05
Table 5 Adjusted residuals of the control group

OG MP ES MC ER AM
Orientating goals (OG) 4.78* 0.22 –0.37 –0.08 –0.78 –0.13

Making plans (MP) 0.23 11.32* –4.87 –0.75 –3.35 –0.62
Enacting strategies (ES) –0.36 –3.75 1.39 0.98 0.28 –0.07
Monitoring and controlling (MC) –0.10 –2.04 1.46 0.08 –0.21 –0.40
*
Evaluating and reflecting (ER) –0.77 –2.96 0.49 –0.91 4.21 1.66
Adapting metacognition (AM) –0.13 –0.63 1.14 –0.38 –0.51 –0.09
*
p < 0.05
5.3 Analysis of coregulated behavioral patterns
Tables 4 and 5 show the adjusted residuals of the experimental group and the con-
trol group, respectively. The columns of the two tables indicate the initial behav-
iors, and the rows represent the subsequent behaviors. If the z score is greater than
1.96, then the behavioral sequence has reached significance (Bakeman & Quera,
2011). The results indicated that there were eight significant behavior sequences
that occurred in the experimental group, including OG → OG (repeatedly setting
goals), MP → OG (setting new goals after forming plans), MP → MP (forming
plans continually), ES → MC (monitoring and controlling after enacting strate-
gies), MC → ES (enacting strategies after monitoring and controlling), MC → MP
(forming new plans after monitoring and controlling), ER → ER (evaluating and
reflecting continually), and AM → AM (adapting metacognition continually).
In contrast, there were only three significant behavior sequences that occurred
in the control group, namely, OG → OG (repeatedly setting goals), MP → MP
(repeatedly forming plans), and ER → ER (repeatedly evaluating and reflecting).
Figure 5 shows the behavior transition diagram of the experimental and control
groups. It is very clear that the transitions of different coregulated behaviors for
13
2.65 7.23 4.78
OG 3.17 MP AM OG MP
2.69 2.58 4.21
6.98
ES MC ER ER
6.72
The behavioral transition diagram of experimental groups The behavioral transition diagram of control groups
Fig. 5 Behavioral transition diagrams of the experimental and control groups
Table 6 Significant behavior sequences that only occurred in the experimental group
OG MP ES MC ER AM
Orientating goals (OG)

Making plans (MP) MP → OG
Enacting strategies (ES) ES → MC MC → MP
Monitoring and controlling (MC) MC → ES
Evaluating and reflecting (ER)
Adapting metacognition (AM)
the experimental group were significantly more than those of the control group.
In addition, Table 6 shows the significant behavior sequences that only occurred
in the experimental group. It was found that orientating goals, making plans,
monitoring and controlling as well as enacting strategies are crucial coregulated
behaviors.
In addition, the interview results indicated that 80% of the interviewees believed
that the comprehensive learning analytics approach was very helpful for monitoring
and controlling the online collaborative learning process. Ninety-three percent of
interviewees reported that the comprehensive learning analytics approach was very
useful for reflection and evaluating processes and outcomes. All of the interview-
ees believed that they could adjust strategies and plans based on the comprehensive
learning analytics results. Therefore, the comprehensive learning analytics approach
had significant and positive impacts on coregulated behavioral patterns.
5.4 Analysis of metacognitive learning engagement
Table 7 shows the metacognitive learning engagement of the experimental and con-
trol groups. The results indicated that there was a significant difference in metacog-
nitive learning engagement between the experimental group and the control group
(t = 5.26, p = 0.000). Furthermore, there was a large effect size for metacognitive
learning engagement according to Cohen (1988). In addition, all of the interviewees
13
Table 7 The t test results of metacognitive engagement of the two groups

Dimensions Groups Number of Mean SD t Effect size
groups
Metacognitive Experimental group 15 458.67 273.47 5.26*** 0.63

engagement Control group 15 124.80 90.99
***
p < 0.001
Table 8 The social interaction analysis results

Dimensions Groups Number of Mean SD U Z p
group
Degree Experimental group 15 7.33 4.18 24 -3.78 0.000

Control group 15 1.33 2.22
Density Experimental group 15 0.77 0.34 16 -4.19 0.000
Centrality Experimental group 15 1.55 0.69 29 -3.65 0.000
***
p < 0.001
(100%) believed that the comprehensive learning analytics approach contributed to

increasing metacognitive learning engagement. Hence, the comprehensive learning
analytics approach had significant and positive impacts on metacognitive learning
engagement.
5.5 Analysis of social interaction
Table 8 shows the social interactions of the experimental and control groups. The
results indicated that there were significant differences in degree (U = 24, Z = -3.78,
p = 0.000), density (U = 16, Z = -4.19, p = 0.000), and centrality (U = 29, Z = -3.65,
p = 0.000) between the experimental group and the control group. Furthermore,
most of the interviewees (80%) believed that the comprehensive learning analytics
approach contributed to facilitating social interaction among peers. Therefore, the
comprehensive learning analytics approach had significant and positive impacts on
social interaction.
6 Discussion
In this section, the main findings of this study are explained and discussed in depth.
Then, the implications of this study are also analyzed and presented in detail.
Finally, the limitations of the present study are clearly described in this section.
13
6.1 Discussion of main findings
In the present study, the impacts of the comprehensive learning analytics

approach on collaborative knowledge building, group performance, coregulated
behavior patterns, metacognitive learning engagement, and social interaction in
an online collaborative learning context were investigated. The qualitative and
quantitative results indicated that the comprehensive learning analytics approach
has significant and positive impacts on collaborative knowledge building, group
performance, coregulated behavior patterns, metacognitive learning engagement,
and social interaction.
The results of this study indicated that the comprehensive learning analyt-
ics approach significantly promoted collaborative knowledge building. The pos-
sible reasons include three aspects. First, the comprehensive learning analytics
approach served as an effective intervention to promote collaborative knowledge
building in this study. The comprehensive learning analytics approach can auto-
matically demonstrate the activated and unactivated knowledge graphs as well as
the activated public knowledge graphs of each group, which serve as an effective
intervention to promote knowledge coconstruction. The unactivated knowledge
stimulates learners to activate more knowledge and build knowledge in depth.
Teo et al. (2018) revealed that the use of learning analytics intervention con-
tributed to understanding and supporting knowledge building. Second, the com-
prehensive learning analytics approach provided personalized feedback for each
group based on the learning analytics results. Resendes et al. (2015) proposed
that the provision of feedback could promote the collaborative knowledge build-
ing level.
The findings of this study revealed that the comprehensive learning analytics
approach significantly improved group performance. There are three possible rea-
sons for this result. First, the comprehensive learning analytics approach was an
effective learning analytics intervention to improve group performance. When learn-
ers engaged in less knowledge building or sent off-topic messages, the comprehen-
sive learning analytics approach could automatically detect this and intervene imme-
diately. Moreover, this result was consistent with previous studies that also revealed
that learning analytics interventions could enhance learning performance (Kew &
Tasir, 2022; Wong & Li, 2020). Second, the present study showed that the compre-
hensive learning analytics approach significantly promoted collaborative knowledge
building. The collaborative knowledge building level was positively related to group
performance (Chai & Zhu, 2021; Zheng et al., 2021). Third, the comprehensive
learning analytics approach provides personalized feedback in real time. Personal-
ized feedback could promote group performance (Zheng et al., 2021). Therefore, the
group performance of the experimental group was significantly higher than that of
the control group.
In addition, the comprehensive learning analytics approach significantly facil-
itated coregulated behavioral patterns. Two possible reasons might explain the
results. First, the comprehensive learning analytics approach was considered
an effective learning analytics intervention to facilitate coregulated behaviors.
The comprehensive learning analytics approach demonstrated analysis results
13
of collaborative knowledge building, metacognitive learning engagement, and

social interaction, which facilitate learners to coregulate their behaviors based
on the analysis results. Second, the comprehensive learning analytics results
serve as a group awareness tool to facilitate coregulated behaviors. Schnaubert
and Bodemer (2022) proposed that the group awareness tool can support coregu-
lation of collaborative learning.
Furthermore, the comprehensive learning analytics approach significantly
enhanced metacognitive learning engagement. There are two possible explana-
tions for this finding. First, the comprehensive learning analytics approach was
an effective learning analytics intervention to promote metacognitive learning
engagement. The comprehensive learning analytics approach can provide real-
time learning analytic results about metacognitive learning engagement, which
can promote metacognitive learning engagement. For example, when there was
less monitoring and control of the collaborative learning progress, the com-
prehensive learning analytics approach would remind learners to monitor and
control the learning process. When there was less reflection and evaluation, the
comprehensive learning analytics approach would remind learners to reflect and
evaluate in depth. Moreover, this finding was in line with Lu et al. (2017), who
found that learning analytics can improve learners’ levels of learning engage-
ment. Second, the comprehensive learning analytics approach provides real-
time feedback for learners, which promotes metacognitive learning engagement.
Wang and Zhang (2020) revealed that feedback could promote students’ learning
engagement.
Finally, the comprehensive learning analytics approach significantly pro-
moted social interaction. There are two possible explanations for this finding.
First, as an effective learning analytics intervention, the comprehensive learning
analytics approach provided immediate social interaction results and social net-
work analysis results, which promote social interaction among group members.
For example, when there was less interaction among group members, the com-
prehensive learning analytics approach would remind learners to increase social
interaction through online discussion. Wise et al. (2015) proposed that learning
analytics results had great potential to facilitate social interaction. Second, the
comprehensive learning analytics approach can provide immediate feedback for
each group. Previous studies have found that feedback can promote social inter-
action (Zheng et al., 2022).
6.2 Implications
The present study has several practical and technological implications for
teachers, researchers, and practitioners. First, the comprehensive learning ana-
lytics approach is very useful and helpful for promoting collaborative knowl-
edge building level, group performance, coregulated behaviors, metacognitive
13
learning engagement, and social interaction. Therefore, it is suggested that

teachers and practitioners adopt the comprehensive learning analytics approach
when conducting online collaborative learning.
Second, the present study revealed that deep neural network models are very effi-
cient and promising in constructing knowledge graphs and text classification. Deep
neural network models can achieve superior performance compared to shallower neural
networks since they are able to learn high-level features with more complexity (Sze
et al., 2017). It is suggested that deep neural network models should be adopted to auto-
matically analyze online discussion transcripts.
Third, teachers and practitioners can adopt the comprehensive learning analytics
approach to detect problems of online collaborative learning and provide interven-
tion for learners. Learning analytics interventions have beneficial impacts on learn-
ing performance (Knobbout & Van Der Stappen, 2020). In addition, tasks, interac-
tion strategies, and assessment methods can also be optimized based on learning
analytics results.
6.3 Limitations
This study has several limitations, and caution should be taken when generalizing
the findings in other contexts. First, the sample size was not large in the present
study. Future studies should expand the sample size to multiple universities to vali-
date the proposed approach. Second, this study selected only one learning domain
to investigate the proposed approach. Future studies should examine the proposed
approach in other learning domains. Finally, the present study investigated the
impacts of the proposed approach on a limited set of variables, namely, collabo-
rative knowledge building, group performance, coregulative behavioral patterns,
metacognitive learning engagement, and social interaction. Future studies should
examine the effects of the proposed approach on other variables, such as higher-
order thinking skills and problem-solving abilities.
7 Conclusions
This study revealed that the comprehensive learning analytics approach is very
effective and useful for improving collaborative knowledge building, group per-
formance, coregulated behavioral patterns, metacognitive learning engagement,
and social interaction in an online collaborative learning context. The main con-
tribution of the present study is to propose and validate the positive impacts of
the comprehensive learning analytics approach. This study also provides insights
into the automated construction of knowledge graphs and analysis of learning
engagement through artificial intelligence technologies. The present study sheds
light on the positive role and value of using a comprehensive learning analytics
approach in online collaborative learning.
13
Appendix
7. Which one is not a multimedia dynamic image file? ()
A. AVI
B. MPG
C. BMP
D. AVS
8. Which one is not a sound file? ( )
A. mid
B. mp3
C. wav
D. avi
Multiple choice questions with multiple answers

1. The major characteristics of multimedia technology include ( )
A. Interactivity
B. Diversity
C. Integration
D. Real-time
2. Which are multimedia? ( )
A. Interactive video games
B. Audio books
C. Color pictorial
D. Stereo music
3. Which are transmission medium? ( )
A. Fiber-optical
B. Twisted-pair
C. Coaxial cable
D. Wireless transmission medium
4. The hardware of multimedia include ( )
A. Computer
B. Digital camera
C. Multimedia board card
D. Multimedia external devices
Fill-in-the-blanks questions
1. Image files can be divided into and .
2. Three elements of color include , , and .
3. Three elements of sound include , , and .
Short answer questions

1. Please briefly describe multimedia and multimedia technology?
2. What are the major types of multimedia material? How can multimedia material be searched for
and obtained?
13
Acknowledgements This study is funded by the International Joint Research Project of Huiyan International
College, Faculty of Education, Beijing Normal University (ICER202101).
Data availability The data of this study cannot be made openly available due to ethical concerns but are
available from the corresponding author on reasonable request.
Declarations
Conflict of interest The authors declare that they have no conflicts of interest.
Notes on contributors Lanqin Zheng currently works as an associate professor at the Faculty of Education
in Beijing Normal University. Her research interests include computer supported collaborative learning,
learning analytics, and AIED. Kinshuk is a professor in University of North Texas. His research interests
include learning analytics and adaptive learning. Yunchao Fan and Miaolang Long are master students at
the Faculty of Education in Beijing Normal University. Yunchao Fan’s research interests include computer
supported collaborative learning and learning analytics. Miaolang Long’s research interests include computer
supported collaborative learning and learning analytics.
References
Agudo-Peregrina, Á. F., Iglesias-Pradas, S., Conde-González, M. Á., & Hernández-García, Á. (2014).
Can we predict success from log data in VLEs? Classification of interactions for learning analyt-
ics and their relation with performance in VLE-supported F2F and online learning. Computers in
Human Behavior, 31, 542–550.
Álvarez-Xochihua, O., Merino, P. J. M., García-Pericuesta, R., González-Fraga, J. Á., Kloos, C. D.,
López, E. G., & Andrade-Aréchiga, M. (2017). Learning analytics implementation in a multidomain
computer-based learning environment. In Proceedings of the Learning Analytics Summer Institute
Spain 2017 (pp. 17–30). https://ceurws.org/Vol-1925/
Avcı, Ü. (2020). Examining the role of sentence openers, role assignment scaffolds and self-determi-
nation in collaborative knowledge building. Educational Technology Research and Development,
68(1), 109–135. https://doi.org/10.1007/s11423-019-09672-5
Bakeman, R., & Quera, V. (2011). Sequential analysis and observational methods for the behavioural
sciences. Cambridge University Press.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychol-
ogy, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa
Chai, S., & Zhu, G. (2021). The relationship between group adoption of Knowledge Building Principles
and performance in creating artifacts. Educational Technology Research and Development, 69(2),
787–808. https://doi.org/10.1007/s11423-021-09986-3
Chan, C. K. (2012). Co-regulation of learning in computer-supported collaborative learning envi-
ronments: A discussion. Metacognition and Learning, 7(1), 63–73. https://doi.org/10.1007/
s11409-012-9086-z
Chen, C. M., Wang, J. Y., & Zhao, R. H. (2022). An effective method for incentivizing groups imple-
mented in a collaborative problem-based learning system to enhance positive peer interaction and
learning performance. Interactive Learning Environments, 30(3), 435–454. https://doi.org/10.1080/
10494820.2019.1663435
Clow, D. (2013). An overview of learning analytics. Teaching in Higher Education, 18(6), 683–695.
https://doi.org/10.1080/13562517.2013.827653
Cohen, J. (1988). Statistical power analysis for the behavioural sciences (2nd ed.). Erlbaum.
Craig, K., Danish, J., Humburg, M., Hmelo-Silver, C., Szostalo, M., & McCranie, A. (2021). Net. Cre-
ate: Network visualization to support collaborative historical knowledge building. International
Journal of Computer-Supported Collaborative Learning, 16(2), 185–223. https://doi.org/10.1007/
s11412-021-09343-9
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional trans-
formers for language understanding. In Proceedings of the 2019 Conference of the North American
13
Chapter of the Association for Computational Linguistics: Human Language Technologies (pp.
4171–4186). Association for Computational Linguistics. https://doi.org/10.18653/v1/N19-1423
Duvall, M., Matranga, A., & Silverman, J. (2020). Designing for and facilitating knowledge-building dis-
course in online courses. Information and Learning Sciences, 121(7/8), 487–501. https://doi.org/10.
1108/ILS-04-2020-0081
Halimi, K., & Seridi-Bouchelaghem, H. (2021). Students’ competencies discovery and assessment using
learning analytics and semantic web. Australasian Journal of Educational Technology, 37(5),
77–97. https://doi.org/10.14742/ajet.7116
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8),
1735–1780.
Hu, X., Ng, J. T., & Chu, S. K. (2022). Implementing learning analytics in wiki-supported collaborative
learning in secondary education: A framework-motivated empirical study. International Journal of
Computer-Supported Collaborative Learning, 17(3), 427–455.
Huang, H. W. (2021). Effects of smartphone-based collaborative vlog projects on EFL learners’ speak-
ing performance and learning engagement. Australasian Journal of Educational Technology, 37(6),
18–40. https://doi.org/10.14742/ajet.6623
Järvenoja, H., Järvelä, S., & Malmberg, J. (2020). Supporting groups’ emotion and motivation regulation
during collaborative learning. Learning and Instruction, 70, 101090. https://doi.org/10.1016/j.learn
instruc.2017.11.004
Karaoglan Yilmaz, F. G., & Yilmaz, R. (2022). Learning analytics intervention improves students’
engagement in online learning. Technology, Knowledge and Learning, 27(2), 449–460. https://doi.
org/10.1007/s10758-021-09547-w
Kew, S. N., & Tasir, Z. (2022). Developing a learning analytics intervention in e-learning to enhance stu-
dents’ learning performance: a case study. Education and Information Technologies, 1–36. https://
doi.org/10.1007/s10639-022-10904-0
Knobbout, J., & Van Der Stappen, E. (2020). Where is the learning in learning analytics? A systematic
literature review on the operationalization of learning-related constructs in the evaluation of learn-
ing analytics interventions. IEEE Transactions on Learning Technologies, 13(3), 631–645. https://
doi.org/10.1109/TLT.2020.2999970
Labonté, C., & Smith, V. R. (2022). Learning through technology in middle school classrooms: Students’
perceptions of their self-directed and collaborative learning with and without technology. Education
and Information Technologies, 27, 6317–6332. https://doi.org/10.1007/s10639-021-10885-6
Lafferty, J., McCallum, A., & Pereira, F. C. (2001). Conditional random fields: Probabilistic models for
segmenting and labeling sequence data. The Proceedings of the 18th International Conference on
Machine Learning (ICML 2001), 282–289. https://dl.acm.org/doi/10.5555/645530.655813
LarrabeeSønderlund, A., Hughes, E., & Smith, J. (2019). The efficacy of learning analytics interventions
in higher education: A systematic review. British Journal of Educational Technology, 50(5), 2594–
2618. https://doi.org/10.1111/bjet.12720
Lei, C., & Chan, C. K. (2018). Developing metadiscourse through reflective assessment in knowledge
building environments. Computers & Education, 126, 153–169. https://doi.org/10.1016/j.compedu.
2018.07.006
Lim, J. Y., & Lim, K. Y. (2020). Co-regulation in collaborative learning: Grounded in achievement goal
theory. International Journal of Educational Research, 103, 101621. https://doi.org/10.1016/j.ijer.
2020.101621
Lim, L. A., Dawson, S., Gašević, D., Joksimović, S., Pardo, A., Fudge, A., & Gentili, S. (2021). Students’
perceptions of, and emotional responses to, personalised learning analytics-based feedback: An
exploratory study of four courses. Assessment & Evaluation in Higher Education, 46(3), 339–359.
https://doi.org/10.1080/02602938.2020.1782831
Liu, C., Wan, P., Hwang, G. J., Tu, Y. F., & Wang, Y. (2021). From competition to social interaction: a
mobile team-based competition approach to promoting students’ professional identity and percep-
tions. Interactive Learning Environments, 1–15. https://doi.org/10.1080/10494820.2020.1823855
Lu, O. H., Huang, J. C., Huang, A. Y., & Yang, S. J. (2017). Applying learning analytics for improv-
ing students engagement and learning outcomes in an MOOCs enabled collaborative programming
course. Interactive Learning Environments, 25(2), 220–234. https://doi.org/10.1080/10494820.
2016.1278391
Melzner, N., Dresel, M., & Kollar, I. (2022). Examining the regulation of motivational and comprehen-
sion-related problems during collaborative learning. Metacognition and Learning, 1–24. https://doi.
org/10.1007/s11409-022-09316-9
13
Minović, M., Milovanović, M., Šošević, U., & González, M. Á. C. (2015). Visualisation of student learn-
ing model in serious games. Computers in Human Behaviour, 47, 98–107. https://doi.org/10.1016/j.
chb.2014.09.005
North, K. (2021). Learning in the Year 2030. Learning in the Year 2030. In Güldenberg, S., Ernst, E.,
North, K. (Eds.), Managing Work in the Digital Economy. Future of Business and Finance. Springer,
Cham. https://doi.org/10.1007/978-3-030-65173-2_14
Ochoa, X., & Wise, A. F. (2021). Supporting the shift to digital with student-centered learning analyt-
ics. Educational Technology Research and Development, 69(1), 357–361. https://doi.org/10.1007/
s11423-020-09882-2
Ozdemir, D., Opseth, H. M., & Taylor, H. (2020). Leveraging learning analytics for student reflection and
course evaluation. Journal of Applied Research in Higher Education, 12(1), 27–37.
Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of class-
room academic performance. Journal of Educational Psychology, 82(1), 33–40. https://doi.org/10.
1037/0022-0663.82.1.33
Quera, V., Bakeman, R., & Gnisci, A. (2007). Observer agreement for event sequences: Methods and
software for sequence alignment and reliability estimates. Behaviour Research Methods, 39(1),
39–49. https://doi.org/10.3758/bf03192842
Reeves, T. C., Herrington, J., & Oliver, R. (2004). A development research agenda for online collabora-
tive learning. Educational Technology Research and Development, 52(4), 53–65. https://doi.org/10.
1007/BF02504718
Resendes, M., Scardamalia, M., Bereiter, C., Chen, B., & Halewood, C. (2015). Group-level formative
feedback and metadiscourse. International Journal of Computer-Supported Collaborative Learning,
10(3), 309–336. https://doi.org/10.1007/s11412-015-9219-x
Saariaho, E., Toom, A., Soini, T., Pietarinen, J., & Pyhältö, K. (2019). Student teachers’ and pupils’ co-
regulated learning behaviours in authentic classroom situations in teaching practicums. Teaching
and Teacher Education, 85, 92–104. https://doi.org/10.1016/j.tate.2019.06.003
Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In R. K.
Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 97–119). Cambridge Univer-
sity Press.
Schnaubert, L., & Bodemer, D. (2022). Group awareness and regulation in computer-supported collabo-
rative learning. International Journal of Computer-Supported Collaborative Learning, 17, 11–38.
https://doi.org/10.1007/s11412-022-09361-1
Shin, Y., Kim, D., & Jung, J. (2018). The effects of representation tool (visible-annotation) types to sup-
port knowledge building in computer-supported collaborative learning. Educational Technology &
Society, 21(2), 98–110. http://www.jstor.org/stable/26388383.
Shin, S., Brush, T. A., & Glazewski, K. D. (2020). Examining the hard, peer, and teacher scaffolding
framework in inquiry-based technology-enhanced learning environments: Impact on academic
achievement and group performance. Educational Technology Research and Development, 68(5),
2423–2447. https://doi.org/10.1007/s11423-020-09763-8
Siemens, G., & Baker, R. S. D. (2012). Learning analytics and educational data mining: towards commu-
nication and collaboration. In Proceedings of the 2nd international conference on learning analyt-
ics and knowledge (pp. 252–254). Association for Computing Machinery. https://doi.org/10.1145/
2330601.2330661
Stahl, G., Koschmann, T., & Suthers, D. (2014). Computer-supported collaborative learning. In R. K.
Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 479–500). Cambridge Univer-
sity Press. https://doi.org/10.1017/CBO9781139519526.029
Sun, Z., Lin, C. H., Wu, M., Zhou, J., & Luo, L. (2018). A tale of two communication tools: Discussion-
forum and mobile instant-messaging apps in collaborative learning. British Journal of Educational
Technology, 49(2), 248–261. https://doi.org/10.1111/bjet.12571
Sun, Z., & Theussen, A. (2022). Assessing negotiation skill and its development in an online collabora-
tive simulation game: A social network analysis study. British Journal of Educational Technology,
1– 25. https://doi.org/10.1111/bjet.13263
Susnjak, T., Ramaswami, G. S., & Mathrani, A. (2022). Learning analytics dashboard: A tool for provid-
ing actionable insights to learners. International Journal of Educational Technology in Higher Edu-
cation, 19(1), 1–23. https://doi.org/10.1186/s41239-021-00313-7
Sze, V., Chen, Y.-H., Yang, T.-J., & Emer, J. S. (2017). Efficient processing of deep neural networks: A
Tutorial and survey. Proceedings of the IEEE, 105(12), 2295–2329. https://doi.org/10.1109/jproc.
2017.2761740
13
Takii, K., Flanagan, B., & Ogata, H. (2021). EFL vocabulary learning using a learning analytics-based
e-book and recommender platform. In 2021 International Conference on Advanced Learning Tech-
nologies (ICALT) (pp. 254–256). IEEE.
Teo, C. L., Chan, C., & Ng, D. (2018). Engaging teachers in discussions around temporality measures from
analytics to inform knowledge building discourse. https://repository.isls.org/bitstream/1/799/1/451.pdf
Unal, E., & Cakir, H. (2021). The effect of technology-supported collaborative problem solving method
on students’ achievement and engagement. Education and Information Technologies, 26(4), 4127–
4150. https://doi.org/10.1007/s10639-021-10463-w
Viberg, O., Hatakka, M., Bälter, O., & Mavroudi, A. (2018). The current landscape of learning analyt-
ics in higher education. Computers in Human Behaviour, 89, 98–110. https://doi.org/10.1016/j.chb.
2018.07.027
Wang, S., & Zhang, D. (2020). Perceived teacher feedback and academic performance: The mediating
effect of learning engagement and moderating effect of assessment characteristics. Assessment &
Evaluation in Higher Education, 45(7), 973–987. https://doi.org/10.1080/02602938.2020.1718599
Wang, X., Xing, W., & Laffey, J. M. (2018). Autistic youth in 3D game-based collaborative virtual learn-
ing: Associating avatar interaction patterns with embodied social presence. British Journal of Edu-
cational Technology, 49(4), 742–760. https://doi.org/10.1111/bjet.12646
Wang, P., Qian, Y., Soong, F. K., He, L., & Zhao, H. (2015). Part-of-speech tagging with bidirectional
long short-term memory recurrent neural network. arXiv preprint arXiv:1510.06168.
Wen, Y., & Song, Y. (2021). Learning Analytics for Collaborative Language Learning in Classrooms:
From the Holistic Perspective of Learning Analytics, Learning Design and Teacher Inquiry. Educa-
tional Technology & Society, 24(1), 1–15.
Wise, A. F., Azevedo, R., Stegmann, K., Malmberg, J., Rosé, C. P., Mudrick, N., et al. (2015). CSCL and
learning analytics: Opportunities to support social interaction, self-regulation and socially shared
regulation. In O. Lindwall, P. Häkkinen, T. Koschman, P. Tchounikine, & S. Ludvigsen (Eds.), The
Computer Supported Collaborative Learning (CSCL) Conference 2015 (pp. 607–614). The Interna-
tional Society of the Learning Sciences. https://doi.org/10.22318/cscl2015.1107
Wong, B. T. M., & Li, K. C. (2020). A review of learning analytics intervention in higher education
(2011–2018). Journal of Computers in Education, 7(1), 7–28.
Xie, T., Liu, R., Chen, Y., & Liu, G. (2021). MOCA: A motivational online conversational agent for
improving student engagement in collaborative learning. IEEE Transactions on Learning Technolo-
gies, 14(5), 653–664. https://doi.org/10.1109/TLT.2021.3129800
Yilmaz, R., Karaoglan Yilmaz, F. G., & Keser, H. (2020). Vertical versus shared e-leadership approach in
online project-based learning: A comparison of self-regulated learning skills, motivation and group
collaboration processes. Journal of Computing in Higher Education, 32(3), 628–654. https://doi.
org/10.1007/s12528-020-09250-2
Zheng, L., Li, X., Zhang, X., & Sun, W. (2019). The Effects of group metacognitive scaffolding on group
metacognitive behaviors, group performance, and cognitive load in computer-supported collaborative
learning. The Internet and Higher Education, 42, 13–24. https://doi.org/10.1016/j.iheduc.2019.03.002
Zheng, L., Zhong, L., Niu, J., Long, M., & Zhao, J. (2021). Effects of personalized intervention on col-
laborative knowledge building, group performance, socially shared metacognitive regulation, and
cognitive load in computer-supported collaborative learning. Educational Technology & Society,
24(3), 174–193. https://www.jstor.org/stable/pdf/27032864.pdf
Zheng, L., Niu, J., & Zhong, L. (2022). Effects of a learning analytics-based real-time feedback approach
on knowledge elaboration, knowledge convergence, interactive relationships and group performance in
CSCL. British Journal of Educational Technology, 53(1), 130–149. https://doi.org/10.1111/bjet.13156
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps
and institutional affiliations.
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under
a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted
manuscript version of this article is solely governed by the terms of such publishing agreement and
applicable law.
13

SOS KALO PAPER Gia LA in CSCL Two Groups Etc

Uploaded by

Copyright:

Available Formats

SOS KALO PAPER Gia LA in CSCL Two Groups Etc

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

SOS KALO PAPER Gia LA in CSCL Two Groups Etc

Uploaded by

Copyright:

Available Formats

Education and Information Technologies

The impacts of the comprehensive learning analytics

Lanqin Zheng1 · Kinshuk2 · Yunchao Fan1 · Miaolang Long1

Received: 10 October 2022 / Accepted: 5 May 2023

Keywords Learning analytics · Online collaborative learning · Collaborative

As an effective pedagogy, online collaborative learning has been widely adopted

RQ1: What is the impact of the comprehensive learning analytics approach on

RQ4: What is the impact of the comprehensive learning analytics approach on

2.1 Online collaborative learning

Learning analytics is defined as the measurement, collection, analysis, and report-

(2017) adopted learning analytics to improve learning engagement and outcomes

3 The comprehensive learning analytics approach

3.1 Design of the comprehensive learning analytics approach

In this study, a comprehensive learning analytics approach was proposed to promote

Phase 1:Collect data Collect data through online collaborative learning

Analysis of social interaction

Phase 3: Feedback about collaborative knowledge building

Feedback about social interaction

Recommend learning resources

Fig. 1 The flowchart of the comprehensive learning analytics approach

3.2 Implementation of the comprehensive learning analytics approach

In this study, the implementation of the comprehensive learning analytics approach

Fig. 2 Screenshot of the online collaborative learning platform

Fig. 3 Screenshot of comprehensive learning analytics

The participants were 90 college students in a top-10 public university in China.

A quasi-experimental design was employed in this study to examine the impacts of

Experimental group Control group

Introduce how to use online collaborative learning

Conduct online Conduct traditional

Submit group products

Fig. 4 The experimental procedure

4.3 Data collection and analysis methods

5.1 Analysis of collaborative knowledge building

To examine the impacts of the proposed approach on collaborative knowledge

Table 2 ANCOVA results of collaborative knowledge building

Experimental group 15 511.68 347.26 504.07 68.65 16.057*** 0.373

Table 3 ANCOVA results of group performance

Experimental group 15 92.40 3.04 92.35 0.45 21.93*** 0.20

5.2 Analysis of group performance

To examine the impacts of the proposed approach on group performance, one-way

Table 4 Adjusted residuals of the experimental group

Orientating goals (OG) 2.65* –0.92 –1.13 2.24 –1.69 –0.64

Table 5 Adjusted residuals of the control group

Orientating goals (OG) 4.78* 0.22 –0.37 –0.08 –0.78 –0.13

5.3 Analysis of coregulated behavioral patterns

2.65 7.23 4.78

Fig. 5 Behavioral transition diagrams of the experimental and control groups

Orientating goals (OG)

5.4 Analysis of metacognitive learning engagement

Table 7 The t test results of metacognitive engagement of the two groups

Metacognitive Experimental group 15 458.67 273.47 5.26*** 0.63

Table 8 The social interaction analysis results

Degree Experimental group 15 7.33 4.18 24 -3.78 0.000

(100%) believed that the comprehensive learning analytics approach contributed to

5.5 Analysis of social interaction

6.1 Discussion of main findings

In the present study, the impacts of the comprehensive learning analytics

of collaborative knowledge building, metacognitive learning engagement, and

learning engagement, and social interaction. Therefore, it is suggested that

2.1 Online collaborative learning

3 The comprehensive learning analytics approach

3.1 Design of the comprehensive learning analytics approach

3.2 Implementation of the comprehensive learning analytics approach

4.3 Data collection and analysis methods

5.1 Analysis of collaborative knowledge building

5.2 Analysis of group performance

5.3 Analysis of coregulated behavioral patterns

5.4 Analysis of metacognitive learning engagement

5.5 Analysis of social interaction

6.1 Discussion of main findings