2015 Book ResponsiveOpenLearningEnvironm PDF

Sylvana Kroop
Alexander Mikroyannidis
Martin Wolpers Editors
Responsive
Open Learning
Environments
Outcomes of Research from the ROLE
Project
Responsive Open Learning Environments
ThiS is a FM Blank Page
Sylvana Kroop • Alexander Mikroyannidis •
Martin Wolpers
Editors
Responsive Open Learning

Environments
Outcomes of Research from the ROLE Project
Editors
Sylvana Kroop Alexander Mikroyannidis
ZSI Vienna Centre for Social Innovation The Open University
Vienna Milton Keynes
Austria United Kingdom
Martin Wolpers
Fraunhofer Institute for Applied Information
Technology (FIT)
Sankt Augustin
Germany
ISBN 978-3-319-02398-4 ISBN 978-3-319-02399-1 (eBook)

DOI 10.1007/978-3-319-02399-1
Springer Cham Heidelberg New York Dordrecht London
Library of Congress Control Number: 2014955579
© The Editor(s) (if applicable) and the Author(s) 2015. The book is published with open access at
SpringerLink.com.
Open Access This book is distributed under the terms of the Creative Commons Attribution
Noncommercial License which permits any noncommercial use, distribution, and reproduction in any
medium, provided the original author(s) and source are credited.
All commercial rights are reserved by the Publisher, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication
of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for commercial use must always be obtained
from Springer. Permissions for commercial use may be obtained through RightsLink at the Copyright
Clearance Center. Violations are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)

Acknowledgement
The publication of this open access book has received funding from the European
Community under grant agreement no. 318499 (weSPOT project).
v
Contents
Personal Learning Environments (PLEs): Visions and Concepts . . . . . . 1

Alexander Mikroyannidis, Sylvana Kroop, and Martin Wolpers
Supporting Self-Regulated Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Alexander Nussbaumer, Ingo Dahn, Sylvana Kroop,
Alexander Mikroyannidis, and Dietrich Albert
A Multidimensional Evaluation Framework for Personal Learning
Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Effie Lai-Chong Law and Fridolin Wild
Case Study 1: Using Widget Bundles for Formal Learning in Higher
Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
Helmut Vieritz, Carsten Ullrich, Erik Isaksson, Hans-Christian Schmitz,
Bodo von der Heiden, Kerstin Borau, Ruimin Shen, Matthias Palmér,
Thomas Lind, and Mikael Laaksoharju
Case Study 2: Designing PLE for Higher Education . . . . . . . . . . . . . . . 115
Denis Gillet and Na Li
Case Study 3: Exploring Open Educational Resources for Informal
Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
Alexander Mikroyannidis and Teresa Connolly
Case Study 4: Technology Enhanced Workplace Learning . . . . . . . . . . 159
Michael Werkle, Manuel Schmidt, Diana Dikke, and Simon Schwantzer
Lessons Learned from the Development of the ROLE PLE
Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Sten Govaerts, Katrien Verbert, Evgeny Bogdanov, Erik Isaksson,
Daniel Dahrendorf, Carsten Ullrich, Maren Scheffel, Sarah Léon Rojas,
and Denis Gillet
vii
viii Contents
Commentary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
ROLE Consortium: Research Institutions . . . . . . . . . . . . . . . . . . . . . . . 241
Supplementary Material . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
List of Authors: Members of the ROLE
Consortium
Dietrich Albert is professor of psychology at

University of Graz, Austria, senior scientist at the
Knowledge Technologies Institute at Graz Uni-
versity of Technology, and key researcher at the
Know-Center, a competence centre for knowledge
management in Austria. His research topics cover
several areas in cognitive psychology, mainly
focusing on knowledge and competence structures
and their applications. He has been involved in
more than two dozen EU-funded research projects
on technology-enhanced learning (e.g. ROLE,
NEXT-TELL, and 80Days).
Evgeny Bogdanov is a post-doctoral researcher

in Computer Science at REACT group of Ecole
Polytechnique Fédéral de Lausanne (EPFL) in
Switzerland. He received his Ph.D. degree in
Computer Science from EPFL in August 2013.
His research interests are in the area of widgets
and widget bundles with focus on portability and
interoperability. He is currently involved in the
European Go-Lab project that started in
December 2012.
ix
x List of Authors: Members of the ROLE Consortium
Kerstin Borau is a certified German and

English teacher at the School of Continuing
Education of Shanghai Jiao Tong University.
She obtained her M.A. in English Linguistics at
Saarland University with a thesis on First Lan-
guage Acquisition that received the highest
grade. Her current research focuses on
non-linear learning, mobile learning and on
exploring new methods for language learning,
such as using Web 2.0 services in innovative
ways to increase active participation.
Teresa Connolly works at the University of

Oxford on the EU FP7 funded BioFresh project.
She has extensive professional experience in
research and teaching in the areas of Geographic
Information Systems (GIS), Academic Practice,
Educational Technology and Open Educational
Resources (OER). Teresa has worked as a Lec-
turer in OER on the OpenLearn project and on a
variety of EU-funded technology-enhanced
learning (TEL), projects (including ROLE) at
the Open University. She also has worked on
the JISC/HEA funded Open Resource Bank for Interactive Teaching (ORBIT)
project at the University of Cambridge. Teresa has also carried out a number of
consultancies for UNESCO, the Imperial College NHS Trust and the UK Higher
Education Academy (HEA) where she is an Associate.
Ingo Dahn leads the Knowledge Media Institute

of the University Koblenz-Landau in Koblenz since
2001. With a habilitation in Mathematical Logic
from Humboldt University Berlin, he started his
career teaching mathematics and doing research in
non-standard logics and model theory. In the late
1980s he moved to artificial intelligence, in partic-
ular automated and interactive theorem proving.
This led to his work on textbook personalisation
and e-Learning after he joined the Artificial Intelli-
gence group in Koblenz in 1998. Major areas of interest in e-Learning are technical
standards, conformance testing, and content recommender systems. He was involved
in numerous European projects, e.g. ROLE, TAS3, and RadioActive Europe.
List of Authors: Members of the ROLE Consortium xi
Daniel Dahrendorf obtained a M.Sc. degree in

computer science and a Staatsexamen in com-
puter science and mathematics from Saarland
University. After graduation he started to work
at IMC in the Innovation Labs department,
where he was involved in national and interna-
tional research projects as a researcher and
senior software engineer. His research areas
and main expertise are Web Application Devel-
opment, Product Management, Social Software,
Learning Management Systems, MOOCs and
Personalised Learning Software. Currently he
manages the development of IMC’s new learning interface for open online courses
and cooperate learning scenarios.
Diana Dikke is working as a research profes-

sional for IMC AG conducting research in the
fields of competency-based personnel develop-
ment and learning management, self-regulated
learning with web applications, virtual learning
environments, and inquiry-based learning with
online labs. Ms. Dikke is also an experienced
specialist in research project and learning soft-
ware marketing, especially in using web instru-
ments for dissemination activities, as well as web
platform administration.
Denis Gillet leads the Coordination and Inter-

action Systems Group at the Swiss Federal Insti-
tute of Technology in Lausanne (EPFL), where
he received his Ph.D. in Information Systems in
1995. His research interests include Technolo-
gies Enhanced Learning (TEL), Human–Com-
puter Interaction (HCI), Engineering Education,
as well as Dynamic Coordination of Distributed
Systems and Devices. His current research focus
is on the design and deployment of social media
platforms for Science Technology Engineering
and Mathematics (STEM) Education and
xii List of Authors: Members of the ROLE Consortium
Knowledge Management (KM). Denis Gillet is Associate Editor of the

International Journal of Technology Enhanced Learning. He has been an Executive
of previous European research projects on Technology-Enhanced Learning and is
currently the Technical Coordinator of the Go-Lab Integrated European initiative
investigating online labs federation and exploitation for inquiry-based learning
at school.
Sten Govaerts is a post-doctoral researcher at

École Polytechnique Fédérale de Lausanne
(EPFL) in Switzerland in the Real-time Coordi-
nation & Distributed Interaction Systems
(REACT) Group. He is currently responsible
for the technical work package of the Go-Lab
project (http://www.go-lab-project.eu/) that aims
to open up remote science laboratories and their
online labs for large-scale use in education. He
finished his Ph.D. at KU Leuven (Belgium) in
the Human–Computer Interaction (HCI)
research group under supervision of Prof. Erik Duval in 2012. His research interests
include user experience design, information visualisation, ubiquitous computing
and findability applied to both the technology-enhanced learning domain and
contextualised music.
Bodo Von der Heiden studied computer sci-

ence at the RWTH Aachen University until
2008. During his studies he was involved in
programming of children programming lan-
guage, called “Blopp”. From 2009 to 2011 he
was a research assistant in the field of knowledge
management at the Center for Learning and
Knowledge Management and Department of
Information Management in Mechanical Engi-
neering at RWTH Aachen University. Here he
was engaged in the development of a knowledge
map in the EU-sponsored project ROLE. Since
2011 he works at edudip GmbH in the develop-
ment of a portal for online seminars, so-called
webinars.
List of Authors: Members of the ROLE Consortium xiii
Erik Isaksson has a Master of Science degree

in Information and Communication Technology
from the Royal Institute of Technology (KTH),
Stockholm, Sweden. After graduating in 2009,
he has worked within the Knowledge Manage-
ment Research (KMR) group as a research engi-
neer at Uppsala University, Sweden, primarily
with Web technologies and Linked Data, and
how these can be applied to Technology-
Enhanced Learning (TEL). Currently, he works
within KMR, now as a Ph.D. candidate at KTH,
in projects related to TEL as well as Cultural
Heritage. Among earlier work experience, he
has worked as an ICT consultant, with assign-
ments mostly involving software development.
Research interests include metadata, the Web, information systems, security and, in
particular, how these can be integrated with usability and interoperability in mind.
Sylvana Kroop is senior researcher and man-

ages R&D projects in the field of Technology-
Enhanced Learning (TEL) at the Centre for
Social Innovation (ZSI) in Vienna, Austria. She
is senior lecturer at the University of Vienna. In
her past (1991–2001) she worked as executive
assistant CEO in the private sector where she
managed five hotels in Berlin/Germany and sur-
roundings. In 2000 she became part of the
research and teaching staff of the Media Faculty
at Technical University of Berlin. As full-time
researcher in TEL she gained more experience in
R&D projects at Bauhaus University Weimar,
Germany. Since 2005 she works at University
of Vienna. In 2012 she has been elected for the
Advisory Board of the Ferdinand Porsche Uni-
versity of Applied Sciences in Vienna, the only
Distance University in Austria, where she supports development and innovation.
Her research is focused on ICT-based innovative pedagogical approaches and the
relation between technological and social innovations in higher education and
workplace learning.
xiv List of Authors: Members of the ROLE Consortium
Mikael Laaksoharju is a junior lecturer and

director of studies in the Division of Visual
Information and Interaction at the Department
of Information Technology, Uppsala University.
His research focuses on developing processes
that integrate human values in interaction design
practice.
Effie Lai-Chong Law is Reader at the Depart-

ment of Computer Science of the University of
Leicester (UK) and Visiting Senior Researcher
of ETH Zürich (Switzerland). She obtained her
Ph.D. in psychology from the University of
Munich (LMU), Germany. Her research domains
are human–computer interaction (HCI) and
technology-enhanced learning (TEL) with a spe-
cific focus on usability and user experience
(UX) methodologies. Effie has been chairing
two international HCI projects: MAUSE
“Towards the Maturation of Usability Evalua-
tion” and TwinTide “Towards the Integration
of Trans-sectorial IT Design and Evaluation” in which researchers more than
20 European countries have been involved. Effie has also assumed a leading role
in a number of interdisciplinary EU-funded research projects on various TEL topics
such as game-based learning, CSCL, and personalised learning environments. She
is an editorial board member of Interacting with Computers.
Na Li is a Ph.D. student at the Coordination and

Interaction Systems Group of the Swiss Federal
Institute of Technology in Lausanne (EPFL).
She received her Bachelor and Master degrees
from Tongji University in 2007 and 2009,
respectively. Her research interests include trust
and reputation systems, technology-enhanced
learning, and social media platforms. More spe-
cifically, she is interested in the design and
development of social media platforms with
application to learning environments, as well as
information filtering in such platforms using
trust and reputation models. She has been
List of Authors: Members of the ROLE Consortium xv
working on the European research projects on technology-enhanced learning, such

as Responsive Open Learning Environments (ROLE) and Global Online Science
Labs for Inquiry Learning at School (Go-Lab).
Thomas Lind is a Ph.D. student in the Division

of Visual Information and Interaction at the
Department of Information Technology, Upp-
sala University. In his research he is identifying
pitfalls and key principles for facilitating user
adoption of IT in large governmental
organisations
Alexander Mikroyannidis is a post-doctoral

researcher in the Knowledge Media Institute of
the Open University UK. His research areas of
interest are related with knowledge management
and applications of Semantic and Social Web
technologies in Technology-Enhanced Learning
(TEL). Recently, he has been investigating self-
regulated learning and the challenges involved
in the adoption of personal learning environ-
ments by the lifelong learner. He has also been
working on the production of online courses and
open educational resources, delivered through
various educational platforms, such as interac-
tive eBooks. Dr. Mikroyannidis has contributed to several projects of the 5th, 6th,
and 7th Framework Programme of the European Community, including FORGE
(ICT-2013-610889), weSPOT (ICT-2012-318499), Euclid (ICT-2012-296229),
ROLE (ICT-2009-231396), OpenScout (ECP 2008 EDU 428016), DEMO-net
(ICT-2006-27219), CASPAR (ICT-2005-33572), and PARMENIDES (IST-2001-
39023).
xvi List of Authors: Members of the ROLE Consortium
Alexander Nussbaumer is member of the

interdisciplinary Cognitive Science
Section (CSS) of the Knowledge Technologies
Institute at the Graz University of Technologies,
Austria. Before that he joined the Cognitive Sci-
ence Section of the Department of Psychology at
the University of Graz. In the context of these
affiliations he has been participating in
EC-funded projects on technology-enhanced
learning and cultural heritage (e.g. iClass,
GRAPPLE, CULTRUA, and ROLE). Having a
background in computer science, his research
interests include the integration of psychological
research and technical application with a focus
on competence-based knowledge representation models and their applications for
learning purposes in adaptive and self-directed learning contexts.
Matthias Palmér received his Ph.D. in media

technology from Royal Institute of Technology
(KTH) with a focus on technology-enhanced
learning and semantic web. After graduating
from KTH in 2012 Matthias co-founded the
company MetaSolutions which has a focus on
how modern web architecture and linked open
data can be used to create services and web
applications for collaborative information man-
agement. Previously Matthias worked for Upp-
sala University within several EC-founded
projects such as LUISA and ROLE as well as
served as the technical lead of the development
of the Uppsala University Student Portal.
Matthias is an active developer and continues
to be involved in the design and development
of several open source projects.
List of Authors: Members of the ROLE Consortium xvii
Sarah León Rojas received a M.A. in Media

Informatics from the Cologne University of
Applied Sciences. She currently works as a
research associate in the User-centred Ubiqui-
tous Computing (UCC) group at the Fraunhofer
Institute for Applied Information Technology
FIT. Her work focuses on human–computer
interaction and interface design as well as the
analysis and visualisation of usage data.
Maren Scheffel studied at the University of

Edinburgh, UK, and the University of Bonn,
Germany, where she received an M.A. in Com-
putational Linguistics. She is currently working
as a research associate at the Fraunhofer Institute
for Applied Information Technology FIT as a
member of the “Context and Attention in
Personalised Learning Environments” depart-
ment that aims to support teachers and learners
in universities and at the workplace by providing
technical solutions. Additionally, she is a Ph.D.
candidate at the Open University of the Nether-
lands. Her research focuses on the application of
linguistic methodologies to the analysis of usage
data, i.e. inferring higher level information such
as patterns and key actions from low level data.
Manuel Schmidt is working in the e-learning

project of Festo at the Festo Learning Centre
which provides the entire e-learning services for
the Festo Group worldwide. Mr. Schmidt’s
tasks are the provision and operation of the
international learning management system at
Festo, production of e-learning contents and
the future development of e-learning at Festo.
Mr. Manuel Schmidt studies business sciences
at the university of cooperative education Saar-
land (ASW), his Diploma Theses dealt with
xviii List of Authors: Members of the ROLE Consortium
e-learning in intercultural settings. He already gained EU project experiences in the

Symphony project.
Hans-Christian Schmitz is a member of the
Institut für Deutsche Sprache (IDS, Institute of
German Language, Mannheim/Germany) where
he searches for traces of grammar in very large
corpora. Before he joined the IDS, he was a
member of the Fraunhofer Institute for Applied
Information Technology, there serving as the pro-
ject manager of the ROLE project, among others.
Hans-Christian earned his Ph.D. in computational
linguistics at the University of Bonn. He has
substituted for professors several times, in differ-
ent areas, namely computational linguistics, Ger-
man language studies and computer science.
Simon Schwantzer holds a diploma in Com-

puter Science from the University of Kaiserslau-
tern (Germany). Since 2010, he works as
research professional at the innovation labs
department of the IMC information multimedia
communication AG in Saarbrücken. His focus
areas are collaborative learning environments,
social learning, and real-time collaboration
systems.
Ruimin Shen has been working in the Depart-

ment of Computer Sciences of Shanghai Jiao
Tong University since 1991, and worked as a
visiting professor at Waseda University between
1997 and 1998. Currently, he serves as a member
of the Ministry of Education’s Expert Committee
on Long-Distance Education. He is the director
of the Intel-SJTU Long-Distance Education
Research Center and dean of the School of Con-
tinuing Education of Shanghai Jiao Tong Uni-
versity. As a member of Education Ministry
Distance Education Committee, Ruimin Shen is
in charge of the regulation of several Chinese
List of Authors: Members of the ROLE Consortium xix
E-Learning Technology Standards. In 2006, Premier Minister Wen Jiabao awarded

Prof. Shen with the National Award for Science & Technology Progress.
Carsten Ullrich the deputy director of the Cen-

tre for e-Learning Technology (CeLTech) at the
German Research Center for Artificial Intelli-
gence (DFKI), and an associate researcher in
the e-learning lab of Shanghai Jiao Tong Uni-
versity, China. He received his Ph.D. in Com-
puter Science (Saarland University) in 2008.
That year, he became an associate researcher at
SJTU, China, where he investigates web-based
and mobile learning in adult education. From
2009 to 2013, he was the SJTU leader of the
European IP ROLE. In 2013, he became the
deputy director of CeLTech. Dr. Ullrich’s
research covers technology-supported e-learn-
ing, with a focus on personalisation and learner-support. He has published numer-
ous papers on adaptivity (Semantic) Web-based learning and mobile learning. He is
a frequent speaker in conferences and innovation fairs.
Katrien Verbert is an assistant professor at the

Eindhoven University of Technology. She
obtained a doctoral degree in Computer Science
in 2008 at the University of Leuven
(KU Leuven), Belgium. She was a post-doctoral
researcher of the Research Foundation—Flan-
ders (FWO) from 2009 until 2012 at KU Leuven.
Her research interests include content models,
content reusability, context-aware recommenda-
tion, visualisation and personalisation, and appli-
cations thereof in healthcare, science
information systems and technology-enhanced
learning. In that respect, she is currently
involved with the RAMLET IEEE LTSC standardisation project that has developed
a standard to enable interoperability of content packaging specifications for learn-
ing resource aggregations. She was involved in the EU FP7 project ROLE that is
focused on contextual recommendation in responsive open learning environments.
She co-organised several workshops and special issues in this area. She also
co-established the dataTEL Special Interest Group of EATEL that is focused on
data-driven research for learning analytics.
xx List of Authors: Members of the ROLE Consortium
Helmut Vieritz studied Physics and Sociology

at the Humboldt University and Freie Universität
of Berlin. After receiving his diploma, he worked
at the Berlin Institute of Technology and Stuttgart
University. His activities focus on Web develop-
ment in E-Learning and Multimedia. In the
MUMIE project, he paid attention to the pro-
cessing of mathematical content with Semantic
Web technologies. Currently, he is working at
the RWTH Aachen University. Besides teaching
informatics in large classes, his research focuses
on usage-centred Web development with inclusive
design. He was responsible for the DFG-research
project (Integrated Accessibility Models of User
Interfaces for Web and Automation Systems) in
collaboration with the Stuttgart University.
Michael Werkle is responsible for quality man-

agement at Festo learning Centre as well as
national and international research projects,
especially in context of further education and
learning with new media. He works for Festo in
this areas since 2005 and has put his experiences
into practice in different projects. From 2003 to
2006 he studied business sciences (core area:
human resources management) at the University
of Applied Sciences in Saarbrücken. In 2011 he
finished his second degree (extra-occupational)
in vocational studies and economic education at
Saarland University in Saarbrücken.
Martin Wolpers holds a Ph.D. in electrical

engineering and information technology from
the Leibnitz University Hannover. He is head of
the research department “Context and Attention
in Personalised Learning Environments” at
FRAUNHOFER. He also holds the title “visiting
professor” at the Katholieke Universiteit Leuven,
Belgium. He has been strongly involved in a
number of highly successful European Projects,
e.g. coordinator of the ROLE project as well as
partner in OpenDiscoverySpace, OpenScout and
NaturalEurope projects. He is vice-president of
List of Authors: Members of the ROLE Consortium xxi
the European Association of Technology Enhanced Learning (EATEL) and presi-

dent of the MACE Association, result of the successful coordination of the eContent
+ project MACE. His research interests focuses on the improvement of
Technology-Enhanced Learning by relying on new and emerging technologies.
This includes work on infrastructures for information processing, knowledge man-
agement and provision as well as contextualised attention metadata and information
elicitation (among others).
Fridolin Wild is research associate at the

Knowledge Media Institute of the Open Univer-
sity of the UK. Fridolin’s research happens at the
interface of the digital and the real, where learn-
ing, thinking, and cognition meet information
systems. His research fields encompass aware-
ness, engagement, and collaboration at scale.
Fridolin is and has been the principal investiga-
tor and work package leader of several large
EC-funded and nationally-funded projects,
amongst others: TELL-ME, TELmap, STEL-
LAR, LTfLL, ROLE, iCamp, ProLearn,
Edukapp, cRunch. Fridolin is the voted treasurer of the European Association of
Technology Enhanced Learning (EATEL) and the managing editor of the Interna-
tional Journal of Technology-Enhanced Learning.
List of Authors: External Experts
and Commentators of Chapters
Graham Attwell is an Associate Fellow, Insti-

tute for Employment Research, University of
Warwick and a Gastwissenschaftler at the
Insititut Technik und Bildung, University of Bre-
men. His experience includes technology-
enhanced teaching and learning and web-based
learning environment development, specialised
in research and development into pedagogies for
Technology-Enhanced Learning. Graham is a
consultant to OECD and UNESCO on open con-
tent development and a consultant to the Euro-
pean Centre for Vocational Education and Training (CEDEFOP) on virtual
communities and knowledge harvesting. He has experience in national and inter-
national programme evaluation in relation to innovations in learning, including use
of ICT to support learning. His recent work has focused on research and develop-
ment of new applications and approaches to e-Portfolios and Personal Learning
Environments and use of social software for learning and knowledge development.
Jon Dron is researching with the Technology

Enhanced Knowledge Research Institute and
teaches in the School of Computing and Informa-
tion Systems at Athabasca University, Canada.
He also holds an honorary faculty fellowship in
the Centre for Learning and Teaching, University
of Brighton, UK. Jon has received national and
local awards for teaching and research, and is a
regular conference keynote speaker. His research
is cross-disciplinary, straddling social, pedagogi-
cal, technological, systemic and philosophical
boundaries of technology, learning and education.
He used to sing for a living.
xxiii
xxiv List of Authors: External Experts and Commentators of Chapters
Sheila MacNeill has recently taken up a post as

a Senior Lecturer in Blended Learning at Glasgow
Caledonian University. Previous to this post she
was an Assistant Director for Cetis (www.cetis.ac.
uk) where her main areas of interest and work
were around the user experience of using and
implementing technology for teaching and learn-
ing. A key area of work during her time at Cetis
centred around the development and support of
approaches to distributed learning environments,
particularly the underlying models and architec-
tures for the sharing of widgets and apps. In
September 2013 Sheila was named the ALT
Learning Technologist of Year, 2013. Sheila reg-
ularly blogs about the use and potential of educa-
tional technology in her blog: howsheilaseesit.
wordpress.com.
Martin Ebner is currently head of the Depart-

ment for Social Learning at Graz University of
Technology and therefore responsible for all
university-wide e-learning activities. He holds an
Assoc. Prof. on media informatics and works also
at the Institute for Information System Computer
Media as senior researcher. His research focuses
strongly on e-learning, mobile learning, learning
analytics, social media and the usage of Web 2.0
technologies for teaching and learning. Martin
gives a number of lectures in this area as well as
workshops and talks at international conferences.
For publications as well as further research activ-
ities please visit the website: http://martinebner.at
List of Authors: External Experts and Commentators of Chapters xxv
Carlo Giovannella graduated in Physics and is

an expert of complex systems, nowadays can be
considered a ’Designer for the experience’ and
expert in the technology-enhanced learning,
interaction design, computer-mediated commu-
nication, design and management of processes,
process and product innovation. Since 2013 is
the Scientific Director of the Creative Industries
Area at the Consorzio Roma Ricerche. He is
member of the Dept. of Educational Science
and Technology of the University of Rome Tor
Vergata, where he is chairing the ISIM_garage
(Interfaces and Multimodal Interactive Sys-
tems). He has edited more than ten books and special journal issues and published
almost 200 papers in Physics, Photography and communication, Technology-
Enhanced Learning and Design for the Experience. He is the scientific editor of
the Journal IxD&A and has been scientific and artistic chair of six editions of
’Interfacce’ and co-chair of several conferences and workshops (ICALT 2012,
HCIEd2008, CHItaly 2009, DULP 2009, DULP@ICALT2010,
DULP@ICALT2011, Horizon 2020: Smart City Learning @ARV13, etc.).
Marco Kalz is Associate Professor at the

Welten Institute—Research Centre for Learning,
Teaching and Technology at the Open Univer-
sity of the Netherlands. Marco is a fellow of the
Interuniversity Center for Educational Sciences
(ICO) and the Dutch research school information
and knowledge systems (SIKS). He is the chair
of the special interest group on Technology-
Enhanced Assessment (SIG TEA) of the
European Association of Technology-Enhanced
Learning (EATEL). Besides European projects
he was and is regularly involved in educational
innovation projects at the Open University of the
Netherlands and in cooperation with other part-
ners. His research interest lies on the use of pervasive technologies and formative
assessment to support self-directed lifelong learning. He has more than 30 peer-
reviewed journal publications and conference papers in the TEL field.
xxvi List of Authors: External Experts and Commentators of Chapters
Jürgen Mangler is researcher at the Workflow

Systems and Technology Group of the Univer-
sity of Vienna in Austria. His current research is
focused around business processes, and their
automated enactment through workflow man-
agement systems. Specific research topics
include: motivating humans when dealing with
strictly predetermined processes and activities;
measuring how skills and competencies change
when humans work in process controlled envi-
ronments; how to automatically recover from
errors (self-healing). In the past he was working
as a security researcher at Secure Business Aus-
tria (SBA), as well as TEL researcher and still
operates a custom SOA LMS at the Faculty of
Computer Science, University of Vienna. He has contributed to projects in the 6th
and 7th Framework Programme of the European Commission, currently ADVEN-
TURE (FP7-ICT-2011-C 231396) in the Factories of the Future (FoF) Theme.
Margit Pohl studied Computer Science and

Psychology in Vienna. She is associate professor
at the Vienna University of Technology where
she works at the Institute for Design and Tech-
nology Assessment/HCI group. She is mainly
interested in the cognitive aspects of the interac-
tion of humans with computers. Her most impor-
tant research interests are E-Learning, Cognitive
Psychology and Visualisation. She has
conducted many projects in the area of
E-Learning and Visualisation. Her main current
research project is CVAST—Center for Visual
Analytics Science and Technology.
Personal Learning Environments (PLEs):
Visions and Concepts
Alexander Mikroyannidis, Sylvana Kroop, and Martin Wolpers
Abstract Personal learning environments (PLEs) hold the potential to address the
needs of formal and informal learners for multi-sourced content and easily
customisable learning environments. This chapter presents an overview of the
European project ROLE (Responsive Open Learning Environments), which spe-
cialises in the development and evaluation of learning environments that can be
personalised by individual learners according to their particular needs, thus
enabling them to become self-regulated learners.
Keywords Personal learning environment • Self-regulated learning • Responsive

open learning environment
Introduction
An ageing society and a flexible economy need lifelong learning more than ever,
otherwise risking that school kids today know more than employees trained half a
decade ago. Lifelong learning requires learners to actively control their learning
activities while addressing the requirements imposed on them in their respective
life contexts. Life context here can be the school, the university, the workplace, the
hobby, etc. This leads to a shift from a centralised institutional teaching approach to
a more learner-centred decentralised learning approach (Wilson 2005). In order to
support this shift, learning environments must change to be more responsive and
open, allowing effectively addressing individual needs of learners and teachers.
A. Mikroyannidis (*)
Knowledge Media Institute, The Open University, Milton Keynes MK7 6AA, UK
e-mail: [email protected]
S. Kroop
Zentrum for Social Innovation, Vienna, Austria
M. Wolpers
Fraunhofer FIT, Schloss Birlinghoven, Sankt Augustin, Germany 53754
© The Author(s) 2015 1

S. Kroop et al. (eds.), Responsive Open Learning Environments,
DOI 10.1007/978-3-319-02399-1_1
2 A. Mikroyannidis et al.
In this chapter, we will reflect on the approach of the European project ROLE
(Responsive Open Learning Environments1). ROLE enables learners individually
to compile their personal learning environments (PLEs) according to their partic-
ular needs and goals. Consequently, the ROLE approach supports self-regulated
learning (SRL) while taking into account the requirements from the roles of the
learners and the teachers.
The remainder of this chapter is structured as follows: Recent advances in
personalised and SRL are introduced. Subsequently, the widget-based approach
of the PLE is presented, along with the process of building a PLE using widgets.
This is followed by an introduction to the ROLE project and its key innovations,
with particular emphasis on evaluating these innovations via a number of case
studies and test beds. An example of a recent research initiative that builds upon the
results of the ROLE project is provided in the section that presents the weSPOT
project. Finally, the chapter is concluded with a summary of the key ROLE
contributions to technology-enhanced learning and an overview of what is
presented by each chapter in the rest of this book.
Personalised and Self-Regulated Learning
The Learning Management System (LMS) has dominated technology-enhanced

learning for several years. It has been widely used by academic institutions for
delivering their distance learning programmes, as well as for supporting their
students outside the classroom. The LMS has been a powerful tool in the hands
of educators, enabling them to complement face-to-face teaching in the classroom
with remote work by individual students, as well as groups of them (Bri et al. 2009;
Wainwright et al. 2007; Abel 2006; Watson et al. 2007).
However, the advent of Web 2.0 has altered the landscape in technology-
enhanced learning. Learners nowadays have access to a variety of learning tools
and services on the cloud. These tools and services are usually provided by different
vendors and in many cases are open and free. However, augmenting and configur-
ing these diverse and distributed tools and services in order to address the needs and
preferences of individual learners is a significant challenge for modern online
learning environments.
This ongoing transition from the traditional approach of the LMS towards Web
2.0-based learning solutions bears significant benefits for learners. It puts emphasis
on their needs and preferences, providing them with a wider choice of learning
resources to choose from. Learners usually switch learning contexts continuously,
adapting to the respective needs automatically. The LMS is not able to provide
learners with the required flexibility. Furthermore, the LMS is a closed system that
does not allow the learner to take her achievements with her when changing the
1
http://www.role-project.eu
Personal Learning Environments (PLEs): Visions and Concepts 3
LMS-providing learning organisation, e.g. while starting a new job, the previously
used LMS-profile cannot easily be transferred to the new one used at the workplace.
The PLE is a facility for an individual to access, aggregate, manipulate, and
share digital artefacts of their ongoing learning experiences. The PLE follows a
learner-centric approach, allowing the use of lightweight services and tools that
belong to and are controlled by individual learners. Rather than integrating different
services into a centralised system, the PLE provides learners with a variety of
services and hands over control to them to select and use these services the way they
deem fit (Chatti et al. 2007; Wilson 2008).
The emergence of the PLE has greatly facilitated the use and sharing of open and
reusable learning resources online. Learners can access, download, remix, and
republish a wide variety of learning materials through open services provided on
the cloud. Open Educational Resources (OER) can be described as “teaching,
learning and research resources that reside in the public domain or have been
released under an intellectual property license that permits their free use or
repurposing by others depending on which Creative Commons license is used”
(Atkins et al. 2007).
SRL comprises an essential aspect of the PLE, as it enables learners to become
“metacognitively, motivationally, and behaviourally active participants in their
own learning process” (Zimmerman 1989). Although the psycho-pedagogical the-
ories around SRL predate very much the advent of the PLE, SRL is a core
characteristic of the latter. SRL is enabled within the PLE through the learner-
driven assembly of independent resources in a way that fulfils a specific learning
goal. By following this paradigm, the PLE allows learners to regulate their own
learning, thus greatly enhancing their learning outcomes (Steffens 2006; Fruhmann
et al. 2010).
The Widget-Based Approach of the PLE
As online learners become more discerning in terms of the choices related to the
types and styles of their potential study materials, they will potentially seek content
from multiple sources. In addition, because of the flexibility and ease of use that
enables many users to customise that content, those same learners may wish to
personalise their learning environment.
The PLEs presented in this book are primarily enabled by widgets, which are
micro-applications performing a dedicated task. This task can be as simple as
showing news headlines or weather forecasts, but also more complex like facilitat-
ing language learning or collaborative authoring. Widgets can be either desktop
based or web based. Desktop-based widgets reside locally on your computer and
may access the web for information, such as a desktop widget that shows the local
temperature and weather. Web-based widgets reside on the web and can be embed-
ded on a web page, such as an RSS reader widget that fetches news on your start
Fig. 1 Browser- and widget-based PLE concept
page. Web-based widgets have proven quite popular as they enhance the interac-
tivity and personalisation of web sites.
As already mentioned above, the theoretical idea of PLEs is not a specific
software application. A PLE is instead a concept based on the idea to have
learner-centric Web 2.0-based environments individually designed. It is not a
one-size-fits-all learning environment but a personalised environment a learner
takes control over his/her own learning process instead of being controlled by a
pre-delivered orchestration of learning goals, tools, services, and content. In PLE
research it is deemed essential to have a learner challenged by offering her the
ability to create her individually controlled and preferred learning environment.
In the ROLE project, the basic equipment for creating PLEs has been developed
according to the idea of an easy drag-and-drop system of widgets.
On the one hand, a repository (widget store) is necessary to store and adminis-
trate useful widgets. On the other hand an enabler space (widget space) is necessary
to have learners their individually preferred widgets integrated, used, and managed
in their personal style. Figure 1 outlines this approach schematically.
From a user perspective, ROLE is Software as a Service (SaaS) (Mell and
Grance 2011; Vaquero et al. 2008)—the user does not install and run it locally.
This paradigm affects three main aspects of the user-visible parts of ROLE:
• Widget space: The widget space contains a number of personally selected
learning widgets whereby all of them access and use already existing and
established external OER. It is the virtual environment where the user installs
and uses its widgets.
• Single widget: A single widget abstracts (accesses and uses) at least one single
external resource. There are widgets accessing and using just one single external
resource, e.g. a Wikipedia widget or an LEO dictionary widget. Furthermore,
some widgets have been implemented to make use of cloud computing to an
extensive degree. One example is the “ROLE translator widget” which accesses
and displays the results of different popular resources such as LEO.org, dict.cc,
Wikipedia, and Google translator. The results of translating a specific term are
used from all translating resources at the same time and are displayed in the same
place by using the ROLE translator widget. Thus a learner has a better and more
critical overview by being able to quickly compare the provided Web 2.0-based
translation data. One more interesting example of a cloud computing-based
widget is “Binocs” which displays search results by using different external
resources and depending on the used resources of a personal network of trusted
friends, colleagues, and experts (Govaerts et al. 2011). All widgets can be found
in the ROLE Widget Store described in the following section.
• Multiple devices: ROLE widgets and content can be accessed and used with
different devices. Depending on the widgets and content, it can be used by all
kind of browser-based applications on notebooks, smartphones, tablets, etc.
Building a Widget-Based PLE
In order to build a widget-based PLE, the learner will need to access a widget store.
A widget store is a directory of widgets, where widgets are commonly categorised
according to their purpose, e.g. widgets for planning, communication, and collab-
oration. An example of such a categorisation is shown in Fig. 2. The learner can
browse and download the widgets, as well as provide feedback on them in the form
of ratings and comments. The ROLE project has built a widget store dedicated to
widgets for learning purposes.2
After selecting the appropriate widgets, the learner needs to add them to the web
space of their choice and start using them for their learning, either by themselves or
in collaboration with other learners. Widgets can also be embedded inside an LMS,
such as Moodle, thus enhancing its functionality and content, as shown in Fig. 3.
Additionally, ROLE offers a facility for creating a shared learning space and
populating it with widgets.3
For more information on building a PLE and using it to become a self-regulated
learner, one can refer to the following free online courses that have been developed
by ROLE:
• Responsive open learning environments4: This course provides an overview of
the concepts behind PLEs and also demonstrates a selection of ROLE widgets
within learning activities. Figure 4 shows such an activity, where the learner is
invited to use a ROLE widget in order to complete a series of learning tasks.
2
http://www.role-widgetstore.eu
3
http://role-sandbox.eu
4
http://labspace.open.ac.uk/course/view.php?id¼7433
Fig. 2 The ROLE widget store offers widgets (tools) for a variety of learning purposes,
categorised according to learning tasks
Fig. 3 ROLE widgets embedded inside a Moodle course. The learner uses them to search for
learning resources, as well as collaborate with other learners through videoconferencing and a
shared writing pad
• Self Regulated Learning5: This course introduces the concept of SRL and guides
learners into using the ROLE tools in order to apply the SRL principles into their
own learning.
The content of these two courses is also available as a free interactive eBook,
developed for the iPad and MacOS (Mikroyannidis et al. 2013a). The eBook
provides an introduction to the new learning technologies that empower SRL and
PLEs. A selection of widgets that will help readers build their own PLE and become
a self-regulated learner are also demonstrated. Readers have an opportunity to
interact with these widgets through a set of learning activities included in the
eBook.
5
http://labspace.open.ac.uk/course/view.php?id¼7898
Fig. 4 A learning activity of a free online course developed by ROLE, featuring a search widget
and a step-by-step guide on completing a series of learning tasks
The ROLE Project and Its Key Innovations
ROLE is a European-funded initiative with 16 internationally renowned research

groups from six European countries and China. It started on February 2009 with a
duration of 4 years. ROLE was established to research and explore a variety of tools
and services that enable learners to build their PLE, based on their needs and
preferences. ROLE has brought forward the innovations in PLEs and SRL, through
research in the following directions:
• User-centric approach to learning environments with a focus on end-user devel-
opment to design and control a PLE.
• Contemporary pedagogical models for personalisation in learning networks,

SRL and collaboration in networked communities.
• Contemporary engineering frameworks for designing, integrating, orchestrating,
and evaluating learning services, tools, and content.
• Frameworks for evaluating learner interactions in learning networks.
The notion of lifelong learning as discussed today formulates a number of
requirements on the technological basis as well as the associated learning and
business/organisational processes. As our target group ranges from all possible
domains and roles, e.g. learners, teachers, companies, employer, employees, and
learning organisations, opportunities arise that will support the current shift in
education towards more self-regulated learners (Van Harleman 2006) in scenarios,
where the teacher role shifts more towards a mentoring role: the centralised
institutional teaching approach shifts to a more learner-centred decentralised learn-
ing approach (Wilson 2005).
The ROLE project provides solutions to this set of complex challenges by
advancing the state of the art in the technology and methodology. The following
sections outline the ROLE approach in technology and methodology.
Technology: interoperable infrastructure enables PLE composition—ROLE has
provided an infrastructure that enables learners to create their own PLEs, while
maintaining a close link to the rules and restrictions of the education-providing
organisation (Isaksson 2013; Dahrendorf 2013). In essence, the idea is to loosen the
control on the learner while maintaining the ability to certify learner achievements.
For example, the learner chooses the required learning tools and contents from a
wide selection and compiles them into her individual PLE. At the same time, the
education provider can control which tools and contents can be chosen by the
learner.
ROLE tools and content within the PLE are able to communicate with each other
in order to enable tools and contents to react to each other based on the user
interaction. Finally, rather than replacing LMS, the ROLE approach allows the
successful augmentation of existing learning environments. This way, the costs for
introducing the ROLE approach to existing learning environments is significantly
reduced, which fosters its uptake.
Methodology: self-regulation as the key learning paradigm—Learners today are
not aware of the advanced learning paradigm of SRL. In most cases, the basic
components of SRL, that is, cognition, meta-cognition, motivation, affects, and
volition (Efklides 2009), are used by learners intuitively without understanding the
conceptual background. Apart from supporting SRL in PLE creation and use
through respective recommenders, collaboration tools and best practice sharing,
ROLE raises awareness through a number of dedicated learning resources. These
range from short videos explaining the SRL principles (see for example http://
youtu.be/jTa1vOH6JjA), to bespoke online courses about SRL that help teachers
and students understand the mechanics and benefits behind SRL, such as the ones
introduced in the previous section of this chapter.
The ROLE Case Studies and Test Beds
The ROLE innovations in technology and methodology have been proven success-
ful in a number of case studies that investigate the impact of PLEs on different
forms of learning. Each case study has employed large test beds that have run
continuously throughout the lifetime of the project and beyond (Mikroyannidis and
Connolly 2012, 2013). The ROLE test beds cover a wide variety of rich contexts
inside and outside of Europe, in which there is potential for significant impacts on
both personalised and SRL. Each test bed concentrates on researching a large
sample of representative individuals; this has enabled ROLE as a whole to collect
experiences covering a large variety of learning contexts and requirements. The
ROLE case studies and test beds are presented in detail in subsequent chapters of
this book. A brief overview is provided below:
Case Study 1: Using Widget Bundles for Formal Learning in Higher Educa-
tion—This case study explores the usefulness of the PLE for facilitating and
complementing the learning that happens inside the classroom in Higher Education.
The test beds of this case study are three universities: the RWTH Aachen University
(RWTH) in Germany, the Shanghai Jiao Tong University (SJTU) in China and the
University of Uppsala in Sweden. These test beds present a variety of learning
cultures, different profiles of students, as well as different methods of teaching and
learning, e.g. synchronous versus distant learning. Bespoke widget bundles were
developed by the ROLE project in order to address the learning contexts and
requirements of these test beds. These widget bundles were employed in different
learning domains to support different types of learning in these test beds and were
evaluated by both the teachers and the students.
Case Study 2: Designing PLE for Higher Education—In this case study, the
potential benefits associated with enabling teachers and students to design, build,
and use their PLEs collaboratively are investigated. The test beds of this case study
are the Tongji University in China and three Swiss universities, namely the Uni-
versity of Fribourg, the University of Geneva, and the EPFL. In these test beds, a
Web 2.0 platform enabling the construction, the sharing, and the repurposing of
PLEs has been introduced. Participatory design and validation activities have been
carried out in the framework of Higher Education, aiming at understanding the
benefits of PLEs in academic institutions.
Case Study 3: Exploring OER for Informal Learning—This case study concerns
the learner’s potential transition from formal to informal learning. The test bed of
this case study is OpenLearn,6 an OER repository offered by the Open University in
the UK. OpenLearn users are primarily informal learners, who want to find and
study OER either individually or in collaboration with others. The ROLE interven-
tion in the OpenLearn test bed has been about improving the informal learning
experience in a number of ways. First of all, by enabling individuals to build and
personalise their learning environment, thus gaining more control over the use and
6
http://www.open.edu/openlearn
manipulation of study materials. Additionally, the adoption of certain ROLE tools

inside OpenLearn is offering further value to learners through fostering learning
communities. This presents an opportunity to individual informal learners to be part
of a shared learning experience instead of a lone study.
Case Study 4: Technology Enhanced Workplace Learning: Learning in the
workplace is targeted by this case study, which explores the challenges and
opportunities associated with SRL in the workplace and the sharing of best prac-
tices among employees. The test bed in question is Festo Lernzentrum Saar GmbH
in Germany. Festo has experimented with the notion of the Personal Learning
Management System (PLMS), a crossover between the PLE and the LMS. The
PLMS aggregates learning resources and applications available in the web and
selected by the learner. It facilitates learners in planning their learning activities,
searching for learning content and tools, training and testing, as well as in reflecting
and evaluating their learning progress.
The evaluation results from the ROLE test beds are presented in detail in the
chapters of this book that discuss each case study. Overall, the evaluation results
indicate the best suitability of the ROLE approach for self-regulated learners while
providing significant improvements even in traditional learning scenarios where
ROLE tools are used for homework-like assignments. Additionally, the successful
evaluation of the ROLE approach has led partners to include it in their commercial
products and consulting practices.
PLEs for Inquiry-Based Learning: The weSPOT Project
The ROLE project has been a pioneering initiative in PLE research. It has paved the
way for more national and international research initiatives that explore the poten-
tial applications and benefits of PLEs in different learning contexts. Both the
theoretical and technological frameworks developed by ROLE have been taken
upon and extended by recent research projects. A prominent example of such an
initiative is the weSPOT project, which is investigating the potential impact of
PLEs in Inquiry-Based Learning.
Inquiry-based learning (IBL) enables learners to take the role of an explorer and
a scientist as they try to solve issues they came across and that made them wonder,
thus tapping into their personal feelings of curiosity. IBL leads to structured
knowledge about a domain and to more skills and competences about how to
carry out efficient and communicable research.
The European project weSPOT7 adopts a PLE-based approach in order to
support learners and educators in IBL (Mikroyannidis et al. 2013c). The project
focuses on IBL with a theoretically sound and technology-supported personal
inquiry approach. weSPOT supports the meaningful contextualisation of scientific
7
http://wespot.net
concepts by relating them to personal curiosity, experiences, and reasoning.

weSPOT addresses several challenges in the area of science education and tech-
nology support for building personal conceptual knowledge (Mikroyannidis
et al. 2013b).
These principles have driven the development of the weSPOT inquiry space,8 a
personal and social IBL environment that reuses and extends the Elgg open-source
social networking framework.9 The weSPOT inquiry space has been built based on
the following requirements (Mikroyannidis 2014):
• A widget-based architecture enables the personalisation of the inquiry environ-
ment, allowing teachers and students to build their inquiries out of mashups of
inquiry components.
• Students can connect with their peers and form groups in order to build, share,
and perform inquiries collaboratively.
Inquiries in the weSPOT inquiry space are consistent with the weSPOT peda-
gogical IBL model (Protopsaltis 2013). According to this model, an inquiry consists
of the following six phases: (1) question/hypothesis, (2) operationalisation, (3) data
collection, (4) data analysis, (5) interpretation/discussion, and (6) communication.
The weSPOT inquiry space enables its users (teachers and students) to create
mashups of their preferred inquiry components, assign them to different phases of
an inquiry, share them with other users, and use them collaboratively in order to
carry out an inquiry. When creating a new inquiry, users are provided with a set of
recommended inquiry components for each phase of the inquiry. They can then
customise these sets of components by adding, removing, and arranging inquiry
components for each phase of the inquiry.
The weSPOT inquiry space offers a variety of inquiry components to teachers
and students, enabling them to create, edit, and share hypotheses, questions,
answers, notes, reflections, mind maps, etc. Some of these components communi-
cate with the APIs of REST web services offered by external tools. One of these
external tools is the ARLearn mobile app,10 which allows students to collect
different types of data (photos, videos, measurements, etc.) with their smartphones
and share them with other inquiry members via the weSPOT inquiry space. A
Learning Analytics dashboard visualises all the activities taking place within an
inquiry, enabling teachers to monitor the progress of their students and students to
self-monitor their progress.
Figure 5 shows an example mashup of inquiry components for a particular phase
of an inquiry that explores the everyday uses of batteries. The phase is labelled
“Discuss the findings” and corresponds to the “Interpretation/Discussion” phase of
the weSPOT IBL model. In this phase, the members of the inquiry use collabora-
tively three inquiry components in order to discuss and interpret their findings.
8
http://inquiry.wespot.net
9
http://elgg.org
10
http://portal.ou.nl/en/web/arlearn
Fig. 5 A mashup of inquiry components for discussing and interpreting the findings of an inquiry
They use the “Inquiry discussion” component to exchange their views asynchro-
nously in discussion forums. They also use the “Questions” component in order to
provide answers to the key research questions of this inquiry and vote for the best
answers. Finally, they create and share mind maps containing interpretations of
their findings via the “Mind maps” component.
Additionally, users have access to external resources and widgets and can use
them in their mashups together with the inquiry components offered by the
weSPOT inquiry space. These resources and widgets originate from external
LMSs, such as Moodle or Blackboard, or from external repositories of widgets,
such as the one offered by the European project Go-Lab.11 In order to integrate
external resources originating from LMSs, we have implemented the IMS Learning
Tools Interoperability (LTI) specification,12 thus allowing teachers to include in
their inquiries either course components from LMSs, such as discussion forums or
quizzes, or entire LMS courses.
Conclusions and Book Overview
In summary, the vision of the ROLE project has been to provide the necessary
infrastructure and processes for any learner across the world to assemble their own
PLE, while enabling the education provider to exercise the necessary control to
facilitate the certification of the learning achievements. From a technical point of
view, the approach taken by ROLE enables the flexible composition of technologies
by the end user in the sense of mashing-up learning tools and technologies at the
“clients” side.
Today’s rapidly changing education and employment conditions demand a
lifelong learner who is flexible, motivated, and in control of his or her learning.
The ROLE initiative has significantly advanced the state of the art in technology-
enhanced learning, by providing lifelong learners with the tools and support they
need for personalising their learning and developing a wide range of SRL skills.
The rest of this book presents in detail the theoretical and technological advances
of the ROLE project, with particular emphasis on the applications of these advances
in the case studies investigated by the project, as well as the evaluation results
obtained by the project’s test beds. More specifically, Chap. 2 introduces the SRL
approach of the project, consisting of a formal framework that describes the SRL
process, related competences, and guidelines. Additionally, the methods offered by
the ROLE SRL framework on supporting learners in order to learn in a self-
regulated way are presented.
Chapter 3 presents the evaluation framework and methodology of the project.
The ROLE evaluation framework is case study-based and follows a multi-method
approach. It integrates technological, organisational, psycho-pedagogical, and
social aspects. At the same time, it provides a flexible and adaptive methodology,
capable of accommodating the changes that are inevitable in the emerging field of
personalised and SRL.
11
http://www.golabz.eu/apps
12
http://www.imsglobal.org/lti/index.html
Chapters 4–7 present the project’s case studies and the test beds employed in
each case study. As mentioned earlier in this chapter, the ROLE case studies and
test beds cover a wide variety of learning contexts, ranging from formal to informal
learning and from higher education to distance learning and workplace learning.
Each test bed is presented according to its specific learning scenarios and require-
ments, the stakeholders involved, as well as the PLE intervention achieved by the
project and the evaluation results.
Finally, Chap. 8 discusses the lessons learned from the development of the
project’s PLE framework. In particular, the overall architecture and its components,
as well as the platforms in which the ROLE technological framework has been
integrated are presented. In addition, the experiences and lessons learned from the
design and development of the ROLE technological framework are discussed,
together with the lessons learned from the collaboration both internally within the
ROLE development team and externally with other open-source projects.
Acknowledgement The research leading to these results has received funding from the European
Community’s Seventh Framework Programme (FP7/2007-2013) under grant agreement
no. 231396 (ROLE project) and under grant agreement no. 318499 (weSPOT project).
Open Access This chapter is distributed under the terms of the Creative Commons Attribution
Noncommercial License, which permits any noncommercial use, distribution, and reproduction in
any medium, provided the original author(s) and source are credited.
References
Abel RJ. Best practices in open source in higher education study: the state of open source software;
2006. Lake Mary: The Alliance for Higher Education Competitiveness. http://www.methodist.
edu/csc/ossresearch/applications_vendors/OSS%20Report.pdf
Atkins DE, Brown JS, Hammond AL. A review of the open educational resources (OER)
movement: achievements, challenges, and new opportunities. Menlo Park: The William and
Flora Hewlett Foundation. 2007. http://www.hewlett.org/uploads/files/Hewlett_OER_report.
pdf
Bri D, Garcia M, Coll H, Lloret J. A study of virtual learning environments. WSEAS Trans Adv
Eng Edu. 2009;1(6):33–43.
Chatti MA, Jarke M, Frosch-Wilke D. The future of e-learning: a shift to knowledge networking
and social software. Int J Knowl Learn. 2007;3(4/5):404–20.
Dahrendorf D. Service syndication platform. ROLE project deliverable; 2013. http://www.role-
project.eu/wp-content/uploads-role/2009/02/role-deliverable-4.2-updatedM48.pdf
Efklides A. The role of metacognitive experiences in the learning process. Psicothema.
2009;21:76–82.
Fruhmann K, Nussbaumer A, Albert D. A psycho-pedagogical framework for self-regulated
learning in a responsive open learning environment. In: International conference eLearning
Baltics Science (eLBa Science 2010), Rostock, Germany; 2010.
Govaerts S, Helou SE, Duval E, Gillet D. A federated search and social recommendation widget.
In: 2nd International workshop on social recommender systems (SRS 2011) in conjunction
with the 2011 ACM Conference on Computer Supported Cooperative Work (CSCW 2011),
Hangzhou, China; 2011. p. 1–8.
Isaksson E. Personal learning service bundles. In: ROLE project deliverable. 2013. http://www.
role-project.eu/wp-content/uploads-role/2009/02/role-deliverable-4.1-updatedM48.pdf
Mell P, Grance T. The NIST definition of cloud computing (draft). NIST Spl Publ. 2011;800:145.
Mikroyannidis A. The weSPOT inquiry space: a personal and social environment for inquiry-
based learning. In: 9th European Conference on Technology-Enhanced Learning (ECTEL).
Graz, Austria; 2014.
Mikroyannidis A, Connolly T. Best-practice roadmap for ROLE implementation. ROLE project
deliverable. 2012. http://www.role-project.eu/wp-content/uploads-role/2009/02/role-deliver
able-5.5.pdf
Mikroyannidis A, Connolly T. Test bed evaluation report. In: ROLE project deliverable. 2013.
http://www.role-project.eu/wp-content/uploads-role/2009/02/role-deliverable-5.4.pdf
Mikroyannidis A, Connolly T, Berthold M. Self regulated learning: iPad edition. 2013a. http://bit.
ly/self-regulated-learning. The Open University.
Mikroyannidis A, Okada A, Scott P, Rusman E, Specht M, Stefanov K, Boytchev P,
Protopsaltis A, Held P, Hetzner S, Kikis-Papadakis K, Chaimala F. weSPOT: a personal and
social approach to inquiry-based learning. J Univers Comput Sci. 2013b;19(14):2093–111.
Mikroyannidis A, Okada A, Scott P. Personal learning environments for inquiry-based learning.
Berlin, Germany: PLE Conference; 2013c.
Protopsaltis A, Hetzner S, Held P, Seitlinger P, Bedek M, Kopeinik S, Rusman E, Firssova O,
Specht M, Haimala F, Kikis-Papadaki K, Okada A, Mikroyannidis A, Scott P. Pedagogical and
Diagnostic Framework. In: weSPOT project deliverable. 2013. http://portal.ou.nl/documents/
7822028/3b2c3110-98bf-4686-b2bd-5756fcab93f1
Steffens K. Self-regulated learning in technology-enhanced learning environments: lessons of a
European peer review. Eur J Educ. 2006;41(3/4):353–79.
Van Harleman M. Personal learning environment. In: 6th IEEE International Conference on
Advanced Learning Technologies (ICALT). IEEE Computer Society, Kerkrade, The Nether-
lands; 2006.
Vaquero LM, Rodero-Merino L, Caceres J, Lindner M. A break in the clouds: towards a cloud
definition. ACM SIGCOMM Comput Commun Rev. 2008;39(1):50–5.
Wainwright K, Osterman M, Finnerman C, Hill B. Traversing the LMS terrain. In: Proceedings of
the 35th annual ACM SIGUCCS fall conference, Orlando: ACM; 2007. p. 355–9.
Watson WR, Lee S, Reigeluth CM. Learning management systems: an overview and roadmap of
the systemic application of computers to education. In: Neto FM, Brasileiro FV, editors.
Advances in computer-supported learning. Hershey: Information Science; 2007.
Wilson S. Patterns of personal learning environments. Interact Learn Env. 2008;16(1):17–34.
Wilson S. Future VLE—the visual version; 2005. http://www.cetis.ac.uk/members/scott/
blogview?entry¼20050125170206
Zimmerman BJ. A social cognitive view of self-regulated academic learning. J Educ Psychol.
1989;81(3):329–39.
Supporting Self-Regulated Learning
Alexander Nussbaumer, Ingo Dahn, Sylvana Kroop,

Alexander Mikroyannidis, and Dietrich Albert
Abstract Self-regulated learning (SRL) competences are crucial for lifelong learn-
ing. Their cultivation requires the right balance between freedom and guidance
during the learning processes. Current learning systems and approaches, such as
personal learning environments, give overwhelming freedom, but also let weak
learners alone. Other systems, such as learning management systems or adaptive
systems, tend to institutionalise learners too much, which does not support the
development of SRL competences. This chapter presents possibilities and
approaches to support SRL by the use of technology. After discussing the theo-
retical background of SRL and related technologies, a formal framework is
presented that describes the SRL process, related competences, and guidelines.
Furthermore, a variety of methods is presented, how learners can be supported to
learn in a self-regulated way.
Keywords Self-regulated learning • Learning guidance • Learning models • Per-

sonal learning environments
A. Nussbaumer (*)
Knowledge Technologies Institute, Graz University of Technology, Graz, Austria
I. Dahn
Knowledge Media Institute, University of Koblenz-Landau, Koblenz, Germany
S. Kroop
Zentrum for Social Innovation, Vienna, Austria
A. Mikroyannidis
Knowledge Media Institute, The Open University, Milton Keynes, UK
D. Albert
Knowledge Technologies Institute, Graz University of Technology, Graz, Austria
Department of Psychology, University of Graz, Graz, Austria

DOI 10.1007/978-3-319-02399-1_2
18 A. Nussbaumer et al.
Introduction
The ROLE project1 has aimed to achieve progress beyond the state of the art in
user-centric responsive and open learning environments, which included psycho-
pedagogical models beyond instructional design. ROLE was researching and
developing a psycho-pedagogical framework for supporting the individual compo-
sition and usage of learning environments. The most important goal of this frame-
work is to support learners to learn in a self-regulated way in responsive open
learning environments.
The self-regulated way of learning2 is a fundamental aim for ROLE. The reason
for this aim is the orientation towards lifelong learning, which means that learning
with ROLE takes place across institutional boundaries. This includes consideration
of formal, non-formal, as well as informal learning. These dimensions of learning
organisations are seen as fluent, rather than as rigid categories.
In order to manage the challenge of learning on their own in the ROLE context,
learners require certain self-regulated learning (SRL) competences. For this reason,
ROLE aims at enhancing this kind of competences by providing support strategies
on different levels depending on the learner’s needs and competences. Basically the
support strategy depends on the learning situation. In a blended learning situation a
teacher or tutor plays an important role for supporting SRL, in a collaborative
situation peers play a role and have influence on the learning trajectory, and finally
technology-mediated approaches play a role in all learning situations. All of them
can be interconnected and the support strategy depends on the respective situations.
This chapter is widely based on ROLE deliverables, especially Deliverable D6.1
(Nussbaumer et al. 2013) has been taken into account. The remainder of this chapter
is structured as follows: Section “Theoretical Background” gives an overview on
the theoretical background of SRL. Learning technology and its relation to SRL is
described in section “Related Work and Technology”. Section “Models and Frame-
work” presents the general framework on SRL and includes several models. A
variety of guidance strategies are listed in section “Support Strategies”. The experi-
ences made at the ROLE test beds and in several workshops with students and
researchers are reported in section “Evaluation Results, Challenges, and Barriers”.
Finally, the conclusion section summarises the key findings.
Theoretical Background
From a psycho-pedagogical point of view, SRL is a complex field of research that

combines motivational as well as cognitive and personality theories. Components of
SRL are cognition, meta-cognition, motivation, affects, and volition (Kitsantas 2002).
1
http://www.role-project.eu/
2
See self-regulated learning teaser video, http://www.youtube.com/watch?v¼jTa1vOH6JjA.
Supporting Self-Regulated Learning 19
According to Zimmerman (2002), students can be described as self-regulated to the

degree that they are meta-cognitively, motivationally, and behaviourally active par-
ticipants in their own learning process. To define students’ learning as self-regulated,
they have to use specific strategies for attaining their goals and all this has to be based
on self-efficacy perceptions. In this context there are mainly three elements important,
namely the SRL strategies of students, their perceptions of self-efficacy regarding
their performance skills, and the commitment to their goals. The learners are active
and able to control, monitor, and regulate their cognition, motivational state, behav-
iour, and context. Furthermore, the learners set goals and try to achieve them through
progress-monitoring. These self-regulatory activities are mediators between personal
characteristics, contextual features, and actual performance in the learning process. In
a meta-analysis conducted by Hattie (2009), it turned out that performing self-
regulatory activities in the learning process is one of the most effective methods to
reach the learning goals.
Zimmerman has developed a cyclic SRL model (Zimmerman 2002) consisting of
three phases, which are the forethought phase (goal setting or planning), the
performance phase (self-observation processes), and the self-reflection phase
(self-reflection processes). According to this model, learning performance and
behaviour consist of both cognitive and meta-cognitive activities. The cognitive
activities are related to dealing with subject domains, for example, acquiring
domain knowledge through reading. The meta-cognitive activities are related to
thinking about and regulating the cognitive activities, for example, making a plan
about domain knowledge acquisition.
A similar approach is pursued by Boekaerts (1999) who developed the layered
SRL model consisting of three layers. The first layer is about the regulation of the
self, which is related to the choice of goals and resources. The second layer focuses
on the regulation of the learning process that relates to the use of meta-cognitive
skills to direct the learning process. The third layer describes the regulation of the
processing modes, which describes the choice of cognitive strategies. Also this
model deals with cognitive and meta-cognitive activities, as well as with goals and
resources.
A key role in SRL is given to learning activities that are also called learning
strategies or learning processes. Dabbagh and Kitsantas (2004) listed six key
processes that are essential for SRL, namely goal setting, self-monitoring, self-
evaluation, task strategies, help-seeking, and time management.
• The goal setting process is defined as the outcome of a learning process and
identifies strategies how to reach these goals. Goal setting motivates the
learner’s choice of and attention to the relevant tasks and it also motivates to
attain higher effort and higher persistence over the course of time (Zimmerman
2002). Furthermore, goal setting influences learning through affective reactions,
for example, higher self-satisfaction when goals are reached. Also the difficulty
of a goal is an important factor for performance that increases with the difficulty
level of the goal (Locke and Latham 2002).
• Self-monitoring is defined as one’s reflected attention to an aspect of behaviour

that directs the learners’ attention to the task and assists them in evaluating the
outcomes of their efforts. Self-monitoring is important because it helps learners
attaining their goals by defining adequate learning adjustments.
• Self-evaluation is the process where the learner compares the learning outcome
with their own goals. It fosters better skill acquisition, self-efficacy beliefs,
intrinsic interest, and self-satisfaction about performance.
• Task strategies is defined as the process of the learner who applies strategies
which help reach their own goals. Studies indicate that students who applied
strategies for learning had a better performance than students who did not apply
them as much (Pintrich 1990).
• Help-seeking is taking place if a learner identifies and calls upon outside
resources. Thereby not only human help is meant, but also external analogue
and digital resources.
• Time management is the process where learners manage the learning regarding
time. Effective time budgeting highly correlates with academic achievement.
According to Roberts and Erdos (1993), meta-cognition is a key concept in the
study of cognition and it plays an important role in the transfer of cognitive skills
and in problem solving. Meta-cognition refers to knowledge and cognition about
one’s own cognition. According to Treier (2004), meta-cognition is a kind of self-
monitoring, self-observation, and self-regulation related to cognitive and infor-
mation processing. Meta-cognition is the competence of reflecting a mental task
critically and to organise involved learning and thinking processes in an efficient
and effective way. The usage of meta-cognitive learning strategies is an essential
component of SRL and is very important for flexibilisation and personalisation.
Taking into account the learner’s characteristics by individually adapting learn-
ing methods has a big influence on the learner’s performance (Issing 2002). The
importance of the adaptation to the learner’s characteristics (also called
personalisation) was shown in several studies. For example, the adaptive subject
material combined with adaptive styles of presentation supports students to
improve their learning achievements and increases learning efficiency (Tseng
et al. 2008). Furthermore, the importance of adaption to individual learning prefer-
ences of a learner regarding visualisation and verbalisation has been proven (Plass
1998). Through a requirement analysis it has been found out that the learner’s
knowledge, goals and tasks, language, interests, and learning styles are important
factors of personalisation approaches (Hover and Steiner 2009).
Supporting SRL in the right way is a crucial factor. On the one hand, it means
providing enough freedom for the learner, in order to stimulate motivation. How-
ever, on the other hand, too much freedom may be overwhelming and an appro-
priate guidance or even adaptation is usually needed to make the learning process
effective and efficient. The concept of guidance and freedom is important because it
has been recognised that highly motivated learners attain a better learning perfor-
mance if they have more control over their learning and are more autonomous
(Issing 2002). On the other hand, some learners show difficulties in carrying out
concrete meta-cognitive activities, such as planning, goal setting, monitoring,

evaluating, and as a result often perform less successfully than would be anticipated
(Bannert 2006). Such learners are in need of guidance when learning. Furthermore,
less motivated learners can also attain an improved learning performance if they
receive more guidance. Keeping these reported findings in mind, the individual
support for learners should be tailored to suitable degrees of guidance and freedom.
In this respect, the learner should be offered an optimal and balanced level of
control and autonomy for their own learning process.
Motivation is a highly relevant aspect for achieving good learning outcomes and
for performing SRL activities. Winne and Hadwin (2008) showed the positive
impact of motivation on student’s attention to their learning progress, on the
progress itself, and on the experience of satisfaction and positive affect. For the
use of SRL activities, a learner has to be motivated as these activities require
additional time and effort. Ryan and Deci (2000) describe intrinsic motivation as
one of the most important aspects regarding learning because it is the prototypical
manifestation of the human tendency towards learning and creativity. Behaviours
of intrinsically motivated learners are freely applied without the necessity of
separable consequences. For intrinsic motivation to develop, there is need for
autonomy, competence, and relatedness. However, there is also a need for extrinsic
motivation and especially a good balance between extrinsic and intrinsic moti-
vation (Covington 2000)
Another important factor for SRL is collaboration. According to Dillenbourg
(1999), collaborative learning comprises individually performed activities and also
extra activities that are generated by interaction among peers. These collaborative
activities trigger additional cognitive mechanisms, which may appear more fre-
quently in collaborative learning situations than in individual learning. Students
working in cooperative learning situations compared to individualistic or compe-
titive learning situations have a higher performance at the mastery and retention of
material, are more often using focusing, elaboration, and meta-cognition strategies,
and develop ideas or solutions which are not gained if they work on their own
(McConnell 2000). Collaboration can also create both intrinsic and extrinsic moti-
vation and is an essential strategy for stimulating curiosity, emulation, attention,
persistence, opening new perspectives, and increasing self-efficacy (Waite and
Davis 2006).
All these aspects are relevant for the psycho-pedagogical approach of ROLE.
The process model and related cognitive and meta-cognitive learning activities
have been explicitly modelled. The concept of guidance and freedom as well as
adaptation is achieved through recommendations of activities and resources. Other
concepts, such as motivation and collaboration, are implicitly targeted through the
use of technology.
Related Work and Technology
This section gives a short overview on technology-enhanced learning (TEL) solu-

tions discusses them from the SRL perspective. Technology plays an important role
for SRL because it has been reported that TEL environments can provide opportu-
nities to enhance SRL competences for students, especially meta-cognitive com-
petences (Bannert 2006).
Since the early 1970s, researchers have adopted the educational model of human
tutors and started to implement it in intelligent computer-based instructions
(Corbett et al. 1997). The goal of those Intelligent Tutoring Systems (ITS) was to
engage students in learning activities and to interact according to their behaviour.
ITSs were supposed to bring intelligence to computer-based instruction, especially
in the knowledge of the subject domain as well as the tutoring principles and
methods of their application (Anderson 1988). This led to the development of
four basic components: the domain model, the student model, the tutoring model,
and the user interface model. An important research strand is also the work on
adaptive hypermedia and adaptive Web and how it is applied in the educational
context (Brusilovsky et al. 2007). Strategies and systems (e.g. as described in Albert
and Schrepp (1999) and DeBra et al. (2006)) have been developed that aim to adapt
its content and behaviour to the individual student. In the light of SRL, these
solutions do not support the learner. They do not allow freedom or autonomy, but
guide the learner through the learning process. They also do not give any hint or
explanation regarding SRL.
A different development of TEL solutions is Learning Management Systems
(LMS). They primarily focus on distributing learning content, organising the
learning processes, and serving as interface between learner and teacher. In educa-
tional institutions LMSs have become very popular and are used in many univer-
sities and schools (Paulsen 2003). Examples of LMSs are Moodle, CLIX,
Blackboard, WebCT, Sakai, ILIAS, and LRN. They all have in common that
different tools are integrated in a single system, such as discussion forums, file
sharing, whiteboards, chat, and e-portfolios (Dalsgaard 2006). These tools together
with learning content are bundled by teachers or tutors, leading to a centralised and
standardised learning experience (Guo et al. 2010). In contrast to ITSs or adaptive
systems, no automatic guidance is provided by LMS. The guidance the learners get
is provided by teachers or instructors who prepare courses. On the other hand,
learners have more freedom and autonomy through some tools integrated in LMSs.
For example, collaboration tools allow to discuss certain topics.
In contrast to an LMS, personal learning environments (PLEs) strive for a more
natural and learner-centric approach. PLEs are online systems that combine ser-
vices as well as tools that enable users to create their own customised learning
system. The teacher is no longer the provider of knowledge, but rather a mediator
between knowledge and students. On the other side, the student is responsible for
organising information and their own learning. In recent years, attempts have been
made to build PLEs based on mashup designs. An example based on social media
tools is eMUSE (Popescu and Cioiu 2011), which integrates Web 2.0 tools into a
single system. It claims that such tool integration leads to a sense of community and
thus increases success and retention rates. Furthermore, eMUSE offers support for
self-monitoring and self-evaluation by providing feedback on learning tasks, which
is supposed to increase learning success and motivation. A further example is the
PLE developed at the Graz University of Technology (Ebner and Taraghi 2010).
This PLE allows for selecting widgets from a repository and adding them to a
personal space. Besides some general purpose widgets (similar to the tools in an
LMS), domain-specific widgets have been created by students in university courses.
It also allows for logging and analysing students’ activities performed on these
PLEs (Softic et al. 2013). Users can manage their PLE and, therefore, take greater
control of their own learning approach.
Specific technology that aimed supporting SRL was developed in several
research projects. For example, the iClass project aimed at developing an intelli-
gent, cognitive-based open learning environment that supports the planning, moni-
toring, and reflection processes of a learner and at the same time personalise the
learning process to the respective learners’ preferences and needs (Aviram
et al. 2008). In order to achieve personalised recommendations and reflection
support, a competence model has been used that is basis for individual guidance
(Steiner et al. 2009). This competence model is based on Competence-based
Knowledge Space Theory (CbKST) (Heller et al. 2006) that structures competences
through prerequisite relations. Visualisation tools have been created that display the
competence structures and let the learner select learning goals and learning paths
(Nussbaumer et al. 2008).
The ROLE approach differentiates from these solutions in the way that it pro-
vides a maximum of technical flexibility and provides personalised learning support
at the same time. A psycho-pedagogical model is integrated with the technology,
which leads to range of freedom and guidance to be chosen by the learner.
Models and Framework
Self-Regulated Learning Process Model
A process model (Fruhmann et al. 2010) has been developed that takes into account
requirements of the ROLE approach to create and use responsive open learning
environments that support SRL. The SRL process model describes learning as a
self-regulated process consisting of four learning phases: (1) defining or revising
the learner’s profile information and setting up an initial individual learning plan,
(2) finding and selecting learning resources, (3) working on selected learning
resources, and (4) reflecting on the applied strategies and reached achievements.
The model is depicted in Fig. 1. This approach follows the cyclic SRL model
proposed by Zimmerman (2002) and was slightly extended and modified. First, the
Fig. 1 SRL process model. This diagram depicts the SRL process model and related strategies
phase when learners create their own learning environment is considered as a

separate phase because this is a central part of ROLE. In the Zimmerman model
this phase would be related to the forethought phase. Second, the personalisation
aspect has been included by defining a user model consisting of models for
describing competences, learning activities, and goals.
In addition to these phases, a taxonomy of learning strategies, learning tech-
niques, and activities has been defined, in order to operationalise the four phases.
The phases are associated with learning strategies (see Fig. 1) and each strategy is
related to learning techniques and learning activities. We consider learning acti-
vities as the applications of learning techniques. This approach is based on the work
of Dabbagh and Kitsantas (2004) on SRL processes and Mandl and Friedrich (2006)
on SRL strategies and techniques. These findings have been adapted and integrated,
in order to serve as a comprehensive and integrated model for technology-support.
Using this theoretical background, we have defined nine SRL strategies and
structured them in three groups, which are cognitive strategies, meta-cognitive
strategies, and resource management (see Fig. 2). The group of cognitive strategies
include organisation, elaboration, and rehearsal tasks of learning topics. The group
of meta-cognitive strategies include goal setting, self-monitoring, and regulation
tasks targeting the control of the own learning process. The group of resource
management strategies include time management, help-seeking, and enabling
(or environment preparation), which means that learners take care for their learning
resources. These strategies are connected to SRL phases, which enrich the meaning
of these phases with a clearer notion. For the instantiation of learning strategies and
learning techniques, two complementary approaches have been used.
For each of the nine listed learning strategies, a variable number of learning
techniques are assigned. For instance, elaboration can be done with the following
Fig. 2 SRL Strategies. This diagram displays the SRL strategies and their relations to each other
Fig. 3 SRL techniques. This diagram gives an example of an SRL strategy and six associated
learning techniques
learning techniques: paraphrasing, creating analogies, producing questions, note-

taking, brainstorming, and collaborative learning (see Fig. 3). A complete list of
learning techniques and their assignment to the SRL strategies can be found in the
ROLE ontology (see section “Learning Ontology”).
Applying the SRL Process Model
Using ROLE learning environment learners are interacting with a learning environ-
ment that provides various resources like contacts to other actors, appropriate tools/
services, and suitable artefacts for the acquisition of knowledge, skills, and the
fostering of competence development. For the personalisation and the preparation
of a learning environment, learners have to set up their learning space in a
meaningful. Especially in SRL situations, where learners take responsibility for
the learning process and learning outcomes, the competent application of learning
strategies is considered helpful for learners and can lead to better learning perfor-
mance (Weinstein et al. 2005).
In SRL situations the definition of the learning goal before the selection of
learning tasks and the application of learning activities is very important. If learners
Fig. 4 Applying SRL strategies and techniques. This figure explains how SRL strategies and
learning techniques are applied in the learning process to achieve a learning goal
are not aware of their learning goals and if they do not know how to reach them,
learners incline to apply more or less habitual learning strategies that may not be
appropriate for processing a learning task instead of adapting or making up new
strategies. The SRL process model and the related learning strategies and tech-
niques are used to guide the learning through the learning process (see Fig. 4). First
learners set a goal. Then they choose from a set of cognitive, meta-cognitive, and
resource management strategies. The strategies are related with respected learning
techniques and learning tools. In this way a sequence of learning activities (strat-
egies and techniques) is created that should lead to reach the learning goal. Through
recommender systems (see section “Support Strategies”) the learner can get help
during the selection process.
Competence Model
A competence model consisting of different kinds of competences takes into

account domain knowledge, the ability to learn with tools, and SRL competences.
The distinction of the competence types originates from the fact that learning
happens on different levels. We distinguish between (1) domain knowledge and
related competences, (2) the ability to learn with specific tools, and (3) the ability to
learn in a self-regulated way. Therefore, three types of competences are defined:

domain competences, tool competences, and SRL competences.
The domain competence describes the knowledge of a learner regarding a
certain domain-specific topic. In detail a concept (e.g. from a concept map) is
assigned with a level from the European Qualifications Framework (EQF).3 The
tool competence defines the ability how a learner can apply a learning technique
with a certain tool functionality. Formally spoken, a tool competence is a pair of a
related learning technique and an EQF level. For example, it can be defined how
well a learner can debate with a tool that allows for video conferencing. The SRL
competence defines the ability regarding one of the nine learning strategies
described above.
Learner Model
In adaptive systems, user models are often designed as overlay models where user
information relies on and is described by conceptual information (e.g. concept
maps). Our learner model follows this scheme, but also relies on the learning
taxonomy and its elements. This is necessary as the information about the learner
is not only given at domain level, but also on the level of cognitive and meta-
cognitive activities (the application of the learning strategies and techniques). The
learner model consists of four elements. First, the competence state describes the
available competences of the learner taking into account that there are three
different types of competences. Thus, a learner is assessed regarding domain
knowledge, the ability to learn with tools, and the ability to learn in a self-regulated
manner. Second, the goals of a learner are also described with these competences,
e.g. the learning goal is expressed in terms of the competences a learner wants to
achieve. Third, the learning history is described by the learning activities (applied
strategies and techniques) a learner has performed, the tools or widgets she has
used, and the competences she has attained. In contrast to plain log data, this
learning history provides more insight because it describes the history in terms of
learning not just the particular interactions, but also regarding learning activities
and used resources. Fourth, pedagogical parameters describe individual preferences
and properties of the learner. Examples are information about preferred tools,
learning groups, or guidance mechanisms. An overview is given in Fig. 5.
3
http://ec.europa.eu/education/lifelong-learning-policy/eqf_en.htm
Fig. 5 Competence ontology. This diagram depicts the relevant elements of the competence
model and how they are technically interrelated
Learning Ontology
The formal representation of the models listed above (competences, strategies,

techniques, goals) is expressed as ontology in RDF4/OWL5 format and consists
of the classes and properties. An overview of this ontology is depicted in Fig. 5.
More details on the learning ontology can be found in Dahn et al. (2013). The
Competence class is the superclass of the three types of competences. It allows
specifying title and description via reusing the title and description properties from
the Dublin Core6 terms vocabulary. Furthermore, it provides two properties that
allow for a generic definition of the competence. First, a topic property is used to
relate a specific competence with a generic object (owl:Thing). Secondly, the
proficiencyLevel property is used to express the extent to which a user has a specific
competence. The generic Level class is subclassed by the EQF class to indicate the
competence level.
In order to relate a competence to a learner, a Person class is used that has
the properties acquiredCompetence, goalCompetence, uses, applies, and
hasParameter.
In this way competences are defined that a learner has already acquired and
competences that a learner should attain, the tools a learner has used, the learning
techniques she has applied, and the personal preferences. The DomainCompetence
class allows for defining a domain competence. It is a subclass of the Competence
class and inherits title, description, topic, and EQF level specification. In this way
4
http://www.w3.org/RDF/
5
http://www.w3.org/TR/owl-overview/
6
http://dublincore.org/
competence can be defined by using the topic property to assign the related domain
concept and the proficiencyLevel property to define the EQF level. The
ToolCompetence class allows for defining a tool competence. It is a subclass of
the Competence class and inherits the title and description specification. Following
the definition of tool competences, the topic property is used to relate tool func-
tionalities and the technique property is used to relate a LearningTechnique. The
SRLCompetence class allows for defining SRL competences. It is a subclass of the
Competence class and inherits the title and description specification. Following the
definition of SRL competences, the topic property is used to relate to SRL strategies
and the inherited proficiencyLevel property is used to relate an EQF level. Since
SRL competence can be related to different types of SRL strategies (cognitive,
meta-cognitive, and resource management strategies), appropriate subclasses of the
SRLStrategy class have been defined.
Mashup Guidelines
The mashup of a learning environment is an important feature. It can affect the way
of learning not only by its pure use, but how it is compiled. It is assumed that
compiling widgets to a learning environment or bundle relate to different psycho-
pedagogical aspects and educational components. Taking into account these edu-
cational components, some essential guidelines have been developed to provide the
learner with recommendations for a psycho-pedagogically sound mashup (Berthold
et al. 2012). These guidelines should be taken into account by the learner creating a
bundle for herself and by teachers or tutors creating bundles for learners. It is also a
basis for recommender technology that can apply these guidelines.
Guideline 1: Cover all phases of the SRL process model: As described in section
“Theoretical Background”, learning is more successful if different cognitive and
meta-cognitive strategies are applied. Hence, a mashup design should contain
widgets that support different learning strategies. For example, one widget for
each phase of the SRL process model. This could be materialised in a setting
with a widget for goal setting, a widget for content searching, a widget to learn
how to find content, and a widget where the learning process is reflected.
Guideline 2: Tool competence: Another educational component that comes into
play is tool competence. Tool competences are abilities a learner has if she has
knowledge about a tool, is able to perform a learning activity with a learning tool in
a domain or domain-independent context, and is able to choose a tool or tool
functionalities unassisted according to her learning or working task, e.g. the learner
can use a tool for goal setting. If a tool competence is lacking for a certain widget,
the learner will have problems to effectively use this widget.
Guideline 3: Number of widgets: The number of widgets and their different
functionalities might overtax the learners, especially if they are not intrinsically
motivated to perform the learning task. Hence, only an appropriate number of
widgets should be added to a widget bundle. The range of the appropriate number of
widgets within a widget bundle depends on the learner.
Guideline 4: Domain aspects: In the third phase of the SRL process model, the
actual learning process takes place. This phase is influenced by learning goals.
Learning goals are defined in the first phase of the SRL process model. Such
learning goals are mostly domain specific and refer to a special context. Thus,
widgets in the bundle should be in line with the domain aspects of the predefined
learning goals.
Support Strategies
In the previous section it has been described how SRL can be operationalised. In
this section concrete approaches are reported how SRL is supported by using the
models and framework described above. They establish the ground basis for the
technology and concepts to support learners in a scientifically driven way and
provide learners with according technology and material to guide them through
the learning process.
Learners who are able to understand the SRL process and the related learning
activities and who are able to perform them on their own can navigate freely
through their learning processes. However, this requires the availability of a high
degree of SRL capabilities (availability of the respective SRL competences). Since
it cannot be assumed that all learners already have these abilities, guidance mecha-
nisms are needed. Such guidance is often needed especially when learning with
technology-enhanced environments (Bannert 2006).
According to the experiences made in the test beds and at several workshops (see
section “Related Work and Technology”), a variety of guidance strategies is
necessary. Both technical and human support is needed depending on the learner
and the situation. Additionally training and introductory material turned out as
useful to increase initial motivation for new ways of learning.
Motivational Video
This section explains an initiative to make learners acquainted with SRL.

According to our experience (in test beds and SRL surveys), the concept of SRL
is new to most lectures and learners. Lifelong learning, non-formal learning, etc. are
buzz words society talks about, but SRL is apparently a term more common in
science and pedagogical research. For this reason, SRL needs to be introduced and
actually promoted to learners, teachers, and a broader community to point out
benefits of such a learning approach. An introduction can be done in many
different ways.
Fig. 6 SRL video. This figure is a screenshot of the video that introduces and explains SRL in the
context of ROLE
Teaser videos were used to explain the concept of SRL on a basic level for
learners who are completely new to SRL. A video7 has been created that focuses on
the explanation of SRL by comparing it to a city travel (see Fig. 6). Two people are
examining a city and its sights. While the first one is attending a guided tour, the
second one defines the goals and plans on his own and does a city walk without
external support. This analogy is explaining both the concept of self-regulation and
the concept of guidance. Further videos have been developed in the course of the
ROLE project, in order to explain different aspects of PLEs and SRL8. Especially,
tutorial videos have been created that explain how to use ROLE technology.
Courses and Training Material
An SRL course is another method to reach learners and provide them with assis-
tance and knowledge about SRL and SRL tools in a compact way. The goals of such
a course are to introduce the idea of SRL and enable them to build their own
learning environment. The content of such a course can be a brief SRL explanation,
an explanation of different learning models or learning strategies, and how they can
be used with ROLE technology.
At the Open University UK test bed such a course has been created as Online
Course and as eBook. This course explains the basic concepts and also lets learners
7
http://www.youtube.com/watch?v¼jTa1vOH6JjA
8
https://www.youtube.com/user/ROLEProject
try out to create PLEs on their own. In this way learners can train SRL behaviour
because they get step-by-step explanation and can try it out immediately.
Preconfigured PLEs
Predefined PLEs are already compiled bundles of widgets to fulfil a certain learning
need. Therefore, they are usually assembled with widgets that cover a certain
domain (e.g. history, chemistry, or language learning). Such bundles are typically
created by teachers or peers. Teachers have the chance to prepare a bundle suitable
for the learning topic. In this way, the guidance is based on the preparation of whole
bundles that are suitable for individual learners. A special case is Layered PLEs that
consists of a set of widget bundles. Each set may be dedicated to a certain learning
strategy of an SRL process phase.
Widget Store
The ROLE Widget Store is a Web-based online catalogue that allows to manage
and index widgets. It provides a user-friendly interface to a widget repository that
simplifies the discovery of widgets. The functionality of the widget store includes
listings of widgets, categorisations, searching by widgets or keywords, and compi-
lation of widgets into bundles. Users can add widgets from the widget store to PLE
systems. From a social media point of view, the store is also the place to collect and
share user tags, comments, and ratings. A widget creator and developer can add a
widget to the store by adding its reference (URL) and metadata.
In order to provide guidance for learners in searching and selecting widgets for
their PLEs, widgets can be tagged with metadata describing the purpose of the
widgets. The first type of tags is a widget categorisation consisting of seven
categories. The categories were derived from the SRL learning process model
and are assigned to its phases. As described above a PLE should consist of widgets
not only for one learning strategy, but widgets for different strategies should be
included. The categorisation system is a useful way to follow this guideline because
users get quick access to widgets for the specific purposes. They can browse the
store and add widgets just by navigating to different categories.
In addition to the widget categories, functionalities described in an ontology
are used to represent features of widgets (e.g. text editing, video chat). These
functionalities are derived from a survey of existing widgets and from an analysis
of the ontology. The SRL techniques are related to functionalities so that the
ontology and the Widget Store share the same set of functionalities.
The third type of metadata is the domain concept describing widgets regarding a
knowledge domain they are related to. Widgets can be either generic (e.g. text
editor) or targeting specific learning domains (e.g. French language). As some
widgets can hardly be described by tool categories or functionalities alone, a

categorisation based on learning domains is introduced. The service of DBpedia9
is used to allow users the tagging of widgets by semantically unique learning
domains supporting them in search for specific tools.
The user interface of the Widget Store allows for searching and filtering widgets
according to categories, functionalities, and domain concepts. A list of widgets is
listed according to the applied search. Additional filters can be applied regarding
the metadata available for the listed widgets. The metadata for each widget is
shown in the search result list, that is, category, functionalities, domain concepts,
rating, title, and description.
Mashup Recommender
The Mashup Recommender widget (MR) (Fig. 7) can be seen as a filtering system
that provides widgets that can be added to the PLE depending on the used template.
The MR contains predefined templates, e.g. an SRL template. This template could
include the four phases of the SRL process model. If the user selects such a phase,
related widgets that support this phase are displayed. For example, if the learner
selects the planning phase, calendar widgets, and To-Do-Widgets could be
suggested by the MR. For this purpose, the MR uses SRL entities from the
Fig. 7 Mashup Recommender. This screenshot displays the Mashup Recommender widget in the
middle and a widget that was recommended on the right side
9
http://dbpedia.org/
ontology. The ontology service is questioned for the respective functionalities of

the SRL entities (learning strategies, techniques, and activities) and the widget store
returns the associated widgets. Such templates can be created using a special
authoring tool.
The MR can be used to provide guidance on different levels and for different
stakeholders. A high level of guidance is the preparation of complete predefined
PLEs based on a specific template by a teacher or tutor. Later the tutor can share this
PLE with her students who can use it or modify it. A lower level of guidance can be
provided if the teacher just shares the template with the students, so that they have
to create their own PLE. For example, a teacher could select the SRL entities goal
setting, resource searching, note taking, and reflecting for a template. Teachers or
learners using this template could easily search these SRL entities for widgets and
include them in a PLE. In this way the PLE consists of widgets for each SRL entity.
Learning strategies are on a higher abstraction level, which results in a greater
number of widgets that can be recommended. Learning techniques are on a lower
abstraction level, which leads to a smaller number of related widgets that can be
recommended. While in the first case the learner gets more widgets recommended
and thus less guidance, in the second case the level of guidance is higher because of
the smaller number of recommended widgets. For a detailed description of the MR
and its technical background, see Nussbaumer et al. (2014).
Activity Recommender
The Learning Activity Recommender guides the learner through the learning
process by recommending learning activities related to the SRL process model.
The learner is guided by means of a step-by-step approach of how to cope with a
problem. In contrast to a direct instruction, the learner can decline to accept
learning activities and can choose between alternatives and will not be penalised
for varying his learning steps from what is suggested. The Activity Recommender
(AR) consists of two widgets, the Activity Recommender widget and the To-Learn
List widgets (Fig. 8).
In contrast to collaborative and content-based filtering approaches handling large
community-generated data sets, the Activity Recommender is working with data
predefined and structured by the educational experts according to the educational
approach described in the last section. These experts prepare the recommendations
by defining learning strategies, techniques, and matching activities for learning
tasks using an authoring tool. The Activity Recommender guides the learners
through the learning process by recommending learning activities and assists
them to compile a learning plan. In contrast to direct instructions, the learners
have a free choice which recommended learning activities they want to perform.
When a learner has decided to use a recommended learning strategy, the respective
learning activities can be sent from the Activity Recommender to the To-Learn list
widget. The To-Learn list widget allows learners to compile an individual learning
Fig. 8 Activity recommender. The To-Learn List widget and the Learning Activity Recom-
mender widget are running in the ROLE SDK environment
plan. The widget enables to add, rearrange, delete, and rename recommended
learning activities or to add own activities, e.g. reminding them to take a break
after a brainstorming session. A learning activity is described with a short summary
and a longer descriptive text. Every learning activity has a status that is either not
started, started, completed, or cancelled. Moreover, it is possible to specify the
learning activities by adding sub-activities on the lower hierarchy levels. Some of
the activities are highlighted in red colour, which means there are further
recommendations available for this activity. Displaying these recommendations
can be triggered by clicking on a highlighted entry. Finally, the entries of the
learning plan can be sorted by status, date, or manually.
SRL Text Reader Bundle
The SRL Text Reader Bundle (see Fig. 9) is a predefined widget bundle that
supports SRL by providing feedback on the SRL activities. The bundle captures
certain SRL activities and displays them in a graphical way to make the user aware
about the activities she performs. The main widget is Text Reader where learners
can read and annotate texts. These texts and related concepts are defined in a
domain model on a backend service. The Self-Evaluation widget allows for relating
the assigned tags with concepts from the domain model and to determine the
proficiency level for each concept. In this way, the learner evaluates herself
36
Fig. 9 SRL text reader widget bundle. The four widgets are displayed in this screenshot
A. Nussbaumer et al.
regarding her own domain competences. The search widget allows searching
additional resources for the domain by searching related tags and concepts. All
performed activities are recorded and stored in the user model. The visualisation
widget follows an Open Learner Model approach and gives feedback to the learner
about her learning process. In addition, the visualisation widget displays the texts
that have been annotated and the concepts that have been used for self-evaluation.
Guidance is provided by delivering a complete bundle of widgets that support the
whole SRL process.
From the user model perspective some different types of information is saved for
further usage. The activities a learner performed are saved using the activity schema
outlined in the ontology definition in section “Support Strategies”. The concepts
coming from a domain model in the background and used for self-evaluation are
stored together with the proficiency level as domain competence. The tags related to
certain texts are saved as generic information. All this information is used for
keeping the user data persistent and visualising the analysis in the Self-Reflection
widget.
SRL Monitor
The SRL monitor provides support to develop self-awareness about the performed
learning activities. The goal is not only to monitor and visualise the observable
actions (as saved in log data), but also to monitor the cognitive and meta-cognitive
activities that are not directly measurable. To this end, the measurable actions are
mapped to cognitive and meta-cognitive learning activities from the ontology. To
be precise, the key actions extracted from the log data analysis (based on an
algorithm that clusters the log data) are mapped to elements of the learning
ontology. The mapping is partially done by the learner herself, but also supported
by an algorithm that takes into account the previous manual assignments. The goal
is to make learners aware about their cognitive and meta-cognitive learning
activities.
The screenshot displayed in Fig. 10 shows two views of the SRL Monitor. In the
first view, the SRL Monitor displays the learner’s captured log data in a sequence.
Then the learner can select which learning technique she has actually applied.
Based on these selections, reasoning is done regarding the applied learning strate-
gies. Since there are only nine learning strategies, a comprehensive overview of the
learner’s behaviour can be given. This overview is graphically shown in the second
view of Fig. 10. In this way learners get feedback about their learning behaviour and
might rethink their learning process if some learning strategies never appear on the
graphical profile.
Fig. 10 SRL monitor. Screenshots of the two views of the SRL monitor widget are shown
Evaluation Results, Challenges, and Barriers
This section gives insight in the experiences made with the psycho-pedagogical
model and the support strategies described above. It starts with an overview of
evaluation studies of a few support strategies. Then it describes the experiences
made in the test beds, in summer schools, and at conference workshops. More
detailed evaluation results are described in the Chaps. 4–7 of this book.
Selected Evaluation Results
One evaluation study (Kroop 2013) focused on the usefulness of the developed
Mashup Recommender (MR) widget (see section “Mashup Recommender”, Fig. 7)
and the SRL template it provided. The study compared teachers’ and students’
acceptance for the MR and its underlying SRL strategy. Altogether the findings
show a broad acceptance of the MR concept by both teachers and students. When a
group of teachers and a group of students independently from each other tested a
variety of PLE-scenarios, the MR was recognised to be the only tool which
consistently supports the creation of a PLE in a self-regulated learning way. Most
scenarios for creating a PLE provide too much freedom (especially in selecting
relevant learning tools from a broad variety of open educational resources), which
causes students as well as teachers to be exhausted easily from (individually)
evaluating and selecting learning tools. Consequently, the provided SRL guidance
by the MR was perceived as a relief in creating a PLE. Although neither the
teachers nor the students already felt familiar with the MR widget, teachers and
students independently from each other appreciated the well-conceived idea to
support SRL while creating a PLE. The discussions as well as the final ratings of
the MR concept revealed that teachers were even more optimistic than students
regarding the potential of the MR concept to improve and ease (self-regulated)
learning. The findings of this study confirm the findings of the below described
TPSRL study: Not only do students need guidance how to learn self-regulated but it
is also important to provide support to the teachers and to train them how to
(efficiently) introduce SRL to students. Another study (Dahn et al. 2013) concerns
the Activity Recommender (AR) (see section “Activity Recommender”). The
objective of this evaluation was to uncover strengths, weaknesses, and suggestions
for improvement of the proposed approach and implementation. Therefore, useful-
ness of the learning support, usability of the developed software, and general
feedback were questioned. The evaluation has been conducted in two different
settings: The first evaluation took place in at the University of Koblenz-Landau in
Germany and the second evaluation took place at the Shanghai Jiao Tong Univer-
sity in China. Summarising the results, there was a clear indication that the students
appreciate guidance and help during the learning process. Some of the students
liked the idea of getting recommendations for learning strategies and techniques for
improving their learning. However, some students did not understand that recom-
mendations are suggestions and not “must-do” instructions. Usability turned out as
being the weakest point of AR. Without supervision and human support, many
students had difficulties with the proper usage of the AR. Concluding these results,
the approach of recommending learning activities is useful for and appreciated by
students, whose SRL competences are not very low, but who do not apply SRL
activities during their learning process. However, very weak self-regulated learners
need sound introduction to SRL not only on pure technological level, but also in
terms of human explanation, an introductory course, or videos.
Results from the Conference Workshops
The workshop series at the ICALT conferences in 201110 and 201211 aimed at
investigating SRL in personal learning environments. The target audience consisted
of researchers, developers, and users of learning environments who were interested
10
http://dbis.rwth-aachen.de/SRL-ROLE2011/
11
http://dbis.rwth-aachen.de/S-ROLE2012
in building individual learning environments. The accepted papers were presented

at the workshop and published in the ICALT conference proceedings. The presen-
tation session was followed by a group and plenary discussion.
The interactive format of the workshops helped to identify barriers of SRL.
According to the participants, these barriers are mainly in the traditional educa-
tional system that does not cultivate the self-regulation competences early enough.
This system designed for industrial purposes does not suit modern requirements
anymore, as it kills motivation and creativity in learners. Individualisation of
learning is a major challenge in education and rapid technological development
brings new opportunities how to address it. A good SRL solution should be
personalised and adaptive, providing a right balance between the learner’s freedom
and guidance, in order to both motivate and support the learner. This threshold is
individual and context dependent. Therefore a spectrum of facilities is needed for
various levels of SRL at different levels of education and in different contexts.
Another key challenge is to prepare learners for lifelong learning when the teacher
may not be available. They have to learn meta-cognitive skills, which are highly
important. In this process, suitable scaffolding has to be taken into account.
The PALE workshops at the UMAP conferences in 2011 and 2012 provided
opportunities to identify and discuss various issues related to topics like pedagogic
conversational agents, responsive open learning environments, and learner
modelling.
At PALE 201112 (Perez-Marin et al. 2011) the ROLE-related discussions
focused on the usefulness of PLEs in order to support SRL. A big challenge is to
find an appropriate threshold between the learner control and tutor guidance.
Personalised support can be provided by means of design templates and recom-
mendations. In any case, this support has to be effective but not intrusive, consid-
ering the learner preferences. Moreover, these preferences are not static, but can
change dynamically according to the context. Participants suggested focusing the
research on two scenarios where learners are supposed to have certain capabilities
to guide themselves: higher education and lifelong learning. It was also suggested
to consider meta-cognitive competences as well as results of behavioural and
cognitive psychology when designing learning environments.
To summarise the outcomes from PALE 201213 (Herder et al. 2012), a lot of data
can be collected in the educational process, but we need to find ways how to use it
reasonably and to develop useful services that make the learning process more
effective and efficient, e.g. by predicting student outcomes in order to intervene.
Novel personalised services and environments are needed especially in lifelong and
workplace educational settings, in order to support informal, self-regulated, mobile,
and contextualised learning scenarios. A big challenge is also adaptation consider-
ing both long-term objectives and short-term dynamically changing preferences of
learners. Here, open and inspectable learner models play an important role,
12
http://adenu.ia.uned.es/workshops/pale2011/
13
http://adenu.ia.uned.es/workshops/pale2012/
considering also learner motivation and affective state. In the case of pedagogic
conversational agents, personalisation is fostered by the use of dialogues adapted to
the specific needs and level of knowledge of each student. Here and in mentorship
systems trust and reputation play a crucial role.
At AHA 201214 conference in Vienna a ROLE workshop15 on PLEs and the
support of SRL took place. Workshop participants were mainly teachers and pro-
fessors from Germany and Austria but also school consultants and technical support
people from higher education institutions. At this workshop the participants had the
opportunity to test and discuss a variety of ROLE tools for creating a PLE including
tools for supporting SRL when creating a PLE. This workshop-session as well as an
additional cross-validation workshop with students including teacher candidates
from the University of Vienna revealed and confirmed success criteria as well as
limitations and barriers for the uptake of the provided ROLE tools. The workshop
results which explicitly included the perspectives and backgrounds of German-
speaking countries and cultures were also presented and discussed with profes-
sionals from Guatemala at a Workshop on Cloud Education Environments
(WCLOUD 2012) in Antigua, Guatemala (Kroop et al. 2012). Interestingly the
experts from Guatemala shared the perspective of their colleagues from German-
speaking countries, especially the need for SRL-supporting technology.
The essential success criteria for the uptake of the PLE tools were seen in the
underlying pedagogical learning model: While participants were working with the
provided tools and felt sometimes overwhelmed by the challenge to create a
reasonable PLE, it became clear that a consistent model on SRL strategies and
techniques as described in section “Models and Framework” was most needed. The
idea to connect different stages of SRL (Planning, Searching, Learning, Reflecting)
with corresponding ROLE tools for learning was seen most useful and most
effective by almost all participants. A positive impact on learning by using
SRL-supported PLE tools was especially seen in the following learning activities:
• Getting started with a learning task in a meaningful way.
• Keeping track of the own learning progress by following the provided learning
strategy.
• Improved time management and reflection. Limitations for the uptake of the
provided ROLE tools as well as doubts on the learning improvement by using
these SRL-supported PLE tools were assumed as well and mainly addressed the
personality, attitude, and motivation of a learner:
– Motivated learners will benefit from the provided PLE tools; it will improve
their efficiency and outcome of learning.
14
http://ahakonferenz.at/fruhere-aha/aha-2012/
15
Sylvana Kroop, Marcel Berthold: Personalisierte Lernumgebungen. Unterstutzung von selb-
streguliertem Lernen.
– Very motivated learners will not need the technology-based SRL-support

(e.g. the Recommender Widget, see section “Mashup Recommender”,
Fig. 7).
– Less motivated or weak learners will not benefit from the provided PLE tools.
Unfortunately the last assumption (about weak learners) was not well reasoned
by the experts. But from the discussion context, it can be assumed that an intrinsic
motivation of a learner is seen as a prerequisite to be successful when learning with
PLEs. The idea that SRL-guidance provided by the PLE technology could trigger
the motivation which is missed by weak learners was received with reservation.
Crucial barriers especially teachers raised for the uptake of the provided ROLE
tools were belonging to time-consuming concerns for (learning) activities such as:
• To get in general used to the new PLE technology.
• To create a useful PLE in order to use it for the content taught in school and for
the most ambitious request.
• To create and provide own SRL templates (see section “Mashup Recom-
mender”) which are adapted to a specific course or specific learning content.
Time-related issues in general can be seen as typical criteria for the uptake of any
new technology. However, interesting is the fact that the participating teachers were
aware of the possible time-wise burden and the additional effort compared to their
used traditional learning and teaching but still accepted the uptake of the provided
ROLE tools. They explicitly recommended the SRL-supporting PLE tools. More-
over, at the end of the workshop the involved teachers strongly expressed their wish
to try out a PLE in their daily activities and thus created a mailing list in order to be
informed and provided by further material, tutorials, online courses, or upcoming
workshops on this topic. Altogether the experiences and findings from workshops
gave important directions for the request and need of the research and development
of innovative learning technology.
Lessons Learned from the Test Beds
Within the ROLE project, an instrument for understanding the perceptions of

educators about SRL was devised. A questionnaire called “Teachers’ Perception
of Self Regulated Learning” (TPSRL16) was formulated, containing questions
about how teachers perceive the SRL capabilities of their students, as well as
about the challenges associated with teaching students with varied SRL compe-
tences. This questionnaire was circulated among the teachers of the Higher Educa-
tion ROLE test beds. The TPSRL survey set out to explore which factors potentially
influence teachers’ assessments of their students SRL competence, how they see the
16
http://fit-bscw.fit.fraunhofer.de/pub/bscw.cgi/39770946
relationship between students’ SRL competence and performance as well as which

type of students in terms of SRL level the teachers prefer to teach.
According to the results of the TPSRL survey, the majority of the teachers that
responded had an awareness of SRL and independent learning. Most of them also
recognised the important function of SRL alongside its significance to them as well
as to students. Some respondents related SRL to increased maturity and, therefore,
an acceptance of responsibility for one’s own learning. Several teachers regarded
SRL as a joint venture, i.e. learning together with students and “discovery
together”. SRL was also seen to influence students’ ability to learn faster. Various
teachers recognised that SRL could improve their students’ reasoning or
questioning abilities as well as their concentration power and, therefore, their
capacity to learn. Some alluded to a teacher’s moral duty to guide/show “right”
path using SRL techniques, thus adding to the all round development of students,
i.e. implying that it was an implicit responsibility of all teachers.
The strategies the respondents to the survey use in order to motivate and support
students in becoming self-regulated are quite varied. One of these strategies con-
sists of providing specific academic study skills facilities outside the classroom,
available as face-to-face as well as via online support from the university,
i.e. blended approach. Additionally, several teachers direct students to online
and/or library-based “learning to learn” resources. Some teachers prefer to offer
less help to their students, thus encouraging them to take more initiatives and learn
for themselves. “Leading by example” is also a popular strategy: the teacher
indicates or offers different approaches to resolving subject-based problems but
leaves the students to choose their own learning path. Finally, the majority of the
respondents agreed that encouraging active learning through peer collaboration
helps motivate SRL of their students. In particular, some teachers promote working
together with their students, e.g. through semi-directed projects, or they encourage
group work.
The survey was also quite revealing with regard to some of the challenges in
motivating higher education students to become self-regulated learners. Several
respondents stated that many students are simply not equipped to learn at an HE
level. They also mentioned that their students are reluctant to accept new methods
of learning or change in their outlook on learning. On the other hand, most students
expect to be provided with precisely defined learning materials and strategies by
their teachers. One of the themes that emerged was that inspiring groups of students
that have mixed learning skills is challenging in itself. Students with fewer SRL
skills require more time to assimilate information or discover new methods of
learning. This has implications for the teacher in terms of effort required to meet
the needs of the entire spectrum of learning skills in the classroom. Most impor-
tantly, the teacher may not have enough knowledge, experience, or personal
confidence to include SRL in the delivered curriculum time frame.
Experiences in the Summer Schools
In the years 2010–2012 several workshops on SRL were organised, mainly for Ph.
D. students in the TEL area at the JTEL Summer Schools. Three key target groups
were considered at these events, namely learners, teachers, and developers. At the
workshops the objective was to demonstrate the conceptual and technical solutions
for personalised support of SRL. In the introductory part we aimed to explain how
psycho-pedagogical theories impact the design of learning environments, consid-
ering principles of SRL. We emphasised the role of decision making for the quality
of the learning outcome and that it should be supported properly to optimise
learning for benefit of the learner. Specific support strategies for SRL were
explained taking into account the various SRL concepts. The participants tried
out the ROLE software aiming to create a PLE. Experience from these events has
shown that Ph.D. students understood the ROLE approach quickly and managed to
design their PLEs without bigger problems.
In the practical part they elaborated approaches for personal support in small
groups. The approaches were made with paper-and-pencil or with respective tools
on their laptops. Each group presented their results followed by a discussion with
the other participants. The group work was active and creative. However, it also
turned out that it was really difficult for the students to find good and innovative
solutions. As a conclusion it can be reported that the concept of personalised
support for SRL can be explained to Ph.D. students, but it is very difficult that
they find new solutions in a 3-h workshop.
In 2010 we conducted a survey to test the impact of choice architecture on the
responses of people and to find out what Ph.D. students think on some issues in
TEL. Our respondents (advanced learners) mostly thought that too much freedom
for the learner may be overwhelming and contra productive. Similarly, they agreed
that learners needed pedagogical assistance. Finally, almost all of them appreciated
availability of a competent tutor. In addition to these findings we could easily see
how important it is to find good explanations of SRL concepts in order to achieve a
common understanding. Slight changes in wording may dramatically change
responses. At the same time this demonstrated that to identify real opinions and
preferences of people may be a tricky issue and the concept of choice architecture
can influence them essentially.
Conclusion
The ROLE project has aimed to achieve progress beyond the state of the art in user-
centric responsive and open learning environments. A key objective of this inten-
tion was the development of a psycho-pedagogical framework. The most important
goal of this framework is to support learners to learn in a self-regulated way in
responsive open learning environments. Based on this framework, a variety of
support strategies have been developed. This approach is grounded on a literature

review on SRL and experiences made in the test beds and in workshops with
students, teachers, and researchers.
One of the most important findings was that adopting the self-regulated way of
learning can be very difficult especially for weak learners of learners not used to
freedom in their learning process. In such situation technological support strategies
alone are not sufficient and not successful. Therefore, a holistic support strategy is
needed, which includes several factors: (1) Material (e.g. videos) that motivates
learners and raises their attention for SRL. (2) Courses that provide step-to-step
introduction to SRL, so that learners can practice and get use to a new form of
learning. (3) Preconfigured environments that already support SRL, so that the
learner does not have to start from scratch. (4) Peers should be included in the
learning situation, which leads to communication and reflection on the own learning
process. (5) Finally teachers (if available) should take care to support learners. It
turned out that also teachers need introduction and awareness for SRL, because not
all teachers do fully understand the concept of SRL.
Five key aspects have been identified as essential for the psycho-pedagogical
approach: Personalisation, degree of guidance, motivation, meta-cognition, and
collaboration. Each support strategy takes into account at least one of these aspects.
The SRL process model describes the learning process and thus serves as a
backbone for support strategies. Based on these aspects, several support strategies
have been developed (see section “Support Strategies”). However, these strategies
are just examples demonstrating how support for SRL can be provided. Actually
there is still room for new and more sophisticated support strategies. Future
research and development can (and should) be done in this field. Taking into
account the theoretical approaches described in this chapter, future possibilities
for supporting learners will emerge.
One of the limitations we faced during the project was the technology-based
assessment of SRL competences. It turned out that automatically assessing SRL
competences (e.g. by monitoring the learner when using the learning environment)
is much more difficult than expected. Especially in this field there is room for
further research. In order to overcome this problem, teachers were asked to estimate
the SRL competences of their students and also students were asked to self-estimate
their SRL competences. However, a method to automatically determine the SRL
competences would still improve the learning process and would give new possi-
bilities to support the development of SRL competences and to personalise the
recommendations of learning resources and activities.
Acknowledgements The research work described in this chapter is partially funded through the
ROLE Integrated Project (grant agreement no 231396); part of the Seventh Framework
Programme for Research and Technological Development (FP7) of the European Union in
Information and Communication Technologies.
All people involved in the ROLE project and not listed as authors merit special thank,
especially Milos Kravcik, Bernd Prunster, Marcel Berthold, Karin Fruhmann, Andreas Kiefel,
and Pablo Lachmann.
References
Albert D, Schrepp M. Structures and design of an intelligent tutoring system based on skill
assignments. In: Albert D, Lukas J, editors. Knowledge spaces: theories, empirical research,
and applications. Mahwah: Lawrence Erlbaum; 1999. p. 179–96.
Anderson JR. The expert module. In: Polson MC, Richardson JJ, editors. Foundations of intelli-
gent tutoring systems. Hillsdale: Lawrence Erlbaum; 1988. p. 21–53.
Aviram A, Ronen Y, Somekh S, Winer A, Sarid A. Self-regulated personalized learning (SRPL):
developing iClass’s pedagogical model. eLearning Papers. 2008;9(9):1–17.
Bannert M. Effects of reflection prompts when learning with hypermedia. J Educ Comput Res.
2006;35(4):359–75.
Berthold M, Lachmann P, Nussbaumer A, Pachtchenko S, Kiefel A, Albert D. Psycho-pedagogical
mash-up design for personalising the learning environment. In: Ardissono L, Kuflik T, editors.
Advances in user modeling, lecture notes in computer science, vol. 7138. Berlin: Springer;
2012. p. 161–75.
Boekaerts M. Self-regulated learning: where we are today. Int J Educ Res. 1999;31(6):445–57.
Brusilovsky P, Kobsa A, Nejdl W, editors. The adaptive web. Berlin: Springer; 2007.
Corbett AT, Koedinger KR, Anderson JR. Intelligent tutoring systems. In: Helander MG,
Landauer TK, Prabhu PV, editors. Handbook of human-computer interaction. Amsterdam:
Elsevier; 1997. p. 849–74.
Covington MV. Goal theory, motivation, and school achievement: an integrative review.
Annu Rev Psychol. 2000;51:171–200. doi:10.1146/annurev.psych.51.1.171.
Dabbagh N, Kitsantas A. Supporting Self-regulation in student-centered web-based learning
environments. Int J e-Learn. 2004;3(1):40–7.
Dahn I, Kiefel A, Lachmann P, Nussbaumer A, Berthold M, Albert D. The ROLE learning ontology
and its application in recommenders for self-regulated learning. Int J Innov Qual Learn
(INNOQUAL). 2013;1(2):52–63.
Dalsgaard C. Social software: e-learning beyond learning management systems. Eur J Open Dist
E-Learn. 2006;14(2):50–71.
DeBra P, Smits D, Stash N. The design of AHA! In: Proceedings of the ACM hypertext
conference, 2006, ACM, New York, p. 133–14.
Dillenbourg P. Introduction: what do you mean by “collaborative learning”? In: Dillenbourg P,
editor. Collaborative learning. Cognitive and computational approaches. Oxford: Elsevier;
1999. p. 1–19.
Ebner M, Taraghi B (2010) Personal learning environment for higher education a first prototype.
In: World conference on educational multimedia, hypermedia and telecommunications
(ED-MEDIA), AACE, Chesapeake, p. 1158–66.
learning in a responsive open learning environment. In: Hambach S, Martens A,
Tavangarian D, Urban B, editors. Proceedings of the international conference eLearning
Baltics science (eLBa Science 2010). Rostock: Fraunhofer; 2010.
Guo Y, Rui J, Zhou H (2010) Pervasive and personal learning environment using service-oriented
architecture: a framework design. In: International conference on networking and distributed
computing, IEEE Computer Society, Los Alamitos, p. 153–5.
Hattie JAC. Visible learning a synthesis of over 800 meta-analyses relating to achievement.
London: Routledge; 2009.
Heller J, Steiner C, Hockemeyer C, Albert D. Competence-based knowledge structures for

personalised learning. Int J E-Learn. 2006;5(1):75–88.
Herder E, Yacef K, Chen L, Weibelzahl S, editors. Workshop and poster proceedings of UMAP
2012, CEUR workshop proceedings; 2012, vol. 872. http://ceur-ws.org/Vol-872/
Hover KM, Steiner C. Adaptive learning environments: a requirements analysis in business
settings. Int J Adv Corp Learn (iJAC). 2009;2(3):27–33. doi:10.3991/ijac.v2i3.956.
Issing LJ. Issing J, Klimsa P. In: Informationen und Lernen mit Multimedia und Internet, Lehrbuch
fur Studium und Praxis. 3rd ed. Weinheim: Beltz; 2002. p. 151–78.
Kitsantas A. Test preparation and performance: a self-regulatory analysis. J Exp Educ. 2002;70(2):
101–13.
Kroop S. Evaluation on students’ and teachers’ acceptance of widget-and cloud-based personal
learning environments. J Univers Comp Sci. 2013;19(4):2150–2171, http://www.jucs.org/
jucs_19_14
Kroop S, Berthold M, Nussbaumer A, Albert D (2012) Supporting self-regulated learning in
personalised learning environments. In: Proceedings of the 1st International workshop on
cloud education environments (WCLOUD 2012), CEUR Workshop Proceedings, vol.
945, p. 47–52. http://ceur-ws.org/Vol-945/
Locke EA, Latham GP. Building a practically useful theory of goal setting and task motivation.
Am Psychol. 2002;57(9):705–17. doi:10.1037//0003-066X.57.9.705.
Mandl H, Friedrich H. Handbuch Lernstrategien. Göttingen: Hogrefe; 2006.
McConnell D. Implementing computer supported cooperative learning. 2nd ed. London:
Routledge; 2000.
Nussbaumer A, Steiner C, Albert D. Visualisation tools for supporting self-regulated learning
through exploiting competence structures. In: Proceedings of the international conference on
knowledge management (IKNOW 2008)
Nussbaumer A, Berthold M, Lachmann P, Kiefel A, Kravcik M, Kroop S, Mikroyanidis A. ROLE
deliverable D6.1(M48): common psycho-pedagogical framework. ROLE Project; 2013. http://
www.role-project.eu/Deliverables
Nussbaumer A, Dahrendorf D, Schmitz HC, Kravcik M, Berthold M, Albert D. Recommender and
guidance strategies for creating personal mashup learning environments. J Comp Sci Inf Syst
(ComSIS). 2014;11(1).
Paulsen M. Experiences with learning management systems in 113 European institutions.
Educ Technol Soc. 2003;6(4):134–48.
Perez-Marin D, Kravcik M, Santos OC, editors. Proceedings of the international workshop on
personalization approaches in learning environments (PALE2011), CEUR Workshop Pro-
ceedings, vol. 732; 2011. http://ceur-ws.org/
Pintrich PR. Motivational and self-regulated learning components of classroom academic perfor-
mance. J Educ Psychol. 1990;82(1):33–40. doi:10.1037/0022-0663.82.1.33.
Plass JL. Supporting visual and verbal learning preferences in a second-language multimedia
learning environment. J Educ Psychol. 1998;90(1):25–36. doi:10.1037/0022-0663.90.1.25.
Popescu E, Cioiu D. eMUSE-Integrating Web 2.0 tools in a social learning environment. In:
Leung H, Popescu E, Cao Y, Lau RWH, Nejdl W, editors. Advances in web-based learning-
ICWL 2011, 10th International Conference, Hong Kong, China, December 8-10, 2011,
vol. 7048. New York: Springer; 2011. p. 41–50. doi:10.1007/978-3-642-25813-8.
Roberts MJ, Erdos G. Strategy selection and metacognition. Educ Psychol. 1993;13(3/4):259–66.
Ryan R, Deci E. Intrinsic and extrinsic motivations: classic definitions and new directions.
Contemp Educ Psychol. 2000;25(1):54–67. doi:10.1006/ceps.1999.1020.
Softic S, Taraghi B, Ebner M, Vocht L, Mannens E, Walle R. Monitoring learning activities in
PLE using semantic modelling of learner behaviour. In: Holzinger A, Ziefle M, Hitz M,
Debevc M, editors. Human factors in computing and informatics, lecture notes in computer
science, vol. 7946. Berlin: Springer; 2013. p. 74–90. doi:10.1007/978-3-642-39062-3.
Steiner C, Nussbaumer A, Albert D. Supporting self-regulated personalised learning through
competence-based knowledge space theory. Policy Fut Educ. 2009;7(6):645–61.
Treier M. Personale Voraussetzungen fur das Lernen mit Neuen Medien. Evaluation und
Gestaltung im Zusammenhand mit der Implementierung einer Bildungsplattform in einem
Konzern. Hamburg: Dr. Kovac; 2004.
Tseng J, Chu H, Hwang G, Tsai C. Development of an adaptive learning system with two sources
of personalization information. Comput Educ. 2008;51(2):776–86. doi:10.1016/j.compedu.
2007.08.002.
Waite S, Davis B. Developing undergraduate research skills in a faculty of education: motivation
through collaboration. Higher Educ Res Dev. 2006;25(4):403–19. doi:10.1080/
07294360600947426.
Weinstein CE, Husman J, Dierking DR. Self-regulation interventions with a focus on learning
strategies. In: Boekaerts M, Pintrich PR, Zeidner M, editors. Handbook of self-regulation.
Burlington: Elsevier Academic Press; 2005. p. 727–47.
Winne P, Hadwin A. The weave of motivation and self-regulated learning. In: Schunk D,
Zimmerman B, editors. Motivation and self-regulated learning: theory, research, and appli-
cations. New York: Lawrence Erlbaum; 2008. p. 297–314.
Zimmerman BJ. Becoming a self-regulated learner: an overview. Theory Into Practice. 2002;
41(2):64–70. doi:10.1207/s15430421tip4102\2.
A Multidimensional Evaluation Framework
for Personal Learning Environments
Effie Lai-Chong Law and Fridolin Wild
Abstract Evaluating highly dynamic and heterogeneous Personal Learning Envi-

ronments (PLEs) is extremely challenging. Components of PLEs are selected and
configured by individual users based on their personal preferences, needs, and
goals. Moreover, the systems usually evolve over time based on contextual oppor-
tunities and constraints. As such dynamic systems have no predefined configura-
tions and user interfaces, traditional evaluation methods often fall short or are even
inappropriate. Obviously, a host of factors influence the extent to which a PLE
successfully supports a learner to achieve specific learning outcomes. We catego-
rize such factors along four major dimensions: technological, organizational,
psycho-pedagogical, and social. Each dimension is informed by relevant theoretical
models (e.g., Information System Success Model, Community of Practice, self-
regulated learning) and subsumes a set of metrics that can be assessed with a range
of approaches. Among others, usability and user experience play an indispensable
role in acceptance and diffusion of the innovative technologies exemplified by
PLEs. Traditional quantitative and qualitative methods such as questionnaire and
interview should be deployed alongside emergent ones such as learning analytics
(e.g., context-aware metadata) and narrative-based methods. Crucial for maximal
validity of the evaluation is the triangulation of empirical findings with multi-
perspective (end-users, developers, and researchers), mixed-method (qualitative,
quantitative) data sources. The framework utilizes a cyclic process to integrate
findings across cases with a cross-case analysis in order to gain deeper insights into
the intriguing questions of how and why PLEs work.
Keywords Evaluation • Multi-method • Usability • User experience • Community

of practice • Self-regulated learning • Diffusion of innovation • Cross-case analysis •
Automated monitoring
E.L.-C. Law (*)

University of Leicester, Leicester, UK
F. Wild
The Open University, Buckinghamshire, UK

DOI 10.1007/978-3-319-02399-1_3
50 E.L.-C. Law and F. Wild
Introduction
Among others, a critical success factor in technology-enhanced learning is the

personalization of learning experience. As emphatically pointed out in the Leu-
ven/Louvain-la-Neuve Communiqué of the Bologna Process 2020, “student-cen-
tered learning requires empowering individual learners, new approaches to teaching
and learning, effective support and guidance structures and a curriculum focused
more clearly on the learner” (p.3). Personalization is also a key issue for
implementing mechanisms to foster and increase activities in informal and lifelong
learning networks. This implies a need for new technology-enhanced learning
models that start from the learners and satisfy their unique needs in order to achieve
a personalized learning experience for everyone.
Recent discussions about technologies for learning have shifted from institution-
managed learning management systems (LMS) to user-controlled social software
for learning. Indeed, the advent of Web 2.0 technologies has phenomenally
transformed the way in which users consume, communicate, collaborate, and create
information and knowledge on the Web. These technologies have underpinned the
emergent notion of Personal Learning Environment (PLE), which is characterized
by qualities such as personalization, openness, responsiveness, flexibility, share-
ability, interactivity, and sociability. PLEs can be perceived as both a technology
and a pedagogical approach (Attwell 2007; van Harmelen 2006; Johnson
et al. 2011; Johnson and Liber 2008; Schaffert and Kalz 2009) that aim to empower
students to be in charge of their own learning by selecting tools and resources to
create, organize, and package learning content, thereby meeting their personal
needs and goals (McLoughlin and Lee 2010).
Nonetheless, the high hope held for the PLE to be a key enabler for lifelong
learning is yet to be shown because the research and practice on PLE is still
evolving. Specifically, substantive claims about the power of PLE should be
grounded in relevant case studies, which, however, are limited in number and
scope (Johnson et al. 2011). The paucity of case studies and missing evidence on
the success or usefulness of PLEs can be attributed to the lack of a comprehensive
evaluation framework for PLEs. The difficulties of evaluating PLEs have been
documented (e.g., Gillet et al. 2011; Giovannella 2011). While technical
implementations have demonstrated some significant progress (see Chaps. 5 and
8 in this volume), the empirical evaluation of PLEs lacks behind. Indeed, the
development of an evaluation framework for PLEs poses several major challenges:
– PLEs are not a stable technology that can be prepared and used in a controlled
environment. In fact, PLEs do change over time and can be highly dynamic.
– PLEs integrate other technological artifacts that are designed independently
from each other and can stem from different providers. This leads to possible
(unintended) interdependencies, usability issues, and update state problems.
– PLEs are used to combine formal and non-formal learning contexts. Therefore
the purpose of using a PLE can be highly heterogeneous, rendering systematic
comparisons across different learners very difficult.
A Multidimensional Evaluation Framework for Personal Learning Environments 51
Fig. 1 Four perspectives (the TOPS model) for PLE evaluation
To tackle these challenges, mixed-method and multi-perspective evaluation

approaches are deemed relevant to address the complexity of PLE usage and its
effects on learning behaviors and learning outcomes.
Four main perspectives can be identified: technological, organizational, psycho-
pedagogical, and social (short: “TOPS”), with each being informed by specific
concepts and theories and subsuming certain methods and tools (see Fig. 1). They
are elaborated in the following with reference to related work of the ROLE project
(http://www.role-project.eu/).
The “TOPS” Model for Evaluating PLEs
In this section we delineate the individual perspectives of the TOPS model—with

specific emphasis on their respective underlying conceptual and theoretical
frameworks.
Technological Perspective
The technological perspective comprises two main aspects: utility and usability and
user experience. It is to emphasize that the user-centered design (UCD) approaches
underpin the work of PLEs, so not only end-users’ but also developers’ perspectives
should be taken into account.
Utility
Two major elements of utility to be evaluated are software and documentation,

which are discussed in detail in the following.
Software evaluation pertains to the functionality of different software compo-
nents constituting PLEs, including widgets, widget containers, the widget store,
libraries, services, tools, and the overall interoperability framework. It is essential
to evaluate how useful these components are to enable end-users to accomplish
specific tasks and goals.
As already indicated above, the strict separation of end-users from developers
can be seen as artificial (at least under the UCD approach), thus requiring an
evaluation approach to look also at developers and “power” users who engage in
customization, configuration, or even end-user-driven development (keyword:
mash-ups). Typically, such developers and power users use configuration options,
authoring tools, and APIs allowing for the mash-up of components to customize or
even create new software artifacts.
Specifically, we highlight a list of factors critical for the technical evaluation of
software, which are adopted from constructs of the Information System Success
Model (ISSM) (DeLone and McLean 2003) and the Technology Acceptance Model
(TAM) (Davis 1989; Venkatesh et al. 2003; Venkatesh and Bala 2008) (see Table 1).
Table 1 Constructs relevant to the utility evaluation of PLEs

Construct Factor
System quality Integration
Portability
Availability
Performance
Reliability
Usefulness of features
Completeness of features
Security
Information quality Usefulness of information
Completeness
Correctness
Appropriateness of presentation format
Service quality Responsiveness
Support
Feedback mechanism
Trustworthiness
Use Users, communities
Functionality accessed
Information items accessed
Duration of use
Frequency of use
An increasingly popular data collection method is automated monitoring: mon-

itoring and data logging for capturing how frequently a service or a feature has been
used and how often different significant events have occurred (Helms et al. 2000).
More specifically, raw data of system use are recorded and then aggregated for
computing measures for individual factors. For example, “mean time to failure” is a
measure for the factor “reliability” under the construct of “system quality” that can
be derived from monitoring data, including information about what and when errors
occur.
Contextual information gathering (i.e., information about the current situation
where a learner deploys specific software) is also important. Noteworthy is that
context has a technical and social aspect: which software/browser is used for
accessing a PLE, which types of data are accessed, and which people interact
with each other in a certain community. In principle, physical context information
can also be recorded if the sensor data required would be available (e.g., GPS
coordinates for spatial location). An example of including contextual data in
evaluation is a factor “Browser Compatibility,” where the number of errors
occurred related to a particular browser can be measured. Similarly, a factor
“Widget Container Interoperability” can be measured by associating errors with a
particular widget container where they occur.
Where it is possible and does not infringe privacy and security regulations, it
may often be safer to capture a broad standard range of monitoring data, especially
as capturing technologies typically require no further human effort beyond initial
setup and the setup is often already integrated into the related software. Subse-
quently, such data can be selected, refined, and processed based on the actual needs
and goals of an evaluation project.
Croll and Power (2009) provide an elaborate list of metrics that can be used for
monitoring usage of web-based technologies. Some of the key metrics relevant to
PLEs are: user-generated content, content popularity, loyalty, search effectiveness,
reflection, enrolment, conversion, and abandonment.
There are a number of ways for collecting data for these metrics. Google
Analytics are a free service to generate a comprehensive range of usage statistics
for any web-based application. Following the insertion of a small JavaScript code
snippet into a given web application, Google starts to record usage statistics
(including simple demographic features and events). Some of the key aspects that
Google can currently track are
– Visitor Tracking: Demographics, conversion, uniqueness, loyalty, etc.
– User Profile: browser, OS, screen resolution, Java availability, flash availability,
connection speed, etc.
– Events: frequency of use of specific event categories, events per visit, total
number of events.
One of the drawbacks of using analytics is the limited capability to provide data
describing how users interact with content and tools (known as attention metadata)
within their environments. Collecting contextualized attention metadata (CAM)
will enable us to infer the ways learners use technologies and tools for specific
purposes. The CAM approach proposed by Wolpers et al. (2007) supports such
tracking of attention metadata. This approach helps observe the user at the appli-
cation level, enabling association of tool usage with content-specific behavior in
context. The challenge of collecting observation data of user attention unobtru-
sively can be resolved by the CAM approach through integrating the data-capturing
process into a user’s daily working environment. This approach allows integrating
data from web applications (e.g., by mapping the Apache open log file format to
CAM) as well as from desktop applications. CAM helps track learning content
usage, analyze behavioral patterns, provide similarity measures between users, and
allow inferences about user goals. CAM data can be utilized to measure the
effectiveness of PLE technologies in providing the learner with a highly responsive
and personalized learning environment. CAM data can also be used to track and
infer self-regulatory activities for measuring the effectiveness of the psycho-
pedagogical model (Scheffel et al. 2010a, b).
All measures that cannot be derived with automatic monitoring need to be
obtained from users explicitly. The challenge is to identify appropriate techniques
for survey data acquisition with the possible lowest obtrusiveness and highest
intuitiveness for users.
For instance, a lightweight “Requirements Bazaar” approach is integrated into
the ROLE Widget Store (http://role-widgetstore.eu/) similar to other well-accepted
systems such as Google’s Android Market or the Chrome Extensions marketplace.
This is a valuable source of data since their users provide feedback on the quality of
tools, services, and widgets using means such as rating scales, and—where appro-
priate—free text comment boxes.
Documentation evaluation looks into the availability and quality of technical
documentation—a prerequisite for software to be accepted by end-users as well as
developers. To encourage developers to contribute new learning technologies by
mashing up existing software components, it is necessary to ensure that documen-
tation is correct, complete, and tailored to developers’ needs.
With regard to the development of web-based software components, developer
documentation of the infrastructure usually includes the following items:
– The set of initial documents (e.g., an overview of the underlying principles and
overarching architecture).
– The reference documentation with complete information on all supported fea-
tures, usually in the form of API documentation.
– The set of tutorials demonstrating how to use the technology for developments
on simple and useful examples.
Specifically, technical documentation should be tested by inviting developers to
practical sessions, where they are asked to use the infrastructure and accompanying
documentation to realize a small but motivating use case beyond basic tutorial
contents. In such sessions, the developers who authored the documentation can
serve as tutors to be consulted to discuss any problems arising. Such discussions can
be used as individual interviews or focus groups to collect feedback on the quality
of the software as well as documentation. This approach, however, does not scale to
large groups of developers. This is where the required alternative means such as
online tools are preferred over presence workshops.
Documentation of web-based software is usually supplemented by different
technical means for communicating with the core developers of the original tech-
nology, authors of the documentation (who often are also its developers), and
developers deploying these software artifacts. For instance, developers use online
forums to get in contact with other developers to report problems and ask for help.
Besides bug reports, such comments often contain practical questions about how to
accomplish certain tasks, thus indicating where the existing documentation could
be unclear or incomplete.
Further means to assess the utility of documentation is to directly integrate
ratings, for instance, in the form of 5-star scales, like/dislike buttons or commenting
functions, into the online documentation. In this manner, different factors from the
dimension Information Quality can be surveyed. These (and additional) features are
often already provided by software project management systems such as
SourceForge, GitHub, and the like.
Usability and User Experience
First of all, it is deemed imperative to demarcate usability from user experience

(UX)—two key concepts in the field of human–computer interaction (HCI). One
main distinction is that usability targets instrumental quality, emphasizing the
effectiveness and efficiency of task and goal attainment with interactive technolo-
gies, whereas user experience targets non-instrumental quality (e.g., aesthetics),
going beyond the traditional task-oriented focuses to address users’ affective and
emotional responses (e.g., fun, pleasure, surprise, sad, happy) to interactive tech-
nologies (e.g., Hassenzahl 2013). Hassenzahl’s (2005) oft-cited model on the
pragmatic and hedonic quality illustrates similar arguments. Despite its decade-
long history, some basic conceptual issues in UX are yet to be resolved (Law
et al. 2009; Law, van Schaik & Roto, 2014). While a deeper exploration of such
issues is beyond the scope of this chapter, here we highlight metrics and approaches
relevant to the evaluation of PLEs.
Noteworthy is that usability and user experience evaluations focus on the
interaction design of technological components underpinning PLEs, which none-
theless contribute to the holistic educative experience with PLEs (see also section
“Psycho-pedagogical Aspect”).
Usability
The usability of different technological components of PLEs (section “Utility”) is

to be evaluated based on a combination of metrics identified from the literature
(e.g., Nielsen 1994) and standards (ISO/IEC 25010:20111; ISO/IEC 9241-1102:

2006 and ISO/IEC 9241-210: 20103). The metrics are listed as follows:
– Learnability: The ability of the technology to enable users to learn with great
ease how to assemble a PLE themselves. If users find it difficult to assemble a
PLE, then the acceptance and uptake may be drastically hindered. Hence, the
assembly process for such an open learning environment should be relatively
straightforward for end-users. Some factors that enable us to ascertain
learnability are consistency of user interface design and predictable system
behavior. Learnability of PLEs is equally important for developers as for
end-users. If developers find it difficult to use PLE software, they may not be
able to create new widgets.
– Efficiency: The ability of the technology to support users to be highly productive.
Features such as consistent look and feel, consistent navigation, frequent feed-
back, and availability of templates to help them quickly assemble their environ-
ments can contribute to the overall efficiency of the PLE software.
– Memorability: The ability of the technology not to require users to reinvest time
in remembering how to use it after a period of nonuse. Closely related with
learnability, memorability can influence the uptake and usage of PLE. The key
success factor for PLE is to make the assembly process of the environment
highly intuitive, using relevant standardized visual cues.
– Error Tolerance: The ability of the technology to avoid catastrophic errors by
making users reconfirm critical actions (e.g., deleting a software component) and
to recover from errors by providing the “un-do” feature that allows users to
reverse their actions.
– Effectiveness: The ability of the technology to help users achieve their goals.
Using PLEs, if learners are able to assemble and personalize their environments
with ease, while at the same time they find the recommendations and rated/
ranked content useful for fulfilling their goal, then we can infer that the tech-
nology is effective and that learners are likely to feel satisfied. More explicit
methods are mentioned above in section “Utility.”
– Flexibility: The ability of the technology to offer a range of services so as to be
able to adapt to task changes. The ability of learners to seamlessly integrate and
use a range of web-based tools and services for assembling their learning
environments and for exporting/importing data as well as settings to other
similar technologies.
– Operability: The ability of the platform to allow users to operate and control it.
– Satisfaction: The ability of the platform to be deployed by users without dis-
comfort. It is highly subjective as compared with the other qualities listed above,
which when realized to a sufficiently large extent, can contribute to overall user
1
Systems and software engineering: Systems and software Quality Requirements and Evaluation.
2
Ergonomics of human-system interaction: Part 110: Dialogue principles.
3
Ergonomics of human-system interaction: Part 210: Human-centered design for interactive
systems.
satisfaction. Note that in addition to the system and service qualities, informa-
tion quality can play a key part in user satisfaction, according to the ISSM
(DeLone and McLean 2003).
Usability evaluation methods comprise a range of usability inspection methods,
user-based tests, and user surveys, which can be used to evaluate PLEs using the
metrics described above. Inspection methods rely on experts, whereas user-based
tests and user surveys, as the names suggest, involve end-users (an overview, see
Holzinger 2005).
Two commonly used inspection methods are heuristic evaluation and cognitive
walkthrough. For heuristic evaluations, experts examine a system based on ten
usability heuristics or principles that were originally derived from a large database
of common problems. Violating any of such principles is identified as usability
problem of which the severity is estimated so as to inform the urgency and necessity
of its being fixed (Nielsen 1994). The major advantages of this method are that it
can be applied throughout the whole development lifecycle and is, relatively, less
time-consuming. In a cognitive walkthrough, experts analyze a system’s function-
ality with a set of four questions (e.g., “Will the user notice that the correct action is
available?”) to estimate how the user would interact with the system (Lewis and
Wharton 1997). A negative response to any of the questions suggests the identifi-
cation of a usability problem.
All inspection methods, as prediction methods, are prone to false alarms and
results thereof are typically to be verified with user-based tests, such as think aloud
or field design methods and observation methods (e.g., video observation, screen
sharing, mouse tracking, eye tracking). Usability evaluation feedback is deployed
for further development of the system under scrutiny, as they can provide insights
into where and why usability requirements are not met.
Think aloud is a method that requires end-users to constantly think aloud as they
are using a system individually or collaboratively in order to understand how they
perceive the features of the user interface, identify preferences, and discover any
potential misconceptions at early design stages (Dumas and Fox 2007). The draw-
back of this method is that it can be tiring for end-users who have to focus and
behave in a rather unnatural manner by giving a running commentary on their own
actions.
Field methods are a collection of tools and techniques for conducting user
studies in context. Among others, Contextual Inquiry (Beyer and Holtzblatt 1998)
is commonly used field method in research as well as in practice. The main
advantage of such methods is that they provide a development team with data
about what and how (and why) people carry out their tasks in a given environment,
thereby enabling the production of useful and usable systems that meet people’s
needs and goals. The main disadvantage is that they are time-consuming. Nonethe-
less, such methods can be streamlined with respect to the budget available for
evaluation in a project (Wixon et al. 2002).
Furthermore, while the importance of automated monitoring techniques was
already highlighted above, methods such as CAM and Google Analytics may not
provide sufficient granularity of data to determine the usability of the PLE software.
The ability of CAM to provide granular and contextual data may be useful, but its
appropriateness may not be established unless or until a sufficient amount of data
has been collected. Apart from traditional methods mentioned above, there are two
additional methods that can be useful for small-scale (eye tracking) and large-scale
(mouse tracking) usability evaluations:
– Eye tracking measures visual attention as people navigate through websites. It is
useful in quantifying which sections of an interface are read, glanced at, or
skipped/ignored. Eye tracking is generally carried out in laboratories and at a
small scale. It can provide useful information for evaluating the effectiveness of
the learning design (Schwonke et al. 2009; van Gog and Scheiter 2010) and it
can be used to gather data after every redesign phase before large-scale rollout.
– Mouse tracking is a technique for monitoring and visualizing mouse movements
on any web interface. Mouse movements provide key data about usability issues
on a large scale, as users can be observed in their natural habitat in an unobtru-
sive and continuous manner. In most cases, a JavaScript code snippet is inserted
to track mouse movements. Privacy issues must be considered while adopting
this method. Tools like Crazyegg,4 Userfly,5 and Simple Mouse Tracking6 can
be used for this purpose. It should be mentioned that even more so than eye
tracking, data captured with this method represent only part of the story and,
hence, must be triangulated with other qualitative data to ensure completeness
and correct interpretation.
For summative usability evaluation, user surveys are deployed. They are nor-
mally administered in the final phase of a project after end-users interact with an
executable prototype. Among others, the System Usability Scale (SUS) is widely
used in research and practice, as it is simple with only ten items and standardized
with psychometric properties (Brooke 1996).
To study the usage of PLEs, it is crucial to evaluate whether the associated
services and features can help achieve learning objectives. This can be derived from
evaluation metadata such as ratings, bookmarks, tags, and comments provided by
users (Vuorikari and Berendt 2009): One important aspect here is to investigate
how the PLE usage facilitates social interactions, triggers discussions, and
improves the understanding of learning content (Mason and Rennie 2007; Farrell
et al. 2007; Rollett et al. 2007). Moreover, when it comes to learning material
recommended by the system, ratings and like/dislike evaluation metadata can help
assess unobtrusively to what extent learners deem them useful.
4
http://www.crazyegg.com/
5
http://userfly.com/
6
http://smt.speedzinemedia.com/smt/
User Experience
The literature on UX published since the turn of the millennium indicates that there
are two disparate stances on how UX should be studied (i.e., qualitative versus
quantitative) and that they are not necessarily compatible or can even be antago-
nistic. A major argument between the two positions is the legitimacy of breaking
down experiential qualities into components, rendering them to be measurable. A
rather comprehensive review on the recent UX publications (Bargas-Avila and
Hornbæk 2011) identifies the following observations: UX research studies have
hitherto relied primarily on qualitative methods; among others, emotions, enjoy-
ment, and aesthetics are the most frequently measured dimensions; the products and
use contexts studied are shifted from work to leisure and from controlled tasks to
consumer products and art; the progress on UX measures has thus been slow.
Given that UX has at least to some extent developed from usability, it is not
surprising that UX methods and measures are largely drawn from usability (Tullis
and Albert 2008). However, the notion of UX is much more complex, given a mesh
of psychological, social, and physiological concepts it can be associated with.
Among others, a major concept is emotion or felt experience (McCarthy and Wright
2004). As emotion arises from our conscious cognitive interpretations of
perceptual-sensory responses, UX can thus be seen as a cognitive process that can
be modeled and measured (Hartmann et al. 2008).
Larsen and Fredrickson (1999) discussed measurement issues in emotion
research with reference to the influential work of Ekman, Russell, Scherer, and
other scholars in this area. More recent work along this direction has been
conducted (cited in Bargas-Avila et al. 2011). These publications point to a
common observation that measuring emotion is plausible, useful, and necessary.
However, like most, if not all, psychological measurements, they are only approx-
imations (Hand 2004) and should be considered critically. Employing quantitative
measures to the exclusion of qualitative accounts of user experiences, or vice versa,
is too restrictive and may even lead to wrong implications (Law et al. 2014).
There exist a range of UX evaluation methods (e.g., Vermeeren et al. 2010). For
qualitative data, narrative or storytelling methods (e.g., Riessman 2008) are com-
monly employed. For instance, users’ short descriptions about their positive and
negative interaction experiences can be analyzed with the use of machine learning
as well as manual coding approach (e.g., Tuch et al. 2013). For quantitative data,
validated scales with good psychometric properties such as AttrakDiff2
(Hassenzahl and Monk 2010) and PANAS (Positive Affect and Negative Affect
Scale; Watson et al. 1988) are increasingly used.
Especially challenging is to operationalize a diversity of emotions, be they
positive and negative, because teasing out their nuances proves difficult. Common
methods here are self-assessment manikins and Emocards (for a summary, see
Stickel et al. 2011). It is even more demanding to measure the social aspect of
UX, which has hitherto been defined as highly individual and contextualized (Law
et al. 2009).
Organizational Aspect
With their capability for personalization and plasticity, PLEs help create a rich and
diverse learning technology ecosystem promising perpetual change and innovation.
The uptake and effects of PLEs at an organizational level can be understood in the
light of theory of Diffusion of Innovation, which is advanced by Rogers (1995):
“An innovation is an idea, practice, or object that is perceived as new by an
individual or other unit of adoption” (p.11).
Furthermore, Rogers (1995) states that the “innovation diffusion process” pro-
gresses over time through five stages: knowledge (when adopters learn about the
innovation), persuasion (when they are persuaded of the value of the innovation),
decision (when they decide to adopt it), implementation (when the innovation is put
into operation), and confirmation (when the decision is reaffirmed or rejected).
The ROLE project conducted a study to identify factors that can have an effect
on the adoption and diffusion of PLE-related technologies in organizations (Chat-
terjee et al. 2013). Table 2 presents an overview of the factors identified.
Among the main organizational factors, the outlook of the top management on
introducing technological change matters, as this particularly influences persuasion
strategies for facilitating positive decision-making in terms of PLE adoption. It is
equally important to look at how coherent or unified the views on PLEs of the key
stakeholders within the organization are. With the increasing popularity of social
media within commercial organizations, extensive use of such platforms can have
positive impacts on informing the stakeholders about key concepts and issues
around PLEs.
The top management, as per the findings of the study, is particularly interested in
the cost-effectiveness PLEs offer as compared to existing solutions in place—the
perceived cost-effectiveness thus plays a key role here for evaluation. Compatibil-
ity with the existing technical infrastructure and high learnability are other key
success factors of introducing innovation. These persuasive factors tend to act in a
push–pull mechanism (Shih 2006) before embarking on the decision-making stage.
Table 2 Potential factors influencing organizational uptake

Categories Factors
Organizational Leaderships attitude towards change
Strategic alignment
Learning culture
IT support
Innovation (PLE) Perceived cost-effectiveness
Compatibility with existing system
Perceived effort expectancy
External factors Perceived factuality
Communication channels and influence Line manager
Social networks
Once the key stakeholders within an organization are informed and persuaded about
the usefulness and utility of PLEs within their organization, the top management
may then take the two key factors into account when deciding upon the adoption of
the new learning technologies.
PLEs enable the learners to take control of their own learning depending on their
contextual needs and goals. It is therefore crucial to check whether a framework
exists that allows relating personal goals directly to organizational goals. Similarly,
the learning culture should not be dominated by didactic and trainer-facilitated
approaches, as a healthy sign of PLE adoption is that learners take control of their
own learning and managing the related technologies. It is necessary to look at the
provision of IT support (particularly in the introduction phase), when stakeholders
start using PLEs within their day-to-day activities. Another important factor that
determines the PLE adoption is its use by line managers. If line managers and senior
team do not lead by example, then the likelihood of PLE adoption can be adversely
affected.
Psycho-pedagogical Aspect
From the psychological and pedagogical perspective, the key aspects to look at are
the ability to foster self-regulated learning, the guidance and recommendation
strategy, and the facilities for reflection and monitoring. Moreover, the availability
and documentation of an activity and skill model play an important role—and how
far this is put into practice.
Self-regulated Learning
From the psycho-pedagogical perspective, effective exploitation of PLEs, which

support lifelong learning, hinges crucially upon the learner’s self-regulated learning
competence. The quality of learning outcomes varies with the extent to which
learners are capable of regulating their own learning (Steffens 2006). Self-regulated
learning approaches have been evolving since the 1970s in educational research and
practice (Efklides 2009).
Successful deployment of PLEs relies on a self-regulated learning process model
such as the following one (derived from Zimmerman 2002), where it is seen as a
learner-centric cyclic model consisting of four recurring learning phases: learner
profile information is defined or revised; learner finds and selects learning
resources; learner works on selected learning resources; and learner reflects and
reacts on strategies, achievements, and usefulness.
Note that while cognitive learning activities are rather related to actual learning
(i.e., information receiving, debating, and experimenting), meta-cognitive learning
activities are related to controlling and reflecting on one’s own learning.
With respect to the evaluation of the success and extent of self-regulated

learning, gathering data about the accuracy and usefulness of the learning process
model is crucial. It is particularly relevant to find out, whether learners can actually
follow the process model and whether they comprehend it and its implications.
Another key question is, whether the process model supports the development of
self-regulatory skills.
It should be taken into account that the process model can be applied in different
contexts and situations. For example, learners might be in a collaborative learning
situation, where they may learn together with peers. Or they may learn on their own.
In addition, the actual learning technology mix may make a difference, since
learners might use tools and widgets explicitly built to support self-regulated
learning, whereas in other cases, performance of meta-cognitive learning activities
may happen just in an implicit way (i.e., being aware of them).
One particularly useful instrument to help in the evaluation of self-directed
learning is the questionnaire. While it certainly is supportive of all other aspects
mentioned above and following below, this widely used instrument can help here in
providing structured, often numerical data. Questionnaires can be administered
without the presence of the researcher, and are often comparatively straightforward
to analyze (Wilson and McLean 1994). According to Cohen et al (2000), “Though
there is a large range of questionnaires that one can use, but there is a simple rule of
thumb to follow: the larger the size of the sample, the more structured, closed and
numerical the questionnaire may have to be, and the smaller the size of the sample,
the less structured, more open and word based the questionnaire may be” (p. 247).
Questionnaires are particularly useful when comparison across groups is required
(Oppenheim 1992).
Guidance and Recommendation Strategies
Guidance for learning in the context of PLEs depends on the situation and on who is
providing the guidance. Learners can learn in a blended learning situation with
teachers structuring the learning process. Peers can be involved in the learning
process, if learners collaborate in some way. Learners can also learn on their own
without human interaction. In the first case, teachers can provide guidance. In the
second case, peers can provide guidance either directly or indirectly (e.g., with
peers attempting to master a problem together). In all cases, guidance can also be
provided by the system through personalized recommendations.
Moreover, the scope of guidance can focus on a variety of things, including the
search for learning resources (e.g., widgets, content, or peers), the composition of a
PLE, the control over the learning process, and the improvement of self-regulation
ability. Evaluating the effectiveness and appropriateness of such guidance strate-
gies requires looking into its preconditions: the given abilities of learners are
relevant, since it depends largely on concrete skills of learners, what they can do
on their own and where they need help.
Furthermore, goals and preferences need to be investigated because the scope of

guidance depends on these factors. It should be noted that it depends on who is
delivering guidance, whether certain preconditions can be taken into account, and
to which extent. If the system provides guidance, then this is done usually in terms
of recommendations. Personalized recommendations are based on a learner model
(e.g., goals, skills, learning history, learning progress, background of a learner, and
the learner’s preferred instructional technique), which models the preconditions for
guidance.
The scope of recommendations can include concrete widgets, content resources,
peers, learning activities, and complete learning environments (i.e., sets of learning
resources). By recommending certain meta-cognitive learning activities, guidance
for self-regulated learning can be provided. In case of teacher guidance, learning
environments can be pre-configured. Especially in a blended learning situation,
teachers can support the use of the learning environment and help improve self-
regulated learning, providing further scaffolds to system guidance.
Regarding evaluation, it is important to assess the appropriateness and quality of
guidance strategies. This includes evaluating, whether the respective guidance
strategy helps learning effectively and whether the guidance provided helps over-
come difficulties. Different guidance strategies have different purposes: it requires
an evaluation of whether all purposes are actually achieved.
While of course the questionnaire (see above) can be utilized to evaluate the
success of particular guidance and recommendation facilities in their context, other
qualitative methods are suitable as well—such as focus groups, the nominal group
technique, and a Delphi study. Quasi-experiments using test collections and statis-
tical measurements are the dominant quantitative methods.
A focus group is a small group of people who get together to discuss a certain
issue given to them normally by a researcher. It usually consists of 6–10 members
and meets regularly during the lifetime of a project or in an ad hoc manner when a
need arises (Vaughan et al 1996). The technique relies on interactions among group
members. Focus groups are used to capture qualitative feedback to triangulate
findings from some other data sources.
Two other techniques, namely Nominal Group technique and Delphi technique
may be used to collect group opinion. The Nominal Group Technique was devel-
oped by Delbecq and Van de Van (1971, 1975) in the 1970s. It has been found to be
useful in improving educational programs (Jones and Hunter 1995). There is further
evidence in the literature that it was successfully used for evaluation purposes in
higher education (Nisbet and Watt 1984). Grant et al. (2003) used the technique to
determine the impact of student journals in postgraduate education.
The Delphi technique (Turoff 1970) is, like the Nominal Group technique, a
structured process, but it does not require physical proximity among participants.
The participants may be geographically dispersed and are not required to meet face
to face. Either technique may be instantiated after validation trials to gather group
data, augmenting and triangulating the monitoring or survey data.
Following the tradition of search engine evaluation, the relevance of recommen-
dations can be evaluated in the so-called quasi-experiment with the help of a
specially prepared test collection. In such a case, the learning resources (e.g.,
content, peers) are evaluated by experts or representative users; this allows com-
paring how well the recommender system performs in bringing up the most relevant
and most complete recommendation. Evaluation measures depend on the guidance
strategy: for example, recommendations fostering serendipity have much more
relaxed requirements on accuracy as compared to identifying potential peers who
are currently in a similar learning situation. An overview on possible evaluation
measures (and their application contexts) can be found in Herlocker et al. (2004).
Reflection and Monitoring
Learner information is important for guidance strategies; this can be the assessment
of a teacher, peers, or the learner herself. A teacher and peers might form an opinion
by observing, the learner can do this by self-monitoring or self-reflection, and the
system can do that by tracking the learner’s behavior and building a learner profile
(or recommending profile information). Most importantly, a mixed procedure can
be used if profile information is proposed by the system and the learner has to
modify and update it. In this case the learner is made aware of certain assessment
outcomes, which also stimulates self-reflection. As already mentioned above,
learner profile may contain information about goals, skills, learning progress, etc.
Evaluation should focus on the accuracy of this information.
While an interview can be used for the evaluation of many of the other aspects
listed above and below, it is particularly useful for the evaluation of reflection and
monitoring. An interview is a purposeful discussion between two or more people
(Kahn and Cannel 1957). One of the most distinct advantages of interview over, for
instance, questionnaires is that the researcher has personal contact with the respon-
dent and hence more control over the questions and its context. The researcher is
available to clarify confusing questions (Cohen et al 2000), which is difficult to do
with questionnaires. This same advantage, however, can also turn into a disadvan-
tage, when the researcher knowingly or unknowingly diverts the discussion and
when allowing personal bias to directly impact on outcomes. Interviews consist of a
more direct method that helps easily spot user preferences, satisfaction, and
encountered problems.
Apart from qualitative approaches, quantitative evaluation techniques utilizing
content analysis over learners’ writings are emerging, some of which using auto-
mation techniques from text mining and statistical processing. Ullmann
et al. (2013) provide an overview and a framework for the study of reflection by
hand and with the help of automation techniques; from natural language processing
as well as using crowd-sourcing of human coding on platforms such as
CrowdFlower or Amazon’s Mechanical Turk.
Activity and Skill Model
For successful deployment of PLEs, the underlying skill model is typically com-
plex, since in addition to the developed domain knowledge, self-regulated learning
and the handling of PLE services and tools have to be considered. Any PLE skill
model encompasses at least these three different kinds of skills: domain, tool, and
self-regulation skills:
– Domain skills are skills that a learner possesses, if he or she has a certain level of
expertise in a knowledge domain. For instance, the learner can explain what
percentages she estimates to have attained and, if she prefers, justifies with some
qualitative comments.
– Tool skills are defined as skills which a learner possesses, if she is able to
perform a learning activity with a learning tool in a domain context: for example,
the learner can use a tool for setting goals or can use a tool in order to retrieve
domain knowledge in a certain topic. Different learning activities with the same
tool can require different skills.
– Self-regulated learning skills imply the ability of a learner to regulate her
learning activities by herself: the learner can realistically set own goals, monitor
own progress, apply effective time management, and self-evaluate. Self-regu-
lated learning skills are skills on a meta-level and domain independent.
For the evaluation, focus should be set on documenting and subsequently
assessing accuracy and usefulness of these skill models. Methods for the assessment
of accuracy and usefulness are essentially the same as those valid for evaluating the
utility of PLE utility (particularly automated monitoring and CAM).
Social Aspect
A Community of Practice approach is an effective way of sharing knowledge. They

are usually characterized by anonymity and an addictive, but voluntary behavior,
with a strong sense of belonging (Hampton and Wellman 2001). Trust, loyalty, and
social usefulness are pertinent motivational features identified in the virtual com-
munity context.
Over the last century, a number of motivational theories were proposed (e.g.,
Maslow 1954; Herzberg 1987; Vroom 1964). At the foundation of these theories, it
is claimed, lies the suggestion that each school of thought focuses on certain factors
to the exclusion of all others—for example, reward, social needs, or psychological
growth.
A few key inferences in the context of PLEs from the motivational models are
mentioned below:
– Recognition of a range of individual needs: Learners have varying levels of
motivation depending on their needs.
– Goal alignment in the provision of materials: If a given task does not align with
the learner’s goal, then the motivation to complete the task will obviously
decrease.
– Varying incentives: Incentives can help instill a sense of achievement and
motivation to keep going. Learners will require varying levels of incentives of
different natures to keep themselves motivated (grades, peer recognition, altru-
ism, to mention just a few).
– Connectedness to community performance: Link of these incentives to perfor-
mance at an organizational or community level.
To assess the social aspect of PLEs, Kim’s (2000) application of Maslow’s
Hierarchy of Needs to online communities can be further adapted: Table 3
Table 3 Community building and motivation (extended from Kim 2000)

Offline Online communities (Kim Personal Learning
Need (Maslow) 2000) Environment
Physiological Food System access Access to PLE
technology, widget store,
user profile
Clothing Ability to maintain own identity Use of templates for
Shelter while participating in the assembly of environment
health community
Security and Protection from Protection from hacking and Data security (automated
safety crimes and war personal attacks monitoring data) and
encryption
Sense of living level playing field multi-level privacy
in a fair society maintain varying level of framework
privacy
Social Give and Belonging to the community as Share and consume tools,
receive love whole and within subgroups content, and resources
Feeling of Belongingness
belongingness Ability to collaborate
across several social
networks
Self-esteem Self-respect Contribute to community and Sharing modified PLE
get recognized templates
Ability to earn Altruism
respect from Mentoring
others Giving and receiving
feedback
Rating and ranking
Self- Develop skills Take on community role that Acquiring expert status
actualization and fulfill one’s develops new skills and opens within the community
potential new opportunities Assembly and regulation
of own learning
illustrates which constructs are relevant to the PLE evaluation from a motivational
perspective.
Clustering techniques and social network analysis (SNA) can be used to trace
whether the infrastructure supports the emergence and evolution of self-directed
communities of interest and practice (Wenger 1998). Both rely on either implicit
factors (looking at interaction and usage patterns) or explicit ones (utilizing eval-
uation metadata).
SNA originates from sociology and network analysis that is widely applied in
physics, electrical science, civil engineering, and others. In SNA, entities and
relations among them are mathematically modeled as graphs, (i.e., sets of nodes
and edges connecting them). Nodes and edges can have different semantics: for
instance, nodes can be people and edges between nodes can be based on commu-
nication between people, for example, through e-mails or chats. Edges can also be
used to denote citations of resources that peers own or create. For instance, a peer is
connected with the other one whose work he has cited. According to the Actor
Network Theory (Latour 1991), we can consider every node as an arbitrary actor,
which is not necessarily human. In this sense, it is also possible to analyze networks
consisting of users and tools, both modeled as nodes.
SNA is a basis for assessing social learning and the interaction with tools used in
learning (Klamma 2010). It helps discover information about social relationships.
Based on this, it allows inspecting social presence of learners within their commu-
nities: for example, it helps in evaluating which roles learners adopt or how their
positions evolve over time, positively as well as negatively.
Since 1967 with the discovery of the small world network phenomenon (Milgram
1967), the heterogeneity of networks has been examined intensively. Newman (2003)
showed that in scale-free networks, connections between nodes are distributed
unequally with a certain probability. While most of the nodes have few connections,
there exist a few nodes exhibiting a large number of connections. The connectivity of a
graph representing a network informs about robustness and cohesiveness of the
network (Brandes and Erlebach 2005). Freeman (1979) also pays attention to centrality
measures that help us to reveal special roles of network nodes. Moreover, brokerage
phenomena can hardly be defined without the application of SNA (Barabási 2007).
Considering the irregularity of peer connections of networks, Newman and Girvan
(2004) developed one of the clustering algorithms, which find groups of network nodes
that are densely connected to each other but sparsely connected with other nodes.
Networks typically consist of several groups of learners communicating with
each other and with other groups. SNA techniques and clustering allow unveiling
the structure underlying such a network. For example, networks can include groups
of learners that have connections only to leaders of groups, but don’t have com-
munications with other groups.
SNA techniques allow following behaviors of learners within a time frame by
examining network centrality measures, which reveal expertise or presence of a
learner within a network. This method of evaluation may show us how learners
evolve in their communities over time: do they become experts or brokers of
information from one to the other community or do they lose their position and
lock themselves in a community closed from communication?
In practice, SNA requires the availability of data containing information on the

nodes, i.e., people, groups of people or even tools, and on the edges, i.e., relations
between nodes. One possible source of input for SNA can be the raw monitoring
data. Here, different kinds of interaction between users are captured.
The Unified PLE Evaluation Framework
Based on the TOPS model and the background literature reviewed above, we
propose an integrated evaluation framework for PLEs. Specifically, the framework
incorporates major dimensions with a gradual progression from the individual to
community focus. Figure 2 lists the key dimensions (and its aspects) of this
evaluation framework and shows how they relate to each other: the framework is
organized in three circles from the inner Technological one, which lays the cornerstone
of PLEs, through the middle Psycho-pedagogical circle, which addresses individual
user’s needs and goals, to the outer Organizational and Social circle, which brings in
the social and organizational factors relevant to the exploitation of PLEs.
The constructs highlighted within the three circles are high-level concepts,
which should be translated into low-level variables, selected from the review
brought forward in the previous sections. Operationalizing and estimating such
variables with particular techniques and tools leads to results, which can somehow
and somewhat account for the extent to which PLEs successfully enable users to
attain their learning goals. For instance, the construct usability is translated into two
Fig. 2 The “TOPS”

integrated evaluation
framework for PLEs
metrics—effectiveness and efficiency— which can be measured in terms of number

and type of errors and in the time to complete a specific task with a PLE.
Nonetheless, not every construct can be operationalized in a straightforward
manner. Indeed, it is a challenging task to develop structural and measurement
models, where factors and measures are orthogonal in the ideal case, but at least
exhibit a lowest degree of collinearity. Statistical analysis techniques such as
correlation, regression, and factor analysis deem useful to sample, validate, and
tune the underlying model in early evaluation runs in order to maximize validity
throughout the overall process.
Table 4 relates these three sets of dimensions (with their main criteria) to the
methods reviewed in the previous sections. Each of the dimensions (technological,
psycho-pedagogical, and organizational/social) can be broken down into its main
groups of constructs, as listed in the first column. The second column provides the
selection of methods that have been used in the past and that we deem most
appropriate for their study.
Table 4 Evaluation dimensions and recommended methods

Group of
Dimension constructs Key methods
Technological Openness Questionnaires
Responsiveness Interviews (incl. storytelling)
Security Desk research (documentation)
Scalability Nominal group, Delphi
Documentation Inspection methods
Interoperability User tests
Accessibility Monitoring data (incl. web analytics, CAM)
Availability, Observation
reliability
Quality (content Unit tests
and UI)
Effectiveness user-based evaluation: behavioral measures, observa-
Efficiency, tions, and questionnaires
satisfaction
Enjoyment
Organizational/ Trust Questionnaires
Social Social usefulness Interviews
New connections Desk research
Sharing Monitoring data
Privacy Social network analysis
Clustering
Psycho- Meta-cognitive Questionnaires
pedagogical Motivation Interviews
Behavioral Nominal group, Delphi, focus group
Recommendations Monitoring data
Observation
Quasi-experiments (relevance accuracy)
The PLE evaluation is ideally conducted in cycles of planning, actual evaluation,

and reflection on results. A useful vehicle for this can be found in form of case
studies and—concluding the final cycle—a cross-case analysis. Case study is a
generic term for the investigation of an individual group or a phenomenon (Bogdan
and Biklen 2006). Case studies are often used for exploratory research, but the
technique can be varied and adapted to include the multi-method mix proposed
above for the unified PLE evaluation framework.
While the techniques used may vary, the distinguishing feature of case study is
the assumption that human systems develop a characteristic wholeness or integrity
and are not simply a loose collection of traits. This approach enables researchers to
investigate a given phenomenon to a much greater depth, bringing out the interde-
pendencies of parts and emerging patterns. Besides, case study has the potential to
accommodate the value context of the enquiry, is flexible to accommodate unan-
ticipated events, does not attempt to generalize, and admit the problems of
researcher bias in various ways (Nisbet and Watt 1984). Nonetheless, the inability
to accommodate re-observation is a major cause of concern.
The final cycle of the cyclic evaluation process depicted above in Fig. 3 can then
be concluded with the cross-case analysis. A cross-case analysis is “a qualitative,
inductive, multi-case study that seeks to build abstractions across cases” (Merriam
1998, p.195). It is used to identify and compare patterns of similarities and
differences across individual cases resulting in meaningful connections. Most
importantly it empowers all stakeholders to access new knowledge from a rich
holistic point of view (Khan and van Wynsberghe 2008).
There are two well-known techniques to carry out cross-case analysis, namely,
variable- and case-oriented approaches (Ragin 2004). There are other techniques as
Fig. 3 Evaluation cycle for PLEs

well but are generally derived from the aforementioned ones. The variable-oriented
technique focuses on comparison of identified variables across cases in order to
delineate causal relationships. The case-oriented approach enables researchers to
make sense of causal similarities between different cases by comparing them using
visualization techniques such as stacking cases (Miles and Huberman 1994),
thereby enabling the identification of new social phenomenon.
There are a number of ways in which case-oriented cross-case analysis could be
carried out, namely, most different design (Przeworski and Teune 1982), typolo-
gies, multi-case methods (Smith 2004), and process tracing (George and Bennett
2005). The first two are of particular interest for PLE. The aim for adopting cross-
case analysis for studying the implementation of PLEs across settings is to identify
similarities in a diverse set of cases, which is what most different design offers.
Additionally clustering of cases might also be relevant to identify and compare
patterns and process pathways to seek typological regularity. We recommend the
adoption of an iterative case study design with multi-method data collection to
triangulate empirical findings. Cross-case analysis should be performed towards the
end of a series of evaluations to obtain a holistic view on the outcomes of deploying
PLEs (cf. Fig. 3).
General Discussion: Qualitative Versus Quantitative
In the foregoing sections we present an array of quantitative and qualitative

methods for data collection and analysis. The selection of a particular type of
method depends on individual researchers’ assumptions, values, and expertise.
Some researchers defy the value of quantitative data with the argument that
numbers cannot tell us anything, insisting on capturing solely qualitative data. Any
method fundamentalism is wrong, not least in the light of a postulate for a wide
repertoire of research skills among researchers. Still such standpoint is often found
in practice, particularly by those critics instigating methodological discussions with
the aim to dismantle or even discredit a particular piece of quantitative work they do
not agree with.
It is in our opinion, however, not that simple: Methods cannot be differentiated
into good and bad, and if a particular method fails to provide results (or even more
often: results beyond tautologies), then this probably says more about their com-
petent handling, rather than their validity or reliability. Exceptions prove the rule, of
course.
In our view, there are two aspects to consider that influence methodological
choices. First, it all depends on why the evaluation is needed, what the goal of the
evaluation is, and who the recipient of the evaluation data is. For example, if the
target is to feed back into psycho-pedagogical or technological development,
qualitative means can provide deeper insights on what has gone wrong, what
works, and what leaves room for improvement. Moreover, qualitative methods
bear the potential to discover, why this is the case.
Furthermore, which approach to adopt depends on the phase of a research study.

Qualitative approaches are particularly useful for exploring a topic and its phenom-
ena in their context. They help in forming hypotheses and build understanding.
Once such understanding is reached, however, more targeted questions can be
posed. Also, if a phenomenon or an application is potentially relevant to a larger
number of people, then it is well justified to conduct a quantitative follow-up to see
if the qualitative findings, suspected dependencies, effects, and other observations
hold when scaling out. Qualitative methods do not scale very well, which can pose a
problem when the target is to, for instance, to assess the effects of an intervention on
a full university, an entire company, or the general population.
This chapter aims to support researchers in determining which method they
need, depending on purpose (“TOPS”) and phase (from case-to-case to cross-case).
It provides a rich repertoire of different methods for the multi-method, multi-
perspective mix, and it helps in combining the strength of different approaches
into a unified evaluation.
As can be seen from the review of the methodological state of the art, the
frontiers in technology-enhanced learning are much more complex than the mere
differentiation of quantitative and qualitative suggests: “mediated” observation
using monitoring data, pictogram-based methods for affect measurement, quasi-
experiments for relevance evaluation, and the like start blurring these boundaries
and start claiming their own place in the standard canon of methods.
It is worth mentioning one class of methods listed in the chapter in particular, as
it stands out through the paucity of research in the area of PLEs: While emotions
and affects can play a critical role in influencing a learner’s motivation to engage in
technology-enhanced learning activities, this experiential aspect tends to be not
only overlooked, but also under-researched.
At the turn of millennium, the psychological research on emotions has been
rekindled, thanks to the work of psychologists such as Klaus Scherer (2005;
“emotion wheel”) and James A. Russell (2003“core affect”). Coincidentally, this
resurgence of interest in emotions and affects has resonated with the shift of
emphasis in HCI around the same time, moving from cognitivist-behavioral per-
formance-based usability to phenomenological-reflective experience-oriented user
experience (UX) (Law et al. 2009) .
Alongside with this change of emphasis is the revived tension about the relative
importance of qualitative and quantitative methods. This issue is actually an
age-old debate in the realm of measurement theory. In brevity, some UX
researchers argue that experience is holistic and cannot be reduced into components
to be measured; any attempt to put down a number to infer the type or intensity of an
emotion is methodologically flawed and inherently meaningless. In contrast, some
other UX researchers believe that the process of experiencing/experienced emo-
tions can be modeled like cognitive processes and thus they are measurable. These
arguments have significant implications to the selection of evaluation methods for
assessing the impact of interacting with technologies (Law et al. 2014).
Above all, putting aside the issue about the quantifiability of user experience, the
main point we want to stress is the high relevance of emotions and affects to the
design and evaluation of learning environments. Both positive (e.g., fun, pleasure,
engaged, liberating) and negative (e.g., anxious, defeated, frustrated, fear) emotions
can substantially shape the effectiveness of any type of learning situations, includ-
ing PLEs. Consequently, due attention should be heeded to this overlooked expe-
riential aspect.
Conclusion and Future Work
Developing an evaluation framework for PLEs is challenging, since technological,

organizational, psycho-pedagogical and social aspects need to be considered in an
integrated manner and with a diverse set of stakeholder perspectives being taken
into account.
Our attempt was to propose a unified framework encompassing the main valid
constructs (derived from relevant theoretical models), yet at the same time provid-
ing a flexible and adaptive methodology that is capable of accommodating the
changes that are inevitable in an emerging field.
In order to achieve this, we have elaborated an integrated framework that is by
nature case study based and follows a multi-method approach. Furthermore, we
recommended concluding the cyclic evaluation with a cross-case analysis in order
to consolidate data from different contexts so as to establish a holistic view.
A number of metrics and possible methods have been identified and located in
the proposed unified framework. The metrics, criteria, methods, techniques, and
tools proposed are subjected to further refinement and improvement. A process
model ensures the possibility to do so in a well-defined manner.
Obviously, more research efforts are called for to investigate the complex
phenomenon of PLE—and this contribution provides the methodological basis on
which such future endeavors can be built.
Acknowledgements The research leading to the results presented in this chapter has received
funding from the European Community’s Seventh Framework Programme (FP7/2007–2013)
under grant agreement no. 231396 (the ROLE project) and no. 318329 (the TELL-ME project).
The authors would like to express their gratitude to the partners who have been involved in the
related research work during the course of ROLE and TELL-ME.
References
Attwell G. Personal learning environments—the future of eLearning? eLearning papers. 2007:2.

http://www.elearningpapers.eu/index.php
Barabási A-L. From network structure to human dynamics. IEEE Contr Syst Mag. 2007;27(4):33–
42.
Bargas-Avila JA, Kasper Hornbæk K. Old wine in new bottles or novel challenges: a critical
analysis of empirical studies of user experience. In: Proc. CHI 2011; 2011. p. 2689–98.
Beyer H, Holtzblatt K. Contextual design: Defining customer-centered systems. San Francisco:
Morgan Kaufmann; 1998.
Bogdan RC, Biklen SK. Qualitative research for education: an introduction to theory and methods.
5th ed. Needham Heights: Allyn & Bacon; 2006.
Brandes U, Erlebach T. Network analysis: methodological foundations. LNCS 3418. Berlin:
Springer; 2005.
Brooke J. SUS: A quick and dirty usability scale. In: Jordan PW et al., editors. Usability evaluation
in industry. Boca Raton: CRC Press; 1996. p. 189–94.
Chatterjee A, Law EL-C, Mikroyannidis A, Owen G, Valesco K. Personal learning environments
in the workplace: an exploratory study into the key business decision factors. International
Journal of Virtual and Personal Learning Environments (IJVPLE), 2013;4(4):44–58.
Cohen L, Manion L, Morrison K. Research methods in education. 4th ed. London: Routledge;
2000.
Croll A, Power S. Complete web monitoring. Sebastopol: Oreilly; 2009.
Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information
technology. MIS Quart. 1989;13(3):319–40.
Delbecq AL, van de Ven AH. Nominal versus interacting group processes for committee decision-
making effectiveness. Acad Manage J. 1971;14:203–12.
Delbecq AL, van de Ven AH, Gustafson DH. Group techniques for program planners. Glenview:
Scott Foresman; 1975.
Delone WH, McLean ER. The DeLone and McLean model of information systems success: a
ten-year update. J Manage Inform Syst. 2003;19(4):9–30.
Dumas JS, Fox JE. Usability testing: Current practice and future directions. In: Sears A, Jacko JA,
editors. The human-computer interaction handbook: fundamentals, evolving technologies and
emerging applications. 2nd ed. Boca Raton: CRC Press; 2007. p. 1129–50.
Efklides A. The role of metacognitive experiences in the learning process. Psicothema. 2009;21
(1):76–82.
Farrell S, Lau T, Nusser S, Wilcox E, Muller M. Socially augmenting employee profiles with
people-tagging. In: Proceedings of the 20th annual ACM symposium on user interface software
and technology (UIST ‘07); 2007. p. 91–100.
Freeman LC. Centrality in social networks: conceptual clarification. Soc Netw. 1979;1(3):215–39.
George AL, Bennett A. Case studies and theory development in the social sciences. Cambridge:
MIT Press; 2005.
Gillet D, El Helou S, McCluskey A. Evaluating agile PLE enablers. In: Proceedings of the PLE
conference 2011, 11–13 July 2011, Southampton; 2011. http://journal.webscience.org/655/
Giovannella C. Virtual learning places: a perspective on future learning environments and
experiences. In: Law EL-C, Mödritscher F, Wolpers M, Gillet D, editors. Proceedings of the
1st workshop on exploring fitness and evolvability of personal learning environments; 2011.
p. 10–17. http://ceur-ws.org/Vol-773/
Grant A, Berlin A, Freeman GK. The impact of a student-learning journal: a two-stage evaluation
using the nominal group technique. Med Teach. 2003;25:659–61.
Hampton K, Wellman B. Long distance community in the network society contact and support
beyond Neville. Am Behav Scient. 2001;45(3):476–95.
Hand DJ. Measurement theory and practice. Chichester: Wiley-Blackwell; 2004.
Hartmann J, De Angeli A, Sutcliffe A. Framing the user experience: information biases on website
quality judgment. In: Proc. CHI’08; 2008. p. 855–64.
Hassenzahl M. The thing and I: understanding the relationship between user and product. In:
Blythe M, Overbeeke C, Monk AF, Wright PC, editors. Funology: from usability to enjoyment.
Dordrecht: Kluwer Academic; 2005. p. 31–42.
Hassenzahl M. User experience and experience design. In: Soegaard M, Dam RF, editors. The
encyclopedia of human-computer interaction. 2nd ed. Aarhus: The Interaction Design
Foundation; 2013. http://www.interaction-design.org/encyclopedia/user_experience_and_

experience_design.html
Hassenzahl M, Monk A. The inference of perceived usability from beauty. Hum Comput Interact.
2010;25(3):235–60.
Helms J, Neale DC, Isenhour PL, Carroll JM. Data logging: higher-level capturing and multi-level
abstracting of user activities. In: Proceedings of the 40th annual meeting of the Human Factors
and Ergonomics Society. Santa Monica: Human Factors and Ergonomics Society; 2000.
p. 303–6.
Herlocker J, Konstan J, Terveen L, Riedl J. Evaluating collaborative filtering recommender
systems. ACM Trans Inform Syst. 2004;22(1):5–53.
Herzberg F. One more time—how do you motivate employees? Harv Bus Rev. 1987;65(5):109–
20.
Holzinger A. Usability engineering methods (UEMS) for software developers. Commun ACM.
2005;48(1):71–4.
Johnson M, Liber O. The personal learning environment and the human condition: from theory to
teaching practice. Interact Learn Environ. 2008;16(1):3–15.
Johnson M, Griffiths D, Wang M. Positioning theory, roles and the design and implementation of
learning technology. J Univ Comput Sci. 2011;17(9):1329–46.
Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311.
Kahn RL, Cannel C. The dynamics of interviewing. New York: Wiley; 1957.
Khan S, van Wynsberghe R. Cultivating the under-mined: cross-case analysis as knowledge
mobilization. Forum Qual Soc Res. 2008;9(1). http://www.qualitative-research.net/index.
php/fqs/index
Kim AJ. Community building on the web: secret strategies for successful online communities.
London: Addison Wesley; 2000.
Klamma R. Emerging research topics in social learning. In: Dirckinck-Holmfeld L, Hodgson V,
Jones C, de Laat M, McConnell D, Ryberg T, editors. Proceedings of the 7th International
conference on networked learning 2010, Aalborg, Denmark; 2010. p. 224–31.
Larsen RJ, Fredrickson BL. Measurement issues in emotion research. In: Kahneman D, Diener E,
Schwarz N, editors. Well-being. New York: Sage; 1999.
Latour B. Technology is society made durable. In: Law J, editor. A sociology of monsters: essays
on power, technology and domination. London: Routledge; 1991. p. 103–31.
Law EL-C, Roto V, Hassenzahl M, Vermeeren A, Kort J. Understanding, scoping and defining
user experience: a survey approach. In: Proceedings of CHI 2009; 2009. p. 719–28.
Law EL-C, Van Schaik P, Roto V. Attitudes towards the measurability of user experience. Int J
Hum Comput Studies 2014;72(6):526–541.
Lewis C, Wharton C. Cognitive walkthroughs. In: Helander M, Landauer TK, Prabhu P, editors.
Handbook of human-computer interaction. 2nd ed. Amsterdam: Elsevier; 1997. p. 717–32.
Maslow AH. Motivation and personality. New York: Harper & Row; 1954.
Mason R, Rennie F. Using Web 2.0 for learning in the community. Internet Higher Educ. 2007;10
(3):196–203.
McCarthy J, Wright P. Technology as experience. Cambridge: MIT Press; 2004.
McLoughlin C, Lee MJW. Personalised and self regulated learning in the Web 2.0 era: interna-
tional exemplars of innovative pedagogy using social software. Australas J Educ Technol.
2010;26(1):28–43.
Merriam SB. Qualitative research and case study applications in education. San Francisco: Jossey-
Bass; 1998.
Miles MB, Huberman M. Qualitative data analysis: an expanded sourcebook. Thousand Oaks:
Sage; 1994.
Milgram S. The small world problem. Psychol Today. 1967;1:60–7.
Newman MEJ. The structure and function of complex networks. Siam Rev. 2003;45(2):167–256.
Newman ME, Girvan M. Finding and evaluating community structure in networks. Physical Rev
E. 2004;69(2):026113.
Nielsen J, editor. Usability engineering. Morgan: Kaufmann; 1994.

Nisbet J, Watt J. Case study. In: Bell J, Bush T, Fox A, Goodey J, Goulding S, editors. Conducting
small-scale investigations in educational management. London: Harper & Row; 1984. p. 79–
92.
Oppenheim AN. Questionnaire design, interviewing and attitude measurement. London: Pinter;
1992.
Przeworski A, Teune H. The logic of comparative social inquiry. Malabar: Robert E. Krieger;
1982.
Ragin CC. Turning the tables: how case-oriented research challenges. In: Brandy HE, Collier D,
editors. Rethinking social inquiry: diverse tools, shared standards. Lanham: Rowman &
Littlefield; 2004. p. 123.
Riessman CK. Narrative methods for the human sciences. Los Angeles: Sage; 2008.
Rogers EM. Diffusion of innovations. In: Stacks DW, Salwen MB, editors. An integrated approach
to communication theory and research, vol. 65. New York: Free Press; 1995. p. 519.
Rollett H, Lux M, Strohmaier M, Dosinger G, Tochtermann K. The Web 2.0 way of learning with
technologies. Int J Learn Technol. 2007;3(1):87–107.
Russell JA. Core affect and the psychological construction of emotion. Psychol Rev. 2003;110
(1):145.
Schaffert S, Kalz M. Persönliche Lernumgebungen: Grundlagen, Möglichkeiten und Herausfor-
derungen eines neuen Konzepts. In: Wilbers K, Hohenstein A (Hrsg.), Handbuch E-Learning
(Gruppe 5, Nr. 5.16). Köln: Deutscher Wirtschaftsdienst (Wolters Kluwer Deutschland),
27, Erg.-Lfg; 2009. p. 1–24.
Scheffel M, Beer F, Wolpers M. Analysing contextualized attention metadata for self-regulated
learning. In: Proc. 2nd Int. Conference on computer supported education, Valencia, Spain,
April; 2010a.
Scheffel M, Beer F, Wolpers M. Analyzing contextualized attention metadata with rough set
methodologies to support self-regulated learning. In: Proc. 10th IEEE International conference
on advanced learning technologies (ICALT 2010), July, Sousse, Tunesia; 2010b.
Scherer KR. What are emotions? And how can they be measured? Soc Sci Inform. 2005;44
(4):695–729.
Schwonke R, Berthold K, Renkl A. How multiple external representations are used and how they
can be made more useful. Appl Cogn Psychol. 2009;23(2009):1227–43.
Shih H. Technology-push and communication-pull forces driving message-based coordination
performance. J Strat Inform Syst. 2006;15(2):105–23.
Smith AD. Knowledge management strategies: a multi-case study. J Knowl Manage. 2004;8(3):6–
16.
Steffens K. Self-regulated learning in technology-enhanced learning environments: lessons of a
European peer review. Eur J Educ. 2006;41(3/4):353–79.
Stickel C, Holzinger A, Felfernig A. Measuring emotions: towards rapid and low cost methodol-
ogies. In: Proc. RecSys‘11 workshop on human decision making in recommender systems
(Decisions@RecSys‘11), Chicago; 2011.
Tuch AN, Trusell R, Hornbæk K. Analyzing users’ narratives to understand experience with
interactive products. In: Proc. CHI 2013; 2013. p. 2079–88.
Tullis T, Albert W. Measuring the user experience. San Francisco: Morgan Kaufman; 2008.
Turoff M. The policy Delphi. J Technol Forecast Soc Change. 1970;2(2).
Ullmann T, Wild F, Scott P. Reflection—quantifying a rare good. In: Proceedings of the 3rd
workshop on awareness and reflection in technology-enhanced learning; 2013. http://ceur-ws.
org/Vol-1103/
van Gog T, Scheiter K. Eye tracking as a tool to study and enhance multimedia learning. Learn
Instruct. 2010;20(2):95–9.
van Harmelen M. Personal learning environments. In: Proceedings of the sixth IEEE international
conference on advanced learning technologies; 2006. p. 815–6.
Vaughan S, Schumm JS, Sinagub J. Focus group interviews in education and psychology.
Thousand Oaks: Sage; 1996.
Venkatesh V, Bala H. Technology acceptance model 3 and a research agenda on interventions.
Decis Sci. 2008;39:273–315.
Venkatesh V, Morris MG, Davis GB, Davis FD. User acceptance of information technology:
toward a unified view. MIS Quart. 2003;27(3):425–78.
Vermeeren APOS, Law EL-C, Roto V, Obrist M, Hoonhout J, Väänänen-Vainio-Mattila K. User
experience evaluation methods: current state and development needs. In: Proc NordiCHI;
2010. p. 521–30.
Vroom VH. Work and motivation. New York: Wiley; 1964.
Vuorikari R, Berendt B. Study on contexts in tracking usage and attention metadata in multilingual
technology enhanced learning, GI Jahrestagung; 2009. p. 1654–63.
Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and
negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54(6).
Wenger E. Communities of practice: learning, meaning, identity. Cambridge: Cambridge Univer-
sity Press; 1998.
Wilson N, McLean S. Questionnaire design: a practical introduction. Newtown Abbey: University
of Ulster Press; 1994.
Wixon DR, Ramey J, Holtzblatt K, Beyer H, Hackos J, Rosenbaum S, Page C, Laakso SA, Laakso
K. Usability in practice: field methods evolution and revolution. In: Extended abstracts of
CHI‘2002; 2002. p. 880–4.
Wolpers M, Najjar J, Verbert K, Duval E. Tracking actual usage: the attention metadata approach.
Educ Technol Soc. 2007;10(1):106–21.
Zimmerman BJ. Becoming a self-regulated learner: an overview. Theory into Practice. 2002;41
(2):64–70.
Case Study 1: Using Widget Bundles
for Formal Learning in Higher Education
Helmut Vieritz, Carsten Ullrich, Erik Isaksson, Hans-Christian Schmitz,

Bodo von der Heiden, Kerstin Borau, Ruimin Shen, Matthias Palmér,
Thomas Lind, and Mikael Laaksoharju
Abstract Formal learning in higher education creates its own challenges for
didactics, teaching, technology, and organization. The growing need for well-
educated employees requires new ideas and tools in education. Within the ROLE
project, three personal learning environments based on ROLE technology were
used to accompany “traditional” teaching and learning activities at universities. The
test beds at the RWTH Aachen University in Germany, the School of Continuing
Education of Shanghai Jiao Tong University in China, and the Uppsala University
in Sweden differ in learning culture, the number of students and their individual
background, synchronous versus distant learning, etc. The big range of test beds
underlines the flexibility of ROLE technology. For each test bed, the learning
scenario is presented and analyzed as well as the particular ROLE learning envi-
ronment. The evaluation methods are described and the research results discussed
in detail. The learned lessons provide an easy way to benefit from the ROLE
research work which demonstrates the potential for new ideas based on flexible
e-learning concepts and tools in “traditional” education.
H. Vieritz (*) • B. von der Heiden

RWTH Aachen University, Aachen, Germany
e-mail: [email protected]; [email protected]
C. Ullrich • R. Shen
Shanghai Jiao Tong University, Shanghai, China
E. Isaksson
Royal Institute of Technology, Stockholm, Sweden
H.-C. Schmitz
Institut für Deutsche Sprache, Mannheim, Germany
K. Borau
School of Continuing Education of SJTU, Shanghai, China
M. Palmér • T. Lind • M. Laaksoharju
Uppsala University, Uppsala, Sweden
e-mail: [email protected]; [email protected]; [email protected]

DOI 10.1007/978-3-319-02399-1_4
80 H. Vieritz et al.
Keywords Higher education • Personal Learning Environment • Widget bundle •

Formal learning • Distance learning • Blended learning
Introduction
This chapter focuses on using Personal Learning Environments (PLEs) in formal

learning in higher education (HE). Here, formal learning means that the PLE and its
widget bundles support the established, “traditional” way of teaching in a lecture.
The teaching and learning activities are not newly created and centered on the PLE,
but the PLE extends the existing teaching context and provides additional activities
within. Therefore, the primary audience of PLEs consists of teachers instead of
students. While this might sound as a contradiction to the paradigm of personal
learning, we argue that a valuable goal of ROLE technology is to increase the range
of interactive and social learning opportunities.
In the ROLE project, three test beds served to explore such a setting:
• RWTH Aachen University, whose department of mechanical engineering is
ranked at 17th in the world (best in Germany) by the QS University Subject
Ranking in 2012.
• The School of Continuing Education (SOCE) of Shanghai Jiao Tong University
(SJTU), a blended learning institution whose students are young, working adults
who study part-time.
• Uppsala University, Sweden’s oldest university founded in 1477, which has got
a long tradition of distance education.
Albeit all three test beds are placed within higher education, they cover quite
different contexts. For instance, SOCE is located in China. With its approach to
teaching and learning based on the Confucian tradition (Zhang 2007), it is quite
different to RWTH as an example of a Western university. Also, SOCE students
study part-time, most of them have a job and family, while RWTH students are
younger full-time students. Furthermore, SOCE is a blended institution, where a
significant part of teaching and learning takes place online, while RWTH is a
traditional on-campus university. In contrast, while Uppsala University is also for
the most part an on-campus university, it offers a wide selection of distance courses
with very limited or no on-campus participation required. It proved particularly
interesting to investigate what different forms ROLE technology could take in these
settings, and it speaks of its flexibility that this was possible at all. Last but not least,
the number of participants in the test bed classes differ from 20 (Uppsala Scenario)
and 250 (SOCE Scenario) to 1,600 (RWTH Scenario). Growing numbers of
students combined with limited resources for teaching are often an important
motivation to search for a better support by e-learning tools.
The chapter starts with a review of related work, followed by the description of
the test beds in separate sections. In each section, we describe the learning context,
the tools and bundles employed, the most relevant evaluations we performed and
Case Study 1: Using Widget Bundles for Formal Learning in Higher Education 81
the lessons we learned. We end this chapter with a brief conclusion that summarizes
the main differences and similarities regarding the usage of ROLE technology in
the three test beds.
Related Research
The presented approach addresses various recent research issues such as teaching
and learning in large classes as well as using cloud services and Web 2.0 applica-
tions for e-learning support.
The usage of PLE technology has been investigated by a few studies, albeit in
small-scale environments. Blees and Rittberger (2009) describe the usage of a
learning environment assembled from different Web 2.0 services in a course on
“Social Software.” The 13 participants were familiar with Web 2.0 technology and
rated the usage of the service relatively high. The 26 case studies reported by
Minocha in (2009) mainly cover studies where students worked with a single Web
2.0 service integrated into a PLE. Law and Nguyen-Ngoc (2008) present a social
network and a content analysis of interactions in a collaborative learning environ-
ment. Their data show that some students profit from such environments, but not all
students. The challenge of teaching large classes has been a research issue for many
years (cf. Leonard et al. 1988; Knight and Wood 2005). The more technical
background of building e-learning tools from Web 2.0 components is being
discussed in Palmér et al. (2009). The approach uses six dimensions for the
mapping of Web 2.0 applications to personalized learning environments. The
capabilities of ROLE-based cloud learning services are investigated in Rizzardini
et al. (2012). The evaluation shows that a cloud-based learning support with ROLE
environments is possible but the learners may need introduction and time to be
familiar with interactive e-learning tools. The particular aspect of navigation
guidance for learning questions in Java programming is discussed in Hsiao
et al. (2010).
While these studies shed light on specific questions regarding of PLE, no prior
work has investigated how a single PLE platform can be adapted to suit the needs of
different formal higher learning scenarios and how it performs in such scenarios
over long periods of time and with significant number of users. More specifically,
the case studies described in this chapter add to the mentioned evaluations insofar
they investigate (1) the use of learning environments that contain components apart
from Web 2.0 tools (in a narrower sense), (2) with large user groups (3) of both
teachers and learners (4) from different cultural contexts. The results are ultimately
relevant for ROLE-based environments but can easily be transferred to other kinds
of environments and systems.
RWTH Aachen University: ROLE for Full-Time Students

in Large Classes
Large classes at universities create their own challenges for teaching and learning.
Audience feedback is lacking. Individual needs of students are often hard to address
sufficiently. At RWTH Aachen University, a ROLE-based knowledge map learning
tool was developed and embedded in the context of a large class course for computer
science in mechanical engineering. The objective of this PLE was to support
individual learning of students during exam preparation. Theme-based exercises
have been developed and evaluated. The tool was grounded in the notion of self-
regulated learning (SRL) with the goal of enabling students to learn independently.1
Learning Scenario
The Institute of Information Management in Mechanical Engineering (IMA) of

RWTH Aachen University offers a lecture about computer science in mechanical
engineering, which was attended by 1,600 students in 2012 (see Fig. 1). The lecture
Fig. 1 Lecture for computer science in mechanical engineering given in the RWTH auditorium
maximum (made by David Emanuel)
1
Parts of the content and information of this section have already been published by Vieritz
et.al. (2013).
is part of the curriculum for the bachelor degree in mechanical engineering (second
semester) and business engineering (fourth semester).
In 2012, the lecture focused on object-oriented software development with Java
and on software engineering (for details see Ewert et.al. 2011). The lecture is
accompanied by a programming lab, a group exercise, and exam preparation
courses. In the lab, the students are taught to program Lego NXT Mindstorms
robots with Java. They are working in small teams of two students in problem-based
learning scenarios. They were requested to program a robotic gripper inspired by
industrial robots.
The robots simulated pick-and-place machines (P&Ps) as they are used for
surface-mount devices (SMDs). The resemblance to industrial robots was meant
to result in a better understanding of mechanical engineering principles by the
students. To support the Java programming language implementation on the NXT
controller, LeJOS was used (Solorzano 2012).
The lab took place together with the lecture during the summer term 2012. The
lecture period started in April and ended in July. Exam preparation courses were
provided in September just before the final test. These courses offered the students
the possibility to train the addressed competences in smaller audiences.
All parts of the course received good feedback and results from the students
within the evaluation. Nevertheless, the students were challenged by learning in
large classes in the programming lab. Individual support was often requested, but
the number of supporting tutors was limited.
Therefore, one important objective for the course revision in 2012 was a better
support for individual learning with e-learning tools. The e-learning system L2P of
RWTH2 is already used as a Learning Management System (LMS) in the lecture,
the group exercises, and the lab mentioned above. However, additional learning
support was requested to assist students in and out of class, but particularly when
learning autonomously. Two major challenges of the described scenario are:
• A wide range of pre-course programming skills among the students.
• Individual support for learning with limited resources for teaching personal.
To meet these requirements, a Web-based e-learning test bed was designed and
implemented which supports different kinds of learning situations like SRL, peer-
instruction learning, and email support by tutors. The test bed learning content
ranged from exam preparation exercises for all students to additional background
information for advanced students. It extends the L2P learning room with interac-
tive learning capabilities and is described in the next chapter.
2
http://www2.elearning.rwth-aachen.de/english
The Personal Learning Environment
The development of the interactive e-learning platform was part of the ROLE
project. Beginning with the summer semester 2010, a previous version of a Web
2.0 Knowledge Map (WKM) was enhanced with ROLE technology. In particular, it
was transferred to a widget-based environment (cf. von der Heiden et al. 2011), that
is a bundle of widgets interacting via ROLE Inter-widget Communication (IWC)
(Renzel 2011).
The WKM is an electronic reference book, which can be regarded as a kind of
improved Wiki system. It won the second prize in the 2010 International
E-Learning Association Awards, in the category “Academic Blended Learning.”
The application supports students in looking up factual knowledge. Students can
search for articles by entering topic keywords and by navigating from their current
article to related articles following hyperlinks. It is based on semantic net techno-
logy, where hyperlinks are not just links, but belong to predefined categories, each
bearing a meaning, as a named relation. The object-oriented content organization
knows classes and objects of knowledge. A class is a predefined template for a
knowledge object such as an “Exercise” and its realization. Similar to a Wiki, the
WKM supports the creation of new content. A dedicated rights management allows
the usage of different roles as authors, administrators, and users. Authoring is
currently restricted to lecturers and tutors. Additionally, the content visualization
capabilities based on hypermedia support nonlinear learning approaches.
For the ROLE project, the WKM was redesigned as an interactive learning tool
and as a test bed for ROLE technology in a higher education scenario. The new
design was motivated by the following main goals:
• Guide and support students in a self-regulated and nonlinear learning process.
• Motivate, introduce, and provide high-quality basic knowledge using multime-
dia material.
• Provide an interactive reference book on lecture contents for exam preparation.
• Support interest-based real-time communication and collaboration in learner
communities.
Thus, the former WKM has been extended with a chat to provide theme-based
learning communication between users. A learning history accomplishes the setup.
Built up with ROLE technology, the “new” WKM is composed of three inter-
communicating widgets (see Fig. 2), namely:
• Web 2.0 knowledge map widget for accessing and reading topic articles as well
as exam exercises.
• Chat widget: general or topic-related group chats and presence information for
individual tutor support or peer-to-peer instruction.
• History widget: tracks individual learning activities and shows personal history
of visited topics.
Fig. 2 Screenshot of RWTH testbed with Web 2.0 knowledge map, chat and history widget
The test bed scenario was deployed for the course lab and also for the students’
individual exam preparation in August and September. The WKM aimed to provide
the students with information covered in the lecture and in the lab. It was filled with
additional SRL-adapted content thus focusing on typical SRL situations such as the
exam preparation phase. It contained explanations and motivations for notions,
definitions, or examples, e.g., for basic Java programming constructs. Background
information was provided as well, e.g., about software installation. Exercises for
exam preparation were associated with lecture content. The presentation and
organization of the WKM followed the paradigm of object-oriented analysis and
design in software development. Relations between objects and classes of objects
were visualized (see Fig. 3) to underline knowledge associations. Functionalities
for annotations, remarks, and feedback were provided.
The second widget, a chat widget, was embedded to offer students the possibility
to ask and answer topic-related questions. Other students answered the posed
questions while a tutor moderated the chat.
Finally, a history widget was embedded into the learning environment. It
supported the backward navigation within the environment by offering the last
five activated knowledge objects. The widget uses data from the WKM widget to
support the learner with his or her own learning history.
The WKM was maintained by the IMA, the test bed was hosted by the depart-
ment of information science at RWTH. Access to the WKM was granted via the
login for the course lab. For the first time in the course’s history, this WKM learning
environment gave students the opportunity of individual support during their exam
preparation.
Fig. 3 Screenshot of the Web 2.0 knowledge map RWTH (start page)
Technical realization: The WKM has been bundled with a chat widget and a
personal history widget. The chat widget allows learners to communicate with
instant chat messages and to see the online status of other learners. It is integrated
with the WKM by automatically creating a separate chat room for each topic that is
currently read by the learner. Learners can see the topics of other learners and can
quickly join them in the topic-specific chat room to discuss their understanding of
the topic and how it relates to their current work. The personal history widget
records the topics visited in the knowledge map and allows quickly navigating back
to previous topics. The three widgets interoperate based on IWC; the following
examples of widget communication events demonstrate a selection of implemented
interactions:
• Entering a topic-based chat room on topic selection: When a student selects a

topic from the WKM, a corresponding chat room is entered in the chat widget.
At the same time, the student’s online status is changed to the new topic in real-
time and visible to and clickable for other students.
• Following a user’s activity: When a student clicks the online status of another
student, he navigates to the corresponding topic in the WKM, in turn triggering
an event to enter the corresponding topic-specific chat room.
• Real-time updates of learning history: When a student selects a topic from the
WKM, the selected topic appears at the top of his or her personal history.
Following the overall ROLE approach of open standard compliant widget-based
learning environments, the WKM test bed was implemented involving the follow-
ing enabling technologies:
• OpenSocial3: OpenSocial is a set of common application programming inter-
faces (APIs) for Web-based social network applications developed by Google
along with MySpace and a number of other social networks. Applications
implementing the OpenSocial APIs will be interoperable with any social net-
work system that supports them. The ROLE version of the WKM is deployed in
Apache Shindig,4 the open-source reference implementation of an OpenSocial-
container.
• Extensible Messaging and Presence Protocol5 (XMPP) is an open standard
technology for real-time communication, which powers a wide range of appli-
cations including instant messaging, presence, multiuser chat, and collaboration.
The ROLE version of the WKM offers XMPP-based features such as topic-
based chat rooms and real-time information on current presence and learning
activities.
• Inter-widget Communication (IWC) (cf. Renzel 2011; Zuzak et al. 2011): With
IWC, individual widget functionalities can be combined to realize complete
application workflows. ROLE leverages various forms of both local and remote
collaboration and communication among. The ROLE version of the WKM
demonstrates local IWC using technologies such as PMRPC6 and Google Gad-
get PubSub being part of the OpenSocial specifications. A basic form of remote
IWC was demonstrated with the new WKM chat functionality.
• Monitoring: All learning activities are tracked by the history widget and
persisted as Contextualized Attention Metadata (CAM; cf. Schmitz
et al. 2011). The ROLE version of the WKM was the first test bed producing
real-life usage data, which, later on, served for producing recommendations and
as a basis for further development of the WKM and ROLE technologies in
general.
3
http://www.opensocial.org/
4
http://shindig.apache.org/
5
http://xmpp.org/
6
http://code.google.com/p/pmrpc/
A detailed description of the ROLE framework technology can be found in

chapter VIII “Lessons learned from the development of the ROLE framework.”
Evaluation and Methodology
Additionally to the tool development, a test bed evaluation was designed to analyze
how the environment influenced the students’ learning processes. The RWTH
ROLE test bed work in 2012 was initiated with a Web-based survey that aimed to
collect details about the students’ experience with e-learning and SRL at the
beginning of the lab in April 2012. The ROLE widget environment was introduced
to the students during the second week of their studies. The enriched ROLE-based
learning environment offered additional support for improvement in SRL opportu-
nities. It also provided information about programming in general, related tools,
modeling as well as Java as such. Around 1,600 students participated in the course.
All students were informed about the ROLE-enhanced learning environment via
several announcements during lectures and labs as well as via email. During the
standard midterm teaching evaluation, a short ROLE-related survey was issued. At
the end of the lecture period, the ROLE test bed was also adapted for individual
exam preparation during summer time. Finally, after the exam, educational staff
was interviewed to evaluate the environment and its application within the course.
The lab sessions took place in the largest computer pool of the RWTH which is
equipped with approximately 200 workstations. This, however, restricted the max-
imum number of students that could attend the lab in parallel to 200 students who
then worked with 100 Mindstorms NXT robots. Since those 100 robots could not be
dismounted and reassembled in each lesson, the lab was based on a standardized
and preassembled robot model.
The ROLE environment was used during the lab time from April to June. Usage
grew significantly in September when the students started their individual exam
preparations some weeks before the exam. The access peak was reached in the days
just before the exam when students switched to “power learning.” This is illustrated
by Fig. 4 showing the number (by day) of accessed knowledge objects. The number
of generated views corresponds with the access rate and indicates the intensity of
usage by the students. Figure 4 underlines the exam-oriented learning during the
preparation that restricts the leeway in learning and thus the autonomy of the
learner. This characteristic learning activity trend has been repeated during the
next exam period in March 2013.
Fig. 4 Requested knowledge objects by day in the RWTH testbed
Results
In June 2012, before the summer break (i.e., at the end of the lab session but before
the exam preparation), the students were asked about the usefulness of the
e-learning environment and rated it positively. 162 stated that the application of
the computer-based learning environment was useful. On the given scale from
1 (strongly disagree) to 5 (strongly agree), the arithmetic mean (AM) of the results
was 3.7 with a standard deviation (SD) of 1.3. Since 3 would be neutral, the students
evaluate the environment positively without being overwhelmed.
After the course, the environment has been evaluated by the teaching staff. We
conducted four interviews, three of them with student assistants who acted as tutors
within the practical exercise and the exam preparation. They were responsible for
adding contents to the knowledge map and for solving technical issues. One
interview was conducted with the lecturer who was responsible for the overall
coordination and who was involved in the planning and conception of the whole
course. In the interviews, we asked the participants to rate several statements on a
scale from 1 (strongly disagree) to 5 (strongly agree) and to explain their ratings.
Moreover, we asked them to comment on the strengths and weaknesses of the
environment and to suggest improvements. The students’ positive judgment of the
environment has been corroborated by the teachers. For each statement, the arith-
metic mean (AM) and the standard deviation is given (SD) (while interpreting these
measures, one has to keep in mind that only four persons rated the statements):
• The environment was useful for the students. AM: 4.25, SD: 0.43
• The environment was useful for me in my role as a lecturer/tutor. AM: 4.00, SD:
0.71
• The students reached the learning goals better because of the environment. AM:
4.00, SD: 0.71
• I reached my teaching goals better because of the environment. AM: 3.50, SD:
1.12
• I would advise the students to use such environments more often if they had
access to them. AM: 4.75, SD: 0.43
• I would use such environments more often for teaching if I had access to them.
AM: 4.67, SD: 0.47
• I would use such environments more often for learning if I had access to them.
AM: 3.25, SD: 1.79 (This is an interesting result: Why do the lecturers/tutors
rather advise their students to use such an environment than use it themselves?
The interviewees answered that their personal learning style is not optimally
supported by such an environment, because firstly they prefer not to browse
through learning contents but to study textbooks and other material, in particular
exercises and exam questions from previous semesters, from beginning to end.
Secondly, they prefer using pen and paper over doing all exercises with the
computer. Therefore, they request an export to PDF so that they can print
selected parts of the material.)
• I consider the environment used within this course as a didactically sound
means. AM: 4.50, SD: 0.50
According to the interviewees, the strengths of the environment were, firstly,
that the knowledge map gave a clear overview on the course contents and their
inter-relationships. The students got a starting point for browsing through the
material and exploring the themes independently. Questions could be answered
by pointing to specific objects on the knowledge map, and students could (and did)
answer their follow-up questions themselves by exploring the surrounding/linked
objects. Thereby, the autonomy of the student was effectively supported. Secondly,
the chat widget allowed fast feedback from the students. Questions could be
answered immediately. Since all students could read the answers, questions did
not have to be answered twice. Thereby, the tutors’ explanations became more
efficient. The tutors saved time for helping with truly individual problems. Thirdly,
the environment improved the communication among the students and, thereby, the
collaborative learning. After a short time span, the students began to answer
questions asked by other students. Fourthly, the environment rendered the students
more flexible regarding their time management and learning speed. They were able
to repeat lessons and exercises without losing track of the course or thwarting
others.
Concerning weaknesses, the interviewees mentioned technical and usability
issues; in particular regarding the administration of the environment and the adding
of new contents to the knowledge map. These issues have to be solved, but they do
neither affect the concept nor the general design of the environment. Moreover, the
interviewees propose the following extensions of the environment:
The chat widget should be exchanged or supplemented by a forum for general
questions and by a commentary function for the elements of the knowledge map. This
would improve the linking of contents with questions and comments. They consider a
learning planner consisting of a simple to-do list with links to exam-related material
and topics, self-tests and a visualization of the current level of knowledge/exam

preparation progress (related to the self-test results) as extremely useful. The inter-
viewees agree that the contents are the most important feature of the environment.
These have to be updated regularly. So far, the contents of the knowledge map are
explored by browsing. An additional search engine for the direct search of specific
content would be reasonable. One interviewee deems a recommender system that
recommends related external material useful.
One aim of offering the ROLE environment was to support SRL. Has this goal
been reached, that is, did the environment effectively support self-regulation? The
interviewees claim that this is in fact the case. While in the beginning, a lot of trivial
questions were asked, the students were able to find the answers to such simple
questions themselves soon. (The question is, however, whether we can attribute this
development to an improvement of self-regulation or rather to a learning effect
regarding the course contents.) The interviewees considered it important to support
SRL. They estimated that by far, most of their students had medium SRL-level.
They correlated the SRL-level with the general knowledge level and acknowledged
that students with a high SRL-level learned better and faster. However, as tutors and
lecturers they generally preferred to teach students with a medium SRL-level over
students with a high SRL-level. They justified this preference as follows: A tutor
was supposed to lead interesting discussions with high SRL-level students. How-
ever, they did not need a tutor that much and therefore did not get in close contact
with them. Often, teaching did not really take place. Moreover, these students
tended to be good students that asked difficult questions. A teacher had to be
well-prepared and feel certain on the course topic to cope with these questions.
This made it sometimes harder to teach students with a higher SRL-level.
Medium SRL-level students were intelligent but still requested interaction with a
teacher. The teacher got in contact with them, observed the learning progress and
saw the positive effect of explanations and assistance. The interviewees found this
very rewarding.
The interviewees considered that a low SRL-level is correlated with rather low
learning success.
Teaching students with a general low level was considered to be cumbersome
and not very rewarding. Feedback given through the environment was recognized
by teachers as very important. The interviewees emphasized the role of the chat
(or a forum). Feedback was deemed important for estimating the students’ progress
and thus adjusting interventions. Moreover, it makes teaching more satisfying.
Conclusions
The evaluation proved the necessity of intensive promotion for new and additional
e-learning tools. Tool objectives and advantages must be clearly communicated
(at the right time) to the students. Nevertheless, only a minority of all students had
used the test bed for a longer time. Here, guidance with learning questions as in
(Hsiao et al. 2010) may motivate students and foster communication.
Until now, overview and learning guidance is given by the visualization of topic
relations on the start page, the hierarchical and object-oriented organization of
knowledge in the map and the linking of knowledge objects. The evaluation proved
that the environment supports SRL and collaborative learning in large classes. The
answering of student questions was easier via the chat widget than by email as all
students were able to see the answer. Additionally, the chat fostered student-to-
student support. Even if the test bed offered support for early learning, the peak of
usage was reached just before the exam. It indicates the students’ remaining in
power learning.
The test bed was implemented as a cloud learning application combining
widgets as services in an overall application and using IWC for communication
between the widgets. Since different people were responsible for the particular
widgets, it was sometimes hard to fix problems, e.g., when a server was not
accessible.
So far, the test bed was aimed to demonstrate the possibilities of ROLE techno-
logy in large classes. The demonstration was successful and further development has
to focus more on the learning requirements of students. Therefore, future improve-
ments are seen in better communication and feedback support to strengthen, e.g.,
learning motivation. Suggested improvements comprise firstly a better collaboration
support that can be implemented by adding improving, topic-related communication
(forum, notepad linked to contents of knowledge map) and secondly a better
SRL-support that can be implemented by adding a learning planner that supports
planning (to-do list) and reflection (self-tests, visualization of progress). The offer-
ing of learning strategies such as learning questions (Hsiao et al. 2010) within the
learning tool may provide new advantages and motivation for the students.
Lessons Learned
The evaluation resulted in the following recommendations. These recommenda-

tions are focused on the context of large classes in HE:
1. New e-learning tools—especially if in concurrence to existing solutions—
require intensive promotion. Students need a clear motivation and benefits for
the own learning objectives. Multiple announcing will be helpful incl. situations
when students will “hear” the message. Here, the usage of the exam preparation
tool was fostered with an announcement only some weeks before the exam when
student’s individual learning situations corresponds to the “message.”
2. If a new e-learning tool must fit to an already existing process of learning and
organization in HE, a good idea is better support for student’s individual learning
process, e.g., when the student is not present at the university and not in contact
with tutors.
3. During their learning workflow, students are interested to ask and answer
questions. Therefore, e-learning tools for individual exam preparation are
more attractive if combined with communication services as chats and email.
Peer-to-peer communication within the students is welcome and can reduce the
effort for mentoring by tutors. Even, communication can provide qualitative
feedback for learning content as exercises and solution samples.
4. One chat for all—instead of multiple chats for different content—provides more
communication activity. Chat activities can link to the corresponding content to
clarify the context for questions.
Outlook
The design of the complete course has been analyzed after the exam in September
2012. The usage of the ROLE test bed, the participation and other indicators has
shown that the students’ SRL capabilities do not meet the presumptions for the lab
design. On the one hand, the students’ pre-skills in programming differ signifi-
cantly; as a consequence, the strict schedule of the lab was often too slow or too fast
for them. On the other hand, the capabilities for self-organizing teaching material
from different sources are not very well developed and students expect to have all
pieces of information at one place. The lab learning activities were often frustrating
for the students and tutors.
Therefore, we developed completely new teaching material which guides the
students through the learning process. In the beginning, the basics of programming
are explained in every detail and step by step, the autonomous work phases of the
students get longer and longer. This teaching material supports an individual
schedule of learning activity, as well, which corresponds better to different pro-
gramming skills. First evaluation results show that the students are much more
interested in the learning activities. Even the tutors prefer the new learning design
since the students are motivated and ask interesting questions.
Shanghai Jiao Tong University: ROLE for Employed,

Part-Time Students
The SJTU case study set out to address the issue of using Personal Learning
Environments in adult higher education. It is associated with the SOCE whose
blended classrooms are based on the Standard Natural Classroom model (Shen
et al. 2008) providing face-to-face interaction with the instructor as well as online
courses. Its students, mostly adult learners who have a job, take classes in the
evening or at the weekend, by either attending in person in the classroom or by
watching live over the Web. Improving their competencies via degree education or
certificate training enables them to increase their chances in the highly competitive
Chinese job market—a market that is also characterized by frequent job-hopping.
Teaching and learning in this case study follows a traditional pattern: it has a
teacher-centric focus, with a “broadcast” model, where most students watch the
lectures rather passively. The ROLE project, in this instance, has offered SJTU an
opportunity to explore and investigate how to use existing available ROLE tech-
nologies and tools that could provide a larger amount of opportunities for learner
and teacher interaction enabling further potential creative ideas for both parties too.
For instance, selected bespoke ROLE tools, such as those offering Voice Recording
and Text-to-Speech recognition, allow foreign language students to practice their
pronunciation by recording themselves and comparing their speech to the “original”
one, thus providing the students with an active learning opportunity.
Learning Scenario
One central aspect of PLEs is that learners can assemble their own learning
environments from existing services. They decide which services to use, assembled
them, and use it for learning. Such a usage presupposes active, technical-savvy
students. From our experience we knew that the students at SOCE do not fall into
these categories. Most students have limited knowledge about Web tools (RSS is
virtually unknown), only limited time at their disposal, and limited technical
expertise. Furthermore, in the Confucian culture of China learning is still very
teacher-centered (Zhang 2007), and students are not used to actively contributing to
class. We therefore decided to build a PLE according to the learning scenarios
specified by the teachers of the courses and make the pre-build PLE accessible to
the students.
One example scenario devised by the teacher of the French course is as follows.
His course aims at helping the students mastering the first steps in spoken and
written French as well as learning about and mastering tools that help students in
their working life. These two goals are supported by activities that require using the
tools. For instance, starting with an English (or Chinese) sentence, such as “Hello,
my name is Tianxiang,” the teacher shows how to use a translation tool to get a first
rough French translation, and how it can be refined by using a dictionary and spell
checker. The French sentence is read aloud by a text-to-speech tool and repeated by
the student until it can finally be recorded. This recorded introduction can then be
uploaded to social networks. At a later point in the lecture, the students will receive
a similar task, such as describing their job without being shown how to use the tools.
Figure 5 contains a screenshot of a PLE, whose basic functionality is similar to
the start pages Netvibes and iGoogle. It provides a single page from which the
students can access different language related services and sites. The widgets of this
PLE facilitate learning a foreign language. For instance, the top right widget
performs spell checking; the second one below enables the translation of texts
using Google Translate; the third widget accesses a text-to-speech synthesizer; and
the bottom left widget allows the student to record and playback his voice.
Fig. 5 Screenshot of a SOCE PLE
As another example of a learning scenario, for the course on Data Structures, the
teacher wanted to use a PLE that supports rehearsing for the exam. This PLE
consisted of a large number of exercises, which trained different concepts in this
specific domain, such as linked lists, sorting, etc. The PLE was not tightly inte-
grated into the weekly teaching, but made accessible at the end of the semester, a
few weeks before the exams.
To summarize, the PLE usage serves different purposes: the students have a
chance to acquire knowledge about existing tools that will be helpful even outside
class. Communication in a foreign language becomes facilitated and empowered
when supported by translations tools and text-to-speech. The former allow under-
standing and producing content that learners not yet master due to insufficient
vocabulary. The later enables the students to listen to new texts, copied from any
source or written by themselves. Together with a recording device, they can
compare their speech to the artificially produced one. Furthermore, the PLE pro-
vides opportunities to train domain knowledge, in this case multiple-choice exer-
cises that cover topics taught during the lecture (gender of verbs, prepositions,
linked lists, sorting, etc.). By reusing existing services we were not required to build
our own version of these tools. The free text-to-speech service offers an astonishing
quality close to native speakers that would have been difficult to achieve on our
own. By assembling all the services in one page, access to these services is
facilitated.
During the ROLE project, SOCE moved from a self-developed, proprietary LMS to
Moodle7 as the Online Learning Environment. Thus, in a first phase, ROLE
technology was developed and evaluated in an additional system (Liferay), which
offers widget support. Once the shift to Moodle began, the ROLE evaluations took
part in ROLE Moodle extensions. This section provides additional information on
the technical realization.
Technical realization: The technical realization of the PLE used at SOCE
underwent an early and an advanced phase. In the early phase, SJTU explored
first usage of PLE in a technical environment that allowed only rudimentary PLE
features, as the advanced features possible at a later time by technology developed
in ROLE were not yet available. In the advanced phase, on which we focus in this
chapter ROLE technology was developed and integrated into the SOCE learning
environment.
Starting in 2011, SOCE moved from its proprietary LMS to Moodle. Moodle is a
popular LMS to manage courses that is the de-facto standard among many educa-
tional institutions. It is a plugin-based PHP application that can be extended by
installing additional modules. These modules have to be installed on a Moodle
server by a system administrator. The Moodle view, as shown to students and
teachers, consists of a main center area and a rather narrow left column (see Fig. 5
for an example). The center area contains main course resources, such as a wiki
page, a forum, a lesson, a quiz, etc.
Moodle’s flexibility and adaptability is achieved via visual themes and server
side plugins, thus an intervention of system administrators is required every time a
change should be done. Teachers and students are not involved in the process of the
customization. Teachers, for example, cannot add or remove plugins on their own.
Differently from Moodle plugins, widgets are client-side applications that can be
added to a system by skipping server side installation, which makes them easy
to add.
Our OpenSocial plugin for Moodle allows a simple and teacher/student-driven
extension of Moodle’s functionality. Once the plugin is installed to Moodle, a
teacher can add a “Widget space” to the course, specify a set of widgets for it,
and choose whether 1, 2, or 3 column view should be used for widgets display
(Fig. 6). The resulting outcome (as displayed to students) is the page with widgets
shown in the iGoogle in similar fashion, where students can work with several
widgets simultaneously (see Fig. 5).
7
https://moodle.org/
Fig. 6 A teacher creates a space with widgets for a course
From the implementation perspective, the plugin consists of two main parts
(Bogdanov et al. 2013). The first part is an engine that renders OpenSocial apps on a
page. This engine is Apache Shindig which represents a reference open-source
implementation of the OpenSocial specification. The second part is a PHP module
that is responsible for a configuration of a page with widgets, adding and removing
them to/from the page and gluing Moodle with the Shindig engine. The OpenSocial
API provides the standardized way to retrieve and exchange information between
different Moodle installations and other social networks, which improves data
portability and interoperability. More precisely, widgets can query Moodle for
data via the Shindig engine: they can retrieve the currently logged in user, the
current course, its participants as well as save and get arbitrary data. The privacy
and security are managed via the Shindig engine and it is in the full control of
university administrators. However, a widget installed within a course runs on
behalf of the teacher who added it and can retrieve/update information that teachers
can normally do in their courses. Thus, teachers are responsible for checking the
trustfulness of a widget, before adding it into their environments. The ability to
retrieve course information and its participants is achieved via OpenSocial Space
extension that allows widgets to adapt to the specific context of the course (con-
textual widgets). For example, a wiki widget can save data for a course and restrict
access to only people engaged in this course. The same wiki widget will behave
differently being added to another course: it will have a different wiki history and a
different list of participants.
Bundles
Two bundles were created by SJTU/SOCE: Creating an audio self-presentation and

the SRL bundle. Both are available in the ROLE widget store.8
Creating an audio self-presentation: The bundle for creating an audio self-
presentation in French includes four main widgets: a translator widget, a spell
checker, a text-to speech engine and a recording widget, and some additional
tools such as a CAM widget, a business dictionary and a conjugation tool. The
four main widgets are used to create a self-presentation in French language, the
additional widgets are to assist student in his learning activities and to collect usage
data. This widget bundle is helpful in a language learning context and can be used to
complete different tasks, such as learning vocabulary, improvement of pronuncia-
tion, producing of texts and audio-files, etc. The precise functionality of the widgets
is as follows.
The Translator widget allows user translating terms or sentences entered. The
Translator is linked with a Translator homepage and a Dictionary homepage, which
can be called up by user. User, who is a beginner in French, may work with this tool
to translate his self-presentation text from English or Chinese into French.
The Spell Checker widget serves to fine tune the translation from the Translator
widget or any other (self-produced) text. User may examine his self-presentation
text with help of this tool and correct spelling mistakes occurred by translation. To
check spelling, type or paste a text into a text entry field and click on a “Spell
Check” button.
The Text-to-Speech Engine allows listening to the pronunciation. The user may
listen to the pronunciation of his self-presentation text to create or to check his own
audio-file made up with the Recording widget. To check the pronunciation, enter a
term or a text into a text entry field and click on a “Say it” button. Voice (male or
female), language (also British and US pronunciation for English language) as well
as additional effects can be selected.
The Recording widget can be used to check user’s own pronunciation or to
produce audio-files such as pronunciation samples, audio texts, or presentations.
With this tool the user may record his self-presentation and compare it with the
given pronunciation of the Text-to-Speech Engine.
Some additional tools, such as a Business dictionary and a Conjugation Tool
(to check a modification of a verb from its basic form), can be added to the bundle to
assist user in his/her learning activities.
SRL bundle: The students at the SOCE of Shanghai Jiao Tong University are
young adult learners who typically have a job and family, and this only limited time
at their disposal. Also, the students are average learners in the sense that they have
little knowledge about how to learn, specifically only limited SRL skills.
For that reason, SJTU together with Koblenz and supported by FIT and Uppsala
devised a bundle for supporting SRL (see Fig. 7). The bundle consists of three
8
http://www.role-widgetstore.eu/bundles
Fig. 7 A screenshot of the self-regulated learning bundle
widgets: the To-Learn list, the Activity Recommender, and the Contextualized
Attention Metadata monitor. It also contains a video illustrating the usage of the
widgets.
The Activity Recommender widget (bottom right in Fig. 7) gives hints about
how to process a given task. In this bundle, the task consists of “how to create a
presentation.” With the tool, users compile a learning plan consisting of learning
activities. The tool shows the current task, matching learning strategies, a list of
concrete learning steps, and additional information.
The To-Learn list widget (bottom left in Fig. 7) allows to compile and to modify
a learning plan. Users can add, rearrange, delete and rename recommended learning
activities or add own activities. The Activity Recommender sends learning activ-
ities to the To-Learn list. Students can keep them, or discard them, as they wish.
The CAM monitor tracks how students use the widgets by storing the events
send from the other widgets in a central database. This widget is not directly of use
for the students in this case, but allows evaluating their usage.
The case study done at SJTU that evaluated the bundle highlighted several points
that are of utmost importance when using such a bundle in class. First, due to the
novelty of the widgets, students will probably be unclear about how precisely they
should use each tool on its own. It is thus recommended that extensive support is
provided and that instructions are available that explain the tools. Secondly, the two
main widgets in this bundle interact, which is not frequently observed as a behavior
in existing systems. This has to be explained in the tool itself. The instructions
should cover the usage but also clearly explain the purpose of the widgets, since
again the offered functionality is seldom encountered in existing software.
The evaluations of PLE usage at SOCE started in 2009 and continued until the end
of the ROLE project in 2013. Each semester, PLE technology was used in a number
of courses, mostly language learning courses, namely several English courses
(English Listening, English Speaking, Critical Reading), and 2 two-semester-long
introductions to French and German, but also a Computer Science course (Data
Structures). The number of students varied over the courses and semesters. In
average, for the language learning about 200 students were inscribed to the courses,
about 20–25 came to the lectures in person and about 50 (25%) took the final exams.
These numbers are typical for the second language (which is deemed as rather
unimportant by the students). About 1,200 students per semester took part in the
Computer Science course, with a similar percentage as for the language learning
courses of students participating in the final exams. In each course, an example PLE
was assembled by the teacher, supported by a team of researchers. In the language
learning courses, the PLE was introduced and used in class, and the students were
expected and encouraged to use it outside class. In the Data Structure course, the
PLE was created at the end of the semester and students were supposed to use it for
preparing their exams.
Results
Results from the PUEU surveys: The survey used at SOCE is based on the
Technology Acceptance Model (TAM, cf. Venkatesh and Bala 2008) and consists
of a set of statements that measure the Perceived Usefulness and Ease of Use
(PUEU) of PLEs. The PUEU questionnaire has therefore constituted the “core”
part of the surveys used among the users (educators, learners, employees, trainers,
etc.) of the ROLE test beds. Naturally, the rest of the questions included in these
surveys have been targeting the specific context of each test bed. At SOCE, surveys
were created for delivery in both English and Chinese. Students were informed that
they should complete them once they had finished their course of study. This was
entirely voluntary. For example the survey from the French and German course was
only completed by 20 of the 150 students. This was despite the fact that the bespoke
Moodle spaces were regularly used by students in class and enhanced by being
actively demonstrated by the teacher. The survey results show that 65% of students
indicated that they used the PLE, while 20% stated that they did not use it, with the
remaining 15% reporting that they did not know what was meant by the term PLE.
Similarly it was also noted that students felt that their knowledge of e-learning was
quite good (65%); with the remaining 35% recording their knowledge as high and
better.
Students reported an overall positive experience of using the PLE. Using a
Likert-type rating scale with a range from 1 (strongly disagree) to 5 (strongly
agree) all but two students either agreed or strongly agreed with the specified
statements, such as “I find a PLE useful for my work,” “I accomplish my work
more effectively with a PLE,” “I find the exercises provided in the PLE helpful for
my learning,” “It is easy for me to use a PLE,” “It is easy for me to use a PLE,” and
so on (see Appendix x for the detailed results of this PUEU survey). Interestingly,
the two negatively formulated questions, namely “I find using a PLE frustrating”
and “I find interacting with a PLE requires a lot of my mental effort” received less
positive answers, namely 7 students agreed or strongly agreed, and 13 students were
neutral or disagreed.
Several questions in the PUEU survey inquired whether students would like to
modify their PLE by adding new widgets or by replacing widgets. The majority of
students (15 or more) agreed with this premise. Furthermore, the perceived value of
specific widgets was also investigated. They remarked that they thought that the
preassembled PLE/Moodle space was well designed. This manifested as very
positive ratings for all widgets. The actual Moodle usage logs, however, indicate
that only a few ROLE widgets were actually used by the students (mainly the
translation tool and some of the exercises), although the teacher encouraged usage
in every class. The results from the survey, therefore, can be seen as representative
of potential interest rather than reflecting active ROLE widget usage.
Significantly the English students completed fewer surveys than those on other
courses, and correspondingly their rating was also less positive. In the English
courses it appears that the focus was on the exercises and, in contrast to the German/
French courses, where the English teacher used the PLE/Moodle spaces less
frequently in class. The survey also revealed that the majority of English students
did “not know what was meant by PLE” (more than half of the given answers).
Those English students who declared an understanding of the term PLE also rated
the perceived usefulness and ease of use positively albeit less than those students in
the German/French courses. It also appears that the English students also are less
inclined to assemble or change widget spaces (as evidenced by the answers to the
respective questions). In addition, the responses relating to the perceived value of
the provided tools appear to be mostly average, with only the translator and
exercises being rated slightly more positive.
Results from the Activity Recommender and To-Learn task list: In addition SJTU
also conducted a further evaluation of two specific widgets, namely the To-Learn
list and the Activity Recommender, within the Business English class. The
To-Learn list enabled students to define and work on to-do-lists specific for
learning. The Activity Recommender supported students in preparing a presenta-
tion, and can add tasks to the To-Learn list if the student agrees. For this evaluation,
the students were given the mandatory homework of preparing a set of slides for a
product presentation and also needed to use a PLE space that contained the two
widgets. Due to the usage of the widgets, and the completion of the survey being
mandatory, a large number of students (240) completed the survey.
Initially a preliminary evaluation took place a few months before the deploy-
ment of the second survey evaluating the usage of the implemented ROLE tools. In
this first evaluation, the activity recommender was implemented as a mock-up using
slides to introduce the widget. The immediate student feedback and a recording of
the class’ reaction to the idea of this widget served to inform the ROLE partners of
some potential shortcomings. The second evaluation, using the implemented tools,
took place over a period of 4 weeks. The students received the (previously
described) task to author a set of slides that portray a real or imagined product
that a company would sell. The task was mandatory as homework. Students were
also instructed to use the two ROLE tools during the authoring of their homework.
The evaluation revealed several problems: those related to technology, e.g.,
features did not run due to browser issues and related to task/tool understanding,
e.g., students did not understand the purpose of the tools. Thus, while the overall
number of participation of the student was very high (given that this was a
mandatory homework), the dissatisfaction of the teacher and students
(as conveyed orally and by email) was also high, resulting in the suggestion that
the tools needed to be improved for more effective usage in a classroom
environment.
The quantitative and qualitative outcomes arising from this evaluation are based
on survey results with a sample size of N ¼ 239. It can also be reported that the
answers to the free text question “Did you have any problems using the Activity
Recommender and the To-Learn-List?” revealed a series of technical issues as well
as disclosing a lack of understanding among the student group that manifested as
significant usability problems for them. That may have been influenced by the fact
that SRL was a concept these learners were not used to. Since the level of English of
the learners was relatively high, the problems were not primarily caused by
comprehension problems. It is important, however, to take into account that the
Learning Activity Recommender widget that they were using was designed origi-
nally for a specific group of learners who would have needed substantial support
and guidance related to their cognitive level and, therefore, was not designed with
all learners in mind. Nonetheless the overall survey results can be interpreted as
quite positive with regard to the perceived usefulness of the ROLE widget bundle.
Since the To-Learn List in the evaluated widget bundle offered support for learners
with good meta-cognitive competences it could be used a convenient tool.
Widget Authoring Tool and teacher interviews: One of the major problems
regarding the usage of ROLE technology at SJTU is that most teachers do not
appear to be interested in many of them as they feel that there are too few
appropriate widgets available that they can use in their courses. To overcome this
problem, the ROLE staff at SJTU developed an easy to use authoring tool that
requires little technical knowledge. In order that the authoring tool fulfills the
teachers’ expectations and also meets the ease of use requirement, separate inter-
views with five teachers were arranged in which a mock-up of the tool was
presented to the teachers. During the interview, the mock-up was used to go through
the authoring process. This helped to identify any immediate difficulties and
enabled the immediate collection of suggestions from the teachers. The tool has
now been made available at SJTU. It uses ROLE technology such as inter-widget
communication to capture interaction data and to allow students to rate widgets.
The SJTU teachers involved in the courses using the ROLE technology were
invited for a semi-structured interview with the aim to record their experiences with
ROLE and how they would like to see ROLE improve in the final year. Two of the
teachers accepted to be interviewed online and a third teacher accepted to provide a
paper-based response to the interview questions. Overall the respondents thought
that ROLE was extremely useful as it made possible to allow learners to access
materials in a more flexible manner, enable them to self-assess their skills, enable
them and teachers to have enhanced access to course metadata.
All of the respondents expressed that they will continue to use ROLE tools in
future because they were impressed by the benefits it brought to their learners (e.g.,
access to greater and better resources) and to them (e.g., monitoring, portability,
etc.). The respondents zeroed on in two areas of improvement, which they men-
tioned throughout the interview. Firstly the need to demonstrate the value of
developing more contextual or subject specific widgets and bundles:
I think what we have to do is to show more clearly the value that ROLE can bring to the
teacher, so I think the basic technology is there. But it’s not every visible in the current
widgets, the current tools that are available. I think this is one place where it really needs to
improve.
Maybe the developers should understand the course, because different courses need
different learning pedagogies. The theory and experiment methods for different course are
depended.
Secondly, to make the widgets and associated technologies facing the end-users
much easier to use without the need for any more technical knowledge than using
Facebook for example.
. . . . but maybe just something like Facebook, where I can just upload, share something, and
press some simple buttons. And I think it’s still too complicated for me, and also for my
students, as I described before. I’m getting a lot of questions. So I think the usability
definitely needs to be improved.
Conclusions
Lessons Learned
Two major lessons have been learned by the evaluations at the SOCE test bed. First,
the significant role of guidance in such an HE setting, and second, the importance of
having a sufficient amount of widgets.
The first lesson became quickly obvious after the initial evaluations based on the
Liferay system (cf. Ullrich et al. 2010). The initial approach of PLE usage consisted
of introducing the PLE during class on the basis of an example. Then, the students
received a homework that required them to use the PLE as demonstrated. For
instance, the teacher showed how to record a video using a Web 2.0 service, and
the students’ homework consisted of recording a video, without a given specific
topic. This open approach was motivated by the thought that the more open the task
was, the more motivated the students would feel. However, this initial approach
failed. None of students did this or any other similar open homework, interestingly
although the students rated the PU of the PLE very high. In the next iteration when
given more guidance by the teacher with specific tasks to perform the number of
handed-in homework increased. We believe the low initial uptake despite PU is due
to several reasons. First, students quickly become overtaxed. The concept of a PLE
is unfamiliar, the embedded services are new to them, and they have only limited
experience in Web 2.0 in general. Second, students often do not see the value in
learning how to use these tools. They feel it distracts from learning grammar and
vocabulary, and does not prepare them for the exam. Thirdly, most of the students
(and teachers as well) are not intrinsically motivated to use Web services, and the
majority of our students feel that the time could be spent more effectively. Thus, the
task of the teacher becomes to demonstrate and highlight the advantages of a PLE,
and guide them through it, so that the students can arrive at an understanding of
what a PLE offers.
Secondly, uptake of ROLE was significantly hindered since only few domain-
specific widgets were available. In the case of SJTU, teachers were less interested in
general purpose widgets, but asked for widgets covering very specific domain
knowledge. Content available in existing Learning Object Repositories was not
used in a single case, since these resources were too different from the specific
needs of the teachers. For instance, existing learning objects about French were too
much dependent of the original course book, and not usable in the SJTU courses due
to too different vocabulary. Yet, teachers did find usable resources on Websites not
available in learning object repositories. We therefore had to enable teachers of
turning these Web resources into widgets usable in their PLEs by using a widget
authoring tool. In addition ROLE staff at SJTU offer those courses that use the
ROLE technology extensive support by technical teams who can set up the widget
spaces and create widgets for those teachers who wish to avail of this service. We
could observe that only those teachers who were extensively reported actually used
ROLE technology.
Finally, it is important to note that the discrepancy between the often very
positive ratings given in the PUEU surveys and the often negative vocal feedback
or confusion observed in classes related to ROLE technology, as well as the
recorded low actual usage visible in the logs, indicate that, at least in a Chinese
context at SJTU, the ensuing survey results should be interpreted with caution.
Notwithstanding this word of caution, however, SJTU is still convinced that ROLE
technology can enable the easy creation and usage of interactive activities that
make the overall classroom activities more interesting and, therefore, empower
teachers to offer extra learning activities that go beyond what a standard Moodle
online course can offer.
Outlook
SOCE will continue using ROLE technology after the official ROLE project has
ended. The teachers who were using PLEs in their classes during the project’s
runtime have taken over their PLE spaces into the course Moodle sites of the new
semesters. Also, new teachers have expressed their interest in creating their own
widgets during the presentations of the authoring tool. One teacher from the Social
Science department created widgets of Web games about different political topics.
Uppsala University: ROLE for Distance Students Working

Collaboratively in Small Groups
The Uppsala University test bed was performed within the distance course “Social
software and Web 2.0” at Uppsala University. The course was given during the
summer semester in June and July 2011 with 34 students and in the spring of 2012
with around 20 students. The course corresponds to 5 weeks of full-time study. A
university wide LMS installation was used throughout the course for the main
interaction with the students. The course consists of four assignments each
containing a part to be done individually and a part to be done within a group.
Learning Scenario
The test bed involved one of the four assignments in the course. The assignment in
question aimed to give the students a deeper look into how social interactions are
used in a professional manner, most specifically by a corporation that communi-
cates with its customers via twitter. The students were tasked with finding patterns
of behavior and how the social interactions work in a specific medium (in this case
micro-blogging and customer relations).
The goal of the assignment, as stated by the teacher of the course, was to use a
typology presented by Shaw et al. (2011) to categorize tweets (i.e., 140-character
statements made in the social media platform Twitter) sent to and from the Swedish
train company SJ during the winter of 2010/2011. This particular winter was
unusually problematic for Swedish train traffic, with extreme weather conditions
resulting in severe delays all over the country. The assignment thus meant to
analyze twitter discussions about traffic disruptions, mainly in commuting. Outside
of the study, the students were required to read a paper coauthored by the teacher
(Larsson et al. 2011) where a more summarizing typology was used to categorize
the same tweets. The students were divided into six groups of 3–5 for the whole
duration of the course, and were told to categorize the tweets collaboratively within
these groups.
The assignment was carried out with the ROLE Uppsala prototype (a slight adap-
tion of the ROLE SDK with single sign on for the students), mainly using a widget
bundle that was developed specifically for the assignment. The bundle resulted
from discussions with the course’s teacher about what tools he would find desirable
for the assignment.
Technical realization: The Analyze Tweets bundle9 allows students to take a
closer look at Twitter tweets. Figure 8 shows the bundle’s five widgets. First, tweets
and tweet conversations are presented in a timeline. Second, the same tweets are
presented in a list and can be tagged with 18 different categories. Third, a pie-chart
of the amount of tweets tagged with each category is shown. Fourth, the tweets can
be discussed in a forum and references inserted to individual tweets within posts as
links. This allows students to, at a later time, refocus on the mentioned tweet by
clicking on the link in the post (timeline and tweet list will adjust to show the tweet).
Finally, any related content can be shown in the content viewer, for example, any
links to Web pages mentioned in tweets show up here if clicked upon. This bundle
Fig. 8 The analyze tweets bundle
9
http://www.role-widgetstore.eu/content/analyze-tweets
makes heavy use of inter-widget communications and most interactions performed

inside an individual widget have consequences in other widgets.
The bundle is intended to be useful when investigating tweets and the conver-
sations they form. Students can make sense of the various ways people use tweets to
communicate by using the provided vocabulary to categorize tweets. The forum
allows students to discuss the problems that appear, for instance if some tweets do
not fit into any of the provided categories.
The purpose is to give students a chance of experimenting with one way of doing
research in the area of social science. A teacher can assess students’ performance by
looking into the categorizations that the students make as well as their activity in the
forum.
An initial pre-study was conducted as a survey during the summer course that gave
input for the design of the full study in the spring of 2012. One of the results showed
the importance of providing a prepared environment rather than relying on the
students ability to assemble a suitable environment from scratch. On the positive
side was that the students reported that multitasking and having multiple tools on
the screen at once was nothing new to them (Jonsson 2012).
Based on the pre-study, and the constraints on what we could do within the given
course, a decision was made to test a limited set of functionality provided by ROLE.
The choice was to focus on inter-widget communication, and how students perceive
a user interface where multiple widgets could be used in combination to reach a
predefined goal.
The evaluation method chosen for the test bed was surveys since the context of
the evaluation was a distance course at the university, where it was problematic to
do interviews or use other methods where the investigator needs to be physically
co-located with the users. The survey consisted of 28 questions and was answered
by 16 of approximately 20 students taking the course (~80%). The reason for this
approximation of the total number of students is due to a discrepancy between the
number of students registered for the course and the number actually attending it.
Out of the 28 questions 23 were formulated as statements and the students were
asked to position themselves on a 5-grade Likert scale with the polar values labeled
as “I fully agree” (1) and “I do not agree at all” (5). As we did not specify
intermediate scale steps between the extremes, we can assume an interval scale
between these. In order to detect survey artifacts some of the questions were
formulated with a mirrored scale. Responses to these were inverted a posteriori to
make it easier to interpret averages and correlations between statements. Of the five
remaining questions four were free text and one consisted of multiple-choice
checkboxes.
For more details about the evaluation, please see the extended report concerning
the evaluation of the Uppsala University ROLE prototype by Lind and
Laaksoharju (2012).
Results
Most of the students were at some point annoyed with something about the system
(AM ¼ 2.25, SD ¼ 1.07). This judgment co-varied negatively with the impression
that the system was working flawlessly.
The students did not report any difficulties to learn how the system worked and
generally considered the platform to supply good support in solving the assignment
(AM ¼ 3.50, SD ¼ 0.82). That widget content was changing automatically when
performing different actions in the system was conclusively seen as not confusing
(AM ¼ 4.1, SD ¼ 0.93). They also seemed to think that system was relatively easy
to work in (AM ¼ 3.19, SD ¼ 0.75). However, there is a tendency to judge the
number of required mouse clicks to be too high (AM ¼ 2.44, SD ¼ 1.32).
Students’ overview of their working process was perceived as good with an
average of 3.5 and a standard deviation of 1.0. The fairly high standard deviation
stems from responses being spread over the whole scale; though the majority
(12 respondents) settled for a grading of 3 or 4, as reflected by the average value
above and by the median value of 4. Significant correlations were observed between
this statement and the level of participation and collaboration, the perception of the
system as practical and the system’s impact on the motivation to complete the
assignment.
The students did not seem to have any significant problems understanding how
to use the system for the assignment (AM ¼ 3.75, SD ¼ 0.86). Interestingly, this
statement did not have any significant correlation with the other statements in the
survey.
The support from the system for doing the assignment was perceived as fairly
high (AM ¼ 3.5, SD ¼ 0.82). This statement correlates with 11 of the other state-
ments, making it the second best predictor after the perceived support for
collaboration.
Applicability in other areas was generally seen as high (AM ¼ 3.38, SD ¼ 1.03).
Significant correlations were found between this statement and the perception of the
system as motivating/tiring (inverted), the perceived support for collaboration and
the statement that the student would have preferred another tool (inverted).
Perceived support for collaboration received the highest count of correlations
with other statements, twelve out of 22, making it a good predictor for overall user
perception of system usefulness. The statement itself received an average of 2.75,
with a standard deviation of 1.07, which is on the lower half of the scale, though still
interpreted as relatively high considering the system is still in a development stage.
The statement that the system was practical to use, received an average of 3.31
with a standard deviation of 0.95. This indicates that the students’ perception of the
system’s practicality was fairly high. This statement is also strongly correlated with
the perceived lack of problems in the system, the support for collaboration and the
perception that the system increased the student’s motivation.
The students felt to a high degree like they were part of a team while working in
the system (AM ¼ 3.94, SD ¼ 0.93).
Conclusions
Generally the results of the evaluation were positive. They indicate that the students
were quite satisfied with the overall usability of the system and perceived that the
system increased their motivation for learning and collaboration.
Many of the students were, however, at some point annoyed with the system. We
have seen in other studies that students have a fairly low tolerance to usability
problems in systems that they are expected to use in their studies. The perception of
the system as a whole may thus be biased toward a more negative impression than
what would have been the case if the system setup had been more stable.
An interesting, positive result when it came to the users’ perception of one of the
unique features in this prototype—that widget content was changing automatically
when performing different actions in the system— is that it was conclusively seen
as not confusing; thus supporting this novel avenue toward automating internal
communication between widgets. Initially there were some concerns that this
technologically rather advanced approach would also be perceived as complicated
by the users, a fear which proved to be unfounded. However, since the students did
not get the opportunity to compose their own, unique set of widgets, we cannot
determine whether this acceptance was due to a perception of the system as one
united whole or whether the widget performance will be predictable even when
users pick and choose widgets at will. Future evaluations should address this
question.
The results for how useful the different widgets were for solving the assignment
are interesting, especially the low score for the tweet timeline. The ambition with
this widget is to visualize patterns of interaction between Twitter users over time.
However, either few of the students seem to have had use for this functionality or
the interaction with the widget was not satisfactory. When looking at the classifi-
cation data, almost half of the total classifications regard reactions and discussions,
for which the widget should be a useful tool. The conclusion to draw from this is
that the widget functionality was not apparent to the students, which can be due to
either interaction problems or learning problems.
The best predictor for the perceived value of the platform was how well it
supported collaboration. This means that students who considered the platform to
support collaboration also considered it to be valuable and vice versa. The conclu-
sion we draw from this is that the collaborative aspects of the tool were something
that the students expected, and either perceived as present or not. This suggests that
the system was perceived primarily as a tool to support collaboration in the solving
of an assignment. The fulfillment of the perceived purpose of a tool determines its
value assessment. This is also a good result for the prototype as it shows that the
students who thought the system was a good platform for collaboration also thought
that the system would be useful in the context of other courses.
A perceived good overview seems to be very important for the overall impres-
sion of the system as it correlated with other important features, viz., the level of
participation and collaboration and the system’s impact on the motivation to
complete the assignment.
Generally the students felt highly engaged in the team effort of solving their
assignment. It is not a controversial claim that a good overview in the system also
positively affects the perceived presence of team members. Thus, if an important
goal for the system is to stress the value in collaboration, creating a sense of good
overview should be highly prioritized.
Lessons Learned and Outlook
The evaluation resulted in the following recommendations:

1. Investigate why some participants perceived the platform as good for supporting
collaboration and others not, as this appears to be an important determinant of
the general impression of the platform.
2. Keep the number of required interactions (like mouse clicks) at a minimal level.
This can be achieved by exploiting the widget communication even more. The
participants in the study did not have trouble understanding the automated
behavior of widgets even though it conceptually appears rather complex.
3. Reevaluate the platform in a course where it is possible to fully implement the
SRL pedagogy. Currently we do not know how users of the system would cope if
they were required to choose widgets by themselves. In the current study, the
students might not be aware of the intricate, self-establishing nature of the
widget communication and it is necessary to find out how this will be perceived
when users set up the learning space by themselves.
4. To address how complete SRL platforms are for students, future evaluation
surveys should include questions about whether any other means of contacting
group members were used in parallel, e.g., instant messaging (like MSN Mes-
senger, Facebook chat), conference call (like Skype), collaborative documents
(e.g., Google Drive). The aim should not be to create tools to replace existing,
well-functioning communication solutions but to complement these.
On the whole the results of the evaluation were encouraging and showed that
continued work on the ROLE SDK and platform was motivated.
Widget Bundles for Formal Learning: Lessons Learned
In this chapter, we described usage of ROLE technology in three different settings

for formal learning in higher education. Despite the differences in the three test
beds, the simple fact that ROLE technology was used to support teaching and
learning at each of the locations supports the claim of flexibility made by the ROLE
project. RWTH and SOCE were able to apply ROLE technology rather extensively
and from early on during the ROLE project, and their evaluation results had
significant influence on the later course of the project. The UU test bed took
place at a later stage in the project and was therefore able to use a prototype very
nearly resembling the project’s end product.
The evaluation work done at these three test beds has taught the ROLE consor-
tium valuable lessons that shaped the further work:
• The added value of ROLE has to be clearly visible to its users. What became
apparent in the two test beds SOCE and RWTH is that technology may not be
attractive in itself, but its added value needs to be clearly visible to the users.
This was voiced by users in the SJTU test bed, who have a full-time job, limited
time available for study and low digital literacy, but also found in the RWTH test
bed, which involves students with high digital literacy (similarly, evaluation
outcomes from the BILD test bed pointed to the need for communicating the
benefits of the ROLE project to organizations).
• The value of ROLE that needs to be made visible includes technological
solutions, but also psycho-pedagogical results, i.e., the benefits of SRL, group
work, etc. These results and solutions have to be explained in a way the average
user (organization, student and teacher) understands.
• Uptake of ROLE technology significantly depends on the availability of widgets.
If too few widgets are available for immediate use, then only few teachers will
employ such technology. Having a few general purpose widgets, such as widgets
teaching SRL or enabling discussions, is insufficient to attract users, instead they
ask for very domain-specific resources. Such demand can be fulfilled by offering
authoring tools.
• Finally, the primary users of PLEs in formal learning in higher education in the
three test beds described in this chapter were teachers, not students. In these
settings, teachers used ROLE technology to integrate tools into the daily teach-
ing that enabled additional learning activities. In such a setting, teachers served
as multipliers, who demonstrate the potential of personal learning environments
to their learners. For instance, the SOCE French teacher reported that having
worked with the PLE during the semester for doing homework, his students
asked whether and how they could access the PLE even after the lecture was
over. They said they found it so useful; they want to continue using it.
References
Blees I, Rittberger M. Web 2.0 learning environment: concept, implementation, evaluation.

eLearning Papers 15. 2009.
Bogdanov E, Ullrich C, Isaksson E, Palmér M, Gillet D. Towards PLEs through widget spaces in
Moodle. Comput Sci Inform Syst. 2013;11(1):443–60.
Ewert D, Schilberg D, Jeschke S. Problem-based learning of Object-oriented Programming with
LEGO Mindstorms and leJOS. In: INTED 2011 Proceedings, International Association of
Technology, Education and Development; 2011. p. 3011–7.
Hsiao I-H, Sosnovsky S, Brusilovsky P. Guiding students to the right questions: adaptive naviga-
tion support in an e-learning system for java programming. J Comput Assist Learn. 2010;26
(4):270–83.
Jonsson M. Using IT to improve learning methods: a usability analysis through user survey and
personas. Student thesis, Uppsala University; 2012. http://uu.diva-portal.org/smash/record.jsf?
pid¼diva2:544350
Knight JK, Wood WB. Teaching more by lecturing less. Cell Biol Educ. 2005 Winter;4(4):
298–310.
Larsson AO, Pär A. Right on track? Corporate Twitter use under pressure. In: Paper presented at
the Nordic Academy of Management conference, Stockholm, 22–24 August 2011.
Law EL-C, Nguyen-Ngoc AV. Fostering self-directed learning with social software: social
network analysis and content analysis. In: Proceedings of EC-TEL‘08. Berlin: Springer;
2008. p. 203–15.
Leonard W, Journet A, Ecklund R. Overcoming obstacles in teaching large-enrollment lab
courses. Am Biol Teacher. 1988;50(1):23–8.
Lind T, Laaksoharju M. Evaluation of the Uppsala University ROLE prototype. Uppsala Univer-
sity; 2012. http://uu.diva-portal.org/smash/record.jsf?pid¼diva2:634185
Minocha S. A study on the effective use of social software by further and higher education in the
UK to support student learning and engagement. Tech Rep JISC, Bristol. 2009.
Palmér M, Sire S, Bogdanov E, Gillet D, Wild F. Mapping web personal learning environments.
In: Proceedings of the second international workshop on Mash-Up Personal Learning Environ-
ments (MUPPLE’09), ECTEL. 2009.
Renzel D. An introduction to ROLE interwidget communication. Technical report, RWTH
Aachen University; 2011.
Rizzardini RH, Linares B, Mikroyannidis A, Schmitz H-C. Cloud services within a role-enabled
personal learning environment. In: 1st International workshop on cloud education environ-
ments (WCLOUD 2012).
Schmitz H-C, Kirschenmann U, Niemann K, Wolpers M. Contextualized attention metadata. In:
Roda C, editor. Human attention in digital environments. Cambridge: Cambridge University
Press; 2011. p. 186–209.
Shaw F, Crawford K, Burgess J, Bruns A. Twitter, crisis, and authority: the role of @QPSMedia
during the Queensland floods. Unpublished manuscript. 2011.
Shen L, Shen R. The pervasive learning platform of a Shanghai Online College—a large-scale test
bed for hybrid learning. In: First international conference on hybrid learning, ICHL 2008,
Hong Kong, China, August 13–15, 2008, Proceedings, Springer, 5169, 178-189 (2008)
Solorzano J. LeJOS. Sourceforge. 2012. http://lejos.sourceforge.net/
Ullrich C, Shen R, Gillet D. Not yet ready for everyone: an experience report about a personal
learning environment for language learning. In: Advances in web-based learning ICWL 2010,
Berlin, vol. 6483. Berlin: Springer; 2010. p. 269–78.
Venkatesh V, Bala H. Technology acceptance model 3 and a research agenda on interventions
decision sciences. Decis Sci. 2008;39:273–315.
Vieritz H, Schmitz H-C, Law EL-C, Scheffel M, Schilberg D, Jeschke S. A knowledge map tool
for supporting learning in information science. In: Foley O, Restivo MT, Uhomoibhi J,
Helfert M, editors. Proceedings CSEDU 2013. SCITEPRESS. 2013.
von der Heiden B, Richert A, Renzel D, Klamma R, Ullrich C, Shen R, Wolpers M, Friedrich
M. Early results of experiments with responsive open learning environments. J Univers
Comput Sci. 2011;17:451–71.
Zhang J. A cultural look at information and communication technologies in eastern education.
Educ Technol Res Dev. 2007;55(3):301–14.
Zuzak I, Ivankovic M, Budiselic I. A classification framework for web browser cross-context
communication. CoRR. 2011;1108.4770.
Case Study 2: Designing PLE for Higher
Education
Denis Gillet and Na Li
Abstract In this chapter, the concept of Personal Learning Environment is first

refined taken into account recent advances and the experience gathered in a
European research project dedicated to responsive open learning environments. A
prototypal implementation of a Web 2.0 platform enabling the construction, the
sharing, and the repurposing of personal learning environments is then introduced.
Participatory design and validation activities carried out in the framework of higher
education test beds aiming at understanding the benefits of personal learning
environments in academic institutions are presented. Finally, the broaden applica-
tion framework for the deployment and the adoption of open user-centric environ-
ments for learning and knowledge management is tackled with perspectives in
terms of supporting inquiry learning at school with online laboratories,
implementing connectivist massive open online courses or enabling agile informa-
tion systems for nongovernmental organizations.
Keywords Personal Learning Environment (PLE) • Informal learning • Higher

education • Social Learning • Web 2.0
Introduction
Trends
Higher education is transforming under both top-down and bottom-up pressure. On

one hand, national policies and international practices are pushing higher education
institutions to reinforce their branding to attract more students to prime the
researchers pump and to trigger more citations to boost rankings. On the other
hand, students currently enrolled in higher education study programs are born with
the Internet and have grown as teenagers with ubiquitous access to online resources
(free or considered as such), peers, and communities, thanks to flagship search
engines and social media platforms. A previous study (Vassileva 2008) has pointed
D. Gillet (*) • N. Li
Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne CH-1015, Switzerland

DOI 10.1007/978-3-319-02399-1_5
116 D. Gillet and N. Li
out that, today’s teenagers are used to learn in context, in response to a perceived
demand, or to solve a particular problem. They search the Internet to find articles,
videos, or any related materials, as well as scouting through their social networks to
find a person who may be able to help. These learning experiences are mostly
“solution-driven,” rather than “learning on principle.”
As a consequence, the school- and course-centric closed learning management
systems (LMS) do not fulfill and match anymore the institutions and students’
expectations and practices. They do not provide the necessary international visibil-
ity for the institutions to be recognized as excellence teaching centers and they do
not provide students with the open and persistent free access to resources and peers
they are used to. The activity- and student-centric model of personal learning
environments (PLE) better supports the opportunistic and agile scheme required
by students to interact with resources, experts, and peers for both social and
educational purposes. The LMS rather enforce the old teaching model focusing
mainly on top-down content delivery through lectures, slides, course notes, and
fully packed exercise sessions.
The current trend is to equip institutions, teachers, and students with skills and
technologies to combine local institutional resources with global open content from
the cloud, as well as to rely on internal and external support provided by peers or
experts.
Students have always exploited resources from libraries and relied on peer
interaction for learning outside the formal institutional settings and activities.
However, with the current ubiquitous access to information and communication
technologies, such informal learning activities and modalities are taking a more
important place in the higher education landscape (Gillet 2010).
Activities carried out in the ROLE1 project have contributed to investigate a
pedagogical framework to strengthen self-directed and informal learning, as well as
a technical framework to benefit from Web 2.0 PLE. One should also highlight that
in the informal setting considered in this chapter, the boundary between learning
and knowledge management (KM) is disappearing (Fig. 1).
In the next paragraphs, we provide insights on the current definition of PLE and
discuss alternative design and implementation approaches. Then, in section “Ded-
icated PLE Platform,” we present the social media platform designed in the
PALETTE (El Helou et al. 2009) and the ROLE (Gillet et al. 2010) projects to
enable interaction in online learning communities and agile construction and
exploitation of PLE by teachers and students. In section “Higher Education
Test beds,” test beds set to validate the benefits of such platforms in higher
education are presented together with evaluation results. In the ROLE project,
these test beds also played an important role for the participatory design and the
social requirement engineering processes. Finally, in section “Conclusions and
Perspectives,” conclusions are drawn and perspectives are provided in terms of
supporting inquiry learning at school with online laboratories, implementing
1
Case Study 2: Designing PLE for Higher Education 117
Fig. 1 Trends supporting

the emergence of personal
learning environments in
higher education
connectivist massive open online courses (cMOOCs) or enabling agile information

systems for nongovernmental organizations (NGO).
Definition of a PLE
The concept of PLE is not new. According to Wikipedia,2 it was first coined in the
1970s. It was however rediscovered and consolidated with the emergence of the
social Web (Web 2.0) that enabled users to take control of their online resources
and interact freely with peers worldwide. Nowadays, conferences series such as the
PLE Conference3 are fully dedicated to discuss issues and investigations related
to PLE.
Initially, definitions of PLE were enforcing the combination of the online and
physical resources to define the learning settings. As example, both the computer
applications and the physical books the students may use on a desk at home to carry
out a given activity can be considered as part of the corresponding personal learning
environment. The same could be said for peers sitting around a table in a cafeteria
and discussing or completing homework assignments. However, physical modali-
ties and the associated offline actions are difficult to identify and to track within
supporting online platforms; the tendency is then to ignore them or to consider them
as embodied in the user model. Other definitions were mainly focusing on the
concept of mashup of Web applications (apps), which happened to be too restrictive
(Wild et al. 2008).
The current conceptualization of a PLE is rather corresponding to an opportu-
nistic aggregation of online content, information, services, and people for a given
2
http://en.wikipedia.org/wiki/History_of_personal_learning_environments
3
http://pleconf.org
Fig. 2 The PLE as an

aggregation of information,
content, services, and
people
activity. The variety of physical interfaces enabling the access and the interaction
with the mentioned entities is also evolving and is currently integrating not only
desktop and laptop computers, but also mobile devices such as smart phones and
tablets. Data gathered or published by online smart devices can also be considered
as resources or channels. As a consequence, we can define a PLE as (Fig. 2) (Gillet
2013)
a shared online opportunistic and possibly ephemeral aggregation of communication
channels, cloud resources, Web applications, and communities or peers (directly or through
social media memberships), assembled in an agile way to define an interaction context for a
given learning or knowledge management purpose, and accessed through interactive
devices (computers, tablets, smart phones, . . .).
In this definition, communication channels correspond to live discussion streams

or notifications usually made of short text messages or links. Content is referring to
multimedia resources that exhibit some persistence. Loosely-coupled distributed
services and online tools are accessed using Web applications.
It is important to enforce that any digital ecosystem can be considered as a PLE
if it has been repurposed for learning and knowledge management. It is more the
intention of use than the design itself that defines the nature of the environment
(Charlier et al. 2010). Obviously, the construction of the environment is part of the
learning activities and is an important ingredient in the appropriation of the
resources and the motivation to carry out the corresponding learning activity. As
such, the PLE realm pushes further constructivism, by not only enforcing the
definition of personal activities and resources by learners, but also by shaping the
related interaction environments. As embedded in the PLE term, such environments
are personal. They are however not individual as most of the time interaction with
peers or experts is desired and supported. Finally, one should underline that each
learning or knowledge management activity carried out by a user may rely on
different tools and resources, and, as such may require the construction of a
different PLE. This is the reason why PLE are often ephemeral, i.e., they may be
abandoned once the activity is completed. It is also why their construction should be
agile, i.e., they should be shaped to each context and purpose.
The above definition clearly shows the importance of the agile construction of
the PLE by the users themselves. Such a construction requires quite high IT literacy
and self-directed learning skills. The higher education framework is hence an
interesting setting for validation of the PLE paradigm, as the acquisition of auton-
omy is one of the main educational objective and outcome. Section “Conclusions
and Perspectives” will especially show that students and early-stage researchers
enrolled in graduate studies programs form an interesting community to elicit PLE
requirements and evaluate PLE benefits.
The ROLE project attempted to provide guidance for the design of self-directed
activities and environments to students without the necessary level of autonomy,
but had troubles to provide evidence that IT tools could help students with limited
IT literacy to build IT environments on their own. For more autonomous students,
the definition of personal learning objectives and the selection of the corresponding
resources and services appeared to be generally carried out implicitly, requiring as
such no specific technical support.
Alternative Design and Implementation of PLE Platforms
If one would stick on the PLE definition proposed in the previous paragraph, there
should be neither design nor implementation of platforms enabling the construction
of PLE. Any set of communication channels, cloud resources, Web apps, and
people assembled by a user would become a PLE. As a matter of fact, most of
the students in higher education build their own learning environments without
identifying them as a PLE and even without noticing that they are actually building
learning environments. A simple set of URLs, a LinkedIn4 or a WhatsApp5 group
dedicated to a given topic studied could be considered as a PLE.
As the concept of a free ecosystem assembled by a user fits very well with the
definition of PLE, it makes it difficult to induce changes in the way higher education
institutions shared and exploit knowledge and support students in both their formal
and informal learning activities. It also makes difficult the development of IT
literacy and autonomy for bachelor students in the usage of the social media
platforms and channels for educational purposes. Social media platforms and
channels are mainly considered as social interaction and entertainment tools. Last
but not least, the current digital ecosystem solutions do not facilitate the storage, the
sharing, and the repurposing of personal aggregations related to the study of given
topics or the completion of specific activities. As a consequence, teachers and
students have the tendency to continuously reinvent the wheel in putting together
their learning environments.
4
https://www.linkedin.com
5
http://www.whatsapp.com
Following the above comments, it is still beneficial to adapt current LMS to

enable a more flexible exploitation and enrichment of the learning resources and
contexts by the students, as well as to provide alternative platforms to enable the
construction and the sharing of PLE by the users for the users. The next section
describes the Web 2.0 PLE platform designed and developed in the framework of
successive technology enhanced learning research projects, especially in ROLE, to
support the construction of PLE, their sharing and their repurposing.
Dedicated PLE Platform
Design Objectives and Models
As stated before, a PLE is an aggregation of dedicated channels, resources, apps,

and people by a user, i.e., typically a teacher or a learner, for a specific learning or
knowledge management purpose. So, any platform supporting the construction of
PLE should support the aggregation of the mentioned entities and their hosting. For
the sake of symmetry in the way we treat the various entities being part of a PLE,
we talk about the aggregation of people, which in fact means the ability of sharing
the PLE with peers or experts and the possibility of repurposing it for their own or
collaborative usage. When dealing with entities gathered from the cloud, due to
their plethora, search and recommendation features are required. Finally, in an open
framework where users may prefer different platforms to exploit their own PLE or
the ones shared by others, import and export features in open Web standards are
important (Fig. 3).
The concept of PLE being abstract, for design and implementation purpose it
should be materialized as an online context dedicated to support a selected indi-
vidual or shared activity. As a consequence, we coined the concept of online spaces
as
Fig. 3 Required features

for PLE platforms enabling
the agile construction and
exploitation of personal
learning environments
the personal online places in which communication channels, cloud resources, Web apps
and people are aggregated to support specific individual or shared activities.
This space concept has first been introduced in Gillet et al. (2008), then formal-
ized in El Helou et al. (2010) as the 3A model, and finally standardized in Bogdanov
et al. (2011) as an OpenSocial6 space specification. Hence, in the rest of this
chapter, the notion of PLE, space and learning context are considered as equivalent.
Obviously, a particular case of a share activity is an individual activity and a
specific case of cloud resource is an internal resource (belonging to an institutional
or an enterprise cloud).
The agile creation by aggregation and the repurposing of a PLE being by essence
a user-centric self-directed activity, the user should be able to set easily privacy
settings and to assign specific roles and rights to the various people he or she is
sharing the PLE with, i.e., the people he or she is carrying out the activity with.
Such people are considered as the members of the shared online space. The lack of a
fine control in the privacy settings and in the selection of people for specific
activities is one of the reasons why LMS and flagship social media platforms are
not very convenient to support the creation of PLE. The other reason seems that
users do not want to mix their social and professional or educational networks
online, and do not perceive a connection between their online activities and learning
in classrooms (Greenhow and Robelia 2009).
The notion of open educational resources (OER) is embedded and even extended
in PLE. In effect, in addition to promoting the sharing of resources like multimedia
documents, in the PLE framework we promote the sharing of Web apps offering
dedicated services only required for specific activities and the sharing of the PLE
themselves. In (Gillet and Bogdanov 2013), it is shown how a PLE instantiated as
an OpenSocial space can be openly or privately shared as a meta-widget.
OpenSocial apps (widgets, gadgets or meta-widgets) are in fact a combination of
xml and javascripts. They can be compared to readable digital artifacts and shared
using Creative Commons7 licenses. Such licenses are more suitable for the model of
applications as a service (AaaS) relying on distributed infrastructures than the old
open source licensing schemes working only for software that can be delivered on a
memory stick as a standalone package.
Prototypal Implementation
Taking into account the specifications for a PLE platform stated in section “Ded-
icated PLE Platform,” the Graasp8 platform has been designed and implemented
following an agile development methodology combined with participatory design
6
http://opensocial.org
7
http://creativecommons.org
8
http://graasp.epfl.ch
and validation in test beds, including the ones described in section “Higher Educa-
tion Test beds.” The elicited requirements could be summarized as the following
design objectives for the platform that should:
• Enable the creation of PLE without the intervention of system managers.
• Be free of rigid structures and contexts (such as a course structure with specific
types of activities and resources).
• Enable opportunistic and focused activities.
• Enable the aggregation of cloud resources.
• Enable agile management of resources in contexts.
• Enable the fine control of roles and rights in contexts.
• Provide open search and meaningful recommendation of resources, apps, and
people.
• Rely on open Web standards similar to the ones used by social media and
knowledge management platforms (avoiding carefully any specialized standard
like the one dedicated only to learning).
The core Graasp feature is to enable the creation and enforce the exploitation of
dedicated online spaces as activity contexts (PLE). These spaces are defined,
configured, shared, and populated by users, for themselves and for the audience
they choose. Graasp stands for grasping resources, apps, activity spaces, and
people. As a matter of fact, any space can embed subspaces supporting
subactivities. However, hierarchy is not enforced. Users may decide to create either
a flat or a hierarchical space structure to support their various learning or knowledge
management activities.
Graasp spaces can include members, resources, subspaces, and apps. In addi-
tion, each entity has its own description implemented as a wiki enabling collabo-
rative edition, a dedicated discussion thread, tags, and personal or public ratings,
enforcing in such a way contextual exploitation (see Fig. 10 in Chap. 8).
In Graasp, there are three audience levels. Spaces can be public, i.e., visible to
everybody, closed, i.e., restricted to their members (but external people can request
membership), or hidden, i.e., only accessible by invited members. There are also
three possible roles for the members of a space. They can be owner, which means
that they can add or remove resources, as well as invite members or revoke
memberships. The owner role can be assigned to more than one member of a
space, which is a unique Graasp feature that enables to pass responsibilities over
when required. People can be contributors, which means that that can add resources
and can create subspaces for which they have the full control of the participants.
Finally, people can be viewers, which means they can access but not alter the
content of a space. They can however post comments, which is an important part of
the asynchronous interaction in shared online activities.
The agile aggregation of cloud resources is supported by an open source plugin
architecture and implemented as a GraaspIt! bookmarklet which provides a
one-click aggregation of external resources in the Graasp clipboard for further
integration in spaces (see the Chap. 8 for more details).
Higher Education Test Beds
Offering a PLE platform to teachers and students in a higher education setting is a

way to provide them with more opportunities and facilities to exploit and repurpose
individually or collaboratively learning resources gathered from various sources.
For adoption, such platforms should bring a high added value compared to institu-
tional solutions extensively deployed like LMS. If a teacher wishes to share the
slides of one lecture with students enrolled in a class, using the institutional LMS is
obviously the way to go. However, if the objective is to help the students to develop
teamwork skills by completing autonomously a collaborative project involving not
only classmates, but also alumni or external experts, a PLE platform is a better and
more agile solution.
In this section, two test beds are described. First, the scenario and the evaluation
of the exploitation of a PLE platform for teamwork activities as part of a bachelor
course on Human Computer Interaction offered at Tongji University is presented.
The design of social media platforms being part of the syllabus, the advantage of
using a PLE platform in this case is twofold. It helps to illustrate the subject matter
and it enables an easy management of the self-defined activities and the
co-produced assets by the students themselves. Second, the support of inter-
institutional training activities on Science 2.0 practices and solutions for doctoral
students at the Swiss national level are detailed. Again, the benefits are twofold. The
doctoral students discover how social media platforms can support their daily
research activities and they also exploit the same platforms to exchange best
networking and dissemination practices.
These two cases emphasis the fact that using PLE or social media platforms in
higher education is simultaneously a will to help users in developing IT literacy and
teamwork skills, as well as a way to empower them in their learning practices.
Teamwork at the Bachelor Level
To examine the acceptability of Graasp in terms of supporting teamwork in higher

education, it was used as a collaborative platform in the Human Computer Inter-
action course offered at Tongji University in China during two consecutive years,
i.e., in 2011 and 2012. Twenty-eight undergraduate students were enrolled in the
course during the first year and 26 during the second.
In addition to attending traditional lecture sessions, the students had to complete
a collaborative design project using Graasp as a support platform. Both years, eight
participatory design teams of 3–4 students were freely formed. The students spent
about 10 h to design the mockup of a social media application. They also delivered
a short report and gave a final presentation to their classmates and experts.
After a short introduction to Graasp, students were encouraged to create their
project spaces, share resources with each other, play different roles for the
participatory design (manager, designer, user, developer), and work with different
project-oriented apps (task assignments, mockup edition, . . .). A survey was
conducted aiming at evaluating the acceptability of the platform in sustaining
collaborative learning. Results of the 2011 evaluation have been detailed by Li
et al. (2012). The importance of privacy settings for shared spaces was confirmed by
the fact that in 2011 50 % of the spaces created were set to public, 47 % closed and
3 % hidden, and in 2012, 60 % were set to public, 32 % closed and 8 % hidden. The
students had also to pick five adjectives that closely matched their personal reac-
tions to Graasp from a list of positive and negative words (Benedek and Miner
2002). A word cloud, showing the frequency of the selected adjectives is presented
in Fig. 4 and Fig. 5 for year 2011 and 2012, respectively. The word clouds show that
the personal control, the ubiquitous accessibility and the trust enforcement of the
PLE platform are part of the core added value. Results also show that improvements
were achieved through participatory design between the two evaluations as effi-
ciency and ease-of-use increased.
Students’ answers show that they do not have traditionally support platforms for
teamwork, for which they mainly use instant messaging applications (100 % in
2011 and 59 % in 2012) or email (64 % in 2011 and 38 % in 2012) simultaneously
or not. The introduction of Graasp was hence quite welcome. Students assessed that
thanks to this platform they could accomplish their teamwork more effectively and
that their motivation increased (only 10 % disagree with these statements). The fact
that they can easily share resources with teammates, freely organized them, and
seamlessly aggregate content from different sources were the main useful features
according to the students (Fig. 6).
This evaluation confirms that PLE platforms are useful for self-directed activi-
ties carried out by bachelor students and involving agile resources aggregation,
roles assignment as well as right management.
Fig. 4 A word cloud based

on frequency of selected
adjectives in 2011
Fig. 5 A word cloud based on frequency of selected adjectives in 2012
Fig. 6 Usefulness of the Graasp platform for teamwork
Science 2.0 Literacy in Doctoral Studies
Despite the fact that more and more European universities are putting together
doctoral courses or programs, most of the learning activities for Ph.D. students are
self-directed. As a consequence, doctoral students are really eager to develop their
skills to conduct research and learn from their peers. Acknowledging this need, the
Federation of the French-speaking Swiss universities (CUSO9), which is offering
inter-institutional continuing education programs, has decided to offer a soft skill
workshop series co-organized by the University of Fribourg, the University of
Geneva, and the EPFL to doctoral students of any discipline.
Three of these 2-day workshops were organized in 2011 and 2012. The first one
held in Geneva was described in (Bogdanov et al. 2012). The second and third one
held in Fribourg and in Lausanne, respectively, are discussed here.
9
http://www.cuso.ch
There were three objectives for the introduction of a PLE platform to support
these workshops. First of all, it helped the educators belonging to the organizing
institutions to collect and share their presentation material in advance and makes it
available to the participants for preparation purpose prior the face-to-face sessions.
This is a typical case when an open PLE platform is required as local institutional
platforms cannot be used due to access restrictions for external people. Second, the
PLE platform was exploited during the sessions by the participants to discuss and
share their own resources and to practice their IT skills. Third, the participants were
encouraged to continue to interact using the PLE platform after the events as an
emerging community of practice.
The main topics covered during the workshops were the search and exploitation
of digital resources (especially scientific references), digital intellectual property
rights, as well as Science 2.0 practices using PLE platforms and other Web 2.0
tools.
In the part dedicated to PLE, Graasp was quickly introduced. A 30-min activity
was then organized during which the participants had to build a personal space to
collect references and discuss the state-of-the-art of their Ph.D. thesis with peers
and with their supervisor. The participants created the spaces they deemed appro-
priate, invited selected people with the relevant roles, started to populate the spaces
with chosen resources, and initiated discussions. This activity gave them the
opportunity to discover the features of the Graasp PLE platform and experience
with the benefits of contextual resource aggregation and discussion for a specific
purpose.
The evaluation carried out at the end of the workshops was quite general and
covered all the topics and the platforms presented during the 2-day events. One of
the clear and obvious findings of the evaluation was that the students were
overloaded with new tools. Hence, these tools need to provide a very high added
value and have the potential to be exploited for the all duration of their Ph.D. studies
for convincing Ph.D. candidates to invest time to master and exploit them. The
main added value that was elicited for Graasp was its ability to aggregate resources
from different sources in dedicated spaces; the resources being either Web book-
marks with previews or online documents. The other features were too numerous
for the participants to really discover and benefit from them. Hence, we can
conclude that in this case, the PLE platform was more useful for the educators to
prepare, collect, and disseminate the teaching material in an agile and effective
way, than for the participants. As a matter of fact, this is a general outcome of the
ROLE project to highlight that sharing and repurposing of teaching materials
among teachers, which are indeed self-directed learners, is an essential need
which is effectively fulfilled with PLE platforms.
A side outcome that was expected from this workshop series was that, taking into
account that in Switzerland all doctoral candidates also act as teaching assistants,
the participants would disseminate their best IT practices and preferred PLE
platforms in their teaching activities with undergraduate and master students.
Whether this goal was achieved has not been evaluated.
Conclusions and Perspectives
The main purpose of PLE platforms is to bring more flexibility for organizations,
teachers, and students in the way they aggregate and exploit online resources and
services, while relying on peers and experts outside the formal class settings around
which the traditional curricula are still organized. As a consequence, the application
of the underlying PLE paradigms goes beyond the cases described in the previous
sections. In fact, interesting new application domains have emerged recently and
are described below. First of all, the concept of shared spaces integrating applica-
tions happens to fulfill to need of inquiry-learning education at school using online
labs. This case is described in section “Inquiry Learning Space for STEM Educa-
tion at School Using Online Labs.” As the informal aggregation of resources from
various sources and shared with various people is part of the definition of agile
knowledge management, large NGOs are becoming interested in the PLE platform
paradigm. The NGOs’ requirements and how they can contribute to the enrichment
of PLE platforms are described in section “Supporting NGOs.” Last but not least,
the current trend in delivering Massive Open Online Courses (MOOCs) using
revamped LMS platforms brings back the old question of finding the right balance
between teaching and learning in higher education. While the mainstream MOOCs
platforms like Coursera or EdX emphasis paced video content delivery, the PLE
platforms can enable connectivist MOOCs to be implemented by enabling teachers
or students to assemble and control the exploitation of openly available learning
resources. This case is discussed in section “Supporting Connectivist MOOCs.”
Inquiry-Learning Space for STEM Education at School Using

Online Labs
The European Commission is funding for 4 years (2012–2016) a large-scale

research project called Go-Lab aiming at promoting Science, Technology, Engi-
neering, and Mathematics (STEM) education at school using online labs (Gillet
et al. 2013). The solutions required to achieve this goal are trifold. First, Go-Lab has
to strengthen and support communities of STEM teachers. Second, Go-Lab has to
provide students with inquiry-learning spaces enabling the exploitation of online
labs with proper scaffolds. Third, the online labs should be accessible through open
Web apps facilitating their aggregation and repurposing by the teachers themselves.
The ability of the PLE platform presented in section “Prototypal Implementa-
tion” to support online communities, to create structured spaces, and to aggregate
Web apps fully fulfills the Go-Lab requirements. In Go-Lab, Graasp is exploited by
teachers to create inquiry-learning spaces in which they can organized all the
resources necessary for the students to carry out the five typical phases of inquiry
learning, i.e., orientation, conceptualization, investigation, conclusion, and discus-
sion (Fig. 7).
Fig. 7 The Graasp PLE platform exploited by teachers to create, share, and repurpose inquiry-
learning spaces dedicated to STEM education (in this case to exploit the Hypatia tool provided by
CERN to study to conservation of momentum in collisions of particles)
The lower or higher secondary school students typically carrying out inquiry-
learning activities having to concentrate on exploiting the online labs, obviously do
not interact with the PLE platform. However, the export feature of Graasp enables
the creation of a standalone version of the constructed PLE as an independent Web
app. The teacher can create this app once the PLE is ready by clicking on the share
button. A secret URL that can be shared with the students is then generated. It
provides access to a simple Web page in which the inquiry-learning phases are
represented by tabs under which the resources (including the labs) selected by the
teachers and the instructions given in the embedded spaces’ wiki are accessible
(Fig. 8).
This example shows how a PLE platform can be exploited by teachers to
construct advanced educational resources, which can not only be presented in a
simple form to students, but also shared and repurposed by other teachers for their
own classroom activities.
Fig. 8 The simple Web page corresponding to the inquiry-learning space constructed by a teacher
(Fig. 7) with embedded Web applications for inquiry learning at school
Supporting NGOs
The PLE and online contextual space features introduced in the previous sections
have attracted the interest of global institutions, organizations, and enterprises, and
especially nongovernmental organizations (NGOs), not for personal learning, but
for agile knowledge management. These knowledge-oriented organizations are
operating at a worldwide scale with distributed entities loosely integrated and
relying on local hierarchical structures (Gillet and Bogdanov 2013). NGOs have
also fast staff turnover and strong digital asset dependency, which require effective
knowledge management solutions. The ability to create contextual spaces managed
in an easy and agile way by freelance experts in the field or in operation is hence
very attractive to such organizations. Their main additional requirements compared
to the defined PLE platforms are:
• Being able to integrate aggregation and interaction spaces in matrix structures
(a space corresponding to a given project like Digging a Well in Ouagadougou
should be simultaneously located in a geographical area (Africa as an example),
in an activity domain (like water management), and a hierarchical department

(like operation).
• Managing access rights by roles, i.e., the owner of a space should be identified as
Project X Leader rather than John Smith (or both) to enable an easy transfer of
duties and resources to someone else when required.
• Enabling fine identity management such as fusion of user profiles when people
contribute to spaces with various credential (such as a general gmail account or
social media platform OpenID) for consolidation purpose.
• Facilitating changes in access rights, i.e., having no distinction between the
intranet and the extranet or the internal and external cloud infrastructures
(as an example, when public documents are drafted and then made publically
available, such feature eases the diffusion process).
• Guarantying the secrecy of some resources as NGOs are often dealing with
content which can be politically sensitive. As a consequence, the physical and
geographical location of the cloud resources should be fully defined and
controlled.
• Automating tagging of contributed content to ease search and recommendation
without diverting contributors from their main tasks.
• Enabling proper licensing schemes (such as Creative Commons) for digital
content so that public resources can only be used or shared under specific
conditions defined by the owner.
Such requirements are also very useful in the context of personal learning in
which people often combine digital identities (e.g., when they move from one
educational institution to another), where privacy issues are also critical (especially
when dealing with young students), and where agile role management makes
collaborative learning more effective. This section shows the interplay between
personal learning and knowledge management in terms of supporting platforms and
elicited requirements. In the future, design and validation between these two
domains should be conducted more closely for cross-fertilization.
Supporting Connectivist MOOCs
PLE platforms have recently attracted interest from educators looking for agile
solutions to develop connectivist MOOCs (Connectivist MOOCs 2013), referred as
cMOOCs. Compare to the mainstream MOOC platforms like Coursera or EdX
which are basically LMS open to external students, PLE platforms offer built-in
social media features to boost opportunistic interaction and informal exchanges
between students. These platforms can also help teachers and learners to aggregate
their own MOOCs from resources freely available in the Cloud under Creative
Commons licenses.
The possibility to add Web apps in PLE platforms is enabling an easy adaption
for their exploitation to support connectivist courses, and especially cMOOCs. Kop
et al. (Kop et al. 2011) highlighted that a connectivist course is based on four major
types of activities, i.e., Aggregation, Remixing, Repurposing, and Sharing, which
are the typical actions supported by PLE platforms. What is however missing in
PLE platforms compare to MOOC platforms is the support for the formal activities.
Especially, features to support coaching and assessment are missing, as well as
features to support the timing and the structuring of the activities, including the
associated content delivery and task assignment. Such features can easily be added
in PLE platforms simply by integrating dedicated Web apps. This possible agile
extension of the platforms through apps also enables the implementation of solu-
tions to support a broad range of MOOC models, from the most formal to the fully
connectivist ones.
Requirement elicitation for cMOOC support in PLE platforms carry out with
members of the RESCIF10 Network of Excellence in Engineering Sciences of the
French-speaking countries have highlighted that the following features are espe-
cially required:
• Peer evaluation support.
• Creation of quizzes, collection of the answers and analysis.
• Team building and competence bartering support.
• Formalization of time-based and topic-based structures through spaces (timing
and navigation) using tables of content, syllabuses or calendars for navigation
and exploration.
• Support of additional metadata through internal tags (automatically identified or
inherited from domain ontologies) to ease search and recommendation.
• Customization of the portal spaces hosting cMOOCs with graphical templates
enforcing branding or group identity building.
• Management of multilingual resources (Wikipedia model) supporting a given
activity to broaden the audience and the sharing opportunities with developing
and emerging countries.
• Tagging and subtitling of video sequences.
• Online recording and editing of video sequences.
• Integration of e-textbook standard documents (epub3).
These features can however be provided as specialized Web apps. Once inte-
grated in a space dedicated to a cMOOC, these apps are accessible to all members,
can be personalized and can store or retrieve information related and resources
belonging to this space.
When relying on a PLE platform, a cMOOC implementation facilitates the
co-production and co-exploitation of content between different teachers which
can provide only materials directly related to their core expertise and rely on
colleagues from other institutions for additional OER. Such an approach
implemented using a mainstream platform would require challenging intellectual
property right negotiations and bilateral conventions for exploitation. As such, the
10
http://www.rescif.net/en/rescif
PLE platform not only enables to flip the classrooms (by freeing classroom time for
personal interaction), but also to flip the institutions (by redefining the educational
mission towards collaborative high-quality content edition and accreditation).
Final Words
PLE as considered in higher education correspond simultaneously to a paradigm

change in the ways the information is shared and consumed, as well as a paradigm
change in the supporting technical ecosystem. Recognizing and acknowledging the
large variety of practices and platforms for both formal and informal activities
carried out by teachers and students is already an important institutional change.
Providing support in developing the necessary IT literacy of higher education
stakeholders and even providing alternative PLE platforms helping them to increase
their effectiveness in exploiting knowledge artifacts and exchanging competences
are the next level of the ongoing academic revolution. Contributions to support this
change have been formalized and illustrated in this chapter.
Acknowledgments This work was partially funded by the European Union in the context of the
ROLE (Grant Agreement no. 231396) and Go-Lab (Grant Agreement no. 317601) projects under
the Information and Communication Technologies (ICT) theme of the 7th Framework Programme
for R&D (FP7), as well as the Personal Learning Environment (Phase 3) project of the Swiss
AAA/SWITCH program and the Learning Infrastructure project of the Swiss CUS Programme
P-2 on Scientific Information: Access, Processing, and Storage.
References
Benedek J, Miner T. Measuring desirability: new methods for evaluating desirability in a usability
lab setting. Redmond: Microsoft Corporation; 2002.
Bogdanov E, Salzmann C, Gillet D (2011) Contextual spaces with functional skins as open social
extension. In: Fourth International conferences on advances in computer-human interactions
(ACHI), Feb 23–28, Guadeloupe, France.
Bogdanov E, Limpens F, Li N, El Helou S, Salzmann C, Gillet D (2012) A social media platform
in higher education. In: Global engineering education conference (EDUCON), Apr 17–20,
Marrakesh, Morocco.
Charlier B, Henri F, Peraya D, Gillet D (2010) From personal environment to Personal Learning
Environment. In: Fifth European Conference on Technology Enhanced Learning (EC-TEL),
Workshop on Mash-Up Personal Learning Environments (MUPPLE), Sept 28–Oct 1, Barce-
lona, Spain.
Connectivist MOOCs. 2013. http://www.connectivistmoocs.org. Accessed 27 June 2013.
El Helou S, Gillet D, Salzmann C, Rekik Y. Social software for sustaining interaction, collabo-
ration and learning in communities of practice. In: Karacapilidis N, editor. Solutions and
innovations in web-based technologies for augmented learning: improved platforms, tools,
and applications, Advances in web-based learning (AWBL) book series. Hershey: IGI Global;
2009. p. 300–16.
El Helou S, Li N, Gillet D. The 3A interaction model: towards bridging the gap between formal
and informal learning. In: Third International conference on advances in computer-human
interactions (ACHI), Feb 10–16, 2010, St. Maarten, Netherlands Antilles.
Gillet D. Tackling engineering education research challenges: Web 2.0 social software for
personal learning. Int J Eng Educ. 2010;26(5):1134–43.
Gillet D. Personal learning environments as enablers for connectivist MOOCs. In: 12th Inter-
national conference on information technology based higher education and training (ITHET),
Oct 10–12, 2013, Antalya, Turkey.
Gillet D, Bogdanov E. Cloud-savvy contextual spaces as agile personal learning environments or
informal knowledge management solutions. In: 12th International conference on information
technology based higher education and training (ITHET), Oct 10–12, 2013, Antalya, Turkey.
Gillet D, El Helou S, Yu CM, Salzmann C. Turning Web 2.0 Social software into versatile
collaborative learning solutions. In: First International conference on advances in computer-
human interaction (ACHI), Feb 10–15, 2008, Sainte Luce, Martinique
Gillet D, Law E, Chatterjee A. Personal learning environments in a global higher engineering
education Web 2.0 Realm. In: Global engineering education conference (EDUCON), Apr 14–
16, 2010, Madrid, Spain.
Gillet D, de Jong T, Sotirou S, Salzmann C. Personalised learning spaces and federated online labs
for STEM education at school: supporting teacher communities and inquiry learning. In: 4th
IEEE global engineering education conference (EDUCON), March 13–15, 2013, Berlin,
Germany.
Greenhow C, Robelia B. Informal learning and identity formation in online social networks.
Learn Media Technol. 2009;34(2):119–40.
Kop R, Fournier H, Mak SFJ. A pedagogy of abundance or a pedagogy to support human beings?
Participant support on massive open online courses. Int Rev Res Open Dist Learn. 2011;12(7).
Li N, El Helou S, Gillet D. Using social media for collaborative learning in higher education: A
case study. In: Fifth International conference on advances in computer-human interactions
(ACHI), Jan 30–Feb 4, 2012, Valencia, Spain.
Vassileva J. Toward social learning environments. IEEE Trans Learn Technol. 2008;1(4):
199–214.
Wild F, Mödritscher F, Sigurdarson S. Designing for change: mash-up personal learning environ-
ments. 2008. eLearning Papers 9.
Case Study 3: Exploring Open Educational
Resources for Informal Learning
Alexander Mikroyannidis and Teresa Connolly
Abstract This chapter explores the potential of informal learning within a Per-
sonal Learning Environment (PLE), as well as the identified informal learning
cultures that have evolved from the use of Open Educational Resources (OER). A
variety of research instruments and strategies have been employed to promote the
use of PLEs in this case study and capture a rich variety of feedback from
Communities of Practice. In particular, there is a focus on the active use of a PLE
and its integration with OER available from the OpenLearn project of the Open
University. Additionally, this chapter describes the discovered necessary guidance
conditions, emergent contrasting learning contexts and evolving different scenarios
in use within the selected Communities of Practice. This research has led to the
identification of valuable lessons as well as the documentation of challenges that
are faced by those using PLEs in the context of informal learning scenarios.
Keywords Informal learning • Learning culture • Open Educational Resources • Self-

Regulated Learning • Personal Learning Environment
Description of Case Study
This case study focuses on the analysis of the informal learning opportunities
presented by the Responsive Open Learning Environment (ROLE) project. Essen-
tially, this research contains a series of different informal learning scenarios to
examine, each of which will be assessed separately. The premise of informal
learning (The Organisation for Economic Co-operation and Development
(OECD) 2010) in this chapter relates to learning that has been gained through
experience and not necessarily from an organised standpoint i.e. the opposite of
formal education where pathways are often prescriptive and delivered from an
instructor.
A. Mikroyannidis (*) • T. Connolly

Knowledge Media Institute, The Open University, Milton Keynes MK7 6AA, UK

DOI 10.1007/978-3-319-02399-1_6
136 A. Mikroyannidis and T. Connolly
Additionally the information presented in this chapter relates primarily to the use of,
as well as outputs of the OpenLearn project1—an Open Educational Resources (OER)
repository of the Open University (OU). OpenLearn offers in excess of 12,000 h of self-
study materials in a variety of formats. These include study materials repurposed as
OER from original OU courses, i.e. initially designed for formal delivery, as well as
new bespoke OER created by both OpenLearn academics and other non-OU educators;
in other words, further facilitating OER available for informal delivery. As the majority
of the study materials presented on the OpenLearn platform are made available using a
Creative Commons licence,2 it has been possible not only to reuse selected information
but also to author new OER materials for the ROLE project, which can be freely shared
to a worldwide audience.
This chapter sets out to examine how an existing platform designed for global
OER delivery, i.e. OpenLearn, can be enhanced with the introduction of Personal
Learning Environment (PLE) technologies. It also considers a selection of social
aspects of informal learning because this often plays a key role within a community
of learners (Lane 2008a). These groups may have similar backgrounds or goals, so
that a PLE can be used to support them throughout their informal learning process.
With this in mind, it is also important to remember that the wider OU staff
community is allied to the OpenLearn repository. Thus access to OpenLearn has
enabled further investigations to take place in active Communities of Practice that
contain a cross section of Higher Education (HE) staff: academic, academic-related,
technician and librarian colleagues.
A number of research instruments were deployed to gather data and information
from these groups. This included using a variety of dissemination opportunities
ranging from seminars relating to ROLE and the use of PLEs to the development of
an interactive eBook describing Self Regulated Learning (SRL), plus presentations
and conference publications. Table 1 outlines all the events and activities that were
monitored, as well as their location and date.
In this chapter, a selection of these events will be described in full detail.
Evaluation activities will also be outlined and the subsequent analysis of the
research instruments’ information presented in Chap. 3. Since many of the events
and activities have overlapping learning contexts and objectives, they are illustrated
initially in more general terms in the next section. Representative events will be
described in more detail in subsequent sections.
Learning Context and Objectives
The first research instrument used in this case study is the seminar/workshop type of
events. In terms of a learning context, these events comprised of an introductory
presentation about the ROLE project, followed by the setting of a framework for the
1
http://www.open.edu/openlearn.
2
http://creativecommons.org/licenses/by-nc-sa/3.0/.
Case Study 3: Exploring Open Educational Resources for Informal Learning 137
Table 1 Overview of the events, activities and artifacts that constitute this case study’s findings
Case study Date Location
1. Events (seminars and workshops)
JTEL Summer School workshop May 2011 Crete, Greece
SCORE Seminar July 2011 Milton Keynes, UK
OU eLC Seminar January 2012 Milton Keynes, UK
Build a Widget Day March 2012 Milton Keynes, UK
JTEL Summer School workshop May 2012 Estoril, Portugal
PLE Conference 2012 workshop July 2012 Aveiro, Portugal
Dev8eD conference workshop May 2012 Birmingham, UK
ITCM seminar July 2012 Milton Keynes, UK
International Workshop on Cloud November 2012 Antigua, Guatemala
Education Environments
2. Activities
ICALT 2011 paper July 2011 Athens, USA
Chapter in Collaborative Learning March 2012 Hershey, USA
book
OER World Congress conference June 2012 Paris, France
ICALT 2012 paper July 2012 Rome, Italy
PLE Conference 2012 paper July 2012 Aveiro, Portugal
AACE E-Learning Round October 2012 Montréal, Canada
Table discussion
AACE E-Learning paper October 2012 Montréal, Canada
3. Artifacts
ROLE online course Summer 2011 OpenLearn website: http://tinyurl.
com/role-course
ROLE online SRL course Spring 2012 OpenLearn website: http://tinyurl.
com/role-srl-course
ROLE SRL eBook December 2013 Apple iBook Store: http://bit.ly/
self-regulated-learning
subsequent workshop element of the seminar, i.e. the use of widgets inside a PLE
for learning purposes. All participants had access to a laptop or tablet computer in
order to engage with the provided online learning materials.
The workshop element of the events then allowed participants to be introduced
to a hands-on and guided use of a PLE. Participants used one of the two bespoke
ROLE online courses, generally sharing the experiences in groups of two or more
learners. Initially the presenter gave a short summary of workshop objectives but
thereafter the individual groups explored the ROLE online course(s) for them-
selves, by following the self-study structured learning materials. The presenter
(s) remained nearby in the room and circulated amongst the participants, answering
questions as required. In the final section of the ROLE online courses, there was an
opportunity to evaluate the experience and all participants were, therefore, invited
to complete a short questionnaire.
The second type of informal learning activity involved presenting and, therefore,
disseminating information directly about the ROLE project at a variety of interna-
tional conferences. This activity took the form of a presentation followed by a
question and answer session. Again, this gave the opportunity for the presenter to
illustrate particular aspects of the ROLE project, highlighting individual and
relevant PLE elements, as well as enabling them to indicate the existence of the
bespoke ROLE online learning courses in the form of self-study interactive OER
materials. Attendees were encouraged to visit both ROLE online courses post-
event, in order to learn more about the project and to complete the short question-
naire related to their learning experience of using the bespoke OER materials
(Mikroyannidis and Connolly 2013b).
Dissemination about the ROLE online courses also took the form of using
posters. These were used as a visual medium to promote and signpost relevant
information related to both the PLE aspect and also the availability of the structured
self-study OER materials. Posters were displayed at a number of international
conferences and proved to be popular talking points for those attending the events
and, in effect, an unanticipated informal learning opportunity for enquirers. The
posters gave a visual representation of the ROLE project and provided an oppor-
tunity for individuals to enquire and learn more about PLEs from the ROLE
representative(s) who presented the poster. Further promotional materials in the
form of bookmarks were also available, enabling visitors to “take away” some
tangible and memorable signposts with further details about different aspects
of PLEs.
A third learning context for this case study is a group described as “artifacts” that
have provided bespoke opportunities for informal learning. This group includes the
creation of both ROLE online courses. As indicated previously, these practical
courses were often embedded into events that were focused on raising awareness
about the PLEs. They were also available online for anyone to study at their own
time. A further artifact that provided an innovative informal learning platform has
been the adaptation of the ROLE online courses into an eBook. Initially, the content
of the first ROLE online course was taken and adapted for presentation as an eBook.
As a pioneering development itself the ROLE online course had been through a
number of iterations and changes, ultimately being published with somewhat
different emphasis in terms of content. Likewise, the ROLE eBook followed a
similar pattern of development. Whilst the learning objectives for the eBook
remained relatively similar to those in the original ROLE online course,
i.e. introductory, the actual learning content of the published ROLE eBook was
somewhat different in form. This was because the SRL online course materials
were merged with the original ROLE online course content. The resulting interac-
tive eBook thus contained the selected contents from both courses.
In summary, it can be said that the underlying curricula presented in each of the
previously described themed informal learning events or activities has been,
broadly speaking, very similar: i.e. an introduction to the ROLE project’s purpose
and, in particular, a focus on the use of OER within a PLE to support informal
learning. On occasion, specific topics were presented, as required, to the different
Communities of Practice too. In addition, information relating to any necessary

guidance for the associated workshop events was given where relevant. Further-
more, the specific development of the two ROLE online self-study courses also
enabled either attending participants, or those directed to the courses at a later date,
to study the materials at their own pace and, potentially, at a time or place of their
choice. Finally, the repurposing of the online course contents as an eBook has
enabled the dissemination of information about PLEs and by using associated OER
materials has facilitated it to be widely distributed as well as used in many further,
and unanticipated, informal learning contexts.
Setup and Organisation of Learning Activities
As previously described, the range of informal learning events offered by the OU

has been divided into three broad dissemination categories: seminar and/or work-
shop events, conference presentations and thirdly the development of bespoke
learning activities that have been tailored to meet the identified learning needs
and objectives of particular Communities of Practice. Consequently, the associated
setup and organisation of the related learning activities was also often custom-made
to meet the needs of an event’s anticipated or identified audience. It is reasonable to
state, however, that most of the embedded learning activities were developed from
an original basic master set and then adapted and/or repurposed for each of the
individual contexts thus actively implementing one of the fundamental themes of
OER that of reuse (Hilton et al. 2010). The underlying premise of adaptation
described here having been to identify an appropriate ROLE widget, then create a
structured activity around it and subsequently produce a set of achievable self-study
tasks related to the topic.
How that learning activity was embedded into the individual event or into the
online environment varied according to a number of factors such as adhering to the
adopted pedagogic model of the event; assessing the availability and suitability of
technology for the learning activity, e.g. considering the appropriateness of the
delivery platform, such as Moodle or the eBook. It was also necessary to reflect
upon whether the learning activity could be evaluated at a later date using relevant
and appropriate research instruments too. The mediation of the learning activity
also had to be well thought out. The basic master set of learning activities had been
originally developed for use as self-study materials, thus necessitating the addi-
tional “presence” of structured and carefully crafted feedback for the user to read,
understand and so allowing learning to take place (Conole et al. 2011). Interest-
ingly, in the many face-to-face events it was possible to offer supplementary
feedback and answer participants’ queries instantly, as required, thus providing
another unanticipated blended form of delivery. Mentors, as such, were not pro-
vided for any activities as such but, as previously described, there were several
opportunities in the many seminars, workshops, conference presentations as well as
alongside poster displays for presenters to give supplementary guidance and pro-
vide signposts to further details about the use of PLEs.
Learning Culture
The OER-based informal learning materials that were developed by the OU initially
adhered to distance-learning pedagogic principles. This basic premise was later
adapted to suit a blended model of delivery according to the circumstances of the
individual events. The learning culture as such could, therefore, be described as
promoting or adhering to one of self-study further described as attractive to those
who have a high level of SRL (Zimmerman 1989). Additionally, in general terms of
the types of people taking part in the events it can be reported that the majority of
those participating, online as well as face-to-face, were staff from HE institutions,
often the OU but not exclusively so. As previously described, participants were
drawn from a number of different Communities of Practice, including academics,
academic-related-staff, technicians as well as librarian colleagues.
PLE Intervention
The overall PLE intervention in this case study was essentially established from a
basis of embedding structured learning activities that used selected ROLE widgets
and then delivered them via a web interface. The chosen platform employed to
display both online courses was the LabSpace3 area of OpenLearn, which uses
Moodle for the delivery of materials and associated educational technology
(McAndrew et al. 2009). The PLE in this illustration, thus, involved a Moodle
environment as well as the use of the selected ROLE widgets that were embedded in
the online courses, enabling participants to access a whole array of further OER
materials in their informal learning context.
Table 2 lists the contents of the two ROLE online courses. The first course
introduced the concepts and technologies of the project. It provided a combination
of tools and services that enable learners to build their own PLE based on their
needs and preferences. It also established a course template that was subsequently
reused for the development of the second online course focusing on SRL. The
template was based on an original OpenLearn study unit format that had been
established in the LabSpace area of the website (Lane 2008b). Associated struc-
tured learning activities, using widgets, encouraged participants to explore and
discover further OER materials or tools to enhance their knowledge.
3
http://labspace.open.ac.uk/.
Table 2 The ROLE and SRL online course tables of contents

A. The ROLE online course B. The SRL online course
1.0 Introduction 1.0 Introduction
1.1 Overview 1.1 Overview
1.2 Learning outcomes 1.2 Learning outcomes
1.3 Definitions 1.3 Definitions
1.4 About ROLE 1.4 About ROLE
2. Example ROLE widgets 2. Self-Regulated Learning
2.1 Introduction 2.1 What is Self-Regulated Learning
2.2 Social search widget: Binocs 2.2 A typical learner: Marcus
Activity 1: Search for OER
2.3 Bibliography search widget: ObjectSpot 2.3 Travel scenario
Activity 2: Search for references
2.4 Videoconferencing widget: FlashMeeting 2.4 Flora’s learning approach
Activity 3: Search for FM replays
2.5 Collaborative authoring widget: EtherPad 2.5 Tim’s SRL approach
Activity 4: Use the EtherPad
2.6 Different learning approaches
2.7 ROLE and SRL
Activity 1: Assess your SRL skills
3. Building a PLE 3. An SRL scenario
3.1 Introduction 3.1 Amanda’s SRL journey
3.2 Using Google 3.2 Amanda sets her learning goals
Activity 5: Create a Google account Activity 2: Setting your learning goals
3.3 Adding a ROLE widget to iGoogle 3.3 Amanda looks for learning tools
Activity 6: Add the FM widget to iGoogle Activity 3: Looking for learning tools
3.4 The ROLE widget store 3.4 Amanda uses the learning tools
3.5 The Google gadget directory 3.5 Amanda reflects on the process
Activity 4: Reflecting on your learning
4. Conclusion and bibliography 4. The PPIM
4.1 PPIM overview
Activity 5: Using a PPIM tool
Evaluation questionnaire Conclusion and bibliography
Evaluation questionnaire
The second online course, focusing on SRL, followed a similar pattern in terms
of layout. It also has sections, learning outcomes, and embedded activities. Both
courses were released under a Creative Commons licence, as indicated earlier, thus
enabling their contents to be not only used or studied in situ but also to be “taken
away” and, potentially, repurposed elsewhere within the terms of the licence. In
other words, whilst the intended purpose of the two courses was to raise awareness
about specific aspects of the ROLE project by introducing selected widgets in a
PLE to access further OER resources, the design and deployment of the course
materials as OER also meant that they could be reused by others too. Again this
adheres to a fundamental premise of OER and that materials, tools and technology
should be freely shared and accessible to a wide audience (Hilton et al. 2010).
OpenLearn as a PLE
The OpenLearn project has changed significantly since its launch in October 2006
(Lane and Law 2012). It was originally designed as an OER repository, using two
websites: LearningSpace and LabSpace. It set out to offer a full range of HE
academic subject materials ranging from the arts and history to science and nature,
at all study levels from access to postgraduate. Commencing with 900 h of study
materials in LearningSpace and 900 h in LabSpace, the now enhanced OpenLearn
website currently offers in excess of 1,200 study hours. A number of changes have
taken place since the launch, the most notable being the significant increase of
available types and styles of study materials. In 2010, the OpenLearn brand also
expanded in size and content to incorporate a significant number of both audio and
video materials from the former Open2.net website—a platform that had been
developed to support joint OU-BBC programmes designed to encourage public
engagement with materials related to a variety of HE subjects. It has been reported
that there have been 24 million unique visitors and approximately 320,000 regis-
tered OpenLearn visitors (Lane et al. 2013).
Through offering the original OER study materials, and then further developing
the OpenLearn website to incorporate the Open2.net resources too, the OU has also
endeavoured to add value to its Open Content by deploying leading edge Learning
Management System (LMS) technologies for learner support. At the same time,
using such an approach, it has also actively sought to encourage the creation of
informal collaborative learning communities. Alongside these developments, the
OU has also pursued the development of international research-based knowledge
about modern pedagogies for HE (Sharples et al. 2012). These improvements have
also presented the ROLE project with an excellent opportunity to gain access to a
wide cross section of learners and educators who engage with OpenLearn OER
materials.
The ROLE project has embraced OpenLearn, both in terms of building on its
experience of creating OER study materials, as well as enabling access to some of
its constituent informal learning communities. By using the LabSpace area of
OpenLearn for the development of the two ROLE online courses it has also been
possible to not only introduce the idea of a PLE but also offer direct access to
selected ROLE widgets that permit end-users to create their own PLE. Whilst
materials for the two courses were hosted on OpenLearn as self-study OER units
they can, of course as previously indicated, be used in face-to-face settings too.
Again, this enhances and improves the possibility of accessing a further variety of
audiences that may have an informal learning context. OpenLearn also provided
ROLE with the potential access to an unanticipated large global audience.
Evaluation Objectives and Instruments
This case study consisted of a number of events, activities and artifacts that enabled
different approaches to be taken to explore the use of OER materials for informal
learning. The overall aim of the research was to ascertain how PLEs could be
implemented for different groups of educators in the HE sector. The first evaluation
objective, therefore, was to determine which of the selected ROLE widgets were
appropriate for the different audiences, as well as assess the impact of introducing
the idea of PLEs to those who may not have been aware of these technologies. It
was important to gauge whether people were receptive to the idea of PLEs in
addition to trying to determine if individuals were prepared to adopt any of the
widgets in their teaching delivery.
The basic research instrument deployed in the different events was the Perceived
Usefulness and Ease of Use (PUEU) Survey, which has been based on the Tech-
nology Acceptance Model (TAM) (Venkatesh and Bala 2008; Venkatesh and Davis
2000). The questions in this survey were used in all the test beds of the ROLE
project and were tailored thereafter for each event according to the composition of
the surveyed groups. Necessary changes to the original survey were, in reality,
minimal. The PUEU survey was available online,4 where all data and information
was also collated. It is important to note that each of the ROLE test beds in fact used
the PUEU survey thus allowing further analysis across the project to take place. As
noted earlier, the PUEU survey was also included as an evaluation opportunity
within both of the ROLE online courses. Again it contained the original questions
slightly tailored for these informal Communities of Practice.
Where appropriate, additional research instruments were used. For instance,
during the workshop elements of an event, the ROLE presenters were able to
offer help to individuals or groups as required but in doing so could also observe,
first-hand, any pertinent issues that arose. Whilst this was not a systematic collec-
tion of data, more so observational and unplanned, it did serve as an excellent
opportunity to see how ROLE widgets, in particular, were received, understood and
used. Observational notes were recorded. This view of participants’ engagement, or
not, with a PLE also enabled the presenters to gauge the usability not only of the
selected ROLE widgets but also participants’ interactivity with the two ROLE
online courses. In some respects these observations could be described, therefore,
as informal research instruments as previously indicated.
The presenters also could observe any perceived impact that the ROLE widgets
had on an individual’s informal learning. This was, of course, more difficult to
ascertain as an observer but was often reinforced when participants, on occasion,
requested help repeatedly thus indicating that they were having difficulty in under-
standing some aspect of the course. Furthermore, observation could also be
employed to ascertain the acceptance of the PLE by different HE groups. Again,
4
http://fit-bscw.fit.fraunhofer.de/pub/bscw.cgi/39523090.
whilst this was not a systematic collection of data or information, it did serve to
supplement the recorded responses in the PUEU surveys.
Methodology, Evaluation and Participants
There are three themes for this case study (see Table 1). Firstly, the seminar and
workshop group events. Secondly, the dissemination activities that involved
presenting information directly about the ROLE project at a variety of international
conferences. Finally, the third group contained a selection of mediating artifacts
that provided opportunities for informal learning. Selected examples from each of
these three groups will now be described in terms of research methodology.
Event 1: The eLC Seminar
This was an opportunity to introduce the ROLE project to the e-Learning Commu-
nity (eLC) of the OU and took place in January 2012. The eLC has more than
300 members including those from both academic and related staff in the OU
campus as well as potentially comprising of many Associate Lecturers of the OU
who are based throughout the 4 Nations and 13 regions in the UK. The eLC offers a
regular programme of workshops and seminars to OU staff, also available to invited
visitors, and covering a wide range of innovative e-Learning-related educational
technology projects.
The ROLE seminar presented an opportunity firstly to describe and then, in the
workshop element, encourage the attending 20 eLC members to use the online
ROLE courses. As previously mentioned, the self-study units include introductory
text about the ROLE project and have various structured activities that enable the
learner to use a selection of ROLE widgets. The ROLE widgets that the participants
were invited to use are shown in Fig. 1 and are the following:
• Binocs: A widget used to search for OER in a number of Web 2.0 repositories,
such as YouTube, SlideShare and Wikipedia.
• ObjectSpot: A widget employed for bibliographic searching in popular biblio-
graphic indexes, such as DBLP and Google Scholar.
• EtherPad: A widget used for synchronous collaborative authoring of a docu-
ment, where participants shared the OER they found in the other two widgets, as
well as their experiences from using the widgets.
By using the online self-study ROLE courses in the workshop, the 20 participants
were also able to gain further insights into how to build or adapt a PLE. This was
achieved by completing the online courses, which included guided access to the
ROLE Widget Store5 where further relevant, learning and teaching widgets are
5
http://www.role-widgetstore.eu.
Fig. 1 A selection of ROLE widgets for finding and sharing OER
located. After the initial presentation defining an overview of the ROLE project, its
aims and specifically outlining the objective of PLEs, eLC participants were invited
to follow the structured activities in the ROLE online courses. During the structured
activities, the ROLE presenters moved around the room offering help to
participants as required and attempting to aid them with their engagement with the
ROLE widgets on request. On the whole, this meant fielding a wide variety of
questions, as well as offering constructive advice about how PLEs functioned and
where to find out more information relating to this developing area. The ROLE
presenters did not actively seek to intervene in the participants learning processes
and only responded when requested to do so by individuals. In effect, their presence
in the workshop part of the eLC event offered a blended learning opportunity to
participants and similarly acted as an informal research instrument to observe
participant interactions first-hand.
Event 2: “Build Your Own PLE” Workshop, JTEL Summer School
This event was held in Estoril, Portugal during May 2012. It was established
originally by the EU-funded ProLearn project and subsequently supported by the
European Association for Technology-Enhanced Learning (EATEL) along with
other EU networks of excellence. PhD students in the area of Technology-Enhanced
Learning (TEL) from across Europe spend the JTEL Summer School week learning
about the latest trends in TEL, and exchanging ideas about their Ph.D. projects.
The “Build your own PLE” workshop was delivered during the JTEL event.
Some 14 students attended the session and used a variety of ROLE widgets in order
to find learning resources and start building their own PLE according to their
research interests. Both the ROLE online courses were used in the workshop.
Other ROLE workshops were also held during the JTEL summer school and
these included a coding session using the ROLE SDK (a development service that
focused on communication and collaboration), a widget design session, as well as a
session concentrating on personalised support for SRL. Thus the “Build your own
PLE” event was one of a family of sessions focusing on the ROLE project. Once
again, the “Build your own PLE” workshop enabled participants to use the widgets
previously described in Event 1.
Event 3: “Build Your ROLE” Workshop, PLE Conference
The third annual PLE conference took place simultaneously in Aveiro, Portugal and
Melbourne, Australia in July 2012. Researchers, educators and practitioners in TEL
and PLE were brought together for a lively exchange of ideas, practices and visions.
A number of ROLE partners, including the OU, delivered a half-day workshop
entitled “Build your Responsive Open Learning Environment”.6 Participants were
introduced to the ROLE tools and learning methodologies and were encouraged to
use these tools in order to build their own Responsive Open Learning Environment.
Additionally, they were able to design their desired tools, according to their
learning scenarios and requirements, as well as submit the results to a subsequent
ROLE widget competition.7 Once again, there was a focus on the two ROLE online
courses within the workshop, along with an opportunity to complete the PUEU
survey.
6
http://projects.kmi.open.ac.uk/role/pleconf-workshop.
7
http://www.role-project.eu/WidgetCompetition.
A paper was also presented at the PLE conference, concerning some of the
emergent lessons learned from the OpenLearn test bed of the ROLE project. The
presenter described the use of widget-based PLEs by informal learners who sought
and discovered new OER materials as a result of using the ROLE widgets. The
presentation took the form of a “speed-dating” style i.e. not a ubiquitous
PowerPoint. A recording of this presentation and the slides were made available
after the conference too (Mikroyannidis and Connolly 2012a).
Event 4: The Dev8eD Workshop 2012
Dev8eD is organised by the Developer Community Supporting Innovation
(DevSCI), a community of developers in the learning provider sector. Thus the
Dev8eD event, held in Birmingham, UK in May 2012, was designed for developers,
educational technologists and users working throughout education, who wanted to
further the development of tools, widgets, apps and other resources for education.
The ROLE workshop was attended by ten conference delegates. Participants
were first introduced to the ROLE project through a brief presentation and then had
the chance to use selected ROLE tools during organised group activities. The
purpose of these activities was to enable participants to understand how a PLE
can be used to support them in their everyday learning and research tasks. Addi-
tionally, participants had the opportunity to build their own PLE according to their
own learning and research activities.
Event 5: International Workshop on Cloud Education Environments
A workshop was hosted by Galileo University in November 2012 in Antigua,
Guatemala. It focused on the exchange of the relevant trends and research results,
as well as the presentation of practical experiences gained while developing and
testing cloud education environments, both from a teaching and a learning perspec-
tive. This workshop raised awareness about both the ROLE project and the function
of PLEs in cloud-based environments. Once again, the two ROLE online courses
were used initially to attract the new external stakeholders, as well as to underpin
this workshop. The workshop was focused on a cloud education environment by
examining how such informal OER materials and services can be distributed using
a number of different publication channels.
Activity 1: The ROLE Poster and Other Publicity Materials
The Paris OER Declaration was formally adopted during the 2012 World OER
Congress held at the UNESCO Headquarters in Paris in June 2012. Over 550 del-
egates including representatives of governments, educators, NGOs, and interna-
tional universities attended the Congress, which was organised in full partnership
with the Commonwealth of Learning (COL) and supported by the William and
Flora Hewlett Foundation (USA).
A poster was displayed in the exhibition area, promoting the two ROLE online
courses.8 The poster gave a visual focal point to which congress participants could
8
http://news.kmi.open.ac.uk/11/18424.
attend and enquire about the ROLE project. It was an excellent opportunity to
promote and disseminate information about ROLE to a wide range of international
delegations. This event proved to be an excellent opportunity to promote both the
project and PLE developments in a significant OER global gathering. In turn and, as
recorded earlier, the poster also acted as a visual mediating artifact that enabled
informal learning about the ROLE project to take place.
Activity 2: World Conference on E-Learning Paper
This is an international conference organised by the Association for the Advance-
ment of Computing in Education (AACE) and co-sponsored by the International
Journal on E-Learning. It was held in Montréal, Canada, during October 2012. The
conference serves as a multidisciplinary forum for the exchange of information on
research, development, and applications of all topics related to e-Learning in the
corporate, government, healthcare, and HE sectors.
A paper was presented describing a number of the lessons learned as well as the
best practices that were observed from the findings of the ROLE OU test bed in
summer 2012 (Mikroyannidis and Connolly 2012b). A round table discussion,
involving eight people, also took place hosted by the OU. It set out to explore the
challenges associated with supporting SRL in HE.
Artifact 1: The ROLE SRL eBook
As previously mentioned, a fundamental aspect of OER is the ability to share and
potentially, therefore, try to encourage the reuse of the developed materials (Hilton
et al. 2010). It can also be argued that in doing this, it is possible to reach out,
disseminate and make contact with new and, possibly, unanticipated audiences. The
two ROLE online courses, for example, had been designed with this in mind: reuse
and sharing potentially using multiple formats and platforms. The materials that
were presented in the LabSpace area of OpenLearn were designed using structured
content and XML. This enabled them to be transferable to other platforms as
OpenLearn offers numerous export facilities, for example: Moodle backup,
SCORM, and IMS package. OpenLearn also allows its structured authoring docu-
ments to be used as databases (Hirst 2012).
During 2012–2013, the OU has been exploring as well as taking advantage of
new and innovative ways to widen participation in its courses and associated
informal learning tools (Connolly 2013; Lane and Law 2012). Whilst there has
been a focus on OpenLearn as a vehicle to achieve this, further platforms including
Apple’s iBook Store and YouTube have also been used as opportunities for
informal learning. The advent of the eBook has offered a new opportunity to
harness not only existing structured content but also include levels of interactivity
previously restricted to the LMS platform (Moodle). As a distance teaching insti-
tution, the OU has always endeavoured to extend the boundaries of publishing, as
well as take advantage of educational technology to do so. Thus “rethinking” the
publication of printed books as eBooks has offered the opportunity to not only alter
models of production (i.e. from print to online to mobile) but also to “open out” and
extend the fundamental and familiar idea of a book by creating new and exciting
experiences for the readers.
With this in mind, the publication of the ROLE SRL eBook (Mikroyannidis
et al. 2013) has taken place alongside a growing series of interactive concept
publications produced by the OU.9 It has taken advantage of the HTML5 technol-
ogy to produce an eBook that can include interactive ROLE widgets and other
inline resources. As a consequence, the ROLE eBook provides an introduction to
new learning technologies that empower the reader in terms of SRL and by
providing access to information about as well as using PLEs. In effect, it is selected
content from the previously published two ROLE online courses that makes this
interactivity and raised awareness possible. A selection of learning tools has been
included that will help an individual to build his/her own PLE and encourages
him/her to become a self-regulated learner too. Readers have an opportunity to try
these tools through a set of interactive learning activities included in this eBook.
The Evaluation’s Participants
As previously mentioned the primary research instrument for the majority of the
described events and activities in this case study was the PUEU survey. Essentially,
the survey was used to gain an understanding of how participants from different
Communities of Practice have attended ROLE events and perceived the usefulness
and ease of use for a number of selected and presented ROLE widgets in addition to
capturing their experience of using the two ROLE online courses. As indicated
earlier, a number of further observational notes have been collected. The latter will
only be reported here to verify and support information collected from the PUEU
survey.
The objectives for the overall evaluation, therefore, focused on participants’
understanding of the concept of PLEs, their use of specific widgets and capturing
knowledge of their opinions and interactions with bespoke ROLE products such as
the eBook. In this respect, the PUEU survey has proven to be quite useful as it
embraced all of these objectives. The survey was easily accessible to all workshop
participants and users of the ROLE online courses.
The numbers and profiles of the people who attended each of the OU-led events
varied enormously according to location and timing. In summary, it can be reported
that the groups ranged in size from 10 to 50. The majority of participants were staff
from the HE sector although one group was PhD students. The attendees’ age range
appeared to be between 21 and 50. In general terms most people who completed the
survey had some experience of TEL although few had either practice or full
understanding of the potential that a PLE could offer. It is reasonable to say that
there was a fairly even spread in terms of gender across all events. Table 3
summarises the profiles of the participants in each event or activity.
9
http://projects.kmi.open.ac.uk/ib/.
Table 3 Brief profiles of those participating in selected events or activities

Events Numbers Occupation
JTEL Summer School workshop 2011 25 PhD students
SCORE seminar 10 HE e-Learning teachers
OU eLC seminar 2012 20 HE e-Learning teachers
Build a Widget Day 14 e-Learning trainers, managers
JTEL Summer School workshop 2012 14 Ph.D. students
Dev8eD conference workshop 10 e-Learning developers
ITCM seminar workshop 10 HE e-Learning practitioners
Cloud Education Environments workshop 50 HE e-Learning practitioners
The PLE Conference 2012 20 e-Learning practitioners
Activities
ICALT 2012 paper 20 e-Learning researchers
AACE E-LEARN conference round table 8 e-Learning practitioners
Evaluation Results
The evaluation results will not be described for each of the completed events and
activities, but reported for the first group of events only. As previously mentioned,
the main research instrument to be deployed for this case study was the PUEU
survey. The majority of results of this survey has been recorded for the events and
will, therefore, be presented here. A number of the observations made by the ROLE
presenters will also be included. In addition, comments and quotes recorded by the
participants via the EtherPad widget will also be presented where appropriate.
Table 4 summarises the themed groupings of this case study.
The eLC Seminar
The respondents to the PUEU survey appeared to have an even split of knowledge
amongst them in relation to TEL. Conversely, however, a significant 88 % of those
participants felt that they had “some” rather than a “good” knowledge of PLEs. In
other words it appeared that the group as a whole was relatively new to the idea of
a PLE.
Observation of the interaction with the EtherPad widget revealed that some
participants were hesitant to use this type of technology and required encourage-
ment from their peers or more experienced colleagues (either those in the group or
from either of the two ROLE representatives who were facilitating the workshop).
The types of information recorded by the participants ranged from anxiety to
amazement that such tools could enable individuals to learn collaboratively.
There was also clear evidence, however, that a more experienced group member
took advantage of the EtherPad widget to communicate with a colleague in another
Table 4 The groups used in this case study

Case study groupings Carried out Surveys deployed
1. Seminar and/or workshop 9 Yes
2. Conference dissemination 7 No
3. Bespoke activity 3 No
part of the room about some mutual work unrelated to the subject in hand
(i.e. ROLE and PLEs). It should be noted, nonetheless, that the same person used
the widget firstly to give advice to his colleague regarding more effective use of the
tool before moving onto the separate non-workshop subject. What this also
revealed, of course, is that despite encouragement to explore ROLE widgets it
was not a compulsory activity and at least one participant chose to continue his own
non-ROLE/PLE work as well as participating in the workshop.
It appeared that the EtherPad widget was used more constructively to exchange
as well as record pertinent PLE-related information. For example, one participant
detailed a blog address that they felt to be “an interesting take on PLEs. Just a tad
off topic. . . (Note to self this is the wiki)”. Others noted comments relating to
enquiries about how each ROLE widget functioned. These ranged from “how are
keywords supposed to work” to “cannot get ObjectSpot to show on iPad2”. Indeed
this idea of recording questions was taken somewhat further by one participant who
remarked: “Why do we need two widgets for search?”
The final question in the survey asked participants to record their feelings about
their use of PLEs. There was a 77 % agreement that PLEs would be slightly useful
for participants work, followed by a slight disagreement that the same PLEs would
help participants accomplish their work more effectively than their current use of
learning technology. Again this was not surprising as most group members were
established and experienced users of learning technology and had revealed that they
only had limited use of PLEs. Half the group proffered a neutral response to the
statement relating to “It would be easy for me to use a PLE” whilst the remainder
recorded that there was a slight chance that that would be the case. There was a
more even spread of responses to the statement “It would be clear to me how to
assemble a PLE using widgets” ranging from slight disagreement (11 %) through to
slight agreement (also 11 %). Most participants remained neutral on the subject.
Interestingly, the statement “I would find using a PLE frustrating” invited the
most disagreement to be recorded with the majority (55 %) remaining neutral
alongside 33 % saying they slightly disagreed and 11 % strongly disagreeing.
Once again, the statement “I would find interacting with a PLE requires a lot of
mental effort” statement invited a strong neutrality (55 %) yet 22 % of participants
strongly disagreed with this premise whilst 11 % recorded that they slightly agreed
that this would be the case for them.
The remaining part of the survey related to participants’ motivation to using a
PLE in their learning process whereby 55 % remained neutral in their responses and
44 % slightly agreeing with this statement. The last statement of “I predict that I
would frequently use a PLE if I had access to it” invited an even response (33 %)
between slightly disagreeing through neutral to slightly agreeing.
The JTEL Summer School Workshops
A ROLE workshop took place in both the 2011 and 2012 JTEL Summer Schools.
The participating research students were enthusiastic and willing to try out the
widgets within the ROLE online courses. Each workshop had the same format of an
introduction to the ROLE project, followed by the practical activity of using the
widgets within the ROLE online courses. In general terms the research students’
overall opinion in both workshops was a positive one. They engaged with the
structured activities, actively used all of the provided tools as well as recording
their thoughts (and sometimes frustrations) in the EtherPad widget.
The EtherPad widget was used in many different ways in the 2011 workshop.
Most of the research students used the tool although some were a little surprised by
the real-time aspect of it: “. . .somebody is writing on the screen!!!! I am scared. . .”
Others considered additional aspects to the experience in that it highlighted some
potential gaps in their own skill set: “. . .I probably have to work on my search
skills. . .” By contrast in the 2012 workshop, however, the EtherPad widget was
used actively by only a few of the participants. Generally, it functioned as a means
to record and exchange URLs of relevant resources such as the “Learn Portuguese
language vocabulary” YouTube video10 that the participant described as: “This is a
great video”. It was also used in identifying a Stephen Downes Slideshare presen-
tation about “Personal Learning The Web 2.0 Way”11
All participants in both workshops were aged between 20 and 40 years. There
was also an equal 50 % male/female split. In answer to the question relating to the
participants’ knowledge of TEL, in the first workshop there was a significantly
higher response rate to the “some” option whilst in both workshops most stated that
they had a “good” knowledge of TEL. In the second workshop with regard to the
question related to PLEs, however, there was a greater spread of responses: 50 %
recorded that they had a good knowledge whilst 25 % stated a “good” knowledge
and the remaining 25 % claimed to be an expert in the field of PLEs. The free-text
responses within the questionnaire provided some insight into the participants’
views of PLEs as well as the use of the ROLE widgets.
The question: “What did you think of the widgets of the workshop activities?”
also invoked a variety of responses in both workshops, most acknowledging that the
widgets were interesting. In addition others said: “I have found them very useful for
10
Learn Portuguese language vocabulary YouTube video http://www.youtube.com/watch?
v¼bzR1q3ZAlKQ&feature¼youtube_gdata_player.
11
Stephen Downes Slideshare presentation http://www.slideshare.net/Downes/personal-learning-
the-web-20-way.
my PhD and for my learning” along with a remark that (the widgets were): “Small
apps which can expand your daily routine”. Others simply said that the widgets and
interface: “look good”. It is also important to note a word of caution too though,
summarised by one respondent who remarked: “Found them quite interesting.
Collaborative text editor had lower quality in contrast to GDoc, surely I would
not use it”.
In relation to the question: “Were you able to find suitable widgets for building
your PLE during the second workshop activity?” there was almost unanimous
response in the second workshop alluding to the intermittent internet issues that
appeared to plague the entire session. One quote, perhaps, sums up the frustration
that most participants felt dominated, their experiences: “Internet connection issues
did not let us perform this activity”. Nonetheless, 90 % of the same participants
responded positively to the next question: “Did you find the workshop activities
useful for your research needs and goals?” which can be summarised by one
remark: “I found it an interesting approach to be tested in the future”. One
respondent did, however, offer a rather more circumspect response: “I do not
know. I have to check those pages more when I come home”.
Opinions from both the workshops’ attendees about widget-based PLEs were, in
general terms, evenly spread. One notable exception, however, was the response
from workshop 2: “I would find interacting with a PLE requires a lot of my mental
effort”. Nonetheless, the respondents overwhelmingly recorded that they were
neutral in their feelings about this statement for the second workshop. Research
students in the first workshop appeared to be more discerning in their learning and,
as noted by the ROLE presenter, the majority of the students appeared to focus on
the Binocs widget rather than the ObjectSpot widget. Again this seemed to colour
their view of the overall experience of using the ROLE technology.
The Dev8eD 2012 Workshop
This workshop was attended by developers and learning technologists, predomi-

nantly male and in the age range of 20–50. Most participants recorded that they had
a good knowledge of PLEs but some also declared little or no knowledge of this
area. Once again the EtherPad was used during the event to record notes and
information relating to the workshop that participants wanted to share with each
other. In this event, however, the participants took a more strategic view of the
EtherPad by using it to store personal observations such as:
I’ve used Etherpad before. The problem with these synchronous writing tools is the way the
connection suddenly stumbles and your flow is disturbed—Just had this problem with
Etherpad. It was static for a while so I assumed people were still having connection
problems—then I clicked to type and got a huge update!
In addition others used the EtherPad to record their thoughts about the activities
themselves as well as how the widgets worked or performed, for example:
Some searches return “60 results” apparently—all of them YouTube videos—I untick
YouTube as an option and get 54 results—but there were way more than 6 YT videos in
my previous list.
The term ‘reflection’ isn’t especially useful for search. . . Binocs’ first result may be the
kind of ‘reflection’ I’m looking for but ObjectSpot results cover a range of different types of
‘reflection’.
At the end of the workshop, participants were encouraged to complete the PUEU
survey. Again this gave them an opportunity to record their thoughts about both the
ROLE widgets, the implementation of them in PLEs as well as the activities of the
workshop itself. Responses for the question: “What did you think of the widgets of
the workshop activities?” were mainly positive but with reservations about the
mechanisms that were used to make the widgets function. Participants were con-
structive in their observations saying, for example:
Could be useful, though a few flaws here and there. Binocs had a odd way of searching and
filtering. Etherpad is a great idea but it didn’t always sync correctly and would jump-start
again when clicked. The Mash-Up Recommender is great but not all widgets were
installable to iGoogle!
There was also positive affirmation that the workshop enabled the participants to
be introduced to new widgets, for instance: “The widget can be useful to put
different tools together” and: “Useful because I had never heard of or seen these
widgets”. Recognition, however, was also given to the technical issues such as:
“Some problems with Etherpad on the iGoogle page—would be better if it sized
down. Also the Binocs broke”.
The question: “Were you able to find learning resources that relate to your topic
(s) of interest during the first workshop activity?” also invited a variety of
responses. Some participants: “Found some things but would need more time to
explore—will do that soon” whilst others were circumspect: “yes, though would
like to understand better why two separate search boxes. I’m guessing one is API
driven, one is custom Google search? could they be combined?”. Remaining
responses affirmed that the participants were, generally, happy using the ROLE
widgets, for example: “Yes, I searched for 6lowpan and found some relevant videos
and slides” and “I tried one topic relevant to my institution. The resources were
good”.
There was also overall positive response to the question: “Were you able to find
suitable widgets for building your PLE during the second workshop activity?” once
again, however, the participants did not hesitate to record their actual experiences
of installing widgets or not, for example: “The ones I did find looked useful but
didn’t add to iGoogle (there was an error)”. Another participant also remarked: “I
think so, but I’d need to think about how I’d integrate it with my other tools (like
Evernote, Twitter, Google calendar)” demonstrating that as a developer or educa-
tional technologist that they were giving some thought to the application as well as
implementation of the ROLE widgets.
There was an overwhelming positive response to the question: “Did you find the
workshop activities useful for your research/teaching/learning needs and goals?”
once again summed up by: “Yes, a useful overview/primer of what’s possible” as
well as: “Yes, Very useful”. Similarly the final question of the survey offered
participants the opportunity to add any remaining questions, comments or sugges-
tions that they wished to record. Some insightful comments were made such as:
I still have lots to learn about this area but this was an informative session to get me
started!” as well as: “I can’t help thinking that if I have the digital literacy skills and
confidence to create a PLE then I don’t need a PLE. (a bit of a paradox!)
Lessons Learned
From earlier, interim, research about this case study (Mikroyannidis and Connolly
2013a) three main themes were identified:
• The usability of the learning tools, i.e. the widgets.
• A consideration of the types and styles of related learning activity formats (often
embedded in the ROLE online courses).
• A reflection as well as action upon suitable methods that might encourage future
participants to consider, engage and continue using PLEs for their own learning
purposes.
In respect of the additional events, activities and artifacts described here it can be
reported that a number of further lessons have been learned. These will be consid-
ered in the context of successes and failures. The underlying lesson learned was the
importance of ensuring that all the technologies are stable, available and accessible
at the time of engagement as this leads to successful deployment. Additionally,
planning an event, activity or even an artifact should also include a level of
adaptation or localisation for particular audiences e.g. PhD students have different
requirements to the more experienced researchers.
The creation of the first ROLE online course alongside the development of the
second ROLE online course focused on SRL were invaluable in the success of all
the components described in this case study. Each course had been structured in a
similar manner (using the same template) and thus contained learning outcomes,
defined learning activity opportunities and clear signposts to relevant ROLE wid-
gets. Thus the use of such structured content made its repurposing as an eBook
much easier to carry out. Consequently, it was possible to build on the success of
both ROLE online courses by raising awareness about PLEs and SRL through an
additional informal based educational channel: the interactive eBook was devel-
oped. Similarly the eBook reused the previously developed course template by
providing an introduction to PLEs and SRL as well as giving an opportunity to
readers to try a selection of ROLE widgets through a set of embedded interactive
learning activities.
Successes and Failures
The level of success for this case study can be measured in a number of different
ways. Both qualitative and quantitative data has already been presented that outline
both the positive and negative impacts that were observed by ROLE presenters or
recorded by participants in the PUEU survey. Most participants were willing and
able to take part in the ROLE workshops although some were hesitant to use this
type of technology. Those people sometimes required encouragement from their
peers or more experienced colleagues in order that they make progress. Most of the
free-text responses recorded in the PUEU survey ranged across the possible spec-
trum of experiences from anxiety to amazement in terms of the potential use of such
widgets to enable collaborative learning for example. It is fair to say, with this
supporting evidence, that the case study has been successful in terms of raising
awareness of PLEs to a cross section of HE staff and groups of research students.
It would be unfair to state that there were direct failures in this case study.
Nevertheless, it should be noted that when technology was intermittent in terms of
access this had a dire influence on the experiences gained by those attempting to
complete the workshop activities. Frustrations with widgets not fully working or a
simple breakdown in internet access had a very negative impact on all participants
but in particular proved to be major stumbling blocks for those who were less
confident or competent with the PLE or individual widgets.
Best Practices for PLE Adoption in Informal Learning
As a result of recording the successes and reflecting on some of the perceived

failures in this case study, it is possible to list a set of best practices for the adoption
of PLEs by informal learners:
• Accessible and easy-to-use tools: Best practice in this case study indicates quite
clearly that a simple format for the ROLE tools is required that enables a range
of learners to use them effectively and efficiently.
• Multi-format introductory and guidance learning course materials: Learners are
in need of guided learning materials that will help them understand the value of
the new technologies. Best practice in this respect included the development of
the ROLE online courses and the ROLE eBook.
• Tailor tools to meet the needs of specific subject audiences: Best practice here
should be to enable adjustment or even design for learners studying particular
subjects or, alternatively, educators researching a wide range of topics to be
implemented as required.
• Tools that harvest information from appropriate repositories/platforms: A set of
generic search widgets were fully tested in this case study thus enabling use
across a variety of learning contexts. As a consequence, some ROLE widgets
(e.g. Binocs and ObjectSpot) have been successfully used and repurposed in
other test beds of the ROLE project.
• Fostering a culture with a community willing to engage with new innovative
technology: In order to maximise the adoption of PLEs, a suitable culture
towards new technologies needs to be fostered. This case study demonstrates
that the teacher or trainer may also need to adapt their own approach in order that
they too are receptive to change.
Conclusions
The different components of this case study have enabled us to extend our under-
standing of the potential impact of the ROLE technologies within a wide variety of
informal learning contexts and scenarios. This case study has numerous rich
contexts in which there is potential for significant impacts of both PLEs and SRL.
The challenges faced and lessons learned in each component of this case study have
also been reported here. As with every new technology, some resistance was
expected and initially faced in most cases by the participants be they educators or
managers in HE institutions or likewise e-Learning practitioners elsewhere. Over-
whelmingly, however, most of the ROLE tools were positively received by those
who participated in the events, activities, or used the artifacts. Each event, activity
or artifact that has been explored has also involved a cross section of representative
individuals. This approach has enabled the ROLE project as a whole to collect
experiences covering a large variety of learning contexts and requirements.
Acknowledgement The research leading to these results has received funding from the European
Community’s Seventh Framework Programme (FP7/2007–2013) under grant agreement
No. 231396 (ROLE project).
References
Connolly T. Learning support guides: roles in informal open learning courses. Report for the Open
Media Unit. Milton Keynes, UK: The Open University; 2013.
Conole G, McAndrew P, Dimitriadis Y. The role of CSCL pedagogical patterns as mediating
artefacts for repurposing Open Educational Resources. In: Pozzi F, Persico D, editors. Tech-
niques for fostering collaboration in online learning communities: theoretical and practical
perspectives. Hershey: IGI Global; 2011.
Hilton J, Wiley D, Stein J, Johnson A. The four R’s of openness and ALMS analysis: frameworks
for open educational resources. Open Learn J Open Dist Learn. 2010;25(1):37–44.
Hirst T. Making the most of structured content: data products from OpenLearn content. In: Internet
librarian international conference, Olympia, London, UK; 2012. http://blog.ouseful.info/2012/
11/02/my-ili2012-presentation-derived-products-from-openlearnou-xml-documents/.
Lane A. Am I good enough? The mediated use of OER to empower learners in excluded
communities. In: Fifth pan-commonwealth forum on open learning, London, UK; 2008a.
http://www.pcf5.london.ac.uk/.
Lane A. From pillar to post: exploring the issues involved in re-purposing distance learning
materials for use as Open Educational Resources. In: OpenLearn Working paper no. 2. Milton
Keynes, UK; 2008b. http://kn.open.ac.uk/public/document.cfm?docid¼9724.
Lane A, Law A. Open engagement through open media. Higher education academy. 2012. http://
www.heacademy.ac.uk/assets/documents/oer/OER_CS_Andy_Lane_Open_Engagement_Through_
Open_Media.pdf.
Lane A, Law A, Dewis L. Data, data everywhere, but ne’er time to think: identifying patterns in
user behaviour across changing open media channels. In: Open educational resources confer-
ence (OER13), Nottingham, UK; 2013. https://www.medev.ac.uk/static/applications/
117e064b-015b-4c92-bd35-1b275372b6ef/Lane_Law_OER13_Paper.docx.
McAndrew P, Santos A, Lane A, Godwin S, Okada A, Wilson T, Connolly T, Ferreira G,
Buckingham Shum S, Bretts J, Webb R. OpenLearn research report 2006–2008. Milton
Keynes, UK: The Open University; 2009. http://oro.open.ac.uk/17513/.
Mikroyannidis A, Connolly T. Introducing personal learning environments to informal learners:
lessons learned from the OpenLearn case study. In: PLE conference, Aveiro, Portugal; 2012a.
http://oro.open.ac.uk/34501/.
Mikroyannidis A, Connolly T. Responsive open learning environments at the Open University. In:
World conference on e-learning in corporate, government, healthcare & higher education
(E-Learn), Montreal, Canada; 2012b. http://academicexperts.org/conf/elearn/2012/papers/
37248/. Association for the Advancement of Computing in Education (AACE).
Mikroyannidis A, Connolly T. Introducing personal learning environments to informal learners.
In: Okada A, editors. Open educational resources and social networks. São Luı́s, Brazil:
EdUEMA; 2013a. http://oer.kmi.open.ac.uk/?page_id¼1254.
Mikroyannidis A, Connolly T. Test bed evaluation report. In: ROLE project deliverable. 2013b.
http://www.role-project.eu/wp-content/uploads-role/2009/02/role-deliverable-5.4.pdf.
Mikroyannidis A, Connolly T, Berthold M. Self regulated learning: iPad edition. The Open
University; 2013.
Sharples M, McAndrew P, Weller M, Ferguson F, Fitzgerald E, Hirst T, Mor Y, Gaved M,
Whitelock D. Innovating pedagogy exploring new forms of teaching, learning and assessment,
to guide educators and policy makers. In: Open University innovation report 1. Milton Keynes,
UK; 2012. http://www.open.ac.uk/personalpages/mike.sharples/Reports/Innovating_Peda
gogy_report_July_2012.pdf.
The Organisation for Economic Co-Operation and Development (OECD). Recognising
non-formal and Informal learning: outcomes, policies and practices. 2010. http://www.oecd.
org/edu/country-studies/recognisingnon-
formalandinformallearningoutcomespoliciesandpractices.htm.
Venkatesh V, Bala H. Technology acceptance model 3 and a research agenda on interventions.
Decis Sci. 2008;39(2):273–315.
Venkatesh V, Davis FD. A theoretical extension of the technology acceptance model: four
longitudinal field studies. Manag Sci. 2000;46(2):186–204.
Zimmerman BJ. A social cognitive view of self-regulated academic learning. J Educ Psychol.
1989;81(3):329–39.
Case Study 4: Technology Enhanced
Workplace Learning
Michael Werkle, Manuel Schmidt, Diana Dikke, and Simon Schwantzer
Abstract Within the ROLE research project it has been evaluated how personal
learning environments (PLEs) perform in different surrounding conditions. Com-
panies do often lag behind in the state-of-the-art developments from research
especially in terms of Technology Enhanced Learning. Festo therefore observed
on its internal test bed how PLEs can be implemented in business contexts and how
to involve the learners in this process. Since there is already a broad variety of tools
to organise and manage formal learning processes within companies the test bed
didn’t start by scratch either. The focus was thus on how to open up an existing
learning management system (LMS) towards a PLE. During this process many
experience from both learners as well as administrators, and training organisers
have been gathered. One of the lessons learned is that a pure PLE doesn’t fit the
requirements on personnel development in business context, but certain PLE
aspects can improve individual learning processes significantly. One showcase is
the Festo LearningTube which was developed during ROLE. This is an example for
the successful integration of user generated content into a corporate LMS.
Keywords Technology enhanced workplace learning • Personal Learning

Management System • Evaluation • Test bed
Challenges and Solutions in Technology Enhanced

Workplace Learning
The competitiveness of a company depends strongly on the skills and abilities of its
managers and employees. The development of information and communication
technologies (ICT) offers a wide range of tools and application options or even
completely new forms of learning in this regard. Thanks to the development of the
M. Werkle (*) • M. Schmidt

Festo Lernzentrum Saar GmbH, St.Ingbert-Rohrbach, Germany
D. Dikke • S. Schwantzer
Information multimedia communication AG, Saarbrücken, Germany

DOI 10.1007/978-3-319-02399-1_7
160 M. Werkle et al.
Internet in particular, new multi-media learning scenarios and teaching methods

increasingly complement, compete with and even partially replace classic class-
room-based teaching.
While these are usually large companies in the field of formal learning which
have been able to establish themselves in the area of classic e-learning or blended
learning concepts on the further education market and in the field of staff develop-
ment, technologies and learning scenarios for promoting informal learning in the
workplace are given even less consideration or fail before they have got off the
ground.
Why is this so? The idea of setting learning processes in motion or even
optimising them simply by using new learning technologies for planning,
performing and reflecting on further education courses, is just as harmful as the
complete ignorance of such usage scenarios and potential. Not only technical
requirements, but also media-didactic conception and design aimed at specific
target groups are crucial for the efficient integration of learning technologies,
which are usually web-based educational applications.
Teachers and learners within companies often lack media competence and the
ability to apply self-regulated learning (SRL). Thus, train-the-trainer concepts for
these skills are needed. Something that should not be underestimated is that
innovative approaches often also fail due to the lack of willingness to change or
poor implementation.
Trends in Technology Enhanced Workplace Learning
In the last years, the rapid changes in business environment set new requirements on
the development of learning methods, as well as learning software and content. The
experiences of the last years show that the curriculum-based learning approach,
based on the assumption that the learner “consumes” knowledge in the form of
predefined learning content, was continuously extended with SRL approaches
based on the cognitive learning theory. This happened through the active involve-
ment of the learner into the learning process, that is planning and defining the
learning strategy, searching for appropriate methods, tools and materials, time
management, reflection, and undertaking corrective actions. Such characteristics
as adaptivity, responsiveness, and personalisation became, thus, mandatory for the
modern learning software solutions.
Also, the penetration of mobile media, such as smartphones and tablets, into the
Technology Enhanced Learning (TEL) branch caused significant changes in the
conception and realisation of workplace learning activities. The possibility to use
small software applications and pieces of content (“learning nuggets”) became an
indispensable component of virtual learning environments making the learning
process time and place independent and optimised for mobile devices. Further,
the “socialisation” of the web (e.g. the emergence of social networks, social sharing
platforms, micro blogging, and so on) caused continuously increasing application
Case Study 4: Technology Enhanced Workplace Learning 161
of social media technology in education, using software tools supporting commu-

nication, content sharing, joint learning activities, and providing mutual feedback
between the learners.
Thus, the following trends/challenges in terms of workplace learning have been
observed:
– Increasing networking of the learners (collaborative learning/social learning)
– Creation of smaller, highly concentrated learning content (micro learning)
– Increasing importance of mobile learning (“bring your own devices”)
– Individualisation and self-regulation of the learning processes (PLE approach)
– Videos as the learning medium with increasing potential
– Increasing importance of specific (experience-based) knowledge and user-gen-
erated content
The ROLE Solution
While learning, software met the requirement of becoming user-centred, flexible,

and social, organisational structures needed more time to adapt to the new trends.
The evaluations conducted within the scope of the ROLE Project showed that the
vocational training and workplace learning providers appreciated the SRL approach
and the idea of personalisation of the learning tools and content. On the other hand,
they wanted to keep control over the learning environment, as well as tools and
content provided to the end users. Further, it became obvious that the implemen-
tation of SRL in an organisation needed the development of specific competencies
by the learners, as well as guidance through the learning process from its very
beginning.
Thus, a learning software solution allowing combination of curriculum-based
and SRL approaches, and providing both standard functionalities and content and
personalised tools and materials has become necessary. To address this require-
ment, the ROLE Project developed a Personal Learning Management System
(PLMS), which is an OpenSocial-based Learning Management System (LMS)
combining functionalities of a LMS and a Personal Learning Environment (PLE)
and allowing users to construct their virtual learning environment according to their
learning history, goals, and preferences.
The following sections describe the main pedagogical and technical concepts
underlying the development of the ROLE solution from the point of view of Festo, a
test bed which actively contributed to the development, testing, evaluation, and
application of the ROLE approach and technology. The Festo Showcase describes
(a) the development and application of a learning video sharing portal
(LearningTube), (b) the extension of the corporate LMS with additional web-
based learning applications (widgets), c) the evaluation of the PLMS developed
based on the corporate LMS and including external learning widgets.
Technology Enhanced Workplace Learning at Festo
The Learning Company Festo
The Festo AG &Co.KG was founded in 1925 by Gottlieb Stoll and Albert Fezer in
Esslingen, Germany. Initially, it manufactured wood cutting tools and later diver-
sified into the automation industry. Now Festo is a leading, worldwide supplier of
automation technology with approximately 30,000 catalogue products, customised
solutions, ready-to-install automation systems, and a matching range of before- and
after-sales services. According to the Engineering & Production on-demand con-
cept, users can adapt these solutions to their individual needs, which actually
increases the number of products offered exponentially. Furthermore, Festo is the
performance leader in industrial training and education programs and offers a
comprehensive range of learning systems for industrial training and education—
from seminars, training and consultancy in 26 languages to e-learning and complete
turnkey learning centres. Some 42,000 customers worldwide take part in Festo’s
seminars or are educated at the company’s own training facilities. In 2012, the Festo
Group had about 16,200 employees and a consolidated turnover of €2.2 billion. The
61 independent national companies serve customers in over 250 offices in 176 coun-
tries worldwide.
Festo AG & Co. KG defines itself as a “learning company” which would like to
constantly expand the knowledge and potential of its employees in order to encour-
age technical innovation and product development. Festo is a globally-oriented and
independent family company with its headquarters in Esslingen. The largest pro-
duction and logistics site is in St. Ingbert/Rohrbach, which is home to the cylinder
production, the Customer Service Center and the Festo Learning Center.
The Festo Lernzentrum Saar GmbH (Festo Learning Center)
The Festo Learning Center was founded in 1994 and is located next to Festo’s
production plant and logistics centre in St. Ingbert/Rohrbach, Germany. As an
accredited institution for advanced vocational training, the Festo Learning Center
offers a wide range of personnel and organisational development programmes. The
service portfolio comprises the whole spectrum of further education (seminars,
training courses, the private technical school “Festo Technikum”), subsidy consult-
ing, vocational retraining and qualification, industry consulting, and e-learning. The
customers are international enterprises of all branches and institutions, as well as
Festo employees and private persons.
Within the ROLE project, the Festo Learning Center aimed to define require-
ments of how to create a ROLE environment within a professional industry
business. The Festo Learning Center wanted to demonstrate the use and benefit of
ROLE technologies for the learners by addressing and supporting them in planning,
realisation and evaluation of qualification measures with the help of the web-based
services on the Festo Virtual Academy—Festo’s global e-learning platform.
The Virtual Academy
The Festo Virtual Academy is the central LMS of Festo with a corporate internal
focus. It is based on the software CLIX developed by IMC (see section “IMC and
CLIX Learning Suite”). The LMS supports personnel development processes
within the Festo organisation and offers strategic relevant online courses to the
learners.
The Virtual Academy is open for each Festo employee worldwide for their
personal further education by means of the lifelong learning approach. The LMS
is accessible via the internet to facilitate SRL processes for the employees (learning
independent of time and place). In addition to the self-learning offers, several
blended-learning modules are available. The web-based trainings (WBTs) are
designed according to didactic models in order to make their “consumption” as
easy as possible for the learners.
The Virtual Academy has more than 9,000 users distributed all over the world,
more than 80,000 logins p.a., more than 800 learning contents in different strategic
learning categories with a total learning time of approximately 2,000 h. The
contents are provided mostly in German, English and Spanish. For specific topics
additional languages like Chinese, Japanese, Portuguese and Russian are also
available.
In the ROLE project, the Virtual Academy was one of the five original test beds
and addressed the issue of providing a responsive learning environment within
further education activities in a company. This included not only continuous
technical and media-didactical possibilities of a LMS and the content within, but
also the technical possibilities beyond the LMS approach.
The application of ROLE at Festo led to better support of individual learning
processes (in terms of self-regulation) and also better support of collaborative and
social learning in a company. Therefore, informal learning increases its importance
for the workplace learning and also the possibilities to record, share and save this
expert knowledge within the company.
The following chapters describe the most important TEL-topics for the current
and future developments of the Virtual Academy as well as the ROLE project with
the Festo test bed, especially
– The initial situation
– The project vision
– The specific target group
– The characteristics of the business context
– The main organisation related challenges and the requirements
– The technical implementation
– The evaluation of the project results
Trends of Technology Enhanced Workplace Learning at Festo
Making Learning Environments More Personal
The PLE approach is nothing particularly new, but is still a vital research topic in
the TEL-community. PLE stands for a PLE, i.e. an individual composition of
learning services that helps learners to plan and conduct their learning activities,
and to reflect on the learning process and progress. Considering the fact that
learners are often a very heterogeneous group (with different learning needs and
goals, learning styles, individual learning experiences, knowledge, learning prefer-
ences or different job roles), the potential of the PLE approach is undisputed.
In 2007, Graham Attwell (2007) defined a PLE as being “comprised of all the
different tools we use in our everyday life for learning”. Considering this quote
alone, one can be of the opinion, that in business contexts, especially in e-learning,
every employee has already developed a PLE consisting of standard and company-
specific software as a digital toolbox. But far from it, at the moment the PLE
approach is more theory than it is implemented in further education scenarios or
personnel development processes in companies. It seems that further education is
more and more lagging behind technical developments.
The main hurdles for the TEL implementation in companies are:
– At the moment, learning processes in companies are unilaterally strategic driven
in a top down manner.
– Most of the learning processes are formal. The potential of informal learning and
user generated content is not being exploited. The paradigm change of content
towards the expert as author is not yet turned into practice in companies.
– Companies have already made big investments in learning technologies (perhaps
in e-learning content or an LMS) and promote only these.
– Frequently there are a lot of existing internal learning tools in (bigger) compa-
nies, but these tools are often not synchronised. Therefore, the promotion of new
learning tools across departments is very difficult.
Consequently, the Festo test bed examined in ROLE how the PLE approach can
enhance learning opportunities for employees. The target was not to create a “pure”
PLE, but rather to combine advantages of a PLE with an existing LMS. Thus, it was
not a question of replacement or substitution of an existing, traditional LMS in a
company but rather an approach that enabled the enhancement of their current
delivery mechanisms. It was also a question of how an existing LMS can be
enriched with new information and communications technologies that accordingly
enhance the end-user (the learner’s) experience. This will be explained in detail in
the section “Implementation of the Personal Learning Management System”.
Videos as the Learning Medium of the Future
The power of the image is used on the Internet to disseminate information quickly
and easily. Photos and graphics are omnipresent in the worldwide web today—be it
for advertising or promotional purposes, to inform or simply to spread a bit of
happiness. Today’s technology makes it very easy for each and everyone of us to
create pictures or videos and publish them on the Internet. Individual users already
actively use video production and sharing to exchange experiences and knowledge.
The trend towards video-based learning is being adopted by private companies and
higher education.
Pioneers in this area include the Khan Academy1 with a thematic collection of
over 4,100 instructional videos and Stanford University, which operates its own
YouTube channel.2 However, the YouTube channel alone is not the impressive
fact, but those 215,289 subscribers downloading over 55 million videos on demand,
illustrates its reach. The private online academy Udacity,3 founded in 2012, pro-
vides similar figures. The philosophy here is “Learn. Think. Do.—Higher Educa-
tion for Free”. The first two free video-based online courses alone reached 90,000
participants (Lewin 2012).
The key to the success of these video tutorials is the didactic simplification
through visual representation. Wherever one previously had to describe a subject in
words, written or verbally, or how complex product drawings had to be created,
today it is simple to visualise information using moving, narrated images. This
usually saves the knowledge carriers and producers of video tutorials more than just
time in comparison to creating professional teaching materials or specialist articles.
The creator of a video tutorial is able to use the sound and video track to give his
“knowledge” not only a personal note but also a kind of personal signature.
The knowledge content is closely connected to the knowledge carrier and these
are not as easy to separate as in the case of pure text-based content. Thus,
statements, opinions and experiences can be expressed personally through inter-
views and the interviewed person “signs” it with his name. Ultimately, the web
video medium suites often better to meet the need for communication and self-
expression than “pure” text. From the learner’s point of view, particular application
or process knowledge is taken in more easily via pictures and transferred to “real
life” than when if it is read from texts. The recipients also use their auditory and
visual sensory paths to process information, which helps to anchor it in the memory.
Festo has recognised this trend towards instructional videos. It developed an
award-winning tool, the LearningTube and Recorder, as part of the ROLE research
project. The LearningTube provides all Festo employees with a simple way of
creating video tutorials and sharing them with colleagues in the company in order to
document best practices arising in everyday work and make them available for
1
http://www.khanacademy.org/about
2
http://www.youtube.com/user/StanfordUniversity
3
https://www.udacity.com/us
colleagues, successors, and new employees. The innovative aspect of that is the
combination of the LMS with easy-to-use authoring software which allows content
owners as well as every learner to instantly create learning content on their own.
Compared to traditional LMS that only had “time-consuming” text-based authoring
tools included, the screen recording capabilities of the LearningTube make it much
easier for the authors to capture their knowledge and transform it into learning
content. The reason for the perceived simplification is that experts can easily reuse
the tools and media they use in their everyday life to transport the information such
as PowerPoint slides or recording the software they need to explain. Thus they do
not need to develop new skills, the system is built on the skill-set the experts
already have.
The ROLE Project and the Festo Test Bed
Project Vision
Learning in the Virtual Academy Before ROLE
The learning processes at Festo can be described as self-controlled by the learners

(employees). The users browse offered learning programmes in the catalogue of the
Virtual Academy and select those they need. During the learning process itself, the
learners can decide by themselves what they would like to learn and which parts of
the web-based training content they would like to skip. Since the platform is
available online, the learners can even access the Virtual Academy from home
and participate in trainings on a voluntary basis. The learners are free to define and
plan what and when to learn, and how frequently to work on the web-based training
offered in the catalogue.
The vision of the ROLE project was to improve especially the aspects “respon-
sive” and “open” of the already existing learning environment of Festo—the Virtual
Academy as the central LMS of the company. From Festo’s point of view the main
targets were to improve existing learning systems according to:
– Openness and adaptivity
– Communication with other learners
– Facilitation of collaboration and peer-assisted learning
– Switch collaborated and individual work
– Exploring ways of benefiting from the experience acquired in a company
– Best practice sharing
Challenges and Requirement Analysis of the Test Bed
The Festo Learning Center aims to meet the requirements of learning environments
within a business context. To achieve this aim, Festo effectively opened up its
platforms and tools to “mix and match” and be interoperable through the use of
ROLE tools and technologies.
Organisation Related Challenges and Requirements
Since the Festo Virtual Academy is effectively a corporate learning environment,

some special surrounding conditions have to be considered within the ROLE
process. These special “challenges” depend on the fact that a corporate LMS has
one central main function: the further development of personnel in the company.
Issues relating to these surrounding conditions are, for example:
– The uncertain scope of openness of a corporate learning environment. How wide
can a corporate learning environment be opened up?
– The “job role” of the learner has to be in the focus of all learning processes
– Knowledge sharing is harder to realise in the job context than in non-working
life
– Understanding learning processes during daily work and after work
– System restrictions and data security
Target Group Related Challenges and Requirements
The Festo test bed is focused on LMS users and especially LMS users in a company.
This target group has special needs and the surrounding conditions in companies are
not as flexible as those predominant, for example, in universities. In contrast to
students, business learners are a very heterogeneous target group with big age
differences (from 16 to 65), different educational backgrounds and previous knowl-
edge, job-roles, learning requirements, learning preferences and learning goals.
Further, the learners in business environments have primarily to fulfil their job
role and learning is mostly to support them in doing so. Due to high workloads, it is
often hard to learn on the job or in other words, there is no or just little time for
learning available. It is often not so easy to disengage workers from their daily
practices. Thus, learners at the workplace need to be supported systematically, not
only with new TEL solutions, but also with their development goals, the working
and learning conditions in general and their work life balance.
User Related Challenges and Requirements
There is one more aspect to be considered, which is related to the personal

requirements of the learners. The learners need a set of specific skills, the so-called
“self-regulated learning” (SRL) skill-set. These are skills the learners must have to
be able to successfully plan, conduct, and evaluate their learning activities. Some of
these skills might be new to some learners, whereas others might be present and
used already, but the awareness about that fact is still missing.
To address this requirement, Festo initiated the “Fit for Self-Regulated Learn-
ing” initiative in the ROLE project. The plan was to implement various SRL
learning modules in the offered ROLE service. Some of the modules should
explicitly show that they support learners in getting SRL known as a method.
Others would be implicitly woven into ROLE services to make their impact on
SRL visible, for example within wizards.
Putting the idea behind “Fit for self-regulated learning” in a nutshell, one can
state that SRL should improve the learning outcomes. This is realised by offering
the learners SRL as a manual. This would not be a technical description of the
platform, but rather a manual for learners that shows how to learn effectively and
successfully in the ROLE environment with the help of SRL.
Festo considered the benefit of the approach to be very high and therefore
decided to personalise learning through ROLE and made learning more demand-
and service-oriented for the users of the Festo Virtual Academy.
Implementation of the Personal Learning Management

System
IMC and CLIX Learning Suite
IMC is currently Europe’s largest learning management solution provider that was
established in 1997 by Prof. Dr. h.c. August-Wilhelm Scheer as a spin-off of
Saarland University. IMC offers a comprehensive portfolio of e-learning-related
products, supporting all business processes in training and education. IMC also
provides professional services covering the whole value chain of content design and
production as well as consulting and managed learning services helping clients to
(re)organise their learning processes and to select, implement, adapt and integrate
suitable software systems and technologies.
IMC’s product portfolio includes professional authoring and content recording
tools (Content Studio, LECTURNITY), a Business Process Guidance System
LIVECONTEXT, as well as a Learning and Talent Management System CLIX
supporting HR- and Personnel Development processes at organisations. CLIX
contains two main modules: the Learning Suite, which is a LMS in the classic
sense, and the Talent Suite supporting Talent Management processes, such as
Talent Identification, Competency Management, Career and Succession Planning,

Performance Management, etc.
In scope of the ROLE Project, IMC used CLIX Learning Suite to implement the
concept of the Personal LMS. CLIX Learning Suite in its basic configuration
includes such functionalities as Organisation Management, Process Management,
Content Management, Testing and Assessment, Resource and Capacity Manage-
ment, Report and Compliance, Evaluation, and some others. The Social Learning
module contains communication tools, such as chats, communities, wikis, and
forums allowing making use of the synergy effects of collaborative learning.
In order to increase personalisation of the system and to extend collaborative
functionalities, CLIX Learning Suite has been integrated in scope of the ROLE
Project with an OpenSocial PLE, thus, constituting a so called Personal LMS. The
PLMS aggregates learning resources and applications available in the web and
selected by the users. Its structure aligns phases of the SRL process and assists the
users in planning of their learning activities, search for learning content and tools,
training and testing, as well as in reflection and evaluation of the learning progress.
Addressing the increasing need in acceleration of the knowledge acquisition due
to rapidly changing organisational and technical conditions at organisations, and in
personalisation of the learning process increasing employees’ motivation and
effectiveness of the training measures, an adaptable learning environment
supporting workplace learning had to be developed. The identified requirements
from the user side were: firstly, for the learner’s point of view, to make the system
customisable in terms of tools and content according to the current learning needs,
and secondly, from organisational point of view, to keep control over the learning
environment to ensure transparency of the learning process and the achievement of
planned learning results.
From LMS to Personal Learning Management System
The early development of LMS is aimed at coordinating the learning processes in

organisations in terms of training measures participant and content management,
including creation of course curricula, adding learning materials and tests, as well
as providing meta-data such as course dates and place, procedure of entering the
course, course completion and certification. Over the last years the LMS developed
in the direction of the Human Resource Management Systems (HRMS) providing
functionalities for personnel assessment, competency management, learning and
development, as well as succession planning. While increasing the planning and
organisation capacity of the LMS, the learning process supporting infrastructure has
not changed much.
The new learning approaches, such as SRL, collaborative and social learning
have been realised in PLEs allowing the users to construct their learning environ-
ments themselves by selecting learning tools and content in a way supporting
acquisition of the desired knowledge. In scope of the ROLE project, a PLE is a
web-based infrastructure, where the users can access, aggregate and manipulate
learning applications and resources of their preference, as well as communicate
with other users sharing experiences and collaborating on projects (Overton 2009).
Importantly, the PLE uses web widgets, which are small web-based software
applications, to support particular learning and teaching goals or training of some
specific skills.
In order to create a software solution supporting both approaches, an OpenSocial
PLE has been integrated into LMS combining functionalities of both systems. Thus,
the PLMS provides instructions and pre-defined learning materials allowing the
users to complete learning courses as usual, but also it ensures the learning process
autonomy offering personalised learning spaces, in which the users can add and use
additional applications and resources. Further, the PLMS allows browsing addi-
tional learning content directly out of the system with help of the Media Search
Widget, thus, being more open than classic LMS. Also, it offers guidance through
the SRL process and support to its each phase, which are described in the sections
below.
Psycho-Pedagogical Integration Model and Personalised

Learning Spaces
The OpenSocial directory of the PLMS is structured according to the phases of the
SRL process described in the Psycho-Pedagogical Integration Model (PPIM)
developed in the ROLE Project (Fruhmann et al. 2010). This model identifies
four main steps:
1. Plan: This phase includes definition of the learning strategy, learning goals,
actions to be taken to achieve these goals, as well as preferences in the sense of
tools and types of content that will be used.
2. Search: At this step the learner searches for learning resources and tools within
the learning environment and outside of it. Here, the user may get recommen-
dations from tutors and peers, but also use recommendation systems to find
appropriate resources.
3. Learn: Learning includes studying of the selected tools and materials, attaining
skills, training and testing, as well as assessment by tutors and self-assessment.
4. Reflect: This phase implies gathering feedback from different sources and self-
evaluation, as well as reflection on the learning process and achievements in
order to evaluate the usefulness of the learning strategy and particular actions
and their correction if needed.
These phases are summarised as “plan-search-learn-reflect” loop (see Fig. 1
(Mödritscher and Nussbaumer 2012).
The OpenSocial directory of the PLMS is divided into four learning spaces
corresponding to these four steps. In order to provide necessary guidance to the
Fig. 1 Psycho-pedagogical integration model
users, each learning space is populated with pre-defined learning widgets. These are
Open Source tools that were developed by the ROLE project or found in the web
(all rights reserved) and reviewed concerning their suitability to support one or
more of the learning phases. Further, the PLMS contains a list of additional tools.
Thus, the users may use pre-defined applications and/or supportive tools from the
list arranging them in the learning spaces. This allows an efficient integration of
external tools into the system respecting the interests of both the learners and the
organisation (Fig. 2).
Besides learning applications, each learning space contains an introductory
video to the respective phase of the SRL learning process and a Wizard Widget
containing supporting questions and hints to assist the user in getting started with
using the system. The user acceptance evaluation showed that the availability of
these assisting tools was highly appreciated by the users (see Evaluation section).
Thus, the PLMS aims at providing as much guidance as necessary, and as much
assistance as possible (Schanda et al. 2012). The development of the PLMS and its
technical implementation focused on the personalisation, adaptivity, and user-
friendliness of the PLMS making it responsive to the needs and preferences of
the users.
Technical Implementation: OpenSocial LMS
In scope of the technical implementation, two major aspects were identified in order
to extend the CLIX Learning Suite to be used as PLMS: on the one hand, a
mechanism was required to enable the integration of external components into
Fig. 2 ROLE PLMS search learning space
the CLIX portal. These components had to be implemented using a standardised

technology for the following reasons:
– In order to separate the development processes and technologies for components
of PLMS from those of CLIX.
– To enable external developers to create components without technical knowl-
edge of the environment they are embedded in.
– In order to allow integration and re-use of existing components, e.g. components
which are provided in the repository.
On the other hand, an interface was required to exchange social information
between PLMS components, CLIX as a hosting environment, and external services.
Taking those requirements into consideration, the decision was made to use
OpenSocial4 as an open cross-platform standard specifying a REST-API to
exchange social information. Having the API specified, the technology stack was
completed by supporting the Google Gadget specification5 to create components,
further on called widgets. Widgets are applications designed to only take small
parts of a screen and make use of web technologies (HTML, CSS, and JavaScript).
For the communication between the widget and the environment, a JavaScript API
was provided, which also included the interfaces of the OpenSocial API.
4
http://opensocial.org
5
https://developers.google.com/gadgets/docs/spec
The implementation of the OpenSocial API and Google Gadget specification

was based on the Apache Shindig project.6 Apache Shindig was realised as a
standalone web application, which was connected over a dedicated web service
interface with the CLIX Learning Suite. This architecture allowed separated main-
tenance processes for both platforms. Shindig took two roles within the PLMS: as
rendering server for widgets and as OpenSocial API provider. CLIX acted as data
provider and environment for embedding the widgets rendered by Shindig.
At the next step, an integration point for the widgets inside the CLIX portal had
to be found. As the PLE structure does not necessarily depend on a concrete course,
the integration was realised using dashboard pages. Dashboard pages were designed
to create a mash-up of panels showing user-centred information (e.g. a course list,
news, social media updates, etc.) and therefore built a capable environment for
widgets. The implementation aimed to provide a seamless integration with existing
dashboard features. A new panel type was introduced and each instance of this
panel represented a single widget.
Platform administrators can select which widget panels are available on a
dashboard page and can pre-configure the widgets. They can choose if a widget
has a system-wide or a user-specific configuration. In the latter case, the users have
their own configuration interface for the widget directly integrated in the
dashboard page.
The final layout and content of a dashboard page allow nearly any degree of
personal individualisation. Whilst the pool of available widgets is previously
defined, their visibility, application, and position can be made customisable for
the individual user. The range of self-controlling is therefore very large; both
settings of fully pre-configured widget mash-ups and open-space solutions, where
users combine widgets starting from a blank page, are possible. Learning processes
can also be supported by combining multiple dashboard pages, covering either
different steps in a learning process or different topics addressed by specific pages.
Concept and Evaluation of the PLMS at Festo
The Virtual Academy Case study
This case study reports on how the ROLE environment can be open to a mix of
internal learning applications alongside external ones. This is regarded as a key
success factor for project ideas that emerge from developments like ROLE to
influence the promotion of further education in companies and meets the overarch-
ing premise of this case study namely that it demonstrates “an internal learning
opportunity in a company” environment. In essence, this section, thus, presents an
6
http://shindig.apache.org/
evaluation of a case study relating to the Festo Virtual Academy and demonstrates
how internal learning opportunities can be improved in a company.
This case study therefore sets out to address the issue of providing a more
responsive learning environment within the further education activities of Festo.
As previously indicated, to achieve this aim, Festo opened up its e-learning
platform to be interoperable and open for mixing and matching with ROLE tools.
This combined approach was called a “Personal Learning Management System”
(PLMS), where the PLMS was comprised of the LMS and PLE together, hence
when combined resulting in the acronym PLMS as technically described in section
“Implementation of the Personal Learning Management System”.
The actual implementation of this approach, i.e. the degree to which these PLE
enrichments are embedded, ultimately depended on both company-specific require-
ments and on the individual learning preferences, as well as the anticipated learning
experiences of individual learners. In general terms, however, the main targets of
the PLMS approach were:
– Simplified access and advanced search of relevant content and learning
materials.
– Support and improvement of the planning of learning, incorporating the reflec-
tion phases of the learning activities.
– Enabling learning motivation and promotion of SRL as well as different forms of
cooperative learning.
To achieve this improvement, Festo supported ideas and the development of
prototypical widgets and tools, which can enhance the learning processes in such
corporate learning environments. Initially, the first item to be addressed during the
implementation of the ROLE approach was to improve the “Openness” of the
system.
At the first step, a federated search widget was developed together with two
other ROLE partners, namely IMC and KU Leuven. The idea of the widget was to
enable a more focused search facility in the Festo LMS. It was developed to search
learning content in several external online resources and feed results back directly
to the learner within the Festo LMS. A media-list widget, that was interoperable
with the federated search widget, was developed too. This allowed users to create
media lists out of the resources found with the search widget. Both widgets are
featured in the Festo Virtual Academy.
At the second step, another perspective of openness considered in the context of
this case study was the integration of user generated content as well as encouraging
the possibility for learners to produce content on their own. To achieve these
outcomes, a commercial screen recorder facility was embedded as a widget,
which enabled learners to create their own videos. This widget bundle was given
the names of “LearningTube” and “Recorder”, and together both widgets provided
all learners, trainers and experts with appropriate support for exchanging and
communicating training content on a daily basis.
In this case the Recorder allowed users to create a video, which could then be
uploaded to the LearningTube and shared with Festo colleagues worldwide. Both
Fig. 3 The Festo LearningTube
tools supported bite-sized learning and ensured that content could be published and
distributed quickly. This style of rapid e-learning, therefore, allowed users to
enhance their presentations with the addition of a voiceover and optional webcam
video. The Recorder widget also provided help and support for creating resulting
screenshots, enabling users to add their own commentary to PowerPoint presenta-
tions or even enabling them to emulate software simulations.
The LearningTube widget bundle was acknowledged by the Comenius-
EduMedia organisation, which gave an award for the practical application of
educational, thematic and design excellence in educational media. Festo was the
only industrial enterprise to receive such an award. Equally, the widget has offered
to Festo employees around the globe, an interactive educational video channel that
has also become a valuable learning tool. As a result of this implementation, there
have been over 220 video uploads by Festo employees since July 2011, consisting
mostly of screencasts and recorded presentations. These videos have been accessed
over 15,000 times to date (April 2013) (Fig. 3).
Evaluation
The concept used to evaluate the ROLE solution at Festo consisted of a combina-
tion of questionnaires, selected expert interviews, a focus group for requirement
gathering purposes, a taskforce observation and interviews by project members.
These various elements ensured that there were standardised frameworks for
evaluation and also personal contacts to offer possibilities to clarify confusing
questions and allow a little more depth.
During the project, Festo carried out two main evaluation loops. To this end,
Festo founded a small focus group of 26 colleagues to test and evaluate the ROLE
technologies in the Virtual Academy from the learner’s perspective. Qualitative
data about the usefulness, usability, look and feel, the innovativeness, the relevance,
the clarity and the improvement potential of the presented ROLE technologies were
collected. Therefore, the first evaluation loop was conducted by a specific ques-
tionnaire, whereas the main focus of the second evaluation loop was to observe how
the test persons work with the learning technology.
The use of the focus group meant that although the evaluation was not on a large
scale, the results were of high quality. This was due to the expert knowledge, the
aforementioned method mix and the possibility to read between the lines and to
receive more detailed feedback.
The First Evaluation Loop
As described, the first step was to implement two widgets in the existing Festo
Virtual Academy LMS, thus, enriching it with appropriate PLE elements. The first
evaluation loop consisted of a questionnaire about the developed media search and
the media list widgets. The evaluation of these widgets was not possible with the
existing standard questionnaire of the virtual academy. Therefore, a new question-
naire, asking about personal information, preferred forms of further education, daily
use of the internet, affinity to Web 2.0, benefit of web 2.0 and the described widgets,
was created. This questionnaire was emailed to the members of the previously
described focus group; a screencast introducing the ROLE project, the ROLE
approach, and the developed widgets was attached.
Results
The response rate of 61.5 % (16 of 26) was not as high as expected, but it should be
kept in mind that the members of the focus group had to give priority to their normal
jobs and the effort required in the different evaluation loops had to be reasonable.
The focus group supported the Festo project team on a voluntary basis partly in their
leisure time. So their resources and feedback were important, but also very limited.
All responses to the questionnaire in this case study regarding the look & feel,
usability and perceived usefulness were very positive concerning the applied ROLE
approach in the business context. It showed that 63 % of the users liked the look &
feel as well the usability.
Nineteen percent said that that the look & feel should be improved and 13 %
stated that the usability should be improved. Regarding the quality of the search
results, the performance and the fun factor of the widgets, 38 % of the users said
these issues were in need of improvement. Forty-four percent rated the quality of
the search results as good, the performance got a good grade from 50 % of the users,
the fun factor was rated as good by 38 % and even as outstanding by 13 % (Fig. 4).
Fig. 4 The first impression of the prototypes
Fig. 5 Perceived usefulness of the prototypes
Evaluation of the benefits of the prototypes showed that most of the users saw a
high or very high benefit of the offered tools. The highlights regarding the benefits
were in this case the federated search feature over several knowledge resources.
Fifty percent stated a benefit and as many as 38 % stated a high benefit. For the
rating feature 31 % stated a benefit and as many as 44 % stated a high benefit. The
good overall impression of the evaluation reflects especially in the recommendation
value—in total 88 % of the test users would recommend the tools to their col-
leagues. Last but not least, perceived usefulness and effectiveness was evaluated
with the question “do you think the offered services will help you to work more
effectively in your job then at present?” Thirty-one percent fully agreed, 50 %
agreed and only 6 % denied that they would be more effective when working with
these tools. This was a really good result for this early prototype evaluation from
Festo’s point of view (Fig. 5).
The Second Evaluation Loop
The subject of the second evaluation loop was the PLMS prototype, which was
developed with the goal of integrating the approaches of the LMSs and the PLEs,
and thus of supporting the user during SRL. The evaluation was conducted on a
PLMS prototype, which was integrated into an IMC test environment.
The acceptance of the learning environment was and is an essential aspect in this
regard. In particular, if an introduction of training offers, which will be used on a
voluntary basis, is concerned, it is important for the training provider to determine
the acceptability extend of the learners. Thus, the objective of the evaluation was to
find out how well the PLMS was accepted by the employees, and which measures
could be implemented to further increase its acceptance.
The following usability factors were important:
– The selection and addition of widgets, and navigation learning spaces
– The use of resources, such as wizard widgets and tutorials
– Acceptance of the PLMS
– Handling of the PLMS
– Comparisons with conventional training media and forms of learning
Therefore, several methods were used in order to evaluate the PLMS. Observa-
tion was conducted on the one hand, and the so-called “think aloud” or “question
asking” method was used on the other. In addition, the questionnaire entitled
“Perceived Usefulness and Ease of Use” was used in this evaluation supplementing
data obtained by means of observation and the “think aloud method” (“TAM”). For
reasons of comparability, the questionnaire was developed within the framework of
the ROLE project, which was intended to ascertain usefulness and user-friendliness
of the project results as perceived by the learner. It was also used in other test beds.
The “TAM” was used as a supplementary method in addition to observation. In
particular, this method was used for examinations of the user interface because it is
especially well suited for detecting problems of this sort. With the use of “TAM”,
the thought processes of the learners were investigated while they were dealing with
the PLMS.
Test persons were prompted to describe their actions and thoughts out-loud
during interaction with the learning environment. In this way, the test persons
were not only able to address any problems they were experiencing with the user
interface, they could also explain them and, as a result, could reveal design defects.
Furthermore, the test persons expressed thoughts of satisfaction, enthusiasm or
motivation when commenting on their actions. Thus, data compiled by means of
“TAM” provided in-depth knowledge, not only regarding the actions of the test
persons, but also about their attitudes as well. If applicable, information about their
experiences gained in dealing with the software, could also be gathered.
The “TAM” was used in a slightly modified form, which was more comparable
to an interview technique, by means of which the study director asked the user
targeted questions regarding comprehension, the sequence of the learning activities,
etc. This counteracted the danger that the test person talked less and less while
working with the learning environment, and no longer remembered to comment on
his actions out loud. This problem might have occurred because working with the
PLMS represented a very new and complex task, for which the user required a large
portion of his cognitive capacity.
The evaluation supervisor spoke with the test persons or helped them work
through the learning steps in the PLMS, if requested to do so. However, this
influenced the experiences, the attitudes and the actions of the learners. This
influencing factor was taken into consideration in the evaluation of the results. As
a rule, “TAM” was conducted with help of audio or video recordings. Instead of a
reconstructive form of data collection, “TAM” made use of so-called registrative
data conservation. This simplified evaluation and assured the reliability and validity
of the compiled data. “TAM”, thus, effectively compensated the weaknesses of
participative observation.
Evaluation Procedure
During the evaluation phase 11 interviews were conducted with employees of the
Festo Learning Centre. The employees came from different departments and had
different educational backgrounds. The evaluation was allotted a duration of 45–
60 min per test person. The users were requested to complete a task in the PLMS to
this end. Subsequently, the test persons evaluated the PLMS with regard to accep-
tance, system performance, required effort and user-friendliness, as well as use of
and satisfaction with the application assistance.
The survey “Perceived Usefulness and Ease of Use” consisted of two parts.
Those questions which were considered as learning premise, namely those regard-
ing age, sex and TEL experiences were presented before the PLMS test. The other
part of the survey dealing with the evaluation of the product, in this case the PLMS,
was conducted after the PLMS test.
The evaluation supervisor was available while the questionnaire was being filled
out, and was able to help the test persons with any questions or uncertainties.
However, the test persons were initially only asked to respond to the closed
questions. Then the observation started, during which time the observer took
notes. If it was needed and time allowed it, the observer went deeper with the
open question section in order to clear up any unanswered points.
Results
The evaluation of the questionnaire entitled “Perceived Usefulness and Ease of

Use”, provided the following summary: Eighty-one percent of test persons rated the
learning environment PLMS as useful. Fifty-five percent indicated that they would
achieve their learning goals somewhat more effectively with such a learning
environment. The usefulness of the learning environment was rated diversity.
Fig. 6 Results of the questionnaire “Perceived Usefulness and Ease of Use”
Eighteen percent chose “fully correct” and 36 % rather correct while 27 % were
undecided and 18 % said that the uses were rather not easy.
There was a strong agreement with 100 % that the use of the learning environ-
ment was not frustrating. But the evaluation showed that there was still a need for
improvement. Almost half of the interviewees (45 %) said the use of the learning
environment was strenuous or rather strenuous. However, the vast majority with
72 % of the respondents would use or rather use the learning environment, while
only 9 % said that they rather not use the tested learning technology. The results in
details were shown in the Fig. 6.
Further, through the use of the “TAM” it was possible to document positive as
well as negative statements of the probands. Figure 7 represents a compendium of
the most important statements, divided into the categories effort, performance and
facilitation.
In conclusion, the evaluation results of the PLMS allowed the following
statements:
– The use of the PLMS is deemed highly beneficial
– The PLMS supports the achievement of individual learning goals
– The PLMS would be used by the learners, if access to a tool was available
– Usability, as well as the look and feel of the PLMS prototypes, must be greatly
improved
In terms of challenges, however, some technical hurdles appeared regarding the
usability as well as look and feel of the PLMS for future implementations. These
challenges related to specific computer-based issues and are all undergoing further
investigation.
Fig. 7 Statements from the thinking aloud interviews
Nonetheless the feedback from the evaluation investigation remains very posi-
tive. It appears that people really liked the PLMS approach. Additionally, and since
the test phases took place, albeit on a prototype, the interviewees asked also for
refinements of the system in relation to their user experience.
Conclusion and Outlook
The Future of TEL in Business Context
The ROLE approach and the gained experience showed the potential of TEL
solutions not only in the higher education field, but also in business context. This
sounds quite simple at first, but it is extremely important. Higher education settings
and learning at a company are not two different worlds, but completely different
learning scenarios, with different learning goals and needs, different learners,
different learning conditions and learning possibilities. Often in research projects
initial solutions are presented without entirely knowing the specific surrounding
conditions.
But these kinds of solutions are more innovative than useful. There are a lot of
hurdles and specifications in companies according to organisational, technical and
personnel requirements. Therefore, it is important for TEL researchers to start every
TEL activity with a target group-specific requirement analysis and after this to
transfer the acquired scientific developments into these new learning environments.
On the other hand, for bigger companies it is important to be open to new ideas,
opportunities and learning approaches as that can help to identify, share and archive
important knowledge in a better or additional way than before. Often a big revo-
lution is not necessary. Companies have a lot of different learning technologies and
learning possibilities that are, however, separate, internal closed systems without
synchronisation and different responsible departments and responsible persons.

Thus, these learning technologies are in internal and in external competition with
one another and a systematic integration of technology-enhanced learning tools is a
challenge.
During the ROLE project, Festo has used its chance to learn from the ROLE-
consortium and bring new research results together with specific business require-
ments. During the project the Virtual Academy was opened up through enhancing
CLIX by OpenSocial widgets. The different evaluation loops showed that is not
only a current trend move away from closed LMS to more self-regulated and
individualised learning. Rather, it confirmed the first impression of the project
team to combine the advantages of the existing LMS with the PLE approach. One
outcome of this approach was the Festo LearningTube which enables every Festo
employee to produce and share user (expert) generated learning videos on the LMS.
The good thing is not only the received Comenius award but also the fact that it
is possible to share very easily user generated content in a company now. The
knowledge carriers produce their content on a voluntary basis without the necessity
of additional incentives. This supported Festo significantly in improving this
learning environment with the developed technologies and the intelligent combi-
nation of formal and informal learning.
What Are the Next Steps?
The Virtual Academy, especially the LearningTube will be continuously improved.

In this context three topics will be more and more important:
– Learning analytics
– e-Learning goes mobile
– Full text search in videos
Learning analytics is a very interesting field for Festo in terms of learning
transfer analysis. Researching how this process can be taken beyond the step of
e-tests and e-evaluation is a specific interest of Festo. Just in time analysis of user
data, visualising learning processes to users, context aware services and individual
recommendations based on learning goals or learning needs would be one of the
future scenarios.
When Festo started with the Virtual Academy it was only accessible via the
Intranet, so people could only learn from their workplaces. This was considered as a
bottleneck for the learning processes, so the platform was moved on a server that is
also accessible via the Internet. The goal is to make use of the benefits of e-learning
especially learning independent from time and location. This step, taken in 2005,
can be taken even beyond by bringing content also to mobile devices such as mobile
phones and tablets. This opportunity should be realised as soon as possible,
especially because the demand of the learners for mobile learning services is
continuously growing. A big lever for this approach is that the videos from the
LearningTube are mobile compliant and can therefore be easily “consumed” from
mobile devices.
Video-based learning is more and more an essential source of knowledge and
information in companies. As described before, the Festo LearningTube is a rapidly
growing internal video portal. At present there are more than 220 expert generated
videos and more are being produced every day. Unfortunately the search function is
not yet as developed. The growing number of videos results in a growing need for
state of the art search functionalities to support learners as much as possible find the
learning content they need. The objective is to find a solution that efficiently
combines voice recognition with the videos on the Virtual Academy and make
them thereby “full text searchable”.
Conclusion
The ROLE philosophy of Festo was to open the existing Virtual Academy LMS and
extend it with new technologies supporting individualisation of the learning process
and increasing its efficiency. The LMS and PLE approaches were combined
towards a PLMS fulfilling existing business requirements. This was technically
realised by means of integration of the PLE elements into the LMS platform and
pedagogically with the use of SRL method. As previously described, the first step
was to implement two widgets (media search and media list widgets) in the existing
Festo Virtual Academy LMS. At the second step, this approach was extended and
realised as a PLMS supporting four learning phases of the SRL process (plan-
search-learn-reflect).
The use of the PLMS at workplace supports, on the one hand, curriculum-based
learning allowing completing learning programmes and courses created by an
organisation for its employees. On the other hand, it extends usual personnel
development measures with SRL activities giving the employees an opportunity
to specialise in the most important and interesting fields of knowledge, thus,
developing their competencies and skills in an individualised manner addressing
personal strengths and preferences. This assures gaining of obligatory knowledge
and skills for completing specific tasks by the employees and supports personal
development, thus removing borders between workplace and spare time learning.
Moreover, while using the PLMS SRL competencies, such as goal setting,
planning, time management, resources search, and self-evaluation, are trained.
These skills can be applied by the learners not only in training context but also at
the workplace helping to improve every-day working processes and achieving
better results. Festo sees the need to support learners with the new technology
and to develop SRL and media competence as key factors for successful working
with the PLMS and the ROLE technologies. In this development process Festo
keeps in mind that, in a business context, there are complex requirements and
restrictions, for example, the contrasts between:
– Openness versus data security (e.g. the work council, the data protection officer
and some others had very different opinions and views on the proposed work)
– Companies’ targets versus individual targets
– Implementation strategy: the “Revolution” (completely new technology) versus
the “Evolution” (successive further development of existing tools)
The conclusion, therefore, is that the current vision and deployment of a PLE
towards an integrated PLMS implementation with predefined learning spaces on the
technical side seems to have been warmly welcomed. Nonetheless, from the
psycho-pedagogical perspective it remains essential that learners are able to learn
in a self-regulated manner.
The required technical improvements, therefore, have to be synchronised with
the necessary individual development of specific SRL competences in order to meet
these very real needs. The bigger pedagogical challenge, however, will be to
promote the new approach to learners on a large scale. The evidence that this is
possible is the fact that the LearningTube widget bundle has become an essential
part of the Virtual Academy.
References
Attwell G. The Personal Learning Environments—the future of eLearning? eLearning Papers.

2007;2(1). http://www.elearningeuropa.info/files/media/media11561.pdf
learning in a responsive open learning environment. In: Hambach S, Martens A, Urban B,
editors. Proceedings of the 3rd International eLBa Science Conference, Rostock, 2010.
Lewin T. Instruction for masses knocks down campus walls. The New York Times, March 4, 2012.
http://www.nytimes.com/2012/03/05/education/moocs-large-courses-open-to-all-topple-campus-
walls.html
Mödritscher F, Nussbaumer A. Self-regulated learning—communities-of-practice vs. self-
regulated learning—motivation, backgrounds, and modus operandi of this blog. In: COPSRL,
Nov 23, 2012. http://augur.wu-wien.ac.at/COPSRL/?cat¼5
Overton L. An Introduction to personal learning environments. Making learning personal—using
PLEs to enhance learning. In: Towards maturity, Nov 18, 2009. http://www.towardsmaturity.
org/article/2009/11/18/introduction-personal-learning-environments/
Schanda F, Dikke D, Mueller N. Personal learning management systems (PLMS): concept,
classification, evaluation. In: eLearning Baltics 2012. Proceedings of the 5th International
eLBa Science Conference in Rostock, Germany, June 21–22, 2012.
Lessons Learned from the Development
of the ROLE PLE Framework
Sten Govaerts, Katrien Verbert, Evgeny Bogdanov, Erik Isaksson,

Daniel Dahrendorf, Carsten Ullrich, Maren Scheffel, Sarah Léon Rojas,
and Denis Gillet
Abstract Within the ROLE European research project, an interoperability frame-

work has been developed to support self-regulated learning and to enable learners
and teachers to create personal learning environments (PLEs). This framework
enables learners to assemble tools, services and resources together to create their
own custom learning environment. This chapter discusses the overall architecture,
the specific components of this architecture and the platforms in which we have
integrated the ROLE framework. Additionally, we share the lessons learned from
the design and development. Furthermore, we discuss our experience with the
S. Govaerts (*) • E. Bogdanov • D. Gillet

Ecole Polytechnique Fédérale Lausanne (EPFL), REACT, Station 9, Lausanne CH-1015,
Switzerland
e-mail: [email protected], http://react.epfl.ch; [email protected], http://react.epfl.ch;
[email protected], http://react.epfl.ch
K. Verbert
Technische Universiteit Eindhoven, Information Systems WSK&I, Postbus 513, 5600 MB,
Eindhoven, The Netherlands
Katholieke Universiteit Leuven, Department of Computer Science, Celestijnenlaan 200A,
3001 Heverlee, Belgium
E. Isaksson
KTH—Royal Institute of Technology, Stockholm, Sweden
D. Dahrendorf
IMC Information Multimedia Communication AG, Altenkesseler Str. 17/D3,
Saarbruecken 66115, Germany
e-mail: [email protected], http://www.im-c.de
C. Ullrich
Department of Computer Science and Engineering, Shanghai Jiao Tong University,
1954 Huashan Road, Shanghai 200050, China
M. Scheffel • S.L. Rojas
Fraunhofer Institute for Applied Information Technology FIT, Schloss Birlinghoven,
Sankt Augustin 53754, Germany
e-mail: [email protected], http://www.fit.fraunhofer.de;
[email protected], http://www.fit.fraunhofer.de

DOI 10.1007/978-3-319-02399-1_8
186 S. Govaerts et al.
ROLE development infrastructure and our collaboration within the ROLE devel-
opment team and with several open-source projects.
Keywords Development • Interoperability • Best practices • Lessons learned •

Collaboration • Open source • Widgets • Web apps • Framework • Personal learning
environment • PLE • Informal learning • Self-regulated learning • Social media
platforms
Introduction
The proliferation of Web 2.0 technologies (e.g. wikis and social networks) has
impacted the way users retrieve and use information and how they interact with
each other (Maness 2006; Ullrich et al. 2008; Ashley et al. 2009). The abundance of
Web-based tools and content creates many opportunities for Technology Enhanced
Learning (TEL).
The ROLE project aims to exploit Web-based tools and technologies to
empower learners to construct their own personal learning environments (PLEs).
The overall goal is to create flexible, Web-based, open technologies for the feder-
ation and mash-up of learning services to empower the learner to build her own
responsive learning environment. Responsiveness is defined as the ability to react to
the learner needs—i.e. through recommendation, adaptation or visual analytics
services that support the learner to be aware of and reflect upon her own learning
process (Fruhmann et al. 2010). The project also targets critical transition stages of
lifelong learning, e.g. due to shifts in learner interests or when leaving the university
and entering a company. Chapter 1 elaborates more on the ROLE vision on PLEs.
Learning management systems (LMSs) such as Moodle, CLIX and Blackboard
primarily focus on distributing learning content, organising the learning process and
serving as interface between learners and teachers. Dalsgaard (n.d.) notes that in
LMSs generally different tools, such as discussion forums, file sharing, whiteboards
and e-portfolios, are integrated in a single system that bundles all tools necessary to
manage and run courses. In contrast to PLEs, LMSs place a strong emphasis on how
to centralise and standardise the learning experience (Guo et al. 2010). Learning
activities in an LMS-based course are organised within a centrally managed system,
which is driven by the needs of the institution. On the other hand, a PLE takes a
more natural and learner-centric approach and is characterised by the free-form use
of a set of services and tools that are controlled and selected by individual learners.
In recent years, research on mash-ups has been elaborated, for example widget
mash-ups have been deployed at Graz University of Technology (Ebner and
Taraghi 2010). In addition, researchers have focused on augmenting traditional
LMSs with widgets to provide live-updating and flexible applications. Wilson
et al. (2009) have implemented widget support for Moodle. Their big challenge is
logging student activities with the widgets, as there is no communication between
the widgets and the LMS.
Lessons Learned from the Development of the ROLE PLE Framework 187
The ROLE framework builds on this existing work, but incorporates additional
core technologies such as inter-widget communication (IWC), automated user
activity tracking, collaborative spaces and authentication and authorisation services
to protect data. This is the basis to enable real-time communication between
widgets and users, and to automate user activity tracking from tools and services.
The analysis of such data and IWC provides the basis to develop responsive systems
that can react to learner needs in a coordinated way.
Within the time span of the ROLE project, a new Apache project, called RAVE,1
emerged with the aim to provide an extensible mash-up platform for using, inte-
grating and hosting widgets with personalisation, collaboration and content inte-
gration features. The features of Apache RAVE and the ROLE project are quite
similar, as confirmed by recent research that has been applying RAVE in educa-
tional contexts (Pierce et al. 2011; Chudnovskyy et al. 2012). Since the RAVE
project started during the development of the ROLE framework, ROLE did not
adopt Rave, but rather contributed components to the RAVE project (which is
discussed in more detail in section ‘Contributing ROLE Software to Open-Source
Projects’).
This chapter presents the ROLE interoperability framework, which is a technical
platform to assemble widgets within responsive open learning environments. The
framework allows the assembly of widget bundles with communication channels,
authentication and authorisation mechanisms and services for activity tracking and
analysis. The framework ensures that the widgets have access to the necessary
information to react to learner needs. Furthermore, the platforms, on which the
interoperability framework has been integrated, are discussed and the lessons
learned from the design and development of the framework components are
presented, as well as on the technical collaboration within the ROLE project and
with open-source projects.
This chapter is organised as follows. First, the overall architecture of the
framework is presented in section ‘The Interoperability Framework’, after which
each component is discussed in more detail. Section ‘ROLE Platforms’ elaborates
on the different platforms that integrate the ROLE infrastructure and the repository
of widgets. Afterwards, the organisation of the ROLE developer community and
our contributions to open-source projects are discussed. Finally, the achieved
results are summarised and their dissemination is discussed.
The Interoperability Framework
The purpose of the ROLE interoperability framework is to support assembly of

different widgets in responsive open learning environments. The architecture sup-
ports communication between widgets, authentication and authorisation
1
Apache RAVE, http://rave.apache.org/
mechanisms, services for activity tracking and analysis and widget spaces, which
manage widgets, resources and users. All these services can be accessed via open
and if possible standardised interfaces. These are necessary for third-party devel-
opers who want to create applications based on ROLE technology. The next section
details the overall architecture.
The Architecture
The ROLE architecture is illustrated in Fig. 1. IWC (see section ‘Inter-widget

Communication’) is used and managed by spaces, but is also an autonomous
component. It depends on JavaScript and the XMPP (Saint-Andre 2004a, b)
protocol to provide a user-, community- and space-centred remote IWC. This
allows developers to build powerful collaborative real-time learning tools and
learners to assemble them easily in responsive open learning environments.
Tracking of activities is done via the Contextualised Attention Metadata (CAM)
framework (see section ‘Contextualised Attention Metadata’). An event-based
schema was developed to model user behaviour in learning environments. Events
are tracked and sent to either a central or container-specific repository. IWC is used
to track such events. The data is stored and retrieved via an REST API.
As the CAM service contains sensitive data, an authorisation and authentication
framework has been developed to protect this data (see section ‘Authentication and
Authorisation’). It is also needed for other ROLE services that handle personalised
data. One of the main goals of this framework is to reduce the amount of user
interaction by providing a Single Sign On (SSO) authentication mechanism.
Finally, widget spaces (see section ‘Spaces’) allow learners and instructors to
create portable collaborative learning environments. Spaces consist of learners,
configurable services and sharable resources, within a learning context. The space
features can be provided either by a (OpenSocial) container itself or by a special
widget. Such an approach guarantees container independence. Furthermore, widget
Fig. 1 The ROLE architecture

spaces provide a simplified single point of access to the other background services
via an extended OpenSocial API.
Inter-widget Communication
IWC enables event-based communication between widgets following the Publish–

Subscribe communication pattern (Birman and Joseph 1987; Eugster et al. 2003).
We employ both local inter-widget communication (LIWC) within a PLE and
remote inter-widget communication (RIWC) among different users, computers
and PLEs.
LIWC is realised in the OpenApplication Event API (Isaksson and Palmer 2010,
n.d.) using the HTML5 Web Messaging standard (Hickson 2011) available in most
major browsers, including backwards compatibility for the Google Gadget PubSub
mechanism. Instead of ‘hard-wiring’ widgets with each other (Sire et al. 2009), all
widgets within a PLE are notified of all events and then decide autonomously to
react accordingly. If the widget acts upon the received event, a receipt is sent back.
Supporting containers that receive such a receipt can inform the user, e.g. by
highlighting the tool that sent the receipt. The event payload format is designed
for partial semantic interoperability, i.e. developers use a combination of
established vocabularies in a simplified format with name-spaced properties
(e.g. Dublin Core (DCMI Usage Board 2006)). In practice, this means that when
an event is broadcasted, the originating widget does not indicate what receiving
widgets should do (only the past action is specified, e.g. select). If the originating
widget had to specify the intent (i.e. the desired future action), it would need to have
buttons or menu items for every conceivable proposable action in every conceiv-
able widget (e.g. add to portfolio, share with friends, search in Wikipedia). With
events, we instead choose to split the job: events should be broadcasted for as many
user actions as possible within each widget, without concern for what receiving
widgets ought to do, and receiving widgets provide the affordances (e.g. buttons)
for their own proposed further actions.
RIWC enables communication among widgets in different browsers and on
different machines in order to foster real-time remote communication and collabo-
ration functionality. RIWC is realised with the Extensible Messaging and Presence
Protocol (XMPP) (Saint-Andre 2004a, b), an open standard for real-time communi-
cation. The power of XMPP lies in its built-in federation capabilities and extensi-
bility through XMPP Extension Protocols (XEPs), such as for Publish/Subscribe
(Millard et al. 2010) and Multi-User Chat (Saint-Andre 2008) as applied in respon-
sive and collaborative learning scenarios (Friedrich et al. 2011). Since no
JavaScript XMPP library with PubSub support was available, ROLE extended the
dojo XMPP library by a set of common PubSub operations. Users can discover
nodes, retrieve subscriptions, create, configure and delete nodes, subscribe and
unsubscribe nodes and publish/receive IWC events in an XML-based payload
format across a federated network of XMPP servers. However, current libraries
using XMPP over BOSH (Paterson et al. 2010) are not applicable in public
containers such as iGoogle due to cross-domain issues. Furthermore, they are rather
unstable and unreliable (Friedrich et al. 2011). Our experiments showed that the
upcoming Web Socket API (Hickson 2009) for XMPP (Moffit and Cestari 2010)
outperforms BOSH with considerable performance and stability improvements and
availability in all containers.
IWC enables more responsive, collaborative environments with real-time noti-
fications and richer user experience, although attention to usability is required
(Isaksson and Palmer 2010).
Contextualised Attention Metadata
Tracking of user interactions with widgets is an essential part to enable responsive-

ness in learning environments. User interaction data can be used for data analysis
and the computation of personal, social and contextual information about users and
applications. Additionally, such data of the actual usage of ROLE services in real-
world settings was used to evaluate the framework.
A variety of attention metadata formats exist. These formats differ in scope,
expressiveness, scalability and context awareness. Butoianu et al. (2010) provide a
survey of the following formats: TaskTracer (Dragunov et al. 2005), Swish (Oliver
et al. 2006), CAM (Wolpers et al. 2006), the User Interaction Context Model
(UICO) (Rath et al. 2009), the Context Modelling Language (CML) (Henricksen
and Indulska 2006) and WildCAT (David and Ledoux 2005). TaskTracer and Swish
are least flexible and expressive. UICO, CML and WildCAT are very expressive;
however, the available frameworks using the formats do not scale well and some are
focused on a specific application (Butoianu et al. 2010). On the other hand, CAM
supports scalability and context awareness very well, but is less expressive. Other
examples are the ActivityStreams specification2 and the Experience API (Glahn
2013). The latter took inspiration from ActivityStreams and only became available
in the last years of the ROLE project. ActivityStreams are less focused on contex-
tual information. In ROLE, CAM is used because of scalability and context
awareness.
The CAM schema (Schmitz et al. 2011) can be used to describe computer-
related activities of one or several users—i.e. which objects attract user attention,
which actions users perform and what the user contexts are. CAM was developed to
describe as many types of attention metadata as possible. Therefore, CAM records
of a user cannot merely describe user foci of attention, but rather her entire
computer usage behaviour. Collections of CAM records can be exploited for
generating diverse kinds of profiles like user profiles and object profiles (item
profiles). CAM records represent user–computer-related foci of attention and
actions and thus can instantly constitute profiles of individual usage behaviour.
2
The ActivityStreams specification, http://activitystrea.ms/specs/json/1.0/
Fig. 2 Structure of the CAM schema
CAM records of different users can be exploited for generating attention and usage-
based object profiles.
CAM records can be used to support self-reflection. For instance, visualisation
widgets can support a user to recapitulate what she did and generate a picture of her
competences. Furthermore, statistical metrics can be employed to aggregate and
evaluate CAM records of different users. By this, general trends, for instance in
computer usage, data consumption and communication behaviour, can be detected.
Aggregated CAM records entail information on the behaviour of average users and
on the behaviour of user groups. They also entail usage information on data objects,
such as how often a certain object was used, and by which kinds of users and in
what contexts it was used. In addition, they can reveal in which respect a user
deviates from the average, whether her behaviour conforms to general trends or
not, etc.
The CAM schema has been developed to provide a unified schema for monitor-
ing data across system boundaries (Wolpers et al. 2007; Schmitz et al. 2011). The
CAM schema has been transformed from a once user-centred version to an event-
based version (see Fig. 2) that is better suited for evaluating and analysing user
observations over time.3
3
Information about the schema is available at https://sites.google.com/site/camschema/ and the
CAM API: http://sourceforge.net/projects/camapi/
One major goal of ROLE is to provide personalisation, recommendation and

self-reflection mechanisms. To achieve this, users can be monitored while
interacting with their learning environment. The collected CAM information is
used to generate different patterns and statistics, such as discovering learning trends
and detecting what is currently happening in the learning environment. For an easy
integration of the CAM monitoring into different learning environments, the moni-
toring architecture was divided into a client and a server component. The client
component can be considered as a data collection element, responsible for accumu-
lating and transforming the information into CAM, while the server component is
responsible for the persistence management and data access control.
Figure 3 shows the CAM architecture applied in ROLE and how it works
together with other ROLE components. The picture shows a platform, which uses
ROLE technology by integrating ROLE widgets into their learning environment
Fig. 3 The CAM architecture used in ROLE

Fig. 4 Screenshot of the ROLE CAM Monitoring widget
(container). Furthermore, the picture illustrates that the CAM monitoring compo-
nents must not be integrated into the platform since they are running on an external
application server. First, a user performs an action on a widget (step 1), e.g. clicking
a button. Since this widget supports IWC (see section ‘Inter-widget Communi-
cation’) this action causes an event, which is published via the OpenApp mecha-
nism (Isaksson and Palmer 2010; Isaksson and Palmer n.d.). Thus the event is
broadcasted (step 2) and can be received by all other widgets in the user’s learning
environment (local IWC, see section ‘Inter-widget Communication’). Each receiv-
ing widget can process and react on the event (step 3). The CAM widget can thus
receive all events sent through IWC. Afterwards the CAM widget identifies the
event and forwards all required information to the CAM Web service, which is
responsible for the CAM mapping and persistence. To offer the user an overview
over the activities in the learning space, it contains a graph where past events from
the users of the space can be displayed using different configurations, i.e. one or
several users and several dates (see Fig. 4). As already mentioned, the picture
illustrates that the Web service is not integrated into the platform, but is a stand-
alone service that can be installed for a specific platform.
In addition to storage, the CAM Web service provides a query method, which
consumes an arbitrary SQL select statement and returns the results in the JSON
format. Using arbitrary SQL select statements ensures maximum freedom for the
developers to experiment with the data. Access to the monitoring data causes
privacy issues. These issues have not been completely resolved in the ROLE
project. Some steps have been taken though. For instance, data privacy can be
supported through a stand-alone installation of the CAM Web service, e.g. by
installing the CAM service in the intranet of an institution. The CAM widget also
allows users to disable tracking (see Fig. 4). Additionally, the CAM Edit and Share
widget4 (see Fig. 5) allows the user to filter and export her CAM data from a specific
space into an SQL file. This widget provides the user with full access and larger
control over her CAM records, and she can share her events with others or analyse
them herself. Finally, since the Web methods for querying and storing CAM are
publicly available, the CAM service requires password-based authentication.
From our experience developing the CAM service and using it in real-world
settings various new insights have been gained. One of the main benefits of the
approach is that developers do not have to specifically write code to track user
events, as the CAM widget will collect the IWC events automatically. On the other
hand, this can also limit the data collection since not all important events might be
requiring IWC. This tracking method also allows developers to be agnostic about
the CAM schema and the implementation of the CAM Web service, since they do
just need to send out IWC events. One of the strengths of the OpenApp IWC is the
openness of its data format, which has no mandatory fields and enables developers
to transmit any kind of JSON data from one widget to another. This freedom makes
the mapping of the OpenApp events into CAM harder as such mapping cannot rely
on certain elements to be present. Defining a subset of fixed mandatory IWC fields
and fixing taxonomies of event types would make the CAM mapping more easy and
robust.
Another flexibility issue is due to the high abstraction level of the CAM schema
to allow all kind of events to be stored. This can generate a large number of different
CAM event mappings, which can make it harder to analyse and compare the CAM
records. The different IWC events stored information in different CAM fields,
making the data analysis more complex without knowing the details of the intrinsic
mapping. The ActivityStreams specification provides an extensible, common
vocabulary list of actions that would aid this and when applied properly could
provide more portable data (Vozniuk et al. 2013).
Currently, user activities are only tracked when the CAM widget is added to the
learning environment. Hence, this enables users to have full control over where
their activities are tracked, but also causes that nothing is recorded if the user
forgets to add the CAM widget. This problem could be circumvented by integrating
CAM monitoring directly into the platform.
4
http://www.role-widgetstore.eu/tool/cam-edit-and-share-widget
Fig. 5 Screenshot of the

ROLE CAM Edit and Share
widget
Spaces
In the ROLE framework, a PLE can consist of various spaces. A space is an abstract
concept that materialises the user’s context and aggregates people, resources,
applications and other subspaces. All these artefacts belong to the same activity
that a person (or a group of them) is working on to achieve a common goal. This
common goal is the purpose to create the space. Various people can participate in a
space and might have different access rights and roles within this space, where they
share resources and applications that they need to achieve their goal. A space might
have subspaces that help hierarchical organisation of resources, applications and
people. A space can be seen as a PLE unit. On the one hand, a space is a way for
users to give shape to their PLEs by aggregating information. On the other hand, a
space allows users to share their PLEs with others by inviting them to collaborate.
The space concept exists in all ROLE platforms (see section ‘ROLE Platforms’):
the ROLE SDK (see section ‘ROLE SDK’), Graasp5 (Bogdanov et al. 2012a) (see
section ‘Graasp’), the OpenSocial Moodle plugin (Bogdanov et al. 2012b) (see
section ‘OpenSocial Moodle Plugin’) and the Widget Store (see section ‘The ROLE
Widget Store’). Every platform internally implements this concept in a different
way. In order to allow widgets to use the information about the space and to enable
integration and portability of spaces between the platforms and beyond, we applied
two approaches: Linked Data and OpenSocial.
For the first approach, we created a space ontology for Linked Data. Linked Data
provides a very powerful and extensible way of describing data in a machine-
understandable way. It targets the discovery and integration of data originating
from different sources. Due to its design, it has limitations. First, Linked Data and
SPARQL require a rather steep learning curve, which is a disadvantage compared
to simple RESTful APIs that are used by many Web developers.6 The second
limitation is the performance. Since the data is located on different servers, many
HTTP requests have to be issued to retrieve the complete data. Moreover, the
SPARQL engine requires traversal of a graph, which is much slower than retrieving
data from a relational database. The authors Health and Bizer (n.d.) (see section 6.3)
foresee the use of data crawling for real-time Linked Data applications, rather than
on-the-fly URI dereferencing.
The alternative approach we used is OpenSocial. The OpenSocial specification
consists of three main parts. The first part describes the widget standard. The second
part standardises the model for social network elements (i.e. Person, App and
Document) and relations between them. The third part standardises a set of com-
mon REST and JavaScript APIs to retrieve data from a social platform. Since the
space concept did not exist in OpenSocial, we introduced it into the OpenSocial
specification. The OpenSocial Space extension standardises the space model
(namely a list of fields that a space can contain), and the REST and JavaScript
5
Graasp, http://graasp.epfl.ch
6
Linked Data API, http://code.google.com/p/linked-data-api/wiki/API_Rationale
APIs to work with spaces (Bogdanov et al. 2011). Through this extension, widgets
can retrieve information about their containing space and its content via OpenSocial
APIs. For example, the Person API can be used to retrieve all the members of the
space. The widget can retrieve a list of the resources and widgets available in the
space via the Document and App APIs, respectively.
The main disadvantage of OpenSocial is that the social model cannot be easily
and arbitrarily extended as with Linked Data. The new extensions require going
through the process of standardisation, which can be quite cumbersome. On a
positive note, OpenSocial provides easy-to-use REST APIs with a JSON-based
data representation. The data format and APIs are standardised, which enables
interoperability when data is accessed and processed. OpenSocial does not target
data discovery (as Linked Data) but rather data retrieval and exchange. Since the
data is often centralised in one institution, it is very fast to retrieve the data
compared to the SPARQL engine.
Authentication and Authorisation
CAM contains sensitive and personal data protected by law. Additionally, users
might prefer to keep the content of their spaces private. The data access has to be
trusted and allowed by the users. The data communication occurs at two different
levels: service-to-service and widget-to-service communication.
Service-to-service communication can occur when for example a recommen-
dation service requires CAM relevance feedback on resources. Data can be trans-
ferred across institutions and countries with different laws. Thus, we decided to
leave the decision of service-to-service authentication and authorisation (A&A) up
to the service developers.
Widget-to-service communication occurs when for example a self-reflection
widget wants to query the CAM service. This has been implemented as follows.
A user is authenticated as being the owner of a particular personal space. The user
first authenticates as being the owner of a separate identity, to which the personal
space is linked (or a new personal space will be created), which thereby implies that
the user is the authenticated owner of the space. The personal space then functions
as the identity of the user.7 Authentication is typically done via OpenID, which is
the standard for decentralised authentication on which we have focused, but other
protocols may be supported as well (one test bed, at Uppsala University, has
implemented support for CAS while also keeping support for OpenID
authentication).
Delegated authorisation is used by widgets that access collaborative and per-
sonal spaces. Furthermore, such delegated authorisation may also be used by third-
7
The identifier of the personal space, i.e. a URI, is used as the identifier of the user, cf. WebID (see
http://www.w3.org/wiki/WebID)
party services. The standard for delegated authorisation that we have focused on,
i.e. OAuth, lets the currently authenticated user choose whether to authorise the
widget or service. After authorisation, the service is provided with a token granting
access to spaces on the user’s behalf. For widgets, the token is managed by the
OpenSocial widget container, which allows widgets to perform requests through the
engine’s OAuth proxy that first applies OAuth and then forwards the request.
Currently, OAuth endpoints for OpenSocial widgets must be hard coded in the
widget’s source XML. Therefore, spaces implement a rewriting of the XML so that
the proper endpoints are included. Otherwise, it would be necessary for widget
developers to maintain separate XML files of their widgets for every server where
the widget is deployed. Widgets using the rewritten XML files, however, can be
added to any widget container, such as Liferay8 (Yuan 2009), while still
maintaining the connection to their respective spaces.
ROLE Platforms
The ROLE interoperability framework has been integrated in various platforms.

This section describes these platforms where users create and use their PLEs and
search for widgets.
The ROLE SDK
The ROLE SDK is a collection of software and tools, which allows trying out
ROLE technology and developing new widgets for mash-up PLEs. In total, ten
versions of the ROLE SDK were released, each one packaging the implementation
outcomes of one milestone of an iterative development process (see section ‘The
ROLE Developer Community’).
The central core of the SDK is the reference implementation of a sample PLE,
which allows using ROLE technology in practice and developing new widgets at
the same time. Within the ROLE SDK, a learning space functions as a collaborative
context for learning, consisting of a bundle of widgets, along with a list of
participants. Widgets can interact with other widgets, and participants can interact
with other participants, by using widgets and built-in features of the ROLE SDK
(e.g. chat functionality).
A personal space is defined as a personal context that consists of a person’s user
model. One representation of the user model is a user profile; another representa-
tion, also based on the user model, is a bundle of (personally chosen) widgets. In the
8
Liferay, http://www.liferay.com/
ROLE SDK, the learning space and personal space are combined in one user
interface.
Additionally, there is a social context. The social context offers access to the
communities of which the user is a member. The specific community that is
accessed typically depends on the website where the widget is currently being
used, which may be a different website than that of the PLE. For instance, if a
widget is being used on a social networking website, the community would be that
of the website. Collaborative contexts (i.e. learning spaces) can transcend social
contexts. A widget may be part of a learning space, and at the same time be used on
a social network for inviting people from that network to participate in the space.
OpenSocial standardises APIs for access to what is defined as the social
context here.
Furthermore, the concept of activities was introduced. Activities can be defined
as purposes for which the user structures her learning context and assembles
widgets. In the ROLE SDK, activities are displayed as one group of visible widgets
at a time. In the GUI, activities can also be tabs or pages. However in the ROLE
SDK, the additional semantics that the term ‘activities’ offers is covered. The term
hints to the user that the groupings should be used for focusing on one activity at a
time (such as training English vocabulary or searching for Web resources), using
the tools that are appropriate to that activity, without being distracted by what is
unrelated to the activity.
As mentioned before, the ROLE SDK relies on the concept of spaces. While a
space is, at its most basic level, simply a bundle of resources such as widgets, there
are several aspects that contribute to its usefulness as the basis for a PLE:
• Aggregation: Widgets (or more generically, tools) are bundled with any other
kind of resources that contribute to the space’s goals. The model enables a very
flexible use of spaces, without requiring modifications in the model or its
implementation.
• Contextualisation: A space forms a context for its contained resources. Widgets
can be made context aware, and are then able to interact with the space and its
resources. In addition to being in the context of a space, widgets can be
contextualised further by being given configuration that is specific to their
instantiation within the space.
• Participation: People can join a space, which means that they become members
of that space. Members are notified of the presence of other members, and can
interact with them both asynchronously and synchronously.
• Personalisation: Spaces offer a level of customisability, so that users are able to
personalise the environment according to their needs.
These design requirements are realised within the ROLE SDK. Being a sample
implementation of a PLE platform, the space user interface (see Fig. 6) is a Web
application composed of four parts. First and on top, the header element is
implemented as a top-aligned bar. It provides elements for controlling the Web
application as a whole, such as signing in and out, and navigating to other parts of
the application. Secondly, the sidebar element is a narrow, fixed-width section,
Fig. 6 The ROLE SDK user interface of a learning space
Fig. 7 The ROLE Widget Store, embedded in the ROLE SDK
running along the left side. It provides elements for controlling the space, such as
switching between activities, and adding widgets.
Thirdly, the container element comprises the central area of the user interface,
not covered by the other parts. This is where the main content is located, typically
the space’s widgets. The container can also be used as an embedded browser, which
is how the ROLE Widget Store (see section ‘The ROLE Widget Store’) is inte-
grated (see Fig. 7).
The fourth and last element is the dashboard, a bottom-aligned bar when
collapsed. Expanding the dashboard displays the widgets on the personal space
Fig. 8 The personal space, inside the expanded dashboard
Fig. 9 The personal space, being accessed on a third-party website
right above the container, as illustrated in Fig. 8. The dashboard provides access to
the user’s personal space, which is a space that is private to the authenticated user. It
is available from any other space (and other pages of the Web application) as well
as from third-party websites via a bookmarklet (cf. Fig. 9).
In the case that the space itself is embedded (e.g. on the course page of an LMS),
it is intended that the parts can be hidden or moved (e.g. the Header), because their
functionality (e.g. sign in) can be already provided by the LMS or to adhere to
another design.
Fig. 10 A shared contextual space created in Graasp that integrates resources gathered from the
Cloud, such as YouTube videos, SlideShare presentations, OpenSocial Widgets, Web pages or
PDF documents with previews
Graasp
Graasp9 (Bogdanov et al. 2012a) is a social media platform for collaborative

learning and knowledge management (see Fig. 10). Graasp implements the
OpenSocial space specification (see section ‘Spaces’), which enables the creation
of spaces shared between people belonging to different communities and networks.
Embedded shared resources are gathered across institutional and corporate bound-
aries. Unlike dominant social media, Graasp enables a fine definition of the
audience, as well as the associated rights and roles to ensure trust and privacy
enforcement. In Graasp, people map their personal and shared projects, interests
and activities into public or private contextual spaces integrating invited members,
relevant resources and necessary apps which can be tagged and rated. Any space or
resource in Graasp integrates its own discussion thread to enable contextual inter-
action. Graasp allows learners to construct and manage their own PLEs. Users can
create a PLE for each learning objective, populate it with various resources and
tools, personalise it and share it with others (Bogdanov et al. 2012a).
9
Graasp, http://graasp.epfl.ch
The space concept is at the core of Graasp. A space can represent a PLE and can
contain four types of entities: people, resources aggregated and used within the
space, apps added to the space to extend its functionality and subspaces to organise
the space content in a hierarchical structure. Graasp enables users to manage their
spaces.
The resources and apps can be aggregated into a Graasp space from both local
and remote locations. First, users can easily drag and drop files directly from their
desktops into their spaces. Second, remote resources from the Cloud can be easily
aggregated via an aggregation mechanism called GraaspIt!. Whenever a user
encounters an interesting page, she can simply click on the GraaspIt! bookmarklet
in the browser and the resource will be added into a space (Gillet and Bogdanov
2012). These collected resources can be aggregated as URLs, embed tags or Web
page screenshots. In addition to resources, Graasp allows users to aggregate widgets
into their spaces. Currently, the OpenSocial widget standard is supported, though
other standards (e.g. W3C widgets) can be added in a similar way. Such widgets
either can be added manually or can be aggregated from existing widget reposito-
ries. For example, when the user is browsing through widgets in a widget repository
(e.g. the ROLE Widget Store, see section ‘The ROLE Widget Store’), a widget of
interest can be added by just clicking on GraaspIt!. The second way is to add
widgets from the ROLE Widget Store by exploiting the widget repository search
mechanism provided within Graasp.
Once a space is created and populated in Graasp, the core part of the interface
(see Fig. 10) enables users to interact with the aggregated content and can be further
personalised with the concept of functional skins (Bogdanov et al. 2011). A
functional skin is a client-side plugin for a space that can retrieve space data via
the OpenSocial APIs and provides users with visual and functional features differ-
ent from Graasp and tailored to specific needs. For example, in addition to the
standard view of Fig. 10, Graasp offers two built-in functional skins: the resource
view and the app view. The resource view displays a list of all resources that exist in
a space and provides download links and presents resource previews. The app view
displays all widget instances from a space as a visual mash-up. In this view, widgets
can be resized and their order can be modified through drag and drop. The
possibilities to personalise the space are extensive and through functional skins
the users can further adapt their spaces for their own professional/personal tasks.
Graasp implements several mechanisms to share and exchange spaces and
widget bundles (sets of widgets combined together for a specific purpose). An
app bundle can be extracted from the existing space and exported as an OMDL
file.10 The OMDL file can be imported into another platform (or reused in Graasp)
or shared at the ROLE Widget Store. Additionally, a space created in Graasp can be
shared with other people and with other platforms. The space can be extracted from
Graasp as a secret URL. This URL can be given to other users and allows them to
collaborate anonymously. Alternatively, the space can also be embedded into
10
Open Mashup Description Language, http://omdl.org
Fig. 11 A teacher creates a space with widgets for a course
another Web platform. Chapter 5 provides more information on how Graasp was
used and evaluated for formal learning.
OpenSocial Moodle Plugin
The OpenSocial Moodle plugin (Bogdanov et al. 2012b) enables the use of
OpenSocial widgets within the Moodle LMS11 to create PLEs. By providing
support for PLEs in an existing LMS, the disruption often caused by providing
users with completely new environments decreases. By integrating PLE features in
an existing LMS, users can still continue to use the features of the familiar LMS, but
can personalise their learning environment with widgets.
The OpenSocial plugin for Moodle exists in two variations. The first version
adds a new module to Moodle, which is displayed in the central area in the Moodle
UI.12 The module allows a teacher to add a ‘widget space’ to the Moodle course
page, specify a set of widgets and choose the widget layout on the Moodle page (see
Fig. 11). After this configuration, students can work with several widgets simulta-
neously (see Fig. 12) in the Moodle course.
11
Moodle, http://www.moodle.org
12
OpenSocial Moodle module, https://github.com/vohtaski/shindig-moodle-mod
Fig. 12 OpenSocial widgets displayed within Moodle
The second version13 of the OpenSocial plugin adds a new block in the right
column of the Moodle UI. With this Moodle block, the teacher can add widgets to
the right column of existing Moodle pages. Both versions of the Moodle plugin
make use of the Apache Shindig engine,14 which provides an open-source imple-
mentation of the OpenSocial specification, to render and manage widgets.
One of the main benefits of these plugins is that they enable teachers to easily
extend Moodle with new features and services provided by widgets. Consequently,
once the OpenSocial plugin is installed in Moodle, a teacher can append the
required functionality herself, without the intervention of system administrators.
The plugin enables the flexibility of selecting the resources and tools required for a
specific course. Additionally, the plugins enable reuse of existing educational
resources and tools. Furthermore, teachers and students can continue to operate in
the learning environments they are familiar with but gain the mash-up features of
PLEs. Naturally, the components of the ROLE architecture are compatible with the
Moodle plugin. For instance, IWC and CAM are fully operational in the Moodle
plugin. By extending widely used LMS with PLE features, we aim to achieve a
faster adoption of the PLE paradigm among institutions. Further details on how this
13
OpenSocial Moodle block, https://github.com/vohtaski/shindig-moodle-block
14
Apache Shindig, http://shindig.apache.org
Moodle plugin was put to use for formal learning and evaluated are available in
Chap. 4.
The ROLE Widget Store
The ROLE Widget Store allows users to search and browse for widgets and
compilations of them. The store addresses the issue of categorisation, browsing,
searching and recommending by providing various widgets categorised based on
functionality, learning phases and learning domains. Further, the Widget Store
enables sharing of platform-independent PLE and templates composed of learning
tools and artefacts (or the so-called Widget Bundles). Via these mechanisms, the
Widget Store fosters the development of a community of practice to exchange
learning tools. Regarding the widget bundles, the store provides features to apply
and share bundles across different learning platforms. This section further discusses
different recommendation strategies and the interfaces that enable interoperability
are specified (LMS/PLE system integration). Figure 13 presents an overview of the
Widget Store architecture.
The main focus of the store is to provide a catalogue of widgets by supporting
two commonly used widget specifications: the W3C widget specification (Caceres
n.d.) and the OpenSocial specification (Mitchell-Wong et al. 2007). Developers can
post either their self-developed widgets or widgets based on licenses which allow
further distribution. Where possible metadata are automatically extracted from the
Fig. 13 Components of the ROLE Widget Store

widgets. Widget bundles are compilations of widgets, which are created to share
good practices of widget use in learning environments. They are intended as a fast
and simple way to provide learners with tools, services, content and a detailed
description of how to use these to complete a specific learning task. Learners can
select several tools from the store to create their learning environment. Additional
references to learning resources can be added. For each tool and resource, learners
are able to add learning activities in order to describe what should be done using the
tool or learning resource. Once such a bundle is created by a learner, she can share it
with the community. Such bundles can be reused by teachers and learners for their
learning environments. In order to support learners in selecting applications for
their PLEs three different categorisations are provided.
• Tool categories are derived from the Psycho-Pedagogical Integration Model
(PPIM) (Fruhmann et al. 2010) (more information on PPIM is also available in
Chap. 2), so users can select widgets supporting different learning phases.
• Tool functionalities represent features of widgets (e.g. text editing, video chat)
and are based on an ontology developed in ROLE.
• Learning domains describe, if possible, the domain of the tools by providing
semantic tags using DBpedia.15
The categorisation of bundles differs from the tool categorisation. A bundle can
be designed to cover several phases of the PPIM model and thus refers to several
tool categories. The approach of the Widget Store is that a bundle automatically
inherits functionalities of tools it contains and can be tagged manually by learning
domains from the DBpedia.
To provide an interface for external systems, the ROLE Widget Store offers an
SPARQL endpoint which allows retrieval and insertion of the data of the Widget
Store based on a standardised interface. Furthermore, different formats (Turtle,
RDF/XML and JSON) are supported so that developers can choose their preferred
data format. Another possibility for PLE platforms to integrate the store is to embed
the store in the learning environment. The embedded version provides a simplified
user interface and offers buttons that allow users to directly choose widgets to
assemble their PLE. The store is connected to other ROLE components in the
following ways:
• Graasp queries the store to provide a catalogue of widgets enabling easy
integration of widgets in Graasp spaces.
• The ROLE SDK embeds the store and uses the embedding features to add
widgets to the ROLE spaces.
• The ROLE Pedagogical Recommender (Nussbaumer et al. 2012) queries the
store to provide recommendations based on the ontology of learning activities
and the store categorisations.
15
DBPedia, http://dbpedia.org/
• The ROLE Requirement Bazaar16 (Renzel et al. 2013) uses the data to support
the requirements elicitation and negotiation process being part of the ROLE
Social Requirements Engineering approach. The ROLE Requirement Bazaar is a
collaborative social platform where users can illicit their needs and wishes to
developers who can extract requirements for future implementation.
Widgets and Tools
One major problem regarding the adoption of ROLE in new test beds and increas-
ing the number of users was the limited number of widgets that were available.
One approach to overcome this problem is to enable a very simple transforma-
tion of existing Web resources into widgets (Ullrich et al. 2013). This transforma-
tion can be done by developers through the usage of widget templates as well as by
non-technical people with the help of an authoring tool. Both solutions support the
same ROLE technologies, namely the capturing of interactions via CAM and the
possibility to rate the widgets. Interactions are captured on a very generic level:
basically, whenever a student uses a Web application integrated into a widget for a
period longer than five seconds, then the widget sends out CAM event of the type
‘used’. Of course, a developer can refine the interactions, when required.
The ‘widgetisation’ of a Web application is simplified through the usage of a
template and through JavaScript libraries that can be included in (existing) widgets.
The template defines a widget that embeds the Web application via the iframe
HTML element. This has the advantage that the original Web application does not
need to be modified. In case, a widget of the Web application already exists, the
capturing of interactions via ROLE can be enabled by the inclusion of the
JavaScript library. This library uses IWC to send out the captured interactions,
which can be made persistent on the CAM service via the CAM Monitor widget as
described in section ‘Contextualised Attention Metadata’.
The proposed approach has been implemented and a widget template is available
in which the widget developers have to add the link to the Web page they want to
integrate. To extend an existing widget, one has to include several lines of
JavaScript code. The generation can be automated by using a set of shell scripts.
The scripts take a list of URIs as input and generate widgets for the URIs. This
reduces the authoring time to less than a minute. In summary, while this solution
works very well for advanced software developers, it is still too complex for the
average user. As an example, the code to extend an existing widget with ROLE
technologies looks as follows:
16
ROLE Requirements Bazaar, http://role-is.dbis.rwth-aachen.de:9090/BazaarFrontend/index.
html
//Load two libraries for allowing the user to rate this gadget//and for
capturing interactions in CAM format $.getScript("http://widgets.
onlinesjtu.com/gadgets/libs/rating.js",
function(){
$.getScript(
"http://widgets.onlinesjtu.com/gadgets/libs/interactioncapture.
js",
function(){
var rating ¼ new ROLE_module.rating
("#importedGadget");
var interactioncapture ¼
new ROLE_module.interactioncapture
("#importedGadget");
})
}
);
The first lines load the libraries. The functionality is activated by creating the
appropriate objects. In the example, #importedGadget specifies the HTML element
to which the interaction capturing and rating functionality should be attached
(typically a div element, which is the parent of the iframe element).
In addition to the simplification of the usage of the libraries, the Shanghai Jiao
Tong University (SJTU) created an authoring widget that allows teachers without
Web development expertise to generate widgets from existing Web resources. The
authoring widget asks users to input the URI of the Web application and add some
metadata. Then, the authoring tool generates and uploads the widget to a server.
Through an integration with the OpenSocial Moodle module (see section
‘OpenSocial Moodle Plugin’) users can create widgets without having to leave
the learning environment.
Thanks to these tools, SJTU was able to create several hundreds of very domain-
specific widgets for ROLE. Additional details on the SJTU test bed are presented in
Chap. 4.
ROLE and Open-Source Developer Communities
All technical partners of the ROLE project have been collaborating successfully to
create the ROLE framework. To foster this collaboration, various support mecha-
nisms were set up, consisting of management structures, sub-projects, development
software and developer meetings. This section elaborates on the developer collabo-
ration within the ROLE project and with open-source projects to disseminate ROLE
technologies.
The ROLE Developer Community
Technical cluster structure: To enable the assessment of requirements, exploration

of technologies, creation of early prototypes and their evaluation, the development
process was split up in consecutive sub-projects, each having its specific goals and
deadlines. In total there were five of such projects: the Christmas project (ended on
Christmas 2010), the Easter project (ended on Easter 2011), the Stonehenge project
(ended on December 22, 2011), the Gunpowder project (ended on January
31, 2012) and the Shori project (ended on January 31, 2013). By defining use
cases and goals for each project, the requirements and planning were defined. These
projects also allowed easier planning of evaluations. The longer projects (i.e. the
Gunpowder and Shori project) had a more elaborate planning phase and manage-
ment methodology. For the Gunpowder and Shori project we aimed to apply the
SCRUM (Schwaber 2004) and Kanban (Ladas 2009) methodology. But due to the
large, geographically dispersed team from different organisations, we opted for an
adapted version of SCRUM combined with Kanban, where one or two persons
would manage the development process and report progress to the ROLE general
assembly. The requirements and goals for the projects were often laid out in a face-
to-face developer meeting or developer camps (see below) and follow-up virtual
meetings. The project managers would then plan milestones (or sprints) often based
on evaluation deadlines and showcases at conferences. Certain topics had smaller
teams working on it in task forces, e.g. assessing a solution for authentication or
CAM. Bi-weekly technical virtual meetings were organised to discuss progress, and
to decide on technology and architecture choices. This setup allowed all developers
to work on their own tasks and be involved in the decision making, but also get an
overview of the current project status. Furthermore, it allowed the project managers
to follow up the progress and react quickly where needed. This approach was
received positive by developers, managers and general assembly. Next to this
methodology, the development was also assisted by software.
Development software: To support the developers and the management, several
software packages were set up. To provide access to and version control our source
code we experimented with Git17 and Subversion (SVN).18 Initially, Git on
GitHub19 was used, but at that time GitHub did not fulfil the requirements of the
project. To reduce the complexity, the source code was migrated to Subversion.20
At the end of the project, the source code was migrated again to GitHub,21 since
GitHub has a more flexible scheme where any external developer can reuse the
source code without any intervention from the repository owner. Whereas
17
Git, http://git-scm.com/
18
Subversion, http://subversion.tigris.org/
19
GitHub, https://github.com/
20
The Subversion repository is available on Sourceforge at http://sourceforge.net/projects/role-
project/
21
The ROLE GitHub repository is available at https://github.com/organizations/ROLE
Sourceforge still requires management by a ROLE partner. To manage the projects,

milestones and bug and issue tracking, Atlassian JIRA22 was used. Tasks, feature
requests and issues were collected in JIRA and assigned to projects and milestones.
Open tasks and issues were discussed in the technical meetings. Overall, our JIRA
experience was quite positive, as it enabled a quick overview of the progress and
future work for developers and managers. Clearly, to have a consistent and up-to-
date overview, developers have to be committed to report their work in JIRA.
Developer camps: During the project period, three developer camps were
organised. Originally, the developer camps were meant for internal developers to
discuss the overall ROLE architecture and technical solutions, and plan the pro-
jects. During the first developer camp, a shared vision of the ROLE objectives was
created. At the second and third developer camp, external experts were invited to
provide feedback on the architecture, identify missing use cases and requirements
and provide a broader scope on recent research results that could be applied in
ROLE. At the third developer camp (November 2011), we invited a larger group of
experts, presented the current status of the ROLE framework and had a small
developer competition to develop widgets for the platform. This was only possible
at this time, because the implementation was mature enough. This developer
competition was good both for dissemination to research and open-source projects
(e.g. Apache RAVE), and for getting feedback from external developers on the
ROLE APIs and documentation. Later we organised four more widget competitions
that were open to the public. In general, the developer camps were a good platform
to collaborate with the whole ROLE technical team and external experts for a
couple of days.
Contributing ROLE Software to Open-Source Projects
Several components and specifications of the ROLE framework were integrated in

other open-source projects. This strategy enables further uptake and development of
the research results of the ROLE project. This section highlights some of the
contributions to the open-source community.
OpenSocial and Apache Shindig: In order to standardise the OpenSocial space
extension (see section ‘Spaces’), EPFL worked with the OpenSocial community for
the specification and with the Apache Shindig community for the reference imple-
mentation of the specification. The communication with both communities happens
through mailing lists. After our specification proposal was presented on the mailing
list, it received very positive feedback and representatives of several companies
showed interest in the extension for use in their products. After several discussions
and refinements of the proposal, the work on the specification draft started. Typi-
cally, the procedure to get a proposal accepted is as follows: First, a patch to the
22
Atlassian JIRA, http://www.atlassian.com/software/jira
OpenSocial specification has to be written. Second, the proposal has to be

implemented in an open source, publicly available platform, e.g. Apache Shindig.
Finally, when the proposal is finalised, the community votes on the final inclusion
of the draft into the specification. Consequently, ROLE wrote a patch for the
OpenSocial specification and extended Apache Shindig, which was shared with
the OpenSocial community.
When we started our proposal, the OpenSocial community was finalising
OpenSocial version 2.0. Thus, initially our proposal would be incorporated in the
next version, 2.5. However, later the decision of the community was to have only
limited changes in 2.5 and leave all larger revisions for the upcoming version 3.0.
Hence, due to the large changes that our proposal would cause, it was decided to
postpone its inclusion and it was only incubated in OpenSocial 2.5. Because of
other changes in the specification of OpenSocial 3.0, our proposal had to be
adapted. Eventually, the process that seemed open and efficient turned out to be
quite time-consuming. Currently (December, 2013), the proposal is still on the road
map for the OpenSocial 3.0. The patch for the specification is ready and the code for
Apache Shindig is available. Once the work on the OpenSocial 3.0 is started, the
patch should be evaluated and voted upon final inclusion into the newest version of
the specification.
However, adding a proposal into the specification does not immediately guar-
antee that it can be used in all OpenSocial platforms. To be able to use the space
proposal in widgets and to enable interoperability with other OpenSocial platforms,
all platforms have to implement the latest version of the specification. There can be
latency, since it takes time to upgrade to newer versions of OpenSocial.
Apache Rave: As mentioned in the introduction section, Apache RAVE is an
open-source mash-up platform with similar functionality as the ROLE framework.
Therefore, it was a very interesting project to contribute to. Technical ROLE
partners have joined two Apache RAVE Hackathons in the Netherlands to present
our work and discuss collaboration. The RAVE community received our concepts
and implementation enthusiastically. Their main interest was in our IWC compo-
nent, the space concept and our Linked Data-based APIs to retrieve and store data in
the PLE. As the space specification proposal was already submitted to the
OpenSocial community, ROLE decided to propose the two other components to
the RAVE community. The process to achieve this is quite similar to the
OpenSocial procedure. One has to announce the idea on the public mailing list of
Apache Rave, where the idea and its specification can be openly discussed. The
next phase is to provide an implementation of the component in RAVE and submit a
patch. This patch will be reviewed by the community and after acceptance can be
included in upcoming milestones. At the time of writing, both proposals have not
yet been accepted. We hope to get approval of the RAVE community in the near
future.
Strophe.js: The parts of the IWC component have been contributed to the open-
source JavaScript XMPP library, named Strophe.js.23 We mainly contributed our
implementation of the XMPP protocol over WebSockets.24 This makes the library
more efficient as data can be efficiently pushed from server to client and long
polling is no longer necessary.
Discussion and Conclusion
This chapter presented the architecture of the ROLE framework and the platforms
where this framework has been integrated. The ROLE framework provides several
components to enable responsive open PLEs, such as IWC, automated user activity
tracking, collaborative spaces and authentication and authorisation services to
protect data. These components provide the basis for real-time communication
between widgets and users and automatic user activity tracking from tools and
services. To evaluate the usability and usefulness of the ROLE philosophy and the
ROLE framework, we have integrated the ROLE framework in various platforms,
such as Moodle, CLIX (Govaerts et al. 2011; Rensing et al. 2013), Graasp and the
ROLE SDK. These platforms have been used in various real-world evaluation
settings (Govaerts et al. 2011), which have been documented in Chaps. 4, 5, 6
and 7.
In general, we can conclude that with the ROLE framework we were able to
meet the project requirements and support the test beds. The birth of the Apache
RAVE project with very similar goals indicates the interest and usefulness of the
ROLE philosophy. Furthermore, the framework produced several components that
were of interest to other open-source projects. Some of these open-source contri-
butions have been completed, while others are still in progress. Additionally, results
of the ROLE framework will be reused and extended in other research projects. For
instance, the ROLE Widget Store will be reused in the Go-Lab project25 as a
repository of apps and online laboratories to enable teachers to assemble learning
environments with online laboratories for inquiry-based learning. Additionally,
Go-Lab will also use Graasp and the OpenSocial Spaces specification to enable
inquiry-based learning spaces for STEM education at school. On the other hand,
researchers of the Learning Layers project26 are using and extending the ROLE
SDK as their learning platform (Kovachev et al. 2013). As mentioned, the ROLE
SDK is mainly meant for developers to extend their existing learning environments
23
Strophe.js, http://strophe.im/strophejs/
24
WebSocket, http://www.websocket.org/
25
Go-Lab, http://www.go-lab-project.eu/
26
Learning Layers, http://learning-layers.eu/
or extend the ROLE SDK itself to support their requirements. To support this,
developers can easily contribute or fork the ROLE SDK GitHub repository.27 We
hope that in this way large parts of our efforts will be used beyond the end of the
ROLE project.
Acknowledgments This research is funded by the European Commission’s Seventh Framework

Programme (FP7/2007-2013) under grant agreement no 231396 (ROLE). Katrien Verbert is a
Postdoctoral Fellow of the Research Foundation—Flanders (FWO).
References
Ashley H, Corbett J, Jones D, Garside B, Rambaldi G. Change at hand: Web 2.0 for development.
Participatory Learning and Action. 2009;59:8–20(13).
Birman KP, Joseph TA. Exploiting virtual synchrony in distributed systems. In: ‘SOSP’87:
Proc. of the 11th ACM Symp. on Operating Systems Principles; 1987. p. 123–38.
Bogdanov E, Salzmann C, Gillet D. Contextual spaces with functional skins as OpenSocial
extension. In: 4th International conference on advances in computer-human interactions;
2011. p. 158–63.
Bogdanov E, Limpens F, Li N, El Helou S, Salzmann C, Gillet D. A social media platform in
higher education. In: IEEE Global Engineering Education Conference (EDUCON); 2012.
Bogdanov E, Ullrich C, Isaksson E, Palmr M, Gillet D. From LMS to PLE: a step forward through
OpenSocial apps in Moodle. In: The 11th International Conference on Web-based Learning
ICWL; 2012.
Butoianu V, Vidal P, Verbert K, Duval E, Broisin J. User context and personalized learning: a
federation of contextualized attention metadata. J Univers Comput Sci. 2010;16(16):
2252–2271. https://lirias.kuleuven.be/handle/123456789/289802
Caceres M. Widget packaging and configuration, W3C working draft 22 March; 2011 (n.d.). http://
www.w3.org/TR/widgets/
Chudnovskyy O, Nestler T, Gaedke M, Daniel F,Fernández-Villamor JI, Chepegin V, Fornas JA,
Wilson S, Kögler C, Chang H. End-user-oriented telco mashups: the omelette approach. In:
Proceedings of the 21st international conference companion on World Wide Web, WWW’12
Companion. New York: ACM; 2012. p. 235–38. http://doi.acm.org/10.1145/2187980.2188017
Dalsgaard C. Social software: E-learning beyond learning management systems. Eur J Open
Distance E-learn (n.d.). 2006(2).
David P-C, Ledoux T. Wildcat: a generic framework for context-aware applications. In: Pro-
ceedings of the 3rd international workshop on Middleware for pervasive and ad-hoc comput-
ing, MPAC’05, ACM, New York; 2005. p. 1–7. http://doi.acm.org/10.1145/1101480.1101483
DCMI Usage Board. DCMI metadata terms, DCMI recommendation. Dublin Core Metadata
Initiative; 2006. http://dublincore.org/documents/2006/12/18/dcmi-terms/
Dragunov AN, Dietterich TG, Johnsrude K, McLaughlin MR, Li L, Her-locker JL. Tasktracer: a
desktop environment to support multi-tasking knowledge workers. In: IUI; 2005. p. 75–82.
27
The ROLE GitHub repository is available at https://github.com/organizations/ROLE
Ebner M, Taraghi B. Personal learning environment for higher education a first prototype. In:
World conference on educational multimedia, hypermedia and telecommunications 2010.
Chesapeake: AACE; 2010. p. 1158–66.
Eugster PT, Felber PA, Guerraroui R, Kermarrec A-M. The many faces of publish/subscribe.
ACM Comput Surv. 2003;35(2):114–31.
Friedrich M, Wolpers M, Shen R, Ullrich C, Klamma R, Renzel D, et al. Early results of
experiments with responsive open learning environments. J Univers Comput Sci. 2011;
17(3):451–71.
learning in a responsive open learning environment In: Proc. of the International Conference
eLearning Baltics Science (eLBa Science 2010), Rostock, Germany; 2010.
Gillet D, Bogdanov E. Personal learning environments and embedded contextual spaces as
aggregator of cloud resources. In: Proceedings of the 1st International workshop on cloud
education environments (WCLOUD 2012); 2012.
Glahn C. Using the adl experience api for mobile learning, sensing, informing, encouraging,
orchestrating. In: 2013 Seventh international conference on Next generation mobile apps,
services and technologies (NGMAST); 2013. p. 268–73.
Govaerts S, Verbert K, Dahrendorf D, Ullrich C, Schmidt M, Werkle M, Chatterjee A,
Nussbaumer A, Renzel D, Scheffel M, Friedrich M, Santos Odriozola JL, Duval E, Law
EL-C. Towards responsive open learning environments: the ROLE Interoperability frame-
work. In: Delgado Kloos C, Gillet D, Crespo Garcia RM, Wild F, Wolpers M (eds) Towards
ubiquitous learning—Proceedings of 6th European conference of technology enhanced learn-
ing, EC-TEL 2011. Berlin: Springer; 2011. p. 125–38. https://lirias.kuleuven.be/handle/
123456789/319048
Guo Y, Rui J, Zhou H. Pervasive and personal learning environment using service-oriented
architecture: a framework design. In: International conference on networking and distributed
computing; 2010. p. 153–5.
Health T, Bizer C. Linked data: evolving the web into a global data space (book) (n.d.). http://
linkeddatabook.com/editions/1.0/
Henricksen K, Indulska J. Developing context-aware pervasive computing applications: models
and approach. Pervas Mob Comput 2006;2(1), 37–64. http://dx.doi.org/10.1016/j.pmcj.2005.
07.003
Hickson I. HTML5 web messaging. Working draft, W3C; 2011.
Hickson I. The web sockets API, Technical report, W3C. W3C Working Draft; 2009.
Isaksson E, Palmer M. OpenApplication specification (n.d.). https://docs.google.com/document/d/
1VReR5Aod7HyIAhBXRB_u9mY3vKdUHR9uXk7_w6KBGX0/preview
Isaksson E, Palmer M. Usability and inter-widget communication in PLEs. In: Proceedings of the
3rd Workshop on Mashup Personal Learning Environments; 2010.
Kovachev D, Renzel D, Nicolaescu P, Klamma R. Direwolf—distributing and migrating user
interfaces for widget-based web applications. In: Daniel F, Dolog P, Li Q (eds) Web engi-
neering, vol. 7977, Lecture notes in computer science. Berlin: Springer; 2013. p. 99–113.
http://dx.doi.org/10.1007/978-3-642-39200-9_10
Ladas C. Scrumban—essays on Kanban Systems for Lean Software development. Seattle: Modus
Cooperandi Press; 2009.
Maness JM. Library 2.0 theory: Web 2.0 and its implications for libraries. Webology. 2006;3(2).
Millard P, Saint-Andre P, Meijer R. XEP-0060: publish-subscribe. Technical report. XMPP
Standards Foundation; 2010. Draft Standard. http://xmpp.org/extensions/xep-0060.html
Mitchell-Wong J, Kowalczyk R, Roshelova A, Joy B, Tsai H. Opensocial: from social networks to
social ecosystem. In: Digital EcoSystems and Technologies conference, 2007. DEST‘07.
Inaugural IEEE-IES. p. 361–6.
Moffit J, Cestari E. An XMPP sub-protocol for WebSocket. Technical report, Internet engineering
taskforce; 2010. Draft Standard. http://tools.ietf.org/html/draft-moffitt-xmpp-over-websocket-
00
Nussbaumer A, Berthold M, Dahrendorf D, Schmitz H-C, Kravcik M, Albert D. A mashup

recommender for creating personal learning environments. In: ICWL; 2012. p. 79–88.
Oliver N, Smith G, Thakkar C, Surendran AC. Swish: semantic analysis of window titles and
switching history. In: Proceedings of the 11th international conference on intelligent user
interfaces, IUI‘06, ACM, New York; 2006. p. 194–201. http://doi.acm.org/10.1145/1111449.
1111492
Paterson I, Smith D, Saint-Andre P, Moffitt J. XEP-0124: bidirectional-streams over synchronous
HTTP (BOSH), Technical report, XMPP Standards Foundation; 2010. http://xmpp.org/exten
sions/xep-0124.html
Pierce ME, Singh R, Guo Z, Marru S, Rattadilok P, Goyal A. Open community development for
science gateways with apache rave. In: Proceedings of the 2011 ACM workshop on gateway
computing environments, GCE’11, ACM, New York; 2011. p. 29–36. http://doi.acm.org/10.
1145/2110486.2110491
Rath AS, Devaurs D, Lindstaedt SN. Uico: an ontology-based user interaction context model for
automatic task detection on the computer desktop. In: Proceedings of the 1st Workshop on
context, information and ontologies’, CIAO‘09, ACM, New York; 2009. p. 8:1–8:10. http://
doi.acm.org/10.1145/1552262.1552270
Rensing C, Schwantzer S, Faltin N. Integration selbstgesteuerten ressourcen-basierten lernens in
eine durch instruktion geprägte lernumgebung. In: Andreas Breiter CR, Meier M
(ed) Proceedings der Pre-Conference Workshops der 11. e-Learning Fachtagung Informatik-
DeLFI 2013. Berlin: Logos. p. 69–70. ftp://ftp.kom.tu-darmstadt.de/papers/RSF13-1.pdf
Renzel D, Behrendt M, Klamma R, Jarke M. Requirements bazaar: Social requirements engineer-
ing for community-driven innovation. In: Proceedings of the 21st International requirements
engineering conference; 2013. p. 326–7.
Saint-Andre P. RFC 3920—Extensible Messaging and Presence Protocol (XMPP): Core, Technical
report; 2004a. Jabber Software Foundation. http://www.ietf.org/rfc/rfc3920.txt
Saint-Andre P. RFC 3921—Extensible Messaging and Presence Protocol (XMPP): instant mes-
saging and presence, Technical report; 2004b. Jabber Software Foundation. http://www.ietf.
org/rfc/rfc3921.txt
Saint-Andre P. XEP-0045: Multi-User Chat, Technical report. XMPP Standards Foundation; 2008.
Draft Standard. http://xmpp.org/extensions/xep-0045.html
Schmitz H-C, Kirschenmann U, Niemann K, Wolpers M. Contextualized attention metadata. In:
Roda C, editor. Human attention in digital environments. Cambridge: Cambridge University
Press; 2011. p. 186–209.
Schwaber K. Agile project management with scrum, best practices; 2004. Redmond: Microsoft
Press. http://books.google.be/books?id¼dJlqJfm8FM4C
Sire S, Paquier M, Vagner A, Bogaerts J. A messaging API for inter-widgets communication. In:
Proceedings of the 18th International Conference on World Wide Web, WWW’09. New York:
ACM; 2009. p. 1115–6.
Ullrich C, Borau K, Luo H, Tan X, Shen L, Shen R. Why Web 2.0 is good for learning and for
research: principles and prototypes. In: Proceedings of the 17th International world wide web
conference. New York: ACM; 2008. p. 705–14.
Ullrich C, Shen R, Borau K. Learning from learning objects and their repositories to create
sustainable educational app environments. In: IEEE International Conference on Advanced
Learning Technologies. Los Alamitos: IEEE Computer Society; 2013.
Vozniuk A, Govaerts S, Gillet D. Towards portable learning analytics dashboards. In: International
Conference on Advanced Learning Technologies; 2013. Los Alamitos: IEEE
Wilson S, Sharples P, Popat K, Griffiths D. Moodle wave: reinventing the VLE using widget
technologies. In: Proceedings of 2nd workshop mash-up personal learning environments
(MUPPLE’09). Workshop in conj. with 4th European Conference on Technology Enhanced
Learning (EC-TEL 2009): Synergy of disciplines; 2009. Berlin: Springer, p. 47–58.
Wolpers M, Martin G, Najjar J, Duval E. Attention metadata in knowledge and learning manage-
ment. In: Proceedings of I-Know 2006: 6th International Conference on Knowledge Manage-
ment; 2006. p. 1–8.
Wolpers M, Najjar J, Verbert K, Duval E. Tracking actual usage: the attention metadata approach.
Educ Technol Soc. 2007;3(10):106–21.
Yuan J. Liferay Portal 5. 2 Systems development, from technologies to solutions; 2009. Birmingham:
Packt Publishing. http://books.google.be/books?id¼OaR3JEZJz5EC
Commentary
The following sections offer the comments of experts outside of the ROLE project
consortium about the contents of this book. Each expert was asked to review and
comment upon a chapter of this book that is relevant to their expertise, thus offering
their feedback about a certain aspect of the ROLE research outcomes.
Personal Learning Environments, Self-Directed Learning

and Context
Graham Attwell
Research and development in learning technologies is a fast moving field. Ideas and
trends emerge, peak and die away as attention moves to the latest new thing. At the
time of writing MOOCs dominate the discourse. Yet the developments around
Personal Learning Environments (PLEs) have not gone away. It could be argued
that the development and adoption of PLEs is not so much driven the educational
technology community but by the way people (and not just students) are using
technology for learning in their everyday lives.
Even when Learning Management Systems were in their prime, there was
evidence of serious issues in their use. Teachers tended to use such environments
as an extended file storage system; forums and discussion spaces were frequently
under populated. In other words such systems were used for managing learning,
rather than for learning itself. Learners expropriated and adapted consumer and
productivity applications for their learning. Such trends became more pronounced
with the emergence of Web 2.0 and social software. Social networking applications
in particular, allowed the development of personal learning networks. Rather than
go to the institutionally sanctioned LMS or VLE, learners communicated through
G. Attwell (*)
Pontydysgu Ltd, Pontypridd, UK

DOI 10.1007/978-3-319-02399-1_9
220 Commentary
Facebook or Whats App. PLNs were not longer limited to class or course cohorts
but encompassed wider social and learning networks. Wikipedia has emerged as a
major open resource for learning.
As mobile technologies have become increasingly powerful and, at least in some
countries, internet access has become increasingly ubiquitous, learners use their
own devices for learning and are not confined to institutional facilities. Regardless
of trends in educational technology theory and research, learners are developing
and using their own PLEs.
At the same time, the ongoing rapid developments in technologies are changing
forms of knowledge development and leading to pressures for lifelong learning.
Universities and educational institutions can no longer preserve a monopoly on
knowledge. Notwithstanding their continuing hold on accreditation, institutions are
no longer the only providers of learning, a move seen in the heart-searching by
universities as to their mission and role.
Such changes are reflected in the growing movement towards open learning, be
it in the form of MOOCs or in the increasing availability of Open Educational
Resources. The popularity of MOOCs has revealed a vast pent up demand for
learning and at least in the form of the c-MOOCs has speeded the adoption of PLEs.
MOOCs are in their infancy and we can expect the rapid emergence of other forms
of open learning or open education in the next few years.
Learning is becoming multi-episodic, with people moving in and out of courses
and programmes. More importantly the forms and sources of learning are increas-
ingly varied with people combining participation in face-to-face courses, online and
blended learning programmes and self-directed and peer supported learning using
different Internet technologies.
These changes are reflected in discussion over pedagogy and digital literacies. It
is no longer enough to be computer literate. Learners need to be able to direct and
manage their own learning, formal and informal, regardless of form and source. In
conjunction with More Knowledge Others (Vygotsky 1978) they need to scaffold
their own learning and to develop a personal knowledge base. At the same time as
the dominance of official accreditation wanes, they need to be able to record and
present their learning achievement. PLEs are merely tools to allow this to happen.
All this leads to the issue of the role of educational technology researchers and
developers. In research terms we need to understand more not just about how
people use technology or learning but how they construct a personal knowledge
base, how they access different resources for learning, including people and how
knowledge is exchanged and developed.
At a development level, there is little point in trying to develop a new PLE to
replace the VLE. Instead we need to provide flexible tools, which can enhance
existing technologies and learning provision, be it formal courses and curricula or
informal learning in the workplace or in the community. It can be argued that while
most educational technology development has focused on supporting learners
already engaged in educational programmes and institutions, the major potential
of technology and particularly of PLEs is for the majority of people not enrolled on
formal educational programmes. Not all workplaces or for that matter communities
Commentary 221
offer a rich environment or learning. Yet there is vast untapped potential in such
environments, particularly for the development and sharing of the tacit knowledge
and work process knowledge required in many tasks and occupations. PLE tools
can help people learning in formal and informal contexts, scaffold their learning
and develop a personal learning knowledge base or portfolio.
At both pedagogic and technical levels, context provides a major challenge.
While mobile technologies recognize the context of place (through GPS), other and
perhaps more important aspects of context are less well supported. This includes
time—how is what I learned at one time linked to something I learned later? It
includes purpose—why am I trying to learn something? It includes the physical
environment around me, including people. And of course it includes the social and
semantic links between places, environments, people and objects.
The challenge is to develop flexible applications and tools to enhance peoples’
PLEs and which can recognize context, can support people in scaffolding their
learning and develop their own Personal Learning Networks and enhance their
ability to direct their own learning and the learning of their peers.
Two major European funded projects, ROLE and Learning Layers are
attempting to develop such applications. They both have the potential to make
major inroads into the challenges outlined in this short chapter.
Reference
Vygotsky LS. Mind in society: the development of higher psychological processes.

Cambridge: Harvard University Press; 1978.
Supporting Self-Regulated Learning
Margit Pohl
Current educational theories emphasise the importance of autonomous learning.

Self-regulated learning is one example for such a theory. In the context of this
theory, metacognition and cognitive strategies play a significant role. One of the
goals of the ROLE project was to support metacognition and reflection of learners
specifically. Chapter 2 on “Supporting Self-Regulated Learning” describes the
basic ideas of this approach and its implementation in the project.
M. Pohl (*)
Institute for Design and Technology Assessment, Vienna University of Technology, Vienna,
Austria
222 Commentary
One relevant issue in this context is the question how much support learners need
and how to give appropriate feedback to them. It is well known that autonomous
learning often overwhelms learners and increases drop-out. Still, there are individual
differences related to the ability to learn autonomously. Students with high
metacognitive skills and self-efficacy are better able to plan their learning processes
and learn more efficiently. The ability to structure one’s own learning and to reflect on
the issues raised in the learning material apparently does not come naturally and has to
be communicated to the students. In contrast to some constructivist approaches, self-
regulated learning takes these issues into consideration. In the context of the ROLE
project, specific assistance is given to the learners to increase metacognition and
reflection. There is an adaptive mechanism in the support strategies which adjusts
the learning material provided to the students to their specific needs. This mechanism
also takes care of the fact that some students need more scaffolding than others.
The support process is based on an extremely sophisticated framework
consisting of a process model, a competence model and a learner model. This
framework enables the system to give highly differentiated feedback to the learners
without having to resort to AI methods. The framework enables the researcher to
come up with relevant guidelines for the development and adoption of learning
resources. In my opinion, the fact that the framework used as a basis for the
development process is strongly related to the guidelines is an indication of the
value of the didactic approach used in this project. In many e-learning projects the
relationship between the underlying theory and the actual design is only very loose
which results in a certain arbitrariness of the design.
One of the strengths of the approach adopted in the ROLE project is that the authors
also clarify challenges and limitations of their work. They conducted a survey with
teachers, and they collected data at summerschools and conference workshops. These
data indicate that the approach has advantages and limitations. Teachers described that
advantages might be better learning from the students, more autonomy for the students
and peer collaboration. They also see problems as, for example, the fact that many
students are not equipped for self-regulated learning and reluctant to accept new
methods of teaching. There are also barriers because of the way how universities or
other educational institutions are organized. These problems have also been described
in the literature (Laurillard 1993). The character of assessments at universities, for
example, does not encourage self-regulated learning or reflection or collaboration. In
addition, metacognitive skills are often not taught in schools or universities. The
development of curricula for schools and universities is usually a highly contested
area, and many different stakeholders try to influence this process. The introduction of
more autonomous and self-regulated learning is, therefore, quite a challenging pro-
cess. Projects like ROLE can play an important role in this context to present an
exemplary realization of self-regulated learning.
I think there are many interesting areas for future work posed by this project.
Although some evaluations have already been conducted, a more detailed study of
student’s interaction with the system would be very interesting. There is a pro-
nounced emphasis on meta-cognitive activities of the students. Students have to tag
widgets or formulate their learning goals. I think it is an interesting research
Commentary 223
question how students adopt these activities. Nowadays, students are not rewarded
for this kind of meta-cognitive activities at universities. Therefore, they might see it
as an obstacle to get a certificate. I assume that meta-cognitive activities have to be
integrated into courses and be rewarded in the same sense as other kinds of learning
activities, but these are open questions which have to be investigated.
In general, I think it would be interesting to investigate how students interact with
this system. Choosing learning widgets and integrating their contents to form a
coherent mental model is certainly a demanding task for the students. It would be
very interesting to know how students cope with this task and what can be learned for
the design of similar systems. I think that the approach using widgets which can be
reused and combined flexibly is very promising, but it is also challenging because it is
unusual and forces the students to reflect about their learning processes even if there
are only few widgets to choose from. There are two aspects which I think would be
relevant in this context. On the one hand, there is the investigation of the interactions
and learning processes of the students. On the other hand, it would also be interesting
to find out what kind of design can support students best in such systems. The first
question is more didactic, whereas the second question also addresses usability issues.
Self-regulated learning is an interesting approach because it combines a more
active role of the learner with fairly rigorous learning strategies. Such learning
strategies can be an advantage if supported appropriately. The ROLE project is an
important step to implement a system to encourage a good balance between
freedom and guidance in the learning process.
Reference
Laurillard D. Rethinking university teaching. London: Routledge; 1993.
Multidimensional Evaluation Framework for PLE: Does It

Make Sense and Do We Need It?
Carlo Giovannella
PLEs are a typical expression of our time, a time dominated by the liquidity, that
from one side is a symptom of a profound crisis of values (Bauman 2000) while on
the other may represent a great opportunity (Giovannella 2009), provided you are
equipped with the skills needed to manage complexity. PLEs are virtual environ-
ments in continuous evolution, potentially no-places (Augé 1992) without memory,
C. Giovannella (*)
University of Rome Tor Vergata, Rome, Italy
224 Commentary
containing the promise for highly customizable environment and learning processes
as amply illustrated by this dedicated publication.
PLEs, thus, are not suitable for most of the today’s learning processes and their
actors. Certainly not for present teachers, who do not “shine” for the mastery of an
adequate digital literacy and who, in the vast majority, still consider virtual environments
useful as content repository or message boards. Neither for most of the students that,
although belonging to the so called digital native (Prensky 2001) and showing a
considerable ability/independence in managing interpersonal communication, when
are asked to take the responsibility of their own training path step back and, actually,
prefer to be hetero-directed and evaluated by teachers. PLEs, thus, are not for today, but
represent an interesting laboratory within which one can experiment around the central-
ity of the person and her ability to design her own learning trajectory according to design
based learning approach, her ability to acquire suitable design literacy (Giovannella
2010) and other twenty-first century skills (Giovannella and Baraniello 2013).
A smooth introduction of widgets usage into more “traditional” learning envi-
ronments would be, thus, largely advisable to foster the transition toward more self-
regulated learning paths.
Considering the present conditions the organizational level of the evaluation,
although should be considered to design a general framework, is too far away with
respect to the nowadays urgencies. Since in PLEs the PERSON and her learning
EXPERIENCE is expected to be at the centre of learning process, the evaluation
should focus mainly on interplay and co-evolution of the “characteristics” of both
people and techno-ecosystem.
As well explained by the authors of Chap. 3 one should go well beyond the
standard HCI prescriptions to embrace the whole multidimensional spectrum of the
human experience mediated by the machine. Of course one needs to develop a better
understanding of the learning experience, develop meaningful models (Giovannella
et al. 2011) and try to make these latter as robust as possible. New evaluation methods,
thus, should be developed and integrated within (or made more easily accessible from)
PLEs, and more in general all kinds of learning environments.
The goal should be the multidimensional evaluation of the EXPERIENCE and,
of course, of:
The learner ability to design her learning process (not just to follow the proposed
one).
The acquisition of the relevant competences and literacies and among them the
ability to interpret the analytics and self-evaluate her own evolution and needs.
Accordingly the evaluation and redefinition of PLEs usage has to capitalize on
the large and well established methodological corpus that have been developed
in the past 20–25 years within many disciplinary domains: anthropology, psy-
chology, sociology, computer science, interaction design, design for the experi-
ence, design, etc., and that has been well synthesized in Chap. 3 of this book. A
corpus that can be even enlarged to consider many other methods (the descrip-
tion of which can be easily found on the web) and that should also be integrated
by new approaches and methodologies suitable for the multidimensional
Commentary 225
monitor of the learning experience (Giovannella et al. 2011, 2013). A task, this
latter, that has been accomplished also by some partners of the ROLE project.
The debate on qualitative vs. quantitative methods and subjective vs. objective
data detection can be considered an ill posed one. No one would renounce to
more objective data, collected in an unobtrusive and respectful of privacy
manner, no one would renounce to push the border from qualitative toward
quantitative data detection, when possible. The debate, thus, has better to
concentrate on the quality of data (i.e. “smart data” instead of “big data”) and
on the ability to interpret them.
As an example, an apparently highly objective detection method like the
eye-tracking when not well controlled may produce unreliable results if indi-
vidual visualization styles are not dutifully taken into account.
As additional example, emotions and sentiment, apart the need of well-grounded
and interoperable models, could be both objectively and subjectively detected,
but the choice of the approach strongly depends on the time-window of interest
and cannot avoid to consider both emitters and detectors, whatever communi-
cation modality (voice, text, images, etc.) and medium are involved.
In conclusion, PLEs are learning labs challenging all actors of learning pro-
cesses and researchers in many respects, including a person/people in place
multidimensional monitoring to detect the acquisition of meta-design literacy,
self-direction and self-evaluation skills.
References
Augé M. Non-lieux. Introduction à une anthropologie de la surmodernité. Paris: Le

Seuil; 1992.
Bauman Z. Liquid Modernity Cambridge: Polity; 2000.
Giovannella C. DULP: Complexity, organicity, liquidity, IxD&A J. 2009;7–8:
11–15.
Giovannella C. Beyond the media literacy. complex scenarios and new literacies for
the future education: the centrality of design. IJDLDC. 2010;3:18–28.
Giovannella C, Carcone S, Camusi A. What and how to monitor complex educative
experiences. Toward the definition of a general framework, IxD&A J. 2011;
11–12:7–23.
Giovannella C, Baraniello V. Smart city learning. IJDLDC 2013;3(4):1–15.
Giovannella C, Popescu E, Scaccia F. A PCA study of student performance
indicators in a Web 2.0-based learning environment, ICALT 2013, IEEE;
2013. p. 33–35.
Prensky M. Digital natives, digital immigrants on the horizon. Washington, DC:
MCB University Press; 2001.
226 Commentary
PLE in Formal Education: Challenges for Openness and

Control
Marco Kalz
Abstract This short comment reflects on a critical account of educational tech-

nology and makes reference to the chapter by Vieritz et al. about the use of widget
bundles for formal learning in higher education.
Introduction
Personal Learning Environments (PLEs) have been intensively discussed since the
introduction of the concept without an agreement about their definition and con-
crete focus. In its early development phase PLE has been introduced as learning
technologies under the control of the learner (van Harmelen 2006). Later we have
described the PLE as a learning environment in which learners on the one hand
actively integrate distributed digital information, resources and contacts, on the
other hand document learning progress and learning outcomes based on standards
(Schaffert and Kalz 2008). While the original concept of the PLE has been
introduced as a counter-concept for teacher/instructor-prepared learning environ-
ments like Learning Management Systems (LMS) nowadays this perception of a
PLE seems to have moved into a direction in which all technology that enlarges the
landscape of standard learning technology can be regarded as a PLE.
The authors of the chapter have presented three case studies of widget bundles
that function as an enrichment of the traditional technology-supported learning
environments at these three institutions. These implementations provide interesting
directions for a transition between learning technologies that are designed
according to fixed curricula and prepared content towards more flexible environ-
ments. Especially the activity recommender might offer an interesting direction to
support self-organized learning. But flexibility alone is not the core of a PLE.
Selwyn calls for a critical account of educational technology that takes into
account the societal intertwining of educational technology on the micro-and
macro-level and the study of learning technology in dimensions of “power, control,
conflict and resistance” (Selwyn 2010). We cannot disconnect this wider discussion
and reflection from the implementation level. In this sense, learners need to be able
to actively (co-)design their learning environment to make it a personal one. This is
the important difference between adaptivity and adaptability of a learning environ-
ment (Oppermann and Rasher 1997). While adaptivity can be designed completely
according to rules of teachers or the designer of a piece of learning technology, the
M. Kalz (*)
Open University of the Netherlands, P.O. Box 2960, 6401 DL Heerlen, The Netherlands
Commentary 227
adaptability enables a learner to design the learning environment according to

individual needs. In the context of educational institutions and formal learning
this leads to a number of challenges.
The authors have argued that pre-designed widget bundles have been used to not
confuse users and provide them with too many choices. But this leads to the
contradiction that widget bundles are a result of a design process of teachers
without giving learners any influence on their technology-enhanced learning envi-
ronment. We have described this contradiction as a “competence continuum”
consisting of a number of core skills to be able to use a PLE effectively for self-
directed learning (Wild et al. 2009). The biggest challenge is to come to a setup that
also enables learners without a high level of self-directedness and IT skills to slowly
get used to a more open and flexible learning environment. Pre-defined spaces that
can slowly be extended are one option for this issue, the other option would be to
make available a limited number of widgets that users try first and then decide about
their use and usefulness.
And this leads to a related challenge: Since PLE are dynamic environments that
grow according to the context and needs of the learner their evaluation needs to take
into account a temporal perspective consisting of a number of snapshots of the
environment and their impact on enabling self-directed learning processes. It is
essential for the further development of PLE and their impact in education that the
community develops evaluation frameworks that can systematically handle the
complexity of evaluating a personal environment that changes its status dynami-
cally over time and can thus fulfill different purposes.
One possible theoretical framework for developing such an evaluation approach
is the adaptive structuration theory: „The act of bringing the rules and resources
from an advanced information technology or other structural source into action is
termed structuration. Structuration is the process by which social structures (what-
ever their sources) are produced and reproduced in social life” (DeSanctis and
Poole 1994). Thus can this theory build a good foundation to analyse the interre-
lation between social structures and technological structures developed in a PLE
and the dimensions pinpointed by Selwyn.
References
DeSanctis, Poole MS. Capturing the complexity in advanced technology use:

Adaptive structuration theory. Organization Sci. 1994;5(2):121–47.
Oppermann, Rasher R. Adaptability and adaptivity in learning systems. Knowl
Transf. 1997;2:173–9.
Schaffert S, Kalz M. Persönliche Lernumgebungen: Grundlagen, Möglichkeiten
und Herausforderungen eines neuen Konzeptes. In: Wilbers K, Hohenstein A,
editors. Handbuch E-learning. Deutscher Wirtschaftsdienst (Wolters Kluwer
Deutschland), Cologne; 2008. p. 1–24.
228 Commentary
Selwyn N. Looking beyond learning: notes towards the critical study of educational
technology. J. Comput Assist Learn. 2010;26, 65–73.
van Harmelen M. Personal Learning Environments. In: Kinshuk R, Koper P,
Kommers P, Kirschner P, Sampson DG, Didderen W, editors. Sixth IEEE
international conference on advanced learning technologies (ICALT’06). IEEE
Computer Society, Kerkrade; 2006. p. 815–6.
Wild J, Wild F, Kalz M, Specht M, Hofer M. The MUPPLE competence contin-
uum. In: Wild F, Kalz M, Palmér M, Müller D, editors. Proceedings of 2nd
workshop mash-up personal learning Environments (MUPPLE’09). Workshop
in conjunction with 4th European conference on technology enhanced learning
(EC-TEL 2009): Synergy of disciplines, Sept, 29, 2009, Nice, France: CEUR
workshop proceedings; 2009. p. 80–8. http://sunsite.informatik.rwth-aachen.de/
Publications/CEUR-WS/Vol-506
The Future of PLEs: How Can Higher Education Be Passed?
Martin Ebner
Woolly Thoughts on PLEs for Higher Education
I just read Chap. 5 on “Case study 2: Designing PLE for Higher Education” and
would like to sort my thoughts. On the one side the ROLE (Gillet et al. 2010) as
well as the Go-Lab project (Gillet et al. 2010) took us a step forward to see how the
future of teaching and learning might look like, on the other side we ourselves at
Graz University of Technology also gathered experiences how a PLE is used in
Higher Education (Ebner et al. 2011; Taraghi et al. 2010). From this personal
perspective I would like to enhance the chapter bringing three dimensions in
mind. Three factors have to be considered when introducing a PLE to Higher
Education institution, at least in middle-Europe:
1. Technological perspective: First of all as already written in the chapter a
Personal Learning Environment offers more or less both—freedom and restric-
tion. Learners must be able to choose their personal applications, contents, tools
for their individual learning process, but should be also able to do this in a secure
and private way. In contrast to a teacher-centred Learning Management System
we are talking now about a user-centred, flexible, expandable system. From a
M. Ebner (*)
Department Social Learning, Computer and Information Services, Graz University of
Technology, Graz, Austria
Commentary 229
technological point of view it is a kind of a multi-application monitoring

environment according to the special needs of a specific learner.
To achieve this goal those platforms are following a widget-based MashUp
concept (Taraghi et al. 2011) where different small applications (widgets) can be
arranged by users themselves. The web-based software consists mainly of two
parts—a framework (the widget container) and the widgets themselves. So the
weakness of the concept is maybe also its strength—to run such an environment a
high number of widgets for different purposes or learning goals are needed. Graz
University of Technology follows the concept of users’ programmed widgets,
which means that students of informatics are doing this small applications during
their projects or exercises (Taraghi and Ebner 2010).
2. Organizational perspective: The second major factor of a PLE in Higher Edu-
cation is the question who is running such an environment and what does that
mean to our lecturers? On the one side it seems rather obvious that the system
has to be provided university-wide on the other side it must be brought into the
mind of each single user—lecturers as well as students. First experiences pointed
out that in general such an environment is intuitive and can be well explained
with the “App-store metaphor”. Due to the fact that nearly everyone owns a
smartphone today it is easily imaginable if a Widget is called App and the
Widgetstore is compared with the App-store. First gathered statistics pointed
out that the PLE in general is used if it is provided university-wide, but still more
or less for getting-information issues than teaching and learning purposes
(Selver et al. 2013; Taraghi et al. 2013).
3. Teaching and learning perspective: Finally it must be taken into account that any
system for supporting learning and teaching needs a certain context where it is
used and an embedded didactical scenario (Ebner et al. 2011). As described well
in the chapter using a PLE for teaching and learning will be a switch from
behaviourism to cognitivism. Most of our daily lectures in typical bachelor
programmes are based on a face-to-face education where lectures present their
contents. It is obvious that this kind of teaching is not appropriate for such an
environment where students should aggregate, share, search, recommend etc. It
can be summarized that an arbitrarily effort will be necessary on this issue.
Future of Higher Education will need therefore new concepts, lecturers who
revise their lectures and learners who will adapt their learning styles. The concept
of a Personal Learning Environment and its technical realization is just a first step
and the chapter as well as the whole book a first great tribute to it.
230 Commentary
References
Ebner M, Schön (aka Schaffert) S, Taraghi B, Drachsler H, Tsang P. First steps

towards an integration of a Personal Learning Environment at university level.
ICT 2011(2011):S22–36.
Gillet D, Law E, Chatterjee A. Personal learning environments in a global higher
engineering education Web 2.0 realm. In: Global engineering education confer-
ence (EDUCON), Madrid, Spain, April 14–16, 2010.
Gillet D, de Jong T, Sotirou S, Salzmann C. Personalised learning spaces and
federated online labs for STEM education at school: supporting teacher com-
munities and inquiry learning. In: 4th IEEE global engineering education con-
ference (EDUCON), Berlin, Germany, March 13–15, 2013.
Selver S, Tarahi B, Ebner M, De Vocht L, Mannens E, Van De Walle R. Monitoring
learning activities in PLE using semantic modelling of learner behaviour. In:
Holzinger A, Ziefle M, Hitz M, Debevc M, editors. Human factors in computing
and informatics. Berlin: Springer; 2013. p. 74-90
Taraghi B, Ebner M, Mühlburger H, Till G. Personal Learning Environment—a
conceptual study. Int J Emerg Technol Learn. 2010;5:S25–30.
Taraghi B, Ebner M. A simple MVC framework for widget development. In:
Proceedings of the 3rd Workshop on Mashup Personal Learning Environments
(MUPPLE10); 2010. p. S1–8.
Taraghi B, Altmann T, Ebner M. The world of widgets—an important step towards
a personalized learning environment. In: Proceedings of the PLE Conference
2011; 2011. p. S1–8.
Taraghi B, Softic S, Ebner M, De Vocht L. Learning activities in Personal Learning
Environment. In: Proceedings of world conference on educational multimedia,
hypermedia and telecommunications. Chesapeake: AACE; 2013. p. 2466–75.
Exploring Open Educational Resources for Informal

Learning
Jon Dron
I have been following the ROLE project since its early days and I am delighted to
read this report of some of the resulting insights and ideas about how lifelong
learning may be supported with its tools. I would like to take this opportunity to
interpret some of the findings in this chapter, drawing on both the chapter and my
own experiences with the development of widget-based PLE tools.
J. Dron (*)
Athabasca University, Athabasca, AB, Canada
Commentary 231
One of the most interesting findings related in this chapter is the mixture of
feelings expressed by participants as to whether the PLE would be useful. For
instance, one of the responses from workshop 2 suggests that the effort needed
might not be worth the trouble and, more explicitly, one participant in the Dev8eD
workshop comments on the potential difficulties of integrating the ROLE widgets
with their existing PLE that included EverNote, Twitter and Google calendar. PLEs
are not filling an empty niche: we all assemble our own PLEs, whether we call them
that or not. At a broad level, there are PLEs that seek a high level of integration and
management of disparate learning tools (that I will refer to as iPLEs), and others
that are mostly an aggregation of tools (that I will call aPLEs). The ROLE tools fall
mainly into the iPLE category: their purpose is largely to make it easier to aggregate
and integrate learning spaces and resources. If this is to work, then it must have
extra value not found in other parts of an aPLE. It must be worth the effort to learn
to use them. My own aPLE includes a range of personal and shared aggregation
tools like browser bookmarks, Pocket, RSS readers and EverNote; productivity
tools like calendars, email, Google Search and Apple Widgets; learning objects
everywhere, from Wikipedia pages to StackOverflow answers; telephone, Skype,
Adobe Connect, social networks, Google Hangouts for dialogue; shelves of phys-
ical books as well as virtual collections; a desk, a range of computers and mobile
devices; and, most significantly, a set of methods, procedures and pedagogies from
which I choose to assist my learning process. Altogether, it is a flexible, highly
customized personal learning environment that I use to assemble the things I need
for my own learning. There would need to be a good reason to add more tools to this
mix. This leads to another quite closely related major issue raised in the chapter:
that of usability.
The chapter highlights issues of usability and technical complexity. This is a
wicked problem because PLEs tread a tightrope. They must provide a lot of
flexibility in order to support an indefinitely large number of potential self-guided
learning strategies but they must also make learning easier. For flexibility, they
must be fairly soft technologies, in which orchestration of processes and methods is
performed by their users. Unfortunately, the softer we make our technologies the
harder they are to use, because we must put in the effort to perform the orchestra-
tion. If we harden our toolset then some parts of the orchestration must be handled
by the tools but, the more orchestration that is built into a technology, the less
flexible it becomes. Efficient, demanding less thought, fast: but rigid. Widgets offer
a potential solution, by allowing small hard pieces to be assembled into a vast range
of learning environments. Using any single widget is mostly pretty simple but
knowing which widgets to choose, how they can be configured, how they can be
arranged and what they can be used for is much more complex. Thus, though the
pieces may be relatively hard, the overall assembly remains soft and therefore
difficult to use effectively, requiring an investment in learning and configuring that,
unless proven worthwhile, is unlikely to be attempted.
When we talk of self-guided learning we normally mean it only at a coarse
granularity: essentially, the absence of an overarching course structure. At a smaller
scale, structured learning objects, book chapters, websites, videos and many other
232 Commentary
teacher-created artefacts are the norm (using “teacher” to mean anyone, including a
team of designers or fellow learners, that intentionally or otherwise helps another to
learn). So it is with interest that I read this chapter reporting on personal learning
environments, but talking about them in the context of intentional teaching,
courses, workshops and other planned processes. Self-regulation can occur at
many scales. We may choose to control different aspects of the learning process
but almost always delegate control to others at many stages, whether to the author
of a chapter or learning object, the leader of a workshop, our PLE or the widgets
within it.
Some tools described in this chapter such as Etherpad and Flashmeeting hinge
on social engagement, which entails a need to be at the very least mindful of the
schedules, needs and goals of others. This highlights a tension that exists in nearly
all PLE implementations, that they support our social learning activities, but that
those social learning activities themselves, with our fellow learners and teachers,
provide shape and form to our learning. For instance, I was not surprised to read that
relatively little use was made of Etherpad and Chat in the events described: given
that participants were collocated it would not normally be very useful to provide
alternative real-time collaboration tools, especially as the tasks did not appear to
focus on production of a permanent artefact but were simple part of some active
experimentation to use the toolset.
At the heart of all my reflections on this chapter is the fact that PLEs are more
than just a way to keep things organized in our learning lives. Done well, they are
generative toolsets that can act in some ways like a teacher, offering guidance,
inspiration, motivational support and structure to the learning experience. But, at
the same time, they seek to provide freedom from such a teacher role, to be soft
tools to support self-regulated learning. They are thus both teachers and not
teachers at the same time. Their innate softness is perhaps the reason that the
evaluations performed in this chapter focused on helping people to use the tools
in a manner that is anything but self-regulated and explains why it is so hard to pin
them down. A PLE is personal: every individual builds processes and methods
around them, configures his or her own space but, at the same time, that space is
shaped and influenced by the people, resources, learning objects, tools and expertise
that are available. This tension lies at the heart of education. When we educate
ourselves we choose the parts that we delegate to others more than those who follow
a more guided path but, through the shape of our tools, the people around us and
simple path dependencies, we have many of our decisions made for us and, at a finer
granularity, always delegate at least some of the teaching process to others. Getting
the right balance is a tough task to perform well and partly explains why case
studies like the one presented here have a vital role to perform in helping us to
understand that better.
Commentary 233
Workplace Learning at Festo: Content Creation by Workers

for Workers
Content Creation by Workers for Workers
Juergen Mangler
Abstract As a follow-up to Chap. 7—an interview with Manuel Schmidt and

Michael Werkle was conducted, to delve into how it is possible for Festo to
embrace PLE concepts in their organization, about the motivation of workers to
create and share content, and about future projects.
Keywords Interview PLE Content creation
Introduction
From Chap. 7 we learned that FESTO is to promote informal learning—learning

that happens e.g. through interaction of the learners in social media environments.
Rather than employing a top down approach, where learners are fed learning
material prepared by dedicated content creators, an individualization of learning
is aspired for that covers the learning process (i.e. the what, when and the pace), as
well as the content creation process. For the content creation process FESTO is
focusing on the experts in the fields (i.e. the workers who do something day to day).
In order to find out more about how typically strict learning related policies in a
company fit in with the goals of PLEs, the interview was conducted with a focus on
the philosophy and benefits related to letting workers create their own learning
material during their work-time, and how this fosters learning and collaboration in
the workplace.
Both interviewees are members of Festo Lernzentrum, a separate entity inside
Festo that maintains the companies’ Learning Management System (LMS), oversees
content creation, as well as the creation of didactic designs for the training of workers.
J. Mangler (*)
Faculty of Computer Science, University of Vienna, Vienna, Austria
234 Commentary
Interview
Interviewer: What is the relationship between traditional e-Learning and PLE’s in

your Organization, and how was it shaped by the ROLE project?
Manuel Schmidt: The ROLE project served as an incubator for complementing
the existing LMS with properties of a Personal Learning Environment (PLE). From
the start of the project there was a strong consensus inside Festo, that a pure PLE
was not suitable for the business context. As motivating workers to maintain and
extend their job related skills and knowledge is a primary focus of businesses in
general, they also want to control the goals and granularity (i.e. the when and what)
of learning. Furthermore businesses want to speed up the process of getting started,
which they see as conflicting with the nature of PLE’s, where the user starts from
scratch, e.g. spending lots of time building one’s own learning environment through
widgets.
Interviewer: Can you describe the typical learning scenarios for Festo workers?
Manuel Schmidt: Festo is, like possibly most businesses, very much focused on
individual workers’ career paths and individual competence development plans.
This formal training of workers is accompanied by e-learning. We focused on
allowing workers that have to tutor other workers to create and distribute their
own content, e.g. videos. (. . .) 90% of our users are knowledge workers.
While all workers of course are actively encouraged to participate in seminars,
which are rigidly structured, we provide complementary e-learning content in our
LMS. For this content, workers can decide for themselves if and which parts they
want to consume. So even if a learner does not participate in seminars, he/she can
select from a wide array of on-line learning material.
Interviewer: Do you have an estimation which percentage of users use your LMS
to consume content because of—or complementary to—seminars, and which per-
centage of users are purely self-motivated learners?
Manuel Schmidt: During 2013, about 50 blended learning seminars with oblig-
atory material provided through the LMS took place, but about 600 courses in total
have consumed during the same year.
Michael Werkle: Staff development in Festo relies on two pillars—quantitative
and qualitative goal-setting between workers and their supervisors, and self-
motivation. The facts are: the 600 courses had a total of about 8,000 users consum-
ing them, and learning videos have been consumed over 13,000 times in the
last year.
Interviewer: In Chap. 7, section ‘Implementation of the Personal Learning
Management System’ it is stated that one organizational requirement is to ensure
the transparency of the learning process and the yielded achievements. For sure the
employer is interested in, and encourages its workers, to improve their skills and
competencies. Does Festo have any formal instruments in place for motivating
workers, for example awards?
Manuel Schmidt: There are no awards for learners or content creators. We were
toying with the thought, but so far nothing has been realized. One important aspect
Commentary 235
in our organization is the very open culture regarding learning: we allow and
encourage the learners to use our on-line resources during normal work-time.
This trust vis-a-vis the workers alone translates into motivation.
Michael Werkle: Rewards for content creation can of course also have negative
effects. For example the use of Wiki’s in companies is often very successful when
coupled with reward systems . . . but they immediately die as soon as the rewards
are taken away. The learning-tube philosophy is successful without any extrinsic
rewards.
Manuel Schmidt: We think that systems are successful when the intrinsic reward
is obvious for the workers. For example, knowledge workers that have to train other
workers are much more flexible when they create videos. The can reuse them in
seminars or even refer to them during normal meetings.
Interviewer: Do you track individual learners, their learning progress, . . .?
Michael Werkle: This is not possible due to German privacy protection laws, and
company level agreements.
Interviewer: Currently the whole approach seems much centred on knowledge
workers—workers who do all their work in front of the computer screen—or at least
on the knowledge working part of the job. Is there a planned integration of the
factory floor learning processes into the system? For example when a worker
explains to other workers how certain systems on the factory floor work, or how
to make them work better, it cannot be captured by screen-casts.
Michael Werkle: There are several research projects underway for human-
machine interaction. We are not sure yet which direction these projects will take.
Manuel Schmidt: For me the question is—how to integrate the LMS into the
work environment—or maybe not integrate it at all. For example, content could be
attached to machines, and a learner equipped with technology like ‘google glass’
could access this content directly in front of the machine, in a augmented reality
setting. The goal will be ‘integration into the normal work environment’.
Interviewer: A very simple step, long before producing content for others is:
taking notes for yourself in order to not forget. Learning material is produced for
self-consumption. Are there any signs that the learning facilities inside Festo are
used like this? To what extent?
Manuel Schmidt: Our content creation process (for learning-tube, Ed.) is
two-tiered. First the created content is saved locally, and only in a second step it
is published. I know that some colleagues are using the system for personal notes.
My personal estimation is that the ratio between published videos and local
videos—consisting of videos that the users are not happy with, and videos that the
users created for self-consumption—is about 1:10.
Interviewer: In the conclusion of Chap. 7, it is mentioned that the search
mechanisms inside videos are not yet there—specifically full text search is not
working because not automatic translation of the spoken word to text is possible.
Are there any new developments in this area?
Manuel Schmidt: We tried to set up a project that tackles full-text search for
videos with a semi-automatic approach: machine-translation and human lectors.
236 Commentary
We furthermore wanted to create tag-clouds for each video. Currently our search
only uses tags that have been added by the creator of the video. . .
Interviewer: . . . but are the learners allowed to add own subtitles, tags, and notes
to learning videos—basically crowd sourcing the creation of data for full-text
search? Semantically conceptualized information is after all much more valuable
than just the plain full-text, as it allows to find something according to its meaning,
instead of the words that are used by the creator.
Manuel Schmidt: This is currently not possible, but will be added in the future.
For now we focused version of the platform that supports a recommender system
for videos, including comments.
Michael Werkle: Especially interesting—and related to this topic—is interna-
tionalization. As we are a multi-national organization, we observe the workers—
e.g. colleagues from US and Germany collaborate for bi-lingual videos. It is
especially important that the tools not only support such collaboration, but make
it easy.
Interviewer: As mentioned in the introduction, media-didactic conception and
design is key to the success of learning material. How does Festo tackle the fact that
when the content is produced by experts in the field (i.e. with no extra media-
didactic education)? Is there a support team that helps the workers who are willing
to produce content, without putting an additional post-production burden on him?
Or is the content left unaltered and filtered purely by how well is received by other
learners?
Michael Werkle: Our observation is: the quality is very high. The content
creators are aware that about 15,000 colleagues—including the upper
management—can watch created content. Thus they put lots of efforts into the
created material. Usually we only have to provide technical support regarding the
tool—and more generic tips, like how to best present my desktop. The users
definitely put lots of effort regarding message and scenario into the content creation
process. The users even come up with lots of ideas for the presentation of topics that
we would have not thought about.
Interviewer: This raises the question about granularity? How is ensured that one
video does not contain too much information (that could be split up into smaller
pieces—micro-learning)?
Manuel Schmidt: Videos typically have a duration of 2–10 min, so the content
creators intuitively go for the right granularity. As most created content deals with
solving a specific problem, a storyline is natural: explain the problem, solve the
problem, happy end.
Also one big group of content creators is definitely key (region, E.d.) managers
and product managers—they know how to sell products and thus are also qualified
to create learning material.
On the other hand, also people with no special skill-set, which even could be
described as introverted, created content. In some cases these people work together
on videos, i.e. interact regarding the topic of the video in the form of a question-
response game. In this case I suspect one person alone would not have created a
video.
Commentary 237
Interviewer: Evaluation is always a time-consuming process that is easily

criticisable because of e.g. sample size, or certain questions used. Additionally,
the quality of the properties of a tool is distorted by the very content it provides—in
the case of Festo, the videos produced by workers. But they also only produce a
snapshot—a glimpse into what a limited group of people thought at a certain time.
Is there a permanent evaluation mechanism, that continuously and preferable
without placing additional burden on the users (a passive mechanism)? Quantity
is important, but what about quality?
Manuel Schmidt: We differentiate between LMS and content. We have a con-
tinuous evaluation regarding learning-content, the platform—its functionality—is
evaluated with each new release. Currently questionnaires are created in conjunc-
tion with courses; the functionality for evaluating single learning objects like videos
is included in our next internal release.
Interviewer: Thank you for the interview.
Conclusion
As a conclusion from the interview, it becomes apparent, under the premise that a
learning-friendly culture has been established, that businesses have no problems
finding motivated workers that create high-quality content, and cooperate with
colleagues in content creation. Finding a balance between pure PLE’s and tradi-
tional LMS seems to be a bit of struggle. Due to already existing formal educational
instruments like seminars, and time-restrictions, properties of PLE’s seem to be
hard to integrate into a company strategy. At Festo the idea of PLE’s is manifested
as a comprehensive library of learning objects—videos, courses, material—that the
learners can select from. It will be very interesting to observe if other companies
will move into the direction of allowing learners to create and share own material
and courses at an even more fine-grained level.
Finally, the topic of integration the factory floor—non white-collar workers—
into these systems will be a challenge, with lots of innovative concepts to be tried
out. For example the idea to attach learning material to physical objects including
the consumption in an augmented reality setting seems very intriguing.
238 Commentary
Lessons Learned from the Development of the ROLE PLE

Framework
Sheila MacNeill
Extending the flexibility of learning environments continues to be a challenge for

both users and developers. Over the past decade there has been increased demand to
ensure that institutionally provided learning spaces are easily adaptable and per-
sonalized. Many teachers, learners and VLE administrators have been frustrated by
the lack of flexibility and opportunities for customization and personalization in
VLEs. Recently, there have been a number of developments that are allowing far
more flexible and open approaches to be taken.
In 2010, in response to the increased demands both pedagogically and techni-
cally to integrate more social applications into VLEs, Cetis produced the Distrib-
uted Learning Environments Briefing Paper. The paper outlined the tensions at that
time as:
the requirement for deeper integration with other (administrative) systems gave rise to the
MLE (managed learning environment) concept. Later, the demand for greater
personalisation and the availability of new web tools gave birth to the PLE (personal
learning environment) debate, in which people radically re-conceptualised the notion of a
learning environment. During these phases, however, the VLE still remained a dominant
force within institutions. This has resulted in a tension between the role of the VLE as a
common tool for the institutional community, the desire to make it permeable to the
institutional network and the wider web and to allow greater levels of personalization/
customization for individuals and institutions.
The chapter outlined five potential models for the opening up and integration of
VLEs with a number of other administrative systems and the wider social web and
allowing increasingly flexible access to VLEs from mobile devices
Following the publication of the paper, Jisc funded a small development
programme, Distributed Virtual Learning Environments1 that allowed several rel-
atively small-scale projects to develop solutions based on the models.
Interoperability and flexibility have, and continue to be, central to the work of
Cetis, so as this programme developed we actively engaged with a number of other
communities working in this space including the Apache Foundation and the ROLE
project.
The work of the ROLE project was of particular interest as it provided a useful
intersection and more potential technical solution to some of the outstanding
challenges from the programme. In particular by providing an underlying open
architecture for the creation, deployment and storage of widgets. The areas of
1
www.jisc.ac.uk/publications/briefingpapers/2012/extending-the-learning-environment.aspx
S. MacNeill (*)
Glasgow Caledonian University, Glasgow, UK
Commentary 239
development discussed in this chapter were of particular interest in terms of

providing potential solutions to HEIs in moving forward the development of
more flexible learning spaces.
Interoperability Framework, Architecture
The open framework and architecture developed through ROLE demonstrates the
key functionality and communication mechanisms for the deployment and integra-
tion of widgets. A significant challenge highlighted by the Jisc DVLE programme
was that institutions do not have the capacity to host and manage institutional
widget spaces/stores. However the concept of a central, educational specific “app
store” did have traction. The concept of an educational specific app store had
traction; it just needed a mechanism to make it a reality. Following the Cetis
2012 App Store Conference Session,2 Jisc funded a pilot project for the Role project
to produce a proof of concept store utilizing their developing infrastructure and
architecture.
Inter-widget Communication
For widgets to be integrated within a successful PLE, it is necessary for them to be

able to integrate with other elements of that environment. Collaboration is an
increasing part of many learning experiences. Widgets offer an array of customized
collaborative activities. One of their inherent appeals is the fact that learners/
teachers can utilize a variety of widget combinations. In an educational setting
such as a course delivered primarily via a VLE, widgets need to be able to access
key user information and recognize individuals and groups.
Contextualized Meta-data
As the chapter highlights tracking widget interactions is central to developing

responsive learning environments. The growing interest in learning analytics in
the sector also points to the desire for more detailed information on user activities.
The exploration and instantiation of the CAM schema as described highlight the
affordances (and challenges) both for end users and developers that this method of
data collection can provide.
2
http://blogs.cetis.ac.uk/sheilamacneill/2012/02/26/app-stores-galore-at-cetis12/
240 Commentary
Spaces
Learning spaces are notoriously difficult to comprehensively define. The bound-

aries are constantly evolving and being permeated. The concept of space(s) defined
by the ROLE framework allows the aggregation of people, resources, applications
and spaces. Two approaches—linked data and Open Social are discussed. Both
have their strengths and weaknesses, which have been explored and expanded
through the work of the project.
Authentication and Authorization
Authentication and authorization of users and data is a vital element of widget

deployment in a PLE context. The chapter highlights the two levels of data
communication methods needed for authentication and authorization—service-to-
service and widget-to-widget. More work has been done on the later, in particular
with recognized authorization services such as OAuth, which provides a level of
user control over sharing of data in specific spaces.
The chapter provides a comprehensive overview of the potential of the ROLE
framework in the development of increasingly adaptable and sophisticated personal
learning environments. From a personal point of view, it was very rewarding to play
a small part in joining up developments within the UK HE sector with the wider
European context provided through the ROLE project.
Reference
MacNeill, S, Kraan W. Distributed learning environments, Cetis. http://publica

tions.cetis.ac.uk/2010/46. Accessed Feb 2014.
ROLE Consortium: Research Institutions
Fraunhofer-Institute for Applied Information

Technology—FIT, Germany http://www.fit.
fraunhofer.de
RWTH Aachen University, Germany http://
www.rwth-aachen.de
Technical University of Graz—TUG, Austria

http://www.tugraz.at
Katholieke Universiteit Leuven, Belgium

http://www.kuleuven.be
University of Koblenz, Germany http://www.

uni-koblenz-landau.de
Uppsala University, Sweden http://www.uu.se/
(continued)

DOI 10.1007/978-3-319-02399-1
242 ROLE Consortium: Research Institutions
Ecole Polytechnique Fédérale de Lausanne—

EPFL, Switzerland http://www.epfl.ch
University of Leicester, UK http://www.cs.le.

ac.uk
Open University UK—UKOU, UK http://kmi.

open.ac.uk
Vienna University of Economics and Business,

Austria http://www.wu-wien.ac.at
Festo Lernzentrum Saar GmbH, Germany

http://www.festo-lernzentrum.de/
IMC Information Multimedia Communication

AG, IMC—Germany http://www.im-c.de
(continued)
ROLE Consortium: Research Institutions 243
The British Institute for Learning and

Development—BILD, UK http://www.thebild.org
Shanghai Jiao Tong University—SJTU, China

http://www.sjtu.edu.cn/
Centre for Social Innovation—ZSI, Austria

http://www.zsi.at
U&I Learning NV—UIL, Belgium http://www.

uni-learning.com
© ROLE Consortium 2009-2013. The Role

project was supported by the European
Commission, in the theme ICT-2007
Digital Libraries and technology-enhanced
learning, as a Large-scale integrating
project (IP), under the 7th Framework Programme, Grant agreement no.: 231396.
.
Supplementary Material
Links to relevant websites and software
ROLE Project Website:

ROLE Widget Store:
http://www.role-widgetstore.eu
ROLE Software on GitHub:
http://github.com/ROLE/ROLE
ROLE Software Documentation:
http://role-project.sourceforge.net/wiki/index.php/Main_Page
ROLE Network on LinkedIn (currently 720 members):
http://www.linkedin.com/groupInvitation?gid¼1590487
ROLE Videos:
http://www.role-project.eu/Videos
ROLE Online Courses:
http://tinyurl.com/role-course
http://tinyurl.com/role-srl-course
ROLE eBook:
http://bit.ly/self-regulated-learning

DOI 10.1007/978-3-319-02399-1
Index
A Creative Commons license, 3, 121, 136

Academic Blended Learning, 84 Cyclic SRL model, 19
Activity Recommender (AR)
SRL, 34–35, 39
and To-Learn task list, 101–102 D
widget, 98, 99 Design based learning approach, 224
ActivityStreams, 190, 194 Desktop-based widgets, 3
Adaptive structuration theory, 227 Dev8eD Workshop 2012, 147, 153–155
Analyze tweets bundle, 106 Development software, 210–212
Apache Rave project, 187, 212 Distance learning. See Uppsala University
ARLearn mobile app, 12 Documentation evaluation, 54–55
Audio self-presentation, 98 Domain skills, 65
Authentication and authorization, of users and
data, 240
Automated monitoring method, 53 E
Autonomous learning, 221 e-Learning Community (eLC) seminar,
144–146, 150–152
e-learning tools, 92
B eMUSE tool, 23
Binocs, 5, 144, 153 EtherPad, 144
Exam-oriented learning, 88–89
Extensible Messaging and Presence Protocol
C (XMPP), 87, 189
Case-oriented cross-case analysis, 70, 71 Extrinsic motivation, 21
Chat widget, 84–86, 90 Eye tracking technique, 58
Cloud computing-based widget, 5
Clustering techniques, 67
Cognitive walk-through method, 57 F
Collaborative learning, 21 Festo Learning Center, 162–163, 167
Competence-based Knowledge Theory, 23 Festo LearningTube, 175, 182, 183
Connectivist MOOCs (cMOOCs), 130–132 Festo Virtual Academy, 163
Contextual inquiry method, 57 Festo workplace learning, interview, 233–237
Contextualized attention metadata (CAM), Field method, 57
53–54, 57–58, 188 Formal learning
Contextualized meta-data, 239 research, 81

DOI 10.1007/978-3-319-02399-1
248 Index
Formal learning (cont.) Build your ROLE workshop, 146–147

ROLE technology, 80 context and objectives, 136–139
RWTH Aachen University (see RWTH culture, 140
Aachen University) Dev8eD Workshop 2012, 147, 153–155
Shanghai Jiao Tong University (see eLC seminar, 144–146, 150–152
Shanghai Jiao Tong University) evaluation objectives and instruments,
testbeds, 80 143–144
Uppsala University (see Uppsala International workshop on cloud education
University) environments, 147
widget bundles, 111 JTEL summer school workshops, 152–153
Full-time learning. See RWTH Aachen open educational resources, 230–232
University participant evaluation, 149, 150
personal learning environment, 140–142,
156–157
G World conference on E-Learning paper,
GitHub, 210 148
Go-Lab project, 127–128, 213 Innovation diffusion process, 60
Graasp PLE platform, 121, 204 Inquiry-based learning (IBL), 11. See also
audience levels, 122 weSPOT Project
bachelor course on human computer Inquiry-learning space for STEM education,
interaction, 123–124 127–129
description, 202 Institute of Information Management in
design objectives, 122 Mechanical Engineering (IMA), 82
doctoral studies, 126 Intelligent tutoring systems (ITS), 22
feature, 122 International workshop on cloud education
Go-Lab project, 127, 128 environments, 147
spaces, 122, 203 Interoperability framework, 238–239
usefulness, 124, 125 Inter-widget communication (IWC), 239
local, 189
remote, 190
H WKM testbed, 87
Heuristic evaluation method, 57 Intrinsic motivation, 21
Higher education. See also Formal learning ITS. See Intelligent tutoring systems (ITS)
bachelor course on human computer
interaction, 123–125
organizational perspective, 229 J
Science 2.0 literacy in doctoral studies, JTEL summer school workshops, 152–153
125–126
teaching and learning perspective, 229
technological perspective, 228–229 L
transformation, 115 Large-scale usability evaluations, 58
History widget, 84–87 Layered SRL model, 19
Learner model, SRL, 27
Learning analytics field, 182
I Learning management system (LMS), 2–3, 14,
IBL. See Inquiry-based learning (IBL) 22, 83. See also Personal learning
iClass project, 23 management system (PLMS)
IMC and CLIX learning suite, 168–169 Learning spaces, 239
Informal learning, 135 LearningTube widget bundle, 174
activities, setup and organisation, 139–140 Liferay system, 103
artifacts, 138 Likert-type rating scale, 101
Build your own PLE workshop, 146 Linked Data, 196
Index 249
LIVECONTEXT, 168 P
Local inter-widget communication Part-time learning. See Shanghai Jiao Tong
(LIWC), 189 University
Perceived Usefulness and Ease of Use (PUEU)
survey, 143
M Personal learning environments (PLEs)
Mashup Recommender widget, SRL activity-and student-centric model, 116
evaluation results, 38–39 adoption factor, 51
guidance level, 34 aggregation and hosting features, 120
templates, 33 alternative design and implementation,
Massive open online courses (MOOCs), 127 119–120
Meta-cognition, 20 challenges, 50
Meta-cognitive activities, 222–223 characterization, 50
Mixed-method, PLE, 51 community building and motivation, 66
Mobile technologies, 219 connectivist MOOCs, 130–132
Moodle, 186 definition, 117–119
flexibility and adaptability, 96 description, 22
OpenSocial plugin, 96, 97, 196, domain-specific widgets, 23
204–206 emergence, 116, 117
ROLE widgets, 5, 7 eMUSE tool, 23
Mouse tracking technique, 58 formal education, 226–227
Graasp platform, 121–122
guidance strategies, 62–63
O higher education, 228–229
ObjectSpot, 144 import and export features, 120
OER. See Open educational resources (OER) informal learning, 140–142, 156–157
OpenApp IWC, 194 inquiry-based learning, 11 (see also
Open educational resources (OER), 3, 220 weSPOT Project)
informal learning, 10–11, 230–232 (see learner-centric approach, 3
also Informal learning) lifelong learning, 1
OpenLearn project (see OpenLearn project) mixed-method, 51
ROLE online learning courses, 138 motivational models, 65–66
OpenLearn project, 136, 147 multidimensional evaluation framework,
Creative Commons licence, 136 223–225
LabSpace area, 140, 148 multi-perspective evaluation, 51
personal learning environment, 142 nongovernmental organizations, 129–130
ROLE online courses, 142 online spaces, 120–121
self-study materials, 136 OpenLearn project, 142
OpenSocial, 87 recommendation strategies, 63–64
and Apache Shindig, 211–212 ROLE innovations, 8–9
directory of PLMS, 170–171 RWTH Aachen University, 84–88
disadvantage, 197 search and recommendation features, 120
learning management system, 171–173 self-regulated learning process, 3, 61–62
Moodle plugin, 96, 204–206 Shanghai Jiao Tong University, 96–97
space, 121 sharing and repurposing features, 120
specification, 206 TOPS model (see TOPS model, PLE)
Open-source developer communities unified evaluation framework, 68–71
Apache Rave, 212 Uppsala University, 106–107
developer camps, 211 widget-based approach, 3–7
development software, 210–211 Personal learning management system (PLMS)
OpenSocial and Apache Shindig, 211–212 collaborative and social learning, 169
Strophe.js, 213 first evaluation loop, 176–177
technical cluster structure, 210 human resource management systems, 169
250 Index
Personal learning management system (PLMS) widgets and tools, 208–209

(cont.) widgets embedded inside Moodle course,
IMC and CLIX learning suite, 168–169 5, 7
OpenSocial LMS, 171–173 widget store, 6, 32–33, 200, 206–208
PPIM, 170–172 ROLE. See Responsive open learning
pre-defined learning materials, 170 environments (ROLE)
second evaluation loop, 178–181 RWTH Aachen University
Virtual Academy, 173–175 cloud learning application, 92
PLEs. See Personal learning environments communication, 93
(PLEs) e-learning tools, 91, 92
PLMS. See Personal learning management evaluation and methodology, 88–91
system (PLMS) lab learning activities, 93
Power learning, 88–89 learning scenario, 82–83
Pre-designed widget bundles, 227 PLE, 84–88
Psycho-pedagogical integration model (PPIM),
170–172
S
School of Continuing Education (SOCE), 80
R Self-evaluation process, 20
RAVE project, 187 Self-guided learning, 231
Recorder widget bundle, 174 Self-monitoring process, 20
Recording widget, 98 Self-regulated learning (SRL), 82, 168,
Remote inter-widget communication 221–223
(RIWC), 189 activity recommender, 34–35, 39
Requirements Bazaar approach, 54 adaptation to learner’s characteristics, 20
Responsiveness, 186 basic components, 9
Responsive open learning environments bundle, 98–99
(ROLE) collaboration, 21
architecture, 188 competence model, 26–27
authentication and authorisation, 197–198 competence ontology, 28–29
case studies and test-beds, 10–11 components, 18–19
contextualised attention metadata, 191–195 courses and training material, 31–32
description, 8–9 cyclic model, 19
free online course learning activity, 8 description, 18
fundamental aim, 18 eBook, 7
Graasp, 202–204 goal setting process, 19
inter-widget communication, 189–190 guidance and freedom, 20
OpenSocial Moodle plugin, 204–206 help-seeking, 20
open-source developer communities, ICALT conferences, 39–40
209–213 iClass project, 23
personal learning environment, 8–9, JTEL summer schools, 44
237–238 layered model, 19
poster and other publicity materials, learner model, 27
147–148 learning strategies/process, 19
SDK mashup guidelines, 29–30
concept of spaces, 199 Mashup Recommender widget, 33–34,
learning space, 198 38–39
personal space, 198, 201 meta-cognition, 20
social context, 199 monitor, 37, 38
space user interface, 199–200 motivation, 21
spaces, 196–197 motivational videos, 30–31
SRL eBook, 148–149 PALE workshops, 40–41
translator widget, 4, 5 and personal learning environment, 3
Index 251
preconfigured PLEs, 32 personal learning environments, 164

process model, 23–26 PLMS (see Personal learning management
ROLE innovations, 8–9 system (PLMS))
ROLE test-beds, 11 Project Vision, 166
ROLE widget store, 32–33 ROLE solution, 161
self-evaluation process, 20 target group related challenges and
self-monitoring process, 20 requirements, 167
SRL-supported PLE tools, 41–42 trends, 160–161
task strategies, 20 user related challenges and requirements,
teaser videos, 31 168
technology-enhanced learning solutions, videos, 165–166
22–23 Technology-supported learning environments,
text reader widget bundle, 35–37 226
time management process, 20 Text reader widget bundle, SRL, 35–37
time-related issues, 42 Text-to-Speech Engine, 98
TPSRL survey, 42–43 Think aloud method (TAM), 57, 178–179
Service-to-service communication, 197 To-Learn list widget, 34–35, 98
Shanghai Jiao Tong University, 93–94 Tool skills, 65
bundles, 98–99 TOPS model, PLE
domain-specific widgets, 104 CAM analytics, 57–58
evaluation and methodology, 100–103 clustering techniques, 67
learning scenario, 94–95 cognitive walk-through method, 57
Liferay system, 103 cost-effectiveness, 60
outlook, 105 cross-case analysis, 70
PLE, 96–97 cyclic evaluation process, 70
Single sign on (SSO) authentication dimensions and recommended methods, 69
mechanism, 188 effectiveness, 56, 69
Single widget, 4–5 efficiency, 56, 69
Small-scale usability evaluations, 58 error tolerance, 56
Social network analysis (SNA), 67–68 evaluation metadata, 58
Social Software course, 81 eye tracking technique, 58
Software evaluation, 52–54 field method, 57
Spell Checker widget, 98 flexibility, 56
SRL. See Self-regulated learning (SRL) guidance strategies, 62–63
Standard Natural Classroom model, 93 heuristic evaluation method, 57
Strophe.js, 213 integrated evaluation framework, 68
System Usability Scale (SUS), 58 learnability, 56
memorability, 56
mouse tracking technique, 58
T operability, 56
Teacher/instructor-prepared learning organizational factors, 60–61
environments, 226 quantitative vs. qualitative methods, 71–73
Teachers’ Perception of Self Regulated satisfaction, 56–57
Learning (TPSRL) survey, 42–43 self-monitoring/self-reflection, 64
Technology-enhanced learning (TEL) self-regulated learning process, 61–62
personalization of learning experience, 50 skill model, 65
SRL, 22–23 social network analysis, 67–68
Technology enhanced workplace learning think aloud method, 57
business context, 181–182 user experience evaluation method, 59
Festo Learning Center, 162–163 user surveys, 58
Festo Virtual Academy, 163 utility evaluation, 52–55
Learning Company Festo, 162 TPSRL survey. See Teachers’ Perception of
LearningTube, 182 Self Regulated Learning (TPSRL)
organisation related challenges and survey
requirements, 167 Translator widget, 98
252 Index
U Web-based widgets, 3–4

Uppsala University Web 2.0 Knowledge Map (WKM), 84–88
evaluation and methodology, 107–109 weSPOT pedagogical IBL model, 12
learning scenario, 105 weSPOT Project
outlook, 110 ARLearn mobile app, 12
PLE, 106–107 inquiry space, 12
User experience (UX) evaluation method, 59 mashup of inquiry components, 12–14
User surveys, 58 sound and technology-supported personal
Utility evaluation, PLE, 52–55 inquiry approach, 11
Widget authoring tool, 102–103
Widget bundles, formal learning, 111
V Widget space, 4, 96–97
Variable-oriented cross-case analysis, 70, 71 Widget-to-service communication, 197
Video-based learning, 165–166 Wikipedia, 4, 5, 219
Wiki widget, 97
World conference on E-Learning paper, 148
W
W3C widget specification, 206
Web 2.0, 2, 4, 5 X
Web-based trainings (WBTs), 163 XMPP Extension Protocols (XEPs), 189

2015 Book ResponsiveOpenLearningEnvironm PDF

Uploaded by

Copyright:

Available Formats

2015 Book ResponsiveOpenLearningEnvironm PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

2015 Book ResponsiveOpenLearningEnvironm PDF

Uploaded by

Copyright:

Available Formats

Sylvana Kroop

Responsive Open Learning

ISBN 978-3-319-02398-4 ISBN 978-3-319-02399-1 (eBook)

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Personal Learning Environments (PLEs): Visions and Concepts . . . . . . 1

Dietrich Albert is professor of psychology at

Evgeny Bogdanov is a post-doctoral researcher

Kerstin Borau is a certified German and

Teresa Connolly works at the University of

Ingo Dahn leads the Knowledge Media Institute

Daniel Dahrendorf obtained a M.Sc. degree in

Diana Dikke is working as a research profes-

Denis Gillet leads the Coordination and Inter-

Knowledge Management (KM). Denis Gillet is Associate Editor of the

Sten Govaerts is a post-doctoral researcher at

Bodo Von der Heiden studied computer sci-

Erik Isaksson has a Master of Science degree

Sylvana Kroop is senior researcher and man-

Mikael Laaksoharju is a junior lecturer and

Effie Lai-Chong Law is Reader at the Depart-

Na Li is a Ph.D. student at the Coordination and

working on the European research projects on technology-enhanced learning, such

Thomas Lind is a Ph.D. student in the Division

Alexander Mikroyannidis is a post-doctoral

Alexander Nussbaumer is member of the

Matthias Palmér received his Ph.D. in media

Sarah León Rojas received a M.A. in Media

Maren Scheffel studied at the University of

Manuel Schmidt is working in the e-learning

e-learning in intercultural settings. He already gained EU project experiences in the

Simon Schwantzer holds a diploma in Com-

Ruimin Shen has been working in the Depart-

E-Learning Technology Standards. In 2006, Premier Minister Wen Jiabao awarded

Carsten Ullrich the deputy director of the Cen-

Katrien Verbert is an assistant professor at the

Helmut Vieritz studied Physics and Sociology

Michael Werkle is responsible for quality man-

Martin Wolpers holds a Ph.D. in electrical

the European Association of Technology Enhanced Learning (EATEL) and presi-

Fridolin Wild is research associate at the

Graham Attwell is an Associate Fellow, Insti-

Jon Dron is researching with the Technology

Sheila MacNeill has recently taken up a post as

Martin Ebner is currently head of the Depart-

Carlo Giovannella graduated in Physics and is

Marco Kalz is Associate Professor at the

Jürgen Mangler is researcher at the Workflow

Margit Pohl studied Computer Science and

Alexander Mikroyannidis, Sylvana Kroop, and Martin Wolpers

Keywords Personal learning environment • Self-regulated learning • Responsive

© The Author(s) 2015 1

Personalised and Self-Regulated Learning

The Learning Management System (LMS) has dominated technology-enhanced

The Widget-Based Approach of the PLE

Fig. 1 Browser- and widget-based PLE concept

Building a Widget-Based PLE

The ROLE Project and Its Key Innovations

ROLE is a European-funded initiative with 16 internationally renowned research