Role of Reflection in Customization Behaviors

Role of Reflection in Customization Behaviors Mona Haraty, Joanna McGrenere, Charlotte Tang Department of Computer Science University of British Columbia, Vancouver, Canada {haraty, joanna, scctang} @ cs.ubc.ca ABSTRACT Reflecting on everyday behaviors is an effective method of enhancing them. In today’s world, many of these behaviors are mediated by technology. Therefore, making the enhancements resulting from reflection should be supported by the technology through customization. Customizable tools allow users to make changes and appropriate them to their needs. In this position paper, we discuss the importance of support for reflection in customizable tools, especially for tools that are used in everyday practices such as personal task management. Then, we discuss the challenges in designing for reflection including 1) how to identify evidence of reflection, when studying changes in behaviors involving the use of tools, and 2) how to evaluate customizable tools that encourage reflection, i.e. how to determine if customization is caused by reflection. Author Keywords Reflection-on-action, personalization, customization, personal task management (PTM) INTRODUCTION Reflection on one’s own behavior is a critical component to positive changes in behavior. In fact, making any changes in one’s behavior involves some degree of reflection. In today’s world, many of our everyday behaviors involve the use of technology. Technology is mediating our behaviors such as managing our tasks, looking for information, and connecting to other people. Therefore, changes in one’s behavior as a result of reflection often involve making changes to the mediating technology as well as to the way they are used. Customizable tools give users control over their experience with the tools. They allow users to make changes to the interface or deeper aspects of them. This characteristic of customizable tools enables users to implement the desired changes resulting from reflecting on their behaviors. We are interested in understanding the role of reflection in customization behaviors. In particular, we are interested in the following research questions: With respect to a customizable tool, does a user’s reflection on their behaviors involving the use Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CHI’12, May 5–10, 2012, Austin, Texas, USA. Copyright 2012 ACM 978-1-4503-1015-4/12/05...$10.00. of the tool cause them to customize it? Can customizable tools improve customization behaviors by encouraging reflection? By improving customization behavior, we mean the user takes advantage of the customization when doing so is of her benefit. Before delving more into how reflection and customization are related, we first describe what we mean by reflection. Reflection has been interpreted in different ways. We adopt the definition of reflection by Sengers et al. [2] that is grounded in critical theory. According to critical theory, our behaviors are shaped by forces that we are normally unaware of. Sengers et al. defined reflection as “bringing unconscious aspects of experience to conscious awareness, thereby making them available for conscious choice” [2]. Building upon this definition, we define “degree of reflection” as the extent to which the unconscious aspects of experience are brought to conscious awareness. People respond to changes in their environments differently depending on the type of change, their motivation to respond to the change, and their level of awareness of the different aspects of the change and its influence on their goals. Consequently, the responses to a change can take different forms ranging from critical reflection on the change to unconscious adaptation to the change. In addition, changes to the situation can cause people to change their behavior, or to make changes in their environment. If the environment is composed of software systems and applications, making changes to the environment is referred to as customization. A study on customization behaviors has shown that the most common reason for people to customize is to respond to an external change [1]. External changes such as job changes, office moves, and going on trips are the most common reasons for customization. However, it is not clear to what extent customizations can be attributed to reflection. Is customization directly imposed by the external factors, or is it the result of some degree of reflection? We are interested in understanding the role of reflection on customization behaviors. This is important, because by understanding this role, we will gain a better understanding of customization behaviors which can in turn help in improving the design of customizable tools as well as tools that aim at encouraging reflection. In the rest of this position paper, we discuss two challenges in understanding the role of reflection on customization behaviors: identifying the evidence of reflection, and evaluating customizable tools that encourage reflection. research is needed into developing methods for identifying and measuring reflection [3]. IDENTIFYING THE EVIDENCE OF REFLECTION We have started to tackle this challenge by collecting subjective data through a survey. In our survey, we have tried to ask our question in an indirect way by asking about changes and the reasons behind them instead of asking whether they have reflected on their PTM behaviors. The first challenge in understanding the role of reflection is that we need to identify the occurrences of reflection. In this paper, the role of reflection in customization is discussed in the context of personal task management. We conducted a field study to investigate individual differences in personal task management behaviors. One of the findings of our field study was that participants differed with respect to how much they customized their tools. This was in part dependent on how customizable their tools were. However, there were participants who used customizable tools and did not make any changes to the tool. Following up our field study, we are conducting a survey to both assess the generalizability of our findings and delve into the underlying causes of the individual differences we observed. Based on the results of our study, we hypothesize that reflection is one of underlying causes to individual differences in personal task management behaviors. In our survey, to investigate whether making changes to one’s personal task management behaviors and customizing tools are due to reflection, we ask about the changes in people’s PTM behaviors and the reasons behind them. We hope that we can identify the evidence of reflection in the responses to these questions; however, we have not found any standard method of identification of reflection even when both the change in behavior and its reasons are known. Based on the results of our field study, we have come up with a number of factors that might help us to identify refection. Intentionality of a change can be an indicator of some degree of reflection. For example, using a new PTM tool that is installed by default on one’s new computer after buying the computer is an unintentional change that does not seem to be due to any reflection. However, giving up on paper and pen and starting to use a text file for making todo lists because of an increased amount of editing on paper, which was in turn due to an increased number of tasks, involves some degree of reflection. Another factor that can help in identifying the degree of reflection is the degree to which a change originates from purely personal experience rather than from others’ experiences. For example, a change such as switching one’s PTM tool to a new tool that is suggested by a friend may involve less reflection than switching to a new tool found by the person him/herself after actively looking for a tool that better meets his/her needs. To address the challenge of identifying evidence of reflection in our study of PTM behaviors, we have identified the characteristics of behavioral changes that are likely to be due to some degree of reflection. Characteristics such as intentionality of the change and its origin are the two we discussed in this paper. However, these are only preliminary steps toward addressing this challenge. Further EVALUATING CUSTOMIZABLE ENCOURAGE REFLECTION TOOLS THAT Once we address the first challenge of identifying the evidence of reflection, we will gain a better understanding of the role of reflection on customization behaviors. Assuming that reflecting on one’s behavior involving use of a tool can cause people to customize their tools, our goal would then be to design customizable tools that encourage reflection on behavior and use of tools. Therefore, another challenge that we foresee in pursuing this research is the evaluation of such tools. First, we need to assess the effectiveness of the tool in encouraging the users to reflect on their behaviors. Secondly, we need to assess whether the customization behaviors have been due to reflection rather than other factors. Assessing the effectiveness of a tool in encouraging reflection involves the same challenge of identifying the evidence of reflection. In addition, we need to ensure that the observed reflection is caused by the design of the tool rather than confounding factors such as the Hawthorne effect. While conducting a comparative lab study, comparing a simple customizable tool to a customizable tool that encourages reflection, is tempting, between subject designs can be strongly affected by individual differences and within subject design may suffer from strong carry over effects. Longitudinal field evaluations, in which participants use the customizable tool instead of their own tool, may be most appropriate for the purpose of our study for two reasons: they reduce the Hawthorn effect in comparison to lab studies and since we deal with reflection, it is unlikely that participants reflect on their studied behaviors within a short amount of time in a lab study. To assess whether customization behaviors have been due to reflection or other factors, the longitudinal field evaluation should be designed carefully to collect rich contextual data for capturing other potential factors contributing to customization behaviors. While designing a proper study is always challenging, the complex connection between the observable behavior and the process of reflection complicates the study design even further. REFERENCES 1. Mackay, W.E. Triggers and barriers to customizing software. Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, ACM (1991), 153–160. 2. Sengers, P., Boehner, K., David, S., and Kaye, J.J. Reflective design. Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility, (2005), 49–58. View publication stats 3. Sumsion, J. and Fleet, A. Reflection: can we assess it? Should we assess it? Assessment & Evaluation in Higher Education 21, 2 (1996), 121-130.