Fingerprints

Erschienen in: MobileHCI '16 : Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. - New York, NY : ACM, 2016. - S. 1085-1088. ISBN 978-1-4503-4413-5 https://dx.doi.org/10.1145/2957265.2965006 “Fingerprints”: Detecting Meaningful Moments for Mobile Health Intervention Yu nlon g W a n g Je n s M u e lle r Abst ract HCI Group HCI Group Universit y of Konst anz Universit y of Konst anz Personalized and cont ext ual int ervent ion s are pr om ising t echn iques for m obile per suasive t echn ologies in m obile healt h. I n t h is paper , we prop ose t he “ fingerprint s” t echn iq ue t o analyze t he users’ daily behav ior pat t er ns t o fin d t he m eaning fu l m om ent s t o bet t er support m ob ile per suasive t echn olog ies, especially m obile healt h int erv ent ions. We assum e t hat for m any persons, t heir behav ior s have pat t ern s and can be det ect ed t hrough t he sen sor dat a fr om sm art phon es. We develop a t hree- st ep int er act ive m achine learn ing work flow t o describ e t he con cept and approach of t he “ fingerpr int s” t ech n ique. By t h is we aim t o im plem ent a pract ical and lig ht - w eight m ob ile int erv ent ion sy st em w it h out b urdening t he u ser s w it h m anual logg ing. I n our feasibilit y st udy, w e show result s t h at pr ovide fir st in sight s int o t he design of t he “ fingerpr int s” t ech n ique. yunlong.wang@uni- konst anz. de j ens.m u eller@uni- konst anz. de Le D u a n H a ra ld Re it e re r I NCI DE HCI Group Universit y of Konst anz Universit y of Konst anz duan.le@uni- konst anz.de harald.reit erer@uni- konst anz. de Sim on Bu t sch e r HCI Group Universit y of Konst anz sim on.but sch er@uni- konst anz.de Au th or Ke yw ords Mobile p ersuasive t ech nologies; m ob ile int erv ent ion; int er act ive m achine learn ing. ACM Classifica t ion Keyw or ds H.1.2. [ Models and Princip les: User/ Mach in e Sy st em ; I .5.4 [ Pat t ern Recogn it ion] : App licat ions. Konstanzer Online-Publikations-System (KOPS) URL: http://nbn-resolving.de/urn:nbn:de:bsz:352-0-399798 I n t r oduction According t o Fogg’s beh avior m odel [ 3] , a per son w ill perfor m a beh avior on ly w hen he or she is sufficient ly m ot ivat ed, is able t o and is t r iggered t o per for m t he behav ior. Therefore, even if t h e u sers h ave t h e m ot ivat ion and abilit y, t he proper t r igger s ar e necessary in healt h persuasive app licat ions. As m ent ioned by Con solv o in [ 1] , food logging applicat ion user s alw ay s forget t o log t heir food, w h ich lead s t o in correct analysis resu lt s. I n m any cases, t he forget t ing is caused by in approp riat e or m issing rem inders. I n t he dom ain of cont ext - aw are m ob ile com put ing, t h e resear ch focu sed on using d ifferent sensor d at a and m achine lear n ing algor it h m s t o det ect hu m an beh avior [ 4] or m ean ingfu l m om ent s [ 6] for self- m on it or in g or int ervent ion s. Som e research in t he HCI com m un it y has explored int eract ive m ach in e lear ning approaches t o bet t er involve t he user s' int eract ion int o cont ext aw are com put ing [ 2, 5] . But for m ob ile h ealt h int ervent ion t echn olog ies, research ers seldom t alk about adopt ing int er act ive m ach ine learn ing. I n t his w ork, w e ex plore how t o apply int er act ive m ach ine lear ning st rat egy int o cont ext - aw are com put ing for m ob ile healt h int erv ent ions. our feasibilit y st udy, w e show result s t h at pr ovide fir st in sig ht s int o t he design of t he “ fingerpr int s” t ech n ique. Scena r io De script ion To bet t er present t h e “ fingerpr int s” t ech n ique, w e app ly it t o an exam ple of healt h per suasiv e app licat ion. This scen ario illust rat es t he process of how w e m ake hu m an rout ine det ect ion and int ervent ion w it h t he int eract ion of t he user s: One of t he aut h ors alw ays has m uch m eat in his lu nch, w hich is u nhealt hy. Therefore, he w ant s t o get an int erv ent ion when h e g oes t o t he cafet eria t o rem ind him t o eat m ore salad rat her t h an m eat . He st art s t o use t he “ Finger print s” applicat ion on his sm art ph one. I n t he fir st w eek t he applicat ion ru ns in t he background and collect s t he dat a aut om at ically. Then t he applicat ion ask s him t o n am e t he found beh avior pat t erns. He select s t he one “ from t he office t o t he cafet eria” and nam es it t o “ Going t o lu nch.” Meanw hile he add s an int erv ent ion it em in t he applicat ion. Aft er t hat , he get s n ot ificat ions t o rem ind him t o eat m ore salad in st ead of m eat w hen he goes t o t he cafet eria. Concept an d Appr oach To t h is end, we present here t he “ fingerpr int s” t echn iq ue, w h ich is a m et aphor for hu m an beh avior rout ines or pat t erns ( e.g. , g oing t o w ork , g oing t o t he cafet er ia, or dr ink ing w at er) . I n t his w ork, w e use a t hree- st ep appr oach t o design a syst em in order t o n ot on ly enable hig hly per son alized and t im ely int ervent ion s b ut also t o r educe t he bu rden on t he u ser of logging d at a. Our w ork is based on t w o assum pt ions: 1) people h ave d aily r out in es w h ich can be det ect ed by t heir m obile dev ices 2 ) t hese rout ines car ry pot ent ially relevant “ fingerpr int s” t o t rigger behav ior ch ange. I n We design our approach based on t hree consider at ion s: applicabilit y, com plex it y , and t he b urden on t he user . The dat a u sed in our m et h od in cludes t he locat ion ( from t he GPS or t he net work) , t he Wi- Fi SSI D ( t h e nam e of t h e Wi- Fi pr ov ider) , t he physical act iv it y ( w alk ing, runn ing, st ill, t ilt ing, on a bike, on a veh icle, or on foot ) and t he t im e st am ps. We m odel each “fingerprint ” as a sequence of stat e t ransit ions. Each st ate is described as a collect ion of sensor dat a w it h the t im e stam p, e.g., { Locat ion, Wi- Fi connect ion, Physical Act iv ity, Tim e-St am p} . To make the system energy- efficient, instead of calculating in a fixed time interval with a f ixed time window, we collect and process the data on ly when the state changes. The reason is that for persons like students or office workers, the smartphones are not often in motion . Compared to a time interval- based data collection method, the system avoids calculating redundant data . Stepl Most importantly, we want to reduce the burden on the user of manually logging activities to train the classifier. To this end, we do not simply adopt the traditional machine learning strategy of other researchers in the field [5], which asks users to label the data repeatedly. Instead, we f irst collect sensor data for a period of time, followed by a data pre-processing step to find potential "fingerprints." These two steps are executed automatically. The users can then name the potential "fingerprints" in one setting, whereby a collection of labels are gathered for the classifier training. Step2 Step3 Figure 1: The workflow of the ~fingerpts" system. Step 1 is data collection; step 2 is ~fingerpts" In the Data Pre-Processing step, we regard the task of f inding potential "fingerprints" as a common substring problem. We use the generalized suffix tree based algorithm to solve the problem. The system generates the string by encoding the location data. After users naming the potential "fingerprints", the system divides each named "fingerprint" into three sections (the beginning, the middle, and the end) and then uses decision tree model to train the classifier. generation; step3 is ~fingerpts." real-time detection of the In the step of Data Naming, the user is provided with an interface to show the found "fingerprints" and asked to name the activities they are interested in. After labeling the "fingerprints," the classifier is trained with the labels. In next step, the system starts to detect the "fingerprints" and to provide interventions to the users. The classifier is updated according to the users' feedback. The data is continually collected and activities matching the "fingerprints" are labeled when they appear. The workflow of the "fingerprints" system is shown in Figure 1. Feasibility Study We conducted a feasibility study to collect data for offline analysis. We developed an android application (see Figure 2) to collect the data as described in the previous section. We invited three university students as the participants. The study had three sessions, including a participant survey session, a data collection session, and a data validation session. In the participant survey session, we asked all the participants to describe their daily routines in a questionnaire by email. Based on their answers, we selected three common routines and incorporated the functionality of logging these routines in the application. In the data collection session, the participants installed the application on their smartphones and started the background service. This session lasted for four working days. We allowed the participants to stop the service at any time for the sake of privacy . We also asked the participants to manually log the three pre-defined routines by pressing a "START LABEL" button and a "STOP LABEL" button when they start and stop a routine, respectively. Manual logging is not included in the workflow we designed as shown in Figure 1. We added this part in the feasibility study in order to get ground truth data for evaluating the results of our algorithm . In the data validation session, we asked the participants to validate their logging data by checking if the location information and their logging data are matched. From t he locat ion dat a, w e filt ered out t h e em pt y ent ries ( no locat ion prov ided) . Then w e processed t h e dat a t o fin d t he pot ent ial “ fingerpr int s” . We ign ore pat t erns w it h less t han four appearan ces, because daily rout ines shou ld app ear four t im es at least in four days. The resu lt s are sh ow n in Table 1. Our m et h od det ect ed all t hree r out ines for Part icipant s 1 and 2. How ever , for Part icipant 3, on ly rout ine 2 w as det ect ed. Aft er review ing t he raw d at a, t he cause w as found t o be t h at num ber of d at a ent r ies for t his part icip ant t o be m uch sm aller t han t he ot h ers. Figure 2: The user int erface of t he prot ot ype applicat ion. D at a R1 R2 R3 P1 8 13 8 27 69 P2 4 9 4 23 04 P3 0 12 0 32 7 En t rie s Table 1: The num bers of det ect ed From t he u ser logging dat a, w e also fou nd t hat t he part icipant s alway s forget t o label our pr e- defined rout ines. Th is is consist ent w it h t he relat ed w ork [ 1] , w hich is a good r eason w hy w e cann ot r ely on t he user s’ labelin g t o t rain t he classifier at t he beg in n ing of our w ork flow . ent ries of each par t icipant ( P1, P2 and P3) . R1 is “ going t o t he universit y ” ; R2 is t o “ going t o have a m eal” ; R3 is “ going hom e” . 1. Sunny Con solv o, Predrag Klasnj a, Dav id W. McDon ald and Jam es A. Landay . 2 01 4. Desig n ing for Healt hy Lifest y les: Design Con siderat ion s for Mobile Tech nologies t o Encourag e Consu m er Healt h and Wellness. Found at ion s and Trends® in Hum an –Com put er I nt eract ion 6, 3 –4, 16 7- 3 1 5. ht t p: / / dx.doi.org/ 10. 15 61/ 11 0 00 00 04 0 2. Anind K. Dey, Raffay Ham id, Chr is Beck m ann, I an Li, and Dan iel Hsu. 2 00 4. a CAPpella: progr am m ing by dem onst r at ion of cont ext - aw are app licat ions. I n Pr oceedings of t he SI GCHI Conference on Hum an Fact or s in Com put ing Sy st em s ( CHI '0 4) , 33- 40. ht t p: / / dx.d oi. org/ 1 0.1 14 5/ 9 85 69 2.9 85 69 7 3. BJ Fogg. 20 09. A behav ior m odel for persuasive desig n. I n Pr oceedings of t he 4t h I nt ernat ional Conference on Persuasive Tech nology ( Per suasive '0 9) , Art icle 4 0, 7 pages. ht t p: / / dx.doi.org/ 10. 11 45/ 15 4 19 48.1 54 19 9 9 4. Andreas Krau se, Asim Sm ailagic, and Dan iel P. Siew iorek. 2 00 6. Cont ext - Aw are Mob ile Com put in g: Lear ning Cont ext - Dependent Per sonal Preferences from a Wearab le Sen sor Arr ay. I EEE Tr ansact ions on Mobile Com put ing 5, 2 ( Febru ary 20 06) , 1 1312 7. ht t p: / / dx.doi.org/ 10. 11 09/ TMC.2 00 6.1 8 5. Brian Y. Lim and An ind K. Dey. 2 01 0. Toolkit t o support int elligibilit y in cont ext - aw are app licat ions. I n Pr oceedings of t he 1 2t h ACM int ern at ion al con ference on Ubiquit ous com put ing ( Ub iCom p '1 0) , 1 3- 22. ht t p: / / dx.doi.org/ 10. 11 45/ 18 6 43 49.1 86 43 5 3 6. Tauh idur Rahm an, Mary Czerw insk i, Ran GiladBach rach, and Pau l Joh ns. 2 01 6. Predict ing " About t o- Eat " Mom ent s for Just - in- Tim e Eat ing I nt erv ent ion. I n Pr oceeding s of t he 6t h I nt ernat ional Confer ence on Digit al Healt h Conference ( DH '1 6) , 14 1- 1 50. Conclusion a nd Fu tu re W ork I n t h is paper, we present t he “ fingerpr int s” t ech niq ue t o support m ob ile healt h int ervent ion s by det ect ing user s' daily rout ines and m eaning fu l m om ent s. We show our concept and appr oach by w h ich w e aim t o m ake our sy st em pract ical, lig ht - w eight and less burden on u sers of m anual logging. I n our feasib ilit y st udy, w e t est ed our assum pt ions w it h a sm all n um ber of part icipant s using t he sensor dat a from st ock sm art phones. appearances of t hr ee rout ines ( R1, R2 and R3) and t he dat a Re fe ren ces The resu lt s of t he st udy were prom ising and based t hereon, w e w ill im plem ent our w h ole w ork flow and cond uct t he evaluat ion, especially from t h e per spect ive of user exper ien ce.