Managing and evaluating digital repositories

Managing and evaluating digital repositories Page 1 of 28 VOL. 13 NO. 1, MARCH 2008 Cont e n t s | Au t h or in de x | Subj e ct inde x | Se a r ch | H om e M a n a gin g a n d e v a lu a t in g digit a l r e posit or ie s Ale sia Zucca la Ra t he na u I nst it ut e , Scie nce Syst e m Asse ssm e n t Un it , Post bu s 9 5 3 6 6 , 2 5 0 9 CJ D e n H a a g, N e t h e r la n ds Cha r le s Oppe n he im a n d Ra j ve e n D h ie n sa D e pa r t m e nt of I nfor m a t ion Scie nce , Lou gh bor ou gh Unive r sit y, Lou gh borou gh , Le ice st e r sh ir e , LE1 1 3 TU, Un it e d Kin gdom Abstract I n t r odu ct ion . We exam ine t he role of t he digit al reposit ory m anager, discuss t he fut ure of reposit ory m anagem ent and evaluat ion and suggest t hat library and inform at ion science schools develop new reposit ory m anagem ent curricula. M e t hod. Face- t o- face int erviews were carried out wit h m anagers of five different t ypes of reposit ories and a Web- based survey was carried out wit h users. The LexiURL Web link evaluat ion soft ware provided a 'webom et ric' basis for invest igat ing pot ent ial users online. Re sult s. Few m anagers had received any form al t raining. The reposit ories were relat ively new and web st at ist ics had been used by t he m anagers t o m onit or t heir success. The LexiURL analysis indicat ed t hat t he net works associat ed wit h t he reposit ory sit es were predict able and m ade sense t o m anagers because expect ed co- links and known links appeared in t he net work diagram s. Users of t he reposit ories discovered t hem t hrough friends and colleagues. Con clu sion . Digit al reposit ories require ongoing evaluat ion t o det erm ine t heir qualit y and new direct ions for growt h. A LexiURL analysis could be carried out by m anagers every four t o six m ont hs and used as a com plem ent t o t ransact ion log file analyses. Reposit ory m anagers will need form al t raining in t he fut ure and we suggest a set of m odules t hat would be suit able for a specialist program m e. CH AN GE FON T I n t r odu ct ion A significant am ount of digit al reposit ory research and developm ent act ivit y is t aking place in t he Unit ed Kingdom , m uch of which is associat ed wit h t he Joint I nform at ion Syst em s Com m it t ee's ( JI SC) Focus on Access t o I nst it ut ional Reposit ories ( FAI R) program m e. I n 2005, JI SC init iat ed anot her call for reposit ory proj ect s wit h t he int ent ion of: bringing t oget her people and pract ices from across various dom ains ( research, learning, inform at ion services, http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 2 of 28 inst it ut ional policy, m anagem ent and adm inist rat ion, records m anagem ent and so on) t o ensure t he m axim um degree of coordinat ion in t he developm ent of digit al reposit ories, in t erm s of t heir t echnical and social ( including business) aspect s. ( Joint I nform at ion Syst em s Com m it t ee 2006) . I n addit ion t o t he JI SC's call for proj ect s, a num ber of t echnical archit ect ure, m et adat a st andards, copyright and int eroperabilit y issues have been ident ified as crit ical t o t he developm ent , m anagem ent and sust ainabilit y of digit al reposit ories ( Day 2003; Gadd et al. 2003a, b, c; Guy et al. 2004; McLean and Lynch 2004; Medeiros 2003) . Academ ics and ot her professionals are increasingly occupied wit h discussing t hese issues online ( Andrew 2006; Harnad 2006a) , t o share all of t he lat est inform at ion concerning pract ical and t echnical challenges. Here, we focus on t he unique role of t he digit al reposit ory m anager and invest igat e t he evolut ion of t his role in light of current research and pract ice. Qualit at ive and quant it at ive dat a t aken from a JI SC digit al reposit ory proj ect ent it led 'User Needs and Pot ent ial Users of Digit al Reposit ories: An I nt egrat ed Analysis' are used t o enhance our discussion and give support t o t he idea t hat new program m es will soon be needed t o help t rain t he growing num bers of professionals engaged in reposit ory m anagem ent . D e fin in ing r e posit or y a n d t h e r ole of t h e r e posit or y m a n a ge r Cent ral t o t he role of a reposit ory m anager is t he purpose or prim ary at t ribut es of t he reposit ory. Specialised reposit ories are being developed for different purposes, for exam ple, e- print s reposit ories, e- learning reposit ories, dat a reposit ories, et hesis reposit ories and subj ect - based reposit ories; t herefore, a useful definit ion of a reposit ory needs t o encom pass all t ypes. Crow ( 2002) , Gibbons ( 2004) and Heery and Anderson ( 2005) offer very sim ilar definit ions. Crow ( 2002: 16) em phasizes t he value of open access and creat ing a digit al reposit ory 'wit h few if any barriers t o access'. Gibbons explains t hat t he com m on feat ure of digit al reposit ories is t hat t hey 'cont ain digit al cont ent ', adding: The range of different t ypes of digit al cont ent can be vast , including t ext , audio, video, im ages, learning obj ect s and dat aset s. The m at erial m ay be born digit al or of a physical m edium t hat has been digit ized, such as scanned im ages. ( Gibbons 2004: 6) Heery and Anderson ( 2005: 2) specify t hat cont ent is deposit ed in a reposit ory, whet her by a creat or or t hird part y and t hat t he reposit ory archit ect ure m anages cont ent as well as m et adat a and offers a m inim um set of basic services, for exam ple, put , get , search and access cont rol. Moreover, a reposit ory m ust be sust ainable, t rust ed, well support ed and well m anaged. The role of a chief reposit ory m anager should be t o recognise and define t he raison- d'êt re of t he reposit ory so t hat deposit ors, users and m em bers of t he public will be fam iliar wit h it s exist ence and purpose. Once t hese users know about t he reposit ory, it s benefit s m ust be advert ised; hence, t he m anager needs t o underst and t he im port ance of est ablishing a prom ot ional program m e. The reposit ory m anager should also have a clear sense of what const it ut es t he success or failure of a growing reposit ory, including long- t erm financing, inst it ut ional support for docum ent cont ribut ion m andat es, and t im e t o encourage individuals t o cont ribut e m at erials. He needs t o be well educat ed on t he t echnical aspect s of t he reposit ory's const ruct ion, including it s underlying soft ware, st andards adopt ed for m et adat a and st andards for int eroperabilit y. The m anager m ust t hink about what t he reposit ory can do for it s cont ribut ors and creat e an appropriat e evaluat ion program m e using inform et rics, bibliom et rics, log file analyses, or webom et ric analyses when needed. The m anager will have t o keep up t o dat e wit h current inform at ion science research, t hink about how t o im plem ent user- assist ance program m es and m ake sure t hat deposit ed, accessed and used m at erials do not infringe copyright . Clearly, t he developm ent of a reposit ory requires a great deal of work; hence, wit hout a crit ical support t eam of inform at ion and com put ing specialist s, t he digit al reposit ory is not likely t o be successful. Som e ba ck gr ou n d The role and core com pet encies of library professionals have been t he subj ect of r ecent debat e ( Chan 2006; Mullins and http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 3 of 28 Linehan 2005; Sargeant and Harrison 2004) and sim ilar at t ent ion has been paid t o t he role of digit al t echnologies ( Choi and Rasm ussen 2006; Hast ings and Tennant 1996; Spink and Cool 1999; Perry 2005) . When an individual chooses t he professional role of librar ian we t rust and underst and s/ he has a core background of specialised t raining. Reposit ory m anagers generally have not necessarily had specialised t raining, or t raining in library or inform at ion science. The work of a librarian can be m easured against t he t heoret ical underpinnings and st andards of librarianship, whereas reposit ory m anagem ent is evolving from a new vision: a new scholarly com m unicat ion m ovem ent based on t he philosophy and st andards of open access ( see Jacobs 2006) . The t erm open access has been given a variet y of definit ions and it s m eaning is st ill evolving; however, following t he Budapest Open Access I nit iat ive m eet ing, a definit ion was produced: First , open access works are freely available. Second, t hey are 'online', which would t ypically m ean t hat t hey are digit al docum ent s available on t he I nt ernet . Third, t hey are scholarly works... Fourt h, t he aut hors of t hese works are not paid for t heir effort s. Fift h, as m ost but not all aut hors of peer- reviewed j ournal art icles are not paid and such works are scholarly, t hese art icles are ident ified as t he prim ary t ype of open access m at erial. Sixt h, t here are an ext raordinary num ber of perm it t ed uses for open access m at erials; users can copy and dist ribut e open access works wit hout const raint . Sevent h, t here are t wo key open access st rat egies: self- archiving and open access j ournals. ( Bailey 2006: 15) Self- archiving is one st rat egy, which Harnad ( 2003; 2006b) describes as t he 'green rout e t o open access'. When an aut hor provides 'lim it less free “ eprint s” of elect ronic versions of t heir own final draft s on t heir own inst it ut ional Websit es for all pot ent ial users Web- wide who cannot afford t he j ournal version', he or she is said t o be 'self- archiving'. ( Harnad 2006b: 1) . Evidence has been produced t o show t hat open archiving of papers result s in an increase in cit at ions ( Brody and Harnad 2004; Brody et al. 2006; Haj j em et al. 2005; Kurz et al. 2004; Moed 2006) and should cont inue if open access advocat es convince scholars t hat t his is one of t he m ost im port ant rewards associat ed wit h t heir part icipat ion. The reasons for t his ar e uncert ain, but t here seem s t o be a consensus t hat an advant age exist s. I n t he absence of a core t raining program m e for reposit ory m anagers, reposit ory developm ent work is now falling int o t he hands of reference librarians ( Chan et al. 2006) , alt hough Koehler ( 2006: 19) not es t hat t he 'process of organizing OA m at erials can com plicat e t he funct ions of library t echnical services' as well. 'I n addit ion t o quest ioning where art icles m ust reside, one m ust decide who is responsible for m igrat ing and archiving t he works and who will resolve t he links'. Genoni ( 2004: 300) also writ es about t he need for librarians t o 'approach t he t ask of cont ent developm ent in reposit ories by applying som e of t he procedures and skills associat ed wit h collect ion m anagem ent '. When academ ic reference librarians are asked t o init iat e reposit ory proj ect s it is because t hey are engaged in public service ( e.g., a liaison t o academ ic facult y) and sit uat ed in inst it ut ions t hat need t o build a reposit ory. The library adm inist rat ion at t he Universit y of New Mexico, Healt h Sciences Library and I nform at ics Cent er is one exam ple: Faced wit h a decision about which unit of t he library should t ake on t he responsibilit y of planning for and im plem ent ing t he inst it ut ional reposit ory, t he library chose t he Reference and User Support Service unit ... based on t he library's view t hat t he web is a public service rat her t han a collect ion…. The Elect ronic Services Developm ent Librarian posit ion had been creat ed and t hat posit ion included t he [ library's] web sit e as a m aj or com ponent . ( Philips et al. 2005: 3) and, furt her: Placing responsibilit y for t he inst it ut ional reposit ory in t he Reference and User Support Services unit was a logical out growt h of t he philosophy, t he organizat ional st ruct ure and t he personal int erest s and skills of t he incum bent s in t he posit ions. ( Philips et al. 2005: 3) http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 4 of 28 Scholars who em phasize t he 'changing roles reference librarians' are aware of t he fact t hat 'libraries have m oved beyond a cust odial role t o cont ribut e act ively t o t he evolving scholarly com m unicat ion process' ( Crow 2002; cit ed in Chan et al. 2005: 270) . Academ ic librarians, in t he t radit ional sense, t ake on t he dut y of keeping facult y and st udent s at a college or universit y inform ed about recent acquisit ions, including what is new and available in a part icular discipline and including digit al resources. They also t each bibliographic inst ruct ion ( e.g., effect ive online search skills) as part of t heir inform at ion lit eracy program m es. When t he Hong Kong Universit y of Science and Technology Library first creat ed it s inst it ut ional reposit ory, a dram at ic shift occurred in t erm s of what was expect ed of t heir academ ic library professionals. All were ...engaged in all st ages of it s developm ent : t he definit ion of goals and scope, evaluat ion of syst em and cont ent , form ing st rat egies and procedures, int erpret ing publishers' policies, cont act ing and servicing facult y m em bers, acquisit ion of cont ent and prom ot ional effort s. ( Chan et al. 2005: 271) Chan et al. ( 2005: 271) adm it t hat 't he learning curve for [ t he st aff was] st eep. Cert ain individuals 'j uggled m ult iple roles' and 'som e of t hese roles [ were] ext ensions of exist ing ones; ot hers [ were] brand new'. The st aff at t his Library t ook t he opport unit y t o learn about reposit ory m anagem ent as t hey progressed and in m any respect s it was a t rial and error process. For inst ance, t he reference librarians e- m ailed all facult y m em bers and invit ed t hem t o subm it papers t o t he new reposit ory, but 't he response was pat het ic' ( Chan et al. 2005: 275) . They resort ed t o t he new j ob of scanning all depart m ent al hom epages and t hose of individual facult y m em bers t o see how m any had post ed fullt ext publicat ions on t he Web ( 89 out of 450) . I n t he end, perm ission was obt ained t o post 150 docum ent s, but t he reference librarians had t o t ake up an advocacy role, which requir ed t hem t o 'check individual publisher's policies or negot iat e for self- archiving right s' ( Chan et al. 2005: 277) . Advocacy work for open access t o a universit y's research out put does not const it ut e t radit ional academ ic reference work; t hus, in t his case, it is not clear how effort put int o t he developm ent of t he new reposit ory affect ed t he norm al reference services. I f t he reference librarian's role is evolving and changing, t hrough involvem ent in reposit ories, t o what degree should reposit ory m anagem ent becom e an im port ant part of an inform at ion science school's curriculum and when should t his curriculum becom e part of t he agenda? The answer rest s upon t he degree t o which t he first digit al reposit ories are successful. Jones et al. ( 2006: 17) indicat e t hat t he inst it ut ional reposit ory is 'a st rong and im port ant new idea' for academ ic organizat ions because it s 'appeal lies in t he idea of “ groundedness” ; inst it ut ions are t hem selves t he ground from which em erge out put s of research – ideas, proposals, hypot heses, experim ent s, dat a and report ed result s'. Conversely, t he aut hors not e t hat , ...it is not yet clear whet her inst it ut ional reposit ories will t ake root and flourish... The concept of inst it ut ionalit y is an increasingly fragile one when we consider digit al cont ent and digit al libraries and we, t herefore, m ust ask whet her we should be developing inst it ut ional reposit ories at all. ( Jones et al. 2006: 17) Jenkins and Breakst one ( 2005) provide som e int erest ing ideas regarding reposit ory prom ot ional work and suggest t hat librarians avoid library j argon when prom ot ing a new reposit ory, since it is bet t er t o use t erm s t hat are m ore readily underst ood and have m eaning for t he t arget audience ( a sim ilar suggest ion was m ade by Gibbons ( 2004) ) . At t he Universit y of Oregon, Scholars' Bank was t he chosen t erm . Likewise, Ohio St at e Universit y decided t o focus on creat ing a Research Bank or Knowledge Bank for t heir academ ic com m unit y ( Rogers 2003) . Jenkins and Breakst one ( 2005: 317- 318) also direct librarians and reposit ory developers 't o posit ion t he reposit ory as com plem ent ary t o t radit ional publishing'. Whilst t his idea of com plem ent arit y sounds posit ive, t hought has t o be given t o what academ ics previously needed and expect ed from t he t radit ional publishing indust ry and how t his has changed wit h t he developm ent of reposit ories. Jones et al. rem ind us t hat in http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 5 of 28 ...pre- digit al t im es, when researchers wrot e up t heir result s for publicat ion, t hey would have been post ed t o a publisher – t he only agent wit h t he t echnology t o present t he finished paper in pleasing form and t o reproduce it ... I n t he digit al age, t he present at ion and reproduct ion funct ion do not require t he int erm ediat ion of a publisher. ( Jones et al. 2006: 18) I f any int erm ediary work is t o be carried out in t he digit al reposit ory age, it should be t hought of in t erm s of workflow and adm inist rat ion. To properly m anage a reposit ory, all persons associat ed wit h it s developm ent and m aint enance m ust be prepared, ...t o exam ine how [ t o] st ruct ure t he adm inist rat ive t asks so as t o produce individual m odules, or workflow st eps, which t hen allow for a st andardised t reat m ent of t he relevant elem ent s of t he syst em . ( Jones et al. 2006: 86) For an inst it ut ional reposit ory, t here is a predefined list of workflow areas wit h specific t asks t hat need adm inist ering:     Subm ission ( workflow generally ent ered int o by one person, alt hough som et im es a m ediat ed subm ission m odel is appropriat e) :  procedures for acquiring cont ent from cont ent creat ors. Post - subm ission ( workflow involving m any people. e.g., m anagers, collect ion adm inist rat ors, cat aloguers) :  a cont ent verificat ion procedure;  a cat aloguing st ep in which m et adat a are verified and augm ent ed. Preservat ion ( subm ission workflow cont r ibut es t o long- t erm preservat ion) :  short and long- t erm st orage and preservat ion procedures. St ruct ural Managem ent ( a workflow of disparat e t asks which do not necessarily follow in order) :  user and user group m anagem ent ;  archive st ruct ure and cont ent m anagem ent ;  policies and aut horisat ions ( Jones et al.: 86- 92) . A significant port ion of t he digit al reposit ory lit erat ure dem onst rat es a j ust ified concern wit h copyright laws and ot her aspect s of int ellect ual propert y right s, such as m oral right s and dat abase right s ( Gadd et al. 2003a, b, c; Gladney 1999) . Wit hin t his area of responsibilit y, reposit ory m anagers are advised t o 'exam ine t he needs of each of t he m ain st akeholder groups involved in t he creat ion and dissem inat ion of [ scholarly works, m at erials, or dat a] ' ( Jones et al. 2006: 140) . St akeholder groups can include aut hors, inst it ut ions, funding bodies, publishers, users, libraries and m em bers of t he general public and each will have t heir own priorit ies. An aut hor's priorit y, for exam ple, is t o have ot her individuals access, m ake use of and cit e t heir work, for scholarship and learning; t hus s/ he is likely t o be concerned wit h j ust cert ain aspect s of copyright ( i.e., t hat his or her nam e should be associat ed wit h t he work and t he work should not be am ended or exploit ed com m ercially, wit hout perm ission) . At least one m em ber of a reposit ory m anagem ent t eam will have t o discuss t he individual elem ent s required for 'a com prehensive deposit and end- users licence agreem ent , including a deposit or's declarat ion, t he reposit ory's right s and responsibilit ies and [ m at erial] re- use t erm s and condit ions' (Jones et al. 2006: 148) . Case st udies pert aining t o reposit ory m anagem ent are growing and wit h t he dawn of a new reposit ory era it is useful t o draw at t ent ion t o Ray's ( 2001: 4) not e t hat 'case st udies of library work are not prom inent in t he lit erat ure on librarianship'. Why t hen are case st udies so im port ant t o reposit ory work? I n Ray's ( 2001: 4) view 't here is a growing int erest in t he fut ure role of librarians, but it t ypically views t he product ion of new roles as linked t o t echnology'. Reposit ory developm ent work is t ransform ing t he t echnology and cult ure of scholarly com m unicat ion; hence case st udies are needed t o help inform at ion professionals bear wit ness t o t his gradual process. Pinfield et al. ( 2002) , Ashwort h et al. ( 2004) and Hey ( 2004) each writ e about what it was like t o set up inst it ut ional eprint s reposit ories at t he Universit ies of Edinburgh, Not t ingham , Glasgow and Sout ham pt on in t he UK. Pinfield et al.'s st udy explains how t he proj ect m anagem ent t eam t ried t o m ake it , http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 6 of 28 ...as easy as possible [ for researchers] t o cont ribut e. At t he beginning [ t he proj ect t eam ] allowed researchers at t he universit y t o e- m ail papers t o an archive adm inist rat or [ t hus em phasizing t hat ] t he library would do t he work. The t eam felt t hat t he academ ics [ did] not want addit ional bureaucrat ic burdens nor did t hey want t o learn new I T skills. ( Pinfield et al. 2002: 8) I n t he US, Rogers's ( 2003: 127) paper indicat es t hat 'while defining t he scope of [ Ohio St at e Universit y's] Knowledge Bank, t he Planning Com m it t ee considered st eps ot her inst it ut ions [ were] t aking t o m anage t heir digit al cont ent '. I n Aust ralia, Kennan and Wilson encourage reposit ory m anagers t o learn from research and pract ice in I nform at ion Syst em s, i.e., work associat ed wit h t he phrase requirem ent s uncert aint y. The creat ion of a reposit ory can be an increm ent al process or a result s- driven process, m eaning t hat 'ot her inst it ut ional int ellect ual capit al and addit ional funct ionalit y could be added as organizat ional change and learning t akes place, or as m ore resources becom e available'. ( Kennan & Wilson 2006: 11) The t asks associat ed wit h developing and m anaging a reposit ory are becom ing increasingly clear now t hat resources are available t o help new reposit ory m anagers adj ust t o t heir roles. Soon, t he fut ure of reposit ories and t heir success will be left t o t hose who know not only how t o develop t hem , but evaluat e t hem as well. Are digit al reposit ories fulfilling t heir prim ary obj ect ives? How are t hese obj ect ives evolving over t im e and how can we be sure t hat t hey are m eet ing t he needs of users? I n t he next sect ion of t his paper, we discuss t he findings of a Joint I nform at ion Syst em s Com m it t ee- funded Proj ect carried out in 2005 and 2006, which was designed t o evaluat e five different t ypes of digit al resources across t he Unit ed Kingdom from a m anagem ent perspect ive, a user perspect ive and from a Web- based perspect ive using a new link analysis soft ware t ool, LexiURL. The Joint I nfor m a t ion Syst e m s Com m it t e e st udy The JI SC- funded user needs st udy was init iat ed in Sept em ber 2005, short ly aft er t he im plem ent at ion of t he 2005 Digit al Reposit ories Program m e. The following public reposit ories, including one digit al library, were select ed for evaluat ion:      A digit al library – The Nat ional elect ronic Library for Healt h ( NeLH) A subj ect reposit ory – CogPrint s: Cognit ive Sciences e- Print s An inst it ut ional reposit ory – e- Print s Sot on ( Universit y of Sout ham pt on) A dat a reposit ory – The UK Dat a Archive An e- learning reposit ory – The Jorum e- learning reposit ory From a m anagem ent perspect ive, our research goal was t o acquire an in- dept h underst anding of t he creat ion rat ionale for each reposit ory, t he collaborat ive work associat ed wit h t he resource's const ruct ion, t he m anagers' st rat egies for ident ifying users and prom ot ing t he resource and t heir current approach t o using Web st at ist ics for user assessm ent . I n t he second phase, we em ployed an online quest ionnaire t o learn m ore about t he needs and percept ions of t he current users; t hat is, t he fact ors m ot ivat ing t hem t o use ( or not use) t he resource and t heir general usage experiences. Wit h t he int roduct ion of LexiURL, a new Web link evaluat ion program , t he t hird aim of t he st udy was t o provide reposit ory m anagers wit h a Webom et ric plan for invest igat ing pot ent ial users or uncovering hidden user com m unit ies, so t hat t hey m ight work t owards building st ronger links ( i.e., Web and real- world links) bet ween t hem selves and ot her relevant organizat ions or act ivit ies, at nat ional and int ernat ional levels. Re posit or y t ype s a n d m a n a ge m e n t pr a ct ice s The m anagers who agreed t o m eet wit h us for int erviews t ook an average of one and a half hours t o respond t o a set of quest ions list ed from a st ruct ured int erview schedule ( see Appendix) . The quest ions for each int erview session ( five sessions in t ot al) were t he sam e; however, short discussions occurred during our m eet ings when it was valuable t o elaborat e upon specific point s. Som e of t he m anagers m et wit h us on an individual basis and ot hers cam e t o us in t eam s of t wo or t hree people. All r em arks in quot at ion m arks t hat follow have been copied from t he int erview t ranscript s. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 7 of 28 Since t he select ed reposit ories, including one digit al library, were different in t ype, it was int erest ing t o evaluat e t hem from a com parat ive perspect ive. By choosing t o evaluat e t he NeLH, our aim was t o det erm ine if cert ain aspect s of digit al reposit ory m anagem ent could be learned from current pract ices in digit al librarianship. During t he period in which we carried out our m anagem ent int erviews we first learned about t he rat ionale behind each resource's const ruct ion. N a t iona l e le ct r onic Libr a r y for H e a lt h The Nat ional elect ronic Library for Healt h was creat ed in 1998 because of, ...a realisat ion t hat clinicians, doct ors, nurses, speech t herapist s, diet icians and all kind of t herapeut ic professionals, needed access t o inform at ion quickly ( Service m anager, NeLH) . According t o t he service m anager, healt h care professionals across t he UK som et im es find it difficult t o achieve quick and easy access t o m edical inform at ion when t hey need it . Norm ally t his is relat ed t o t he fact t hat t he library of a hospit al or m edical cent re is locat ed in a separat e wing and is not always a convenient place t o get t o in order t o do an inform at ion search. Oft en, m edical professionals are also called t o work out side a t radit ional m edical set t ing; t hereby finding t hem selves in a posit ion where it is t oo t im e consum ing t o get t o a m edical research library. When using a Web- based digit al library, a healt h care pract it ioner will only need t o be in a place where he or she has access t o t he I nt ernet ; hence, t he service was Web- based t o: ...provide clinicians wit h access t o t he best current evidence on condit ions and t reat m ent s t o im prove pat ient care ( Service m anager, NeLH) . At t he t im e it becam e available it was not 'aim ed at t he public' but 'it has a sist er service called t he NHS Direct t hat is'. Bot h were developed 'm ore or less at t he sam e t im e and 't here [ has been] a lot of cross usage … som et hing like 10- 15% '. CogPr in t s CogPrint s, t he subj ect specialt y reposit ory, was creat ed in 1997 for t he cognit ive science research com m unit y, because of t he success of t he Los Alam os physics e- print s arXiv. At our int erview wit h t he m anager, we learned t hat t her e was a background int erest 'in dem onst rat ing t hat [ subj ect specialt y reposit ories] were not j ust for physicist s' and t hat t hey could 'work for ot her disciplines'. The CogPrint s m anager was convinced t hat if t he new subj ect specialt y reposit ory grew t o be successful, it would show t hat 'archives wit h self- archive papers [ were] not j ust a special quirk of physics'. e - Pr int s Sot on e- Print s Sot on was creat ed in 2002/ 2003, at t he sam e t im e as 't he JI SC FAI R program m e was init iat ed' ( and was funded by t he sam e program m e) and short ly aft er t he ECS dat abase was creat ed at t he Elect ronics and Com put er Science depart m ent . The developm ent of e- Print s Sot on was closely associat ed wit h 't he issue of push and pull of t he open access m ovem ent ' ( Service m anagerA, Sot on) . This universit y- based reposit ory t eam felt t hat it was a 'nat ural progression in t he publishing debat e as a whole' and t hat t he creat ion of e- Print s Sot on: ...would enable t he universit y t o organize it s inst it ut ional research out put in a way t hat would allow bet t er analysis of where t he research is going ( Service m anagerB, Sot on) . UK D a t a Ar chive The UK Dat a Archive was creat ed in 1967/ 1968, because: ...t he UK research council t hought it would be a good idea [ t o creat e] a one- st op shop [ for researchers] so http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 8 of 28 t hat … rat her t han having t o individually go t o t he dat a providers, m ainly governm ent depart m ent s and com m ercial dat a providers, [ t hey] would be able t o go t o a cent ral locat ion and obt ain all t heir dat a ( Service m anager, Dat a Archive) . The not ion was t o get one, ...organizat ion brokering access agreem ent s and licensing arrangem ent s and copyright arrangem ent s rat her t han individuals having t o do t hat on a one t o one basis ( Service m anager, Dat a Archive) . Research councils have been m aj or sources of financial support for: ...dat a collect ion exercises; t herefore in order t o m axim ise secondary use of t he dat a, [ sponsored researchers have been] required t o offer dat a t o t he archive ( Service m anager, Dat a Archive) . Essent ially, 't he dat a archive' was first creat ed as an 'archive of invest m ent s, m ade by t he research councils t hem selves'. According t o t he m anager, t his reposit ory 'has becom e m ore im port ant over t he last few years … because of it s change in st at us'. Not only has it gradually becom e a digit ized resource ( since 1999) , it is also: ...a legal place t o deposit ; t he only digit al reposit ory in t he count ry t hat has legal place as st at us, [ which] m eans t hat m em bers of t he public can com e t o [ it t o] acquire digit al m at erials ( Service m anager, Dat a Archive) . Jor um The Jorum e- learning reposit ory, was creat ed in 2005/ 2006 and funded by JI SC t o host : ...cont ent creat ed for t he [ higher and furt her educat ion] com m unit y [ as well as] t o st im ulat e a com m unit y of users for t eaching resources ( Service m anagerA, Jorum ) . Out side t he Unit ed Kingdom 'ot her t eaching and learning reposit ories' have been creat ed, ...but none t hat were doing quit e t he sam e as Jorum . MERLOT is… anot her int ernat ional reposit ory, [ which is] essent ially a library cat alogue syst em where people can com e and search for cont ent but t he cont ent isn't cont ained wit hin t he reposit ory. This is not t he case wit h Jorum , since… it houses m et adat a records t hat describe t he cont ent t hat can be found elsewhere, but it can also be held in t he reposit ory it self ( Service m anagerA, Jorum ) . Colla bor a t ive w or k When we asked our int erviewees t o provide a brief explanat ion of who was or is current ly involved in t heir proj ect , all confirm ed t he im port ance of collaborat ion or t eam work. CogPrint s, for exam ple, was: ...creat ed by an Elect ronics and Com put er Science PhD st udent at Sout ham pt on Universit y. The second version, post Open Access I nit iat ive ( OAI ) , was rewrit t en by anot her PhD st udent t o m ake CogPrint s OAI com pliant . The t hird version [ was] t aken over by anot her PhD st udent at Sout ham pt on ( Service m anager, CogPrint s) . and since t hen CogPrint s has been t he proj ect of, http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 9 of 28 a very savvy populat ion of com put er scient ist s and pret t y good hardware resources ( Service m anager, CogPrint s) . When it was t im e for t he Universit y of Sout ham pt on t o develop anot her, m uch larger e- print s archive, a proj ect t eam was form ed by t he Sout ham pt on Oceanography Cent re library, which included t he Cent re, t he School of Elect ronics and Com put er Science and I nform at ion Syst em s Services. The m em bers of t his library proj ect t eam found t hat 'it was [ m uch m ore of a] collaborat ive effort wit hin [ different part s] of t he inst it ut ion'. We were t old t hat : ...aft er a period of init ial developm ent ... backing from t he Deput y Vice- Chancellor for Research was secured [ because] t here was a furt her 'buy - in' from t he inst it ut ion regarding [ t he reposit ory's value for] t he UK Research Assessm ent Exercise. I t was possible t o see how a reposit ory would help m anage t he Research Assessm ent process and aid t he m anagem ent of inform at ion m ore generally in relat ion t o research ( Service m anagerA, Sot on) . I de nt ifying a nd unde r st a nding u se r s To populat e a digit al reposit ory wit h useful m at erials, a professional developm ent t eam needs t o ident ify and sufficient ly underst and t he needs of t heir service's prim ary users. At t he Nat ional elect ronic Library for Healt h, a num ber of user groups were ident ified from t he pilot work, when t he m anagem ent t eam ...had panels of people reviewing m at erial and com ing t o som e sort of consensus [ regarding t he m at erial's value] . The user groups consist ed of a wide range of people, including doct ors, nurses, various ot her allied healt h professionals and library and inform at ion workers as well ( Service m anager, NeLH) . All of t he users have now becom e key part ners because t hey act as advocat es on [ t he Nat ional elect ronic Library for Healt h's] behalf by get t ing people t o use t he library and t hey also do a lot of t raining… in lit erat ure searching, for exam ple and use of dat abase ( Service m anager, NeLH) . Through t hese user groups t he m anagem ent t eam has ident ified a key qualit y resour ce t hat t he clinicians feel t hey need t o access quickly. Hit t ing t he Headlines, for exam ple, ...is a review of t he coverage of healt h issues in t he press. Two or t hree t im es a week, a st ory is picked up from t he press and exam ined, conclusions are t hen drawn as t o t he validit y or ot herwise of t he newspaper report ing. Clinicians find t his very useful as pat ient s pick up on t hese st ories from t he TV or newspapers and oft en clinicians are not aware of what t he posit ion is. So it helps clinicians t o help pat ient s ( Service m anager, NeLH) . At e- Print s Sot on, t he developm ent t eam , ...want ed t o capt ure t he whole out put of t he Universit y, but 'saw t hat [ t hey] < > needed t o st art in a specific area. Research was [ t heir] key focus, [ and t his included] conference papers, post ers, proj ect report s and all t he different t hings t hat research encom passes ( Service m anagerA, Sot on) . The researcher was t he prim ary user t hat t hey had in m ind; t herefore one m anagem ent int erviewee said: 'what t he researcher t hinks is im port ant is what goes int o t he reposit ory' ( Service m anager C, Sot on) . Anot her int erviewee added: When we t alk about users we m ean… people who are deposit ing t he work, such as aut hors. Users were want ed from a spread of areas across t he universit y but we st art ed wit h [ t hose] we knew were int erest ed. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 10 of 28 Ot her groups were t arget ed which would set good exam ples for t he rest of t he universit y. For exam ple, Educat ion was t arget ed as t hey would not have a dat abase t hat t hey could regularly deposit m at erial int o, so t hey would be encouraged t o self archive via t he reposit or y and hence show ot her facult ies t hat it is a good idea t o be proud of t heir research and have it m ade visible ( Service m anager B, Sot on) . Pr om ot ion a l w or k Aft er a reposit ory is creat ed, people are expect ed t o becom e users; however, new users will not necessarily recognize a service's value unless it is sufficient ly publicised. At t he UK Dat a Archive leaflet s are available t o t he general public concerning all branches of it s service: Publicit y t akes on a variet y of form s, including t he dist ribut ion of hard copy docum ent s, elect ronic docum ent s and specialist docum ent s aim ed at specialist audiences. The publicit y m at erials inform users and pot ent ial users – e.g., t he Archive's annual report – but t hey also but t hey serve anot her purpose of showing sponsors what [ t he m anagem ent t eam ] is act ually doing ( Service m anager, Dat a Archive) . Users of t he Dat a Archive are invit ed t o regist er and provide cont act det ails so t hat t hey can access all m at erials. A newslet t er is available in hardcopy and as a .pdf version on t he Web. The Archive has '20,000 regist ered users, but [ approxim at ely] only 4000 have asked for a hard copy' ( Service m anager, Dat a Archive) . Mailing list s are also used t o inform users of new releases of dat a and a lot of prom ot ional m at erial such as paper brochures are produced and dist ribut ed at workshops and conferences. The Jorum e- learning reposit ory launched it s resource for public use in t wo st ages. First , new deposit ors and cont ribut ors were given an opport unit y t o becom e fam iliar wit h t he reposit ory ( in Novem ber 2005) , t hen, short ly aft er, ...t he user service, [ which allows] people t o download cont ent , went live in January 2006. The t wo separat e services were st aggered slight ly t o allow som e cont ent t o build up ( Service m anagerA, Jorum ) . Throughout t he t wo launches, art icles were writ t en, newslet t ers were produced and m ailing list s were t arget ed. The Jorum e- learning reposit ory is also prom ot ed at event s, som e on invit at ion; ot hers organized by t he m anagem ent t eam . We prom ot e t he service t o e- learning, I LT people, learning resource st aff in inst it ut ions. We do t his in a variet y of ways in at t em pt t o get at end users, so we prom ot e t o t he right people in t he right places t o encourage upt ake ( Service m anager B, Jorum ) . Wit h respect t o user t raining, ...a t rain- t he- t rainer approach is used, whereby t raining and out reach event s are held all over t he count ry t o give an overview of what Jorum is and showcase som e of t he [ deposit ed] m at erials. These are t ypically half day event s, for int erm ediaries who will in t urn pass t he inform at ion on t o end users. The int erm ediaries are provided wit h t he resources t o deliver sessions t o users at t heir inst it ut ions ( Service m anager B, Jorum ) . M e a su r ing succe ss Since m any different assessm ent program m es and t ools m ay be used t o m easure t he success of a new digit al resource, one of our obj ect ives was t o ask t he library/ reposit ory m anagers if and how t hey had been obt aining act ionable inform at ion from t he Web t he bet t er t o underst and users. The Nat ional elect ronic Library for Healt h service m anager dem onst rat ed a high degree of awareness regarding his users: http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 11 of 28 We know t hat [ t hey are] in m any cases are overworked, except ionally busy and have a num ber of com pet ing priorit ies; t herefore our st rat egy is really t o t ry and sell NeLH t o t hem , by t elling t hem what 's in it for t hem . The key m essages are t hat we're always available, t hat we're easy t o find and you can find t he inform at ion wit hin a few m inut es of going ont o our sit e ( Service m anager, NeLH) . Asked if usage st at ist ics were collect ed for assessm ent purposes: Yes, we do, on a m ont hly basis. The st at ist ical soft ware used t o t rack users is called WebTrends® and it enables inform at ion such as what are t he m ost visit ed pages, t he average t im e spent on t he sit e, ent ry and exit pages, so it enables, t o a cert ain ext ent , t he m apping of a users' j ourney t hrough t he sit e ( Service m anager, NeLH) . We asked if t he service t races where users com e from and he said: Yes, Google was one of t he highest ent ry point s t o [ t he Nat ional elect ronic Library for Healt h Websit e] ( Service m anager, NeLH) . Regular use of t he service, ...breaks down som et hing like 40% General Pract it ioners ( GP's) , 30- 35% nurses and 15% professions allied t o m edicine. The rem ainder is st udent s and t he general public. The st udent s are from a variet y of relat ed areas, such as life sciences ( Service m anager, NeLH) . We asked if t he m anagem ent t eam had com e across any benefit s t o current or new users: Yes, we have and do. Success st ories are a key part of NeLH's public relat ions. For exam ple, we m ake a point of publicizing t he fact t hat som eone saw som et hing on our sit e t hat direct ly benefit ed or cont ribut ed t o pat ient care. Som e individuals have said: I 've changed, or im proved m y pract ice t hrough som et hing I 've read. Test im onials of t his nat ure dem onst rat e t hat people are finding t he Nat ional elect ronic Library for Healt h useful. Som e people volunt eer t his inform at ion t hrough t he feedback facilit ies available on t he sit e. Posit ive feedback is received on what people found on t he sit e, for exam ple: I was able t o do t his, because I found t his. Ot her inform at ion is sought by asking client s in t he user com m unit y and t he library com m unit y and t hey relay t he feedback t hat users have given ( Service m anager, NeLH) . The e- Print s Sot on m anagem ent t eam spoke about collect ing som e usage st at ist ics for a user assessm ent , but adm it t ed t hat t his has not been a m aj or part of t heir focus yet . Yes, we have done a lit t le bit of t his, [ but ] our m ain focus has been t o work on t he [ developm ent of] t he reposit ory. We are very conscious about t he fact t hat we need t o see and show t he vice chancellor som e good st at ist ics. At t he m om ent st at ist ics are m odest but t hey will be m uch m ore sophist icat ed and will t ie in wit h ot her st at ist ics for ot her reposit ories around ( Service m anager A, Sot on) . When asked, 'Who have you ident ified recent ly as t he m ain users of E- Print s Sot on?' t he reply from anot her int erviewee was: Academ ic users [ i.e., facult y] wit hin t he Universit y use it for t heir own reasons, whet her it is t o creat e a bibliography or see what ot her people are doing. There are also users from out side t he universit y, int ernat ionally. We know t his because we get e- m ails from all over t he world, part icularly in nursing ( Service m anager C, Sot on) . http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 12 of 28 The CogPrint s m anager m ent ioned t hat t his reposit ory's user base was locat ed worldwide and t hat a m aj orit y could be ident ified as 'alm ost cert ainly academ ics'. He also st at ed t hat t he subj ect - based reposit ory was 'not t he kind t hat t he laym an would be part icularly int erest ed in'. I n t erm s of collect ing act ionable inform at ion from t he Web, t he m anagem ent t eam at CogPrint s has im plem ent ed an online syst em for collect ing Web st at ist ics, but t he m anager provider sat isfact ion was m ore relevant t han user sat isfact ion: The relevant quest ion is how do you get t he 85% of t he non- providers t o be providers, so t hat t hey can get t he enhanced im pact . CogPrint s should not be looked at ... in fact , open access [ t o published art icles] should not even be looked at from a user st andpoint ; it should be looked at from a provider st andpoint ( Service m anager, CogPrint s) . The Jorum e- learning m anagem ent t eam said t hat t he collect ion of usage st at ist ics was 'one of t he t hings t hat [ t hey were] current ly looking at '. One of t he int erviewees st at ed: ...current ly we are collect ing st at ist ics on who is logging on t o t he service, t he num ber of downloads, et c. ( Service m anager A, Jorum ) . This respondent was able t o t ell us t hat t hey were up t o 140 regist ered higher educat ion and furt her educat ion inst it ut ional m em bers. Le x iURL lin k a n a lysis LexiURL is free soft ware designed t o ret rieve link dat a from search engines, like Yahoo! , Google, or Alt aVist a and calculat e sum m ary st at ist ics for list s of links or URLs. I t s out put is a series of st andard report s t hat convey inform at ion about page URLs, sit es and Web dom ains linking t o a m ain sit e of int erest . Alt hough LexiURL is a flexible, generic program , m any of it s funct ions are useful for a digit al reposit ory link analysis. Before each of t he m anagem ent int erviews, a Web link analysis report was prepared and present ed t o t he m anagers at t he m eet ings for discussion of t he im plicat ions of t he dat a. All link dat a were organized in a uniform form at t hat explained how t he links could be exam ined or m anipulat ed for evaluat ion purposes, or visit ed on t he Web for furt her insight . The inform at ion given t o t he m anagers included a list of t he page URLs linking t o t heir reposit ory, a list of all second and t op level dom ains and a co- link net work m ap. Figures 1 and 3 show t wo co- link m ap exam ples: one creat ed for t he e- Print s Sot on m anagem ent t eam ( Oct ober 20t h, 2005) and anot her creat ed for t he Jorum t eam ( April 3, 2006) . Dist ances bet ween t he nodal point s ( Websit es) represent a kind of sim ilarit y- based relat ionship of 'co- linkedness' on t he Web. Co- linked Websit es occur 'when t wo pages bot h have inlinks from a t hird page' ( Thelwall 2004: 5) . Lines leading t o t he sit e of int erest represent direct ed inward links and line t hickness indicat es t he link frequency. Our research int erest in t he m aps was t o give t he reposit ory m anagers a m et hod of visualising t he Web net work in which t heir service was sit uat ed, at t he t im e of t he st udy. The e- Print s Sot on sit e was sit uat ed wit hin an academ ic co- link environm ent , as expect ed, but m any of t he co- linked sit es were not direct ly linked t o e- Print s Sot on. Jorum 's co- linked sit es were eit her universit y sit es, or sit es relat ed t o e- learning ( e.g., t he MERLOT e- learning resource) . Figures 2 and 4, following each co- link m ap, graph t he num ber of different sit es in second or t op level dom ains t hat cont ain at least one page linking t o t he e- Print s Sot on Websit e and one page linking t o t he Jorum Websit e. The responses obt ained from t he m anagers concerning t his dat a were posit ive, given t he fact t hat we were int roducing a Web analysis t echnique t hat t hey had not seen before. A link analysis using LexiURL should ideally be carried out for each of t he reposit ories approxim at ely every four t o six m ont hs. Over t his period a m anager m ay be able t o det ect changes in t he co- link m aps represent ing t he resource's online net work or Web com m unit y. New links m ight appear and a regular review of t heir cont ext ( i.e., where t hey are sit uat ed on http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 13 of 28 a Web page and what t ype of organizat ion is creat ing t he link) would give m anagers an opport unit y t o t hink about places where new users m ight be surfing t he Web and address t he needs of pot ent ial user groups. From our init ial analyses, we discovered t hat a growing proport ion of links t o t he UK resources were com ing from int ernat ional Websit es ( eg. t he Universit y of Queensland Aust ralia direct ed a link t o t he e- Print s SOTON sit e; list ing it as a key resource on t heir Dat abases for Social Sciences page) . As a result of t his inform at ion, we are cert ain t hat m anagers will want t o see t hat t hese links are preserved and will want t o know if such links are being followed as access point s t o t heir resource. LexiURL can also easily be used as a supplem ent t o a log file analysis. Log files provide inform at ion about daily user act ivit ies on t he Web, eit her in t erm s of t he search engines used and phrases/ words users' t ype t o carry out a search, or t he Web URLs ( links) t hat are being followed. A LexiURL analysis is a com plem ent t o log file dat a because it ext ract s list s of links from t he Web ( using Yahoo! ) t hat exist 'in t he wild', which can be com pared t o log file ( followed) URLs. Furt herm ore, we recom m end t hat m anagers consider using LexiURL t o perform com parat ive link analyses wit h 'com pet it or' sit es or ot her int ernat ional reposit ories sim ilar in scope and purpose. I f m ore links or different t ypes of links are found t o be direct ed t o t he sit e of anot her sim ilar resource, t hen perhaps t hese links represent previously unrecognized users, or areas for furt her out reach and cooperat ion. Figu r e 1 : Top 4 9 sit e s co- lin k e d w it h Un ive r sit y of Sout ha m pt on e - Pr int s, in cludin g dir e ct e d link s. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 14 of 28 Figur e 2 : Se cond or t op le ve l dom a in s w it h a t le a st on e pa ge lin k in g t o t h e Un ive r sit y of Sou t h a m pt on e - Pr in t s. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 15 of 28 Figu r e 3 : Top 4 9 sit e s co- lin k e d w it h Jor um , includin g dir e ct e d link s. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 16 of 28 Figur e 4 : Se cond or t op le ve l dom a in s w it h a t le a st on e pa ge lin k in g t o Jor u m Re posit or y u se r s The result s of our user survey provided us wit h current inform at ion concerning t he percept ions som e users have of t he reposit ories, what t hey want or need from t hem and how t hey approach t hem on t he Web. Our survey was carried out on t he I nt ernet using a Web- based quest ionnaire. To obt ain part icipant s we com piled a set of relevant m ailing list s on t he I nt ernet ( e.g., m ailing list s for healt h care professionals, lect urers, educat ors, researchers et c who would likely be int erest ed in t he reposit ory's cont ent ) and sent out announcem ent s regarding our quest ionnaire t hrough t he list s.> We wrot e t o som e of t he school heads at t he Universit y of Sout ham pt on and asked if t hey would agree t o circulat e an announcem ent regarding our quest ionnaire and som e of t he reposit ory m anagers were helpful in encouraging people t o com plet e our survey. Figure 5 shows t he t ot al num ber of survey respondent s corresponding t o each service. 54% of t he respondent s were fem ale; 44% were m ale ( 2% of t he individuals surveyed did not respond t o t he gender quest ion) . As expect ed, t he m aj orit y of individuals who com plet ed t he survey were bet ween t he ages of 25 and 65 ( 92 % ) ( 2% did not respond t o t his quest ion) . 82% of our survey respondent s were resident s of t he UK. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 17 of 28 Figu r e 5 : Tot a l n u m be r of sur ve y r e spon de n t s cor r e sponding t o e a ch se r vice . Sixt eeen percent of respondent s were resident s of ot her count ries, for inst ance, t he Unit ed St at es, Aust ralia, Canada, New Zealand, I srael, China, I ndia, Thailand, Hait i, I ran, Ecuador, Bolivia, Brazil, Mexico, Uruguay, Trinidad and Tobago and part s of Europe ( i.e., I t aly, France, Germ any, Turkey, Hungry, Finland) . Most of t he foreign survey respondent s were associat ed wit h CogPrint s, but t his was expect ed since t his subj ect reposit ory has a great er int ernat ional focus t han t he ot her resources evaluat ed here. Most of t he respondent s were librarians or inform at ion professionals or academ ic st aff and researchers. However, som e m anagers ( i.e., I T or proj ect m anagers) and m any nurses, t eachers, st udent s and physicians and public healt h care pract it ioners also com plet ed t he survey. Figures 6, 7, 8, and 9 and Table 1 below present t he result s obt ained concerning t he quest ions shown in capt ions. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 18 of 28 Figu r e 6 : H ow did you fir st le a r n a bout t h e e x ist e n ce of t h e libr a r y/ r e posit or y? http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 19 of 28 Figu r e 7 : Ar e you a u se r of [ digit a l libr a r y/ r e posit or y n a m e ] ? Figur e 8 : W ha t is your usua l W e b a cce ss poin t t o t h e digit a l libr a r y/ r e posit or y? When we com pared all Web access point answers t o t he digit al library/ reposit ory t o t he frequency of use responses, we observed t hat :    Few users had creat ed a personal hom epage link t o t he reposit ory. Users who creat ed a bookm ark or personal hom epage link t o one of t he select ed digit al resources t ended t o use t he resource a few t im es each week. Less frequent use of a reposit ory was associat ed wit h a habit of searching for it by nam e on t he Web. W e b a cce ss poin t t o t h e digit a l libr a r y or r e posit or y Fr e que ncy of use Book m a r k e d ( 83) Type URL t o r e a ch sit e ( 50) Follow lin k fr om a not he r pa ge ( 4 2 ) Pe r sona l hom e pa ge lin k ( 20) Se a r ch for it by n a m e on W eb ( 1 8 ) Ot he r ( e .g., e - m a il lin k / At h e n a por t a l / D e sk t op icon ) ( 8 ) Everyday 16 2 3 3 0 0 2- 3 t im es a week 14 12 6 8 1 1 Once a week 10 5 5 2 3 1 http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 20 of 28 Approxim at ely every 2 weeks 16 4 6 0 3 1 Once a m ont h 4 5 6 2 2 0 23 22 16 5 9 5 A few t im es a year Ta ble 1 : W e b a cce ss poin t a n d fr e qu e n cy of u se Figur e 9 : The m a t e r ia l on t h e digit a l libr a r y or r e posit or y is u su a lly r e le va n t t o w h a t I n e e d. Two addit ional sect ions of t he user survey were creat ed t o encourage users t o indicat e what t ype of inform at ion t hey would like t o see available at each of t he online resources and st at e also what t ype of benefit s t hey had experienced when using m at erials from t he reposit ory or library sit e. One user of t he Nat ional elect ronic Library for Healt h st at ed t hat s/ he was able t o obt ain inform at ion t hat is clinically relevant m uch fast er t han previously. Anot her user wrot e about t he personal benefit s of using t he healt h library online: I accident ally st um bled on t o som e useful inform at ion concerning a condit ion I suffer from m yself. This inform at ion was com plet ely new t o m e and st art ed m e on t he road t o finding som e m ore, which has offered m e anot her t reat m ent opt ion and im proved m y own healt h ( User C) . A user of CogPrint s said t hat s/ he was, '...get t ing bet t er in [ his/ her] work and feeling m ore com fort able about being in t ouch wit h great resources for free' ( User F) . One unexpect ed benefit relat ed t o us by an e- Print s Sot on user was t hat aft er deposit ing m at erials on t he sit e s/ he had '...received cont act from ot her researchers wit h sim ilar int erest s'. ( User D) . Also, http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 21 of 28 one UK Dat a Archive user said t hat , 't he availabilit y of dat a led t o a new funded st ream in his/ her research program m e' ( User G) . And, finally, we were t old by one Jorum user t hat t he exist ence of t his e- learning reposit ory ( even at it s earlier st ages) had 'increased t he st im ulat ion of t eaching st aff and t heir m ot ivat ion' ( User E) . Am ongst t he individuals who com plet ed our surveys ( excluding t he Nat ional elect ronic Library for Healt h) 50% percent ident ified t hem selves as service users and 50% percent as non- users. Ten percent of non- users com m ent ed t hat t hey were st ill learning about t he services and 22% claim ed t o be int erest ed in using t hem in t he fut ure. Only 8% st at ed t hat t hey did not want t o use t hem at all. Som e of t he survey respondent s who indicat ed t hat t hey were not users of t he services st udied, said t hat t hey were users of ot her t ypes of digit al libraries and/ or reposit ories; hence, when we asked, 'For what purpose did you use anot her digit al library or reposit ory?' t he responses were as follows:     To To To To ret rieve and use t ext - base resources for personal learning purposes ( 48% ) . ret rieve and use im ages, m aps, diagram s, or ot her visual aids ( 17% ) . ret rieve t eaching m at erials ( 27% ) . cont ribut e or deposit m at erial int o t he reposit ory for ot hers t o use ( 25% ) M a n a ge m e n t im plica t ion s Digit al reposit ories are not st at ic and require ongoing evaluat ion t o det erm ine t heir qualit y and t o ident ify new direct ions for growt h. Managem ent t eam s of well- est ablished and well- used reposit ories m ay need t o becom e knowledgeable about collect ing Web link st at ist ics, download st at ist ics or cit at ion st at ist ics in t he fut ure for a variet y of analyt ic purposes, so t hat int erest ed part ies will have an adequat e m easure of a reposit ory's success. Reposit ory uses can be as varied as t he users t hem selves; hence it is im port ant for m anagers t o com m unicat e regularly wit h users ( e.g., t hrough an open forum ) in order t o share inform at ion and obt ain feedback. Reposit ory m anagem ent t eam s who set up and m aint ain regist rat ion dat abases, list servs, or int eract ive newsgroups for users are engaging in an im por t ant m anagem ent pract ice. Alt hough t he developm ent and m anagem ent of a digit al library differs from t he process of creat ing and m anaging a digit al reposit ory, t here are t im es when reposit ory m anagers can and should learn from t he work of t heir digit al library colleagues. Because we included t he Nat ional elect ronic Library for Healt h in t his st udy, we obt ained a valuable point of reference for how it is t hat reposit ory m anagers m ight underst and users. The Nat ional elect ronic Library for Healt h m anagem ent t eam spent a lot of t im e researching t he needs of users; t his proved t o be good pract ice, part icularly in t erm s of users prom ot ing t he digit al library. Reposit ory m anagers are focused on how t o develop t heir reposit ories and are int ent on encouraging individuals t o deposit , but over t im e t hey will have t o focus m ore on underst anding long- t erm user needs. A user - based focus will becom e especially im port ant for m anagers of e- learning reposit ories because t he expect ed value t hat e- learning obj ect s will have on lect urers and st udent s in higher educat ion. Based on t he survey inform at ion generat ed from non- users, reposit ory m anagers should not assum e t hat non- use of t heir resource is due t o an ignorance of or lack of fam iliarit y wit h digit al resources. Pot ent ial users could be using ot her t ypes of digit al libraries and reposit ories; t herefore, it is a good best m anagem ent pract ice t o t ry t o find out m ore about what is at t ract ing t hem t o ot her reposit ories ( online com pet it ors possibly) and develop publicit y program m es t hat will bring people up t o dat e on what m akes t heir resource especially valuable. Digit al reposit ory m anagers m ay need t o give m ore considerat ion t o t he im port ance of personal inform at ion sharing am ong friends and work colleagues ( Rosen 2000) . A significant num ber of individuals surveyed for t his proj ect indicat ed t hat t hey had learned about t he services st udied t hrough a friend or colleague. I nit ial evidence was also found t o suggest t hat http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 22 of 28 reposit ory use can cont ribut e t o collegial net working. For exam ple: 'I have received cont act from ot her researchers wit h sim ilar int erest s' ( User D) . Personal Websit e links t o online digit al resources are norm ally not plent iful ( e.g., Beaulieu 2005) ; however based on t his proj ect 's user survey we discovered t hat persons who frequent ly use t he st udied services som et im es do have a direct ed link from t heir personal Websit e. A regular LexiURL link analysis should give t he m anager new insight int o t he num ber of personal pages linking t o t heir resource over t im e, including som e of t he growing num ber of Weblogs. What is t he relat ionship bet ween t he source of t he link and t he link t arget ? Does t he source sim ply acknowledge t he digit al resource or provide descript ive inform at ion concerning part s t hat t hey appreciat e, recom m end t o ot hers, or have consult ed t o great benefit ? Tr a in in g im plica t ion s Earlier we indicat ed t hat t he basic requirem ent s t o run a library or digit al library successfully are covered t o a great er or lesser ext ent by t radit ional library and inform at ion science schools' curricula, but none, t o our knowledge, focuses on t he part icular needs and requirem ent s of reposit ory m anagers. I t m ay be, of course, t hat som e program m es on digit al libraries include reposit ories as a t ype of digit al resource. Mezick and Koenig's ( 2008) recent review of inform at ion science educat ion draws at t ent ion t o new program m es in knowledge m anagem ent , inform at ion archit ect ure and digit al libraries, but m akes no m ent ion of ot her em erging areas such as social inform at ics ( see Kling 1999) or inst it ut ional reposit ory m anagem ent . Bot h areas are closely relat ed because m anagers clearly need t o recognize t he social cont ext in which new reposit ories are developed before t hey can underst and how t hey will influence t he ways t hat people look for and use inform at ion. Wit h t he increase in reposit ory act ivit y, t here is lit t le doubt m anagers dem onst rat e t hat m any felt t heir way when first possible or desirable ( Dhiensa 2006) . During t he int erview any t raining before set t ing up t heir reposit ories. We found was carried out in- house. t hat m anagem ent t raining will be needed. Surveys of reposit ory start ing, oft en m aking m ist akes t hrough ignorance of what was phase of our st udy we asked t he m anagers if t hey had received t hat m ost had not part icipat ed in any form al t raining, or t hat it For a new curriculum in reposit ory m anagem ent , m at erials could be drawn from exist ing curricula, but m uch of t his inform at ion would need t o focus on issues specific t o reposit ories. I n- house t eaching could be support ed by cont ribut ions from reposit ory m anagers, for inst ance, as guest speakers invit ed t o give present at ions and share pract ical insight s. The issues t hat need t o be t aught apply int ernat ionally; hence t here is no reason why such a program m e could not be provided t o a world- wide audience using e- learning m et hods. The m aj or com ponent s of a new curriculum m ight be ( in t hem at ic order) : The changing electronic publishing environment      Elect ronic publishing developm ent s; Web 2.0. Principles of open access, including green and gold rout es. The developing virt ual and m anaged learning environm ent s learning obj ect s. Scholarly research environm ent , including funding. Elect ronic publishing indust ry – dynam ics, business m odels, current developm ent s, including digit al right s m anagem ent . Repositories   I nst it ut ional reposit ories; learning obj ect reposit ories; subj ect - based reposit ories; blended reposit ories; nat ional and int ernat ional reposit ories. Current research and developm ent work in reposit ories. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 23 of 28 Management issues     Budget ing and long- t erm financing of reposit ories. Hum an resource m anagem ent issues, including how t o m ot ivat e people t o add m at erial t o reposit ories. Market ing, including m arket research, advocacy and prom ot ion. Bibliom et rics, Webom et rics, cit at ion analysis and ot her m et hods of evaluat ing im pact and success; t he open access advant age. Librarianship   Digit al libraries and ot her recent developm ent s in librarianship. The relat ionship bet ween libraries and reposit ories. Technical tools    I nform at ion t echnology and net working: basics and developm ent s, including an evaluat ion of t he various free and priced reposit ory soft ware. Web 2.0 t echnologies and t heir applicabilit y t o reposit ories. Workflows and use cases. Legal issues  Legal issues, including copyright , ot her int ellect ual propert y right s, publishing law ( especially defam at ion and obscene cont ent ) , Creat ive Com m ons and ot her licences. Core reading m at erials for a new reposit ory m anagem ent program m e should include t he books writ t en by Jones et al. ( 2006) , Jacobs ( 2006) and Cockburn ( 2001) ; however, m ost of t he support ing lit erat ure will be j ournal art icles and Web sit es. Con clu sion Many of t he m anagem ent issues t hat reposit ory m anagers are facing are novel and t he t echniques available t o assist t hem wit h long- t erm evaluat ions are eit her in t heir infancy, like LexiURL, or not well known. We have dem onst rat ed t he result s of one fairly general st rat egy t hat can be applied t o different reposit ory t ypes, including digit al libraries, but because t his evaluat ion has com e at an early st age in t he reposit or y era, furt her evaluat ive research will be needed in t he fut ure.> This research shows t hat an overall evaluat ion process should, at t he very least , consider t he reposit ory m anagem ent t eam 's> and t he users' perspect ives and should apply som e t ype of obj ect ive m easure t o det erm ine how t hese int eract ing fact ors are cont ribut ing t o t he reposit ory's success. Most of t he lit erat ure on reposit ory m anagem ent dem onst rat es a concern for inst it ut ional reposit ories and t he effect t hat t hey will have on research out put s or research assessm ent s; t hus furt her research will be needed t o det erm ine how ot her t ypes of reposit ories, e.g., learning obj ect reposit ories, cont ribut e t o higher educat ion and what kind of effect t hey are having on t eaching and learning. I n sum , we believe t here is a st rong case for library and inform at ion science schools t o develop program m es, or at m inim um , specialist m odules, t o assist t he ever increasing num bers of people who wish t o t rain as reposit ory m anagers. Since our proj ect was lim it ed t o reposit ories in t he UK and was an explorat ory st udy, it will becom e increasingly im port ant t o find out how digit al reposit ory m anagers everywhere are learning t heir t rade, keeping up wit h rapid inform at ion t echnology developm ent s and coping wit h t heir t raining needs. New research, including m arket research, is needed t o est ablish t he best m et hods of providing such t raining. Might it , for exam ple, be provided by library schools, com put er http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 24 of 28 science depart m ent s, professional associat ions, or com m ercial t raining pr oviders? Also, how should it be delivered: by m eans of short courses, dist ance learning, or e- learning packages? Wit h t he rapid developm ent and growing im port ance of reposit ories, t hese are issues t hat should not be left t o chance. Ack n ow le dge m e nt s We wish t o t hank t he anonym ous referees for t heir helpful com m ent s. Funding for t his research was provided t o us in t he Unit ed Kingdom by t he Joint I nform at ion Syst em s Com m it t ee's Digit al Reposit ories Program m e. References                  Andrew, T. ( 2006, March 31) . Concern of longevit y and lack of act ive engagem ent from researchers wit h inst it ut ional reposit ories [ Msg 347] . Message post ed t o JI SC- REPOSI TORI ES elect ronic m ailing list , archived at ht t p: / / t inyurl.com / 33qc58 Ashwort h, S., Mackie, M. & Nixon, W. J. ( 2004) . The DAEDALUS proj ect , developing inst it ut ional reposit ories at Glasgow Universit y: t he st ory so far. Library Review , 5 3 ( 5) , 259- 264. Bailey, C.W. ( 2006) . What is open access? I n N. Jacobs ( Ed.) , Open access: key st rat egic, t echnical and econom ic aspect s ( pp. 13- 26) . Oxford: Chandos Publishing. Beaulieu, A. ( 2005) . Sociable hyperlinks: an et hnographic approach t o connect ivit y. I n C. Hine ( Ed.) , Virt ual m et hods: issues in social research on t he I nt ernet . ( pp. 183- 197) . New York, NY: Berg. Bell, S., Fost er, N. F. & Gibbons, S. ( 2005) . Reference librarians and t he success of inst it ut ional reposit ories. Reference Services Review , 3 3 ( 3) , 283- 290. Brody, T. & Harnad, S. ( 2004) . Com paring t he im pact of open access ( OA) vs. non- OA art icles in t he sam e j ournals. D- Lib Magazine, 1 0 ( 6) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.dlib.org/ dlib/ j une04/ harnad/ 06harnad. ht m l. Brody, T., Harnad, S. & Carr, L. ( 2006) . Earlier Web usage st at ist ics as predict ors of lat er cit at ion im pact . Journal of t he Am erican Societ y for I nform at ion Science and Technology, 57( 8 ) , 1060- 1072. Buehler, M. A. & Boat eng, A. ( 2005) . The evolving im pact of inst it ut ional reposit ories on reference librarians. Reference Services Review , 3 3 ( 3) , 291- 300. Chan, D. ( 2006) . Core com pet encies and perform ance m anagem ent in Canadian public libraries. Library Managem ent , 2 7 ( 3) , 144- 153. Chan, D.L.H., Kwok, C.S.Y. & Yip, S.K.F. ( 2005) . Changing roles of reference librarians: t he case of t he HKUST inst it ut ional reposit ory. Reference Services Review , 3 3 ( 3) , 268- 282. Choi, Y. & Rasm ussen, E. ( 2006) . What do digit al librarians do? I n Marchionini, G., Nelson, M.L. & Marshall, C.C. ( Eds.) Proceedings of t he 6t h ACM/ I EEE- CS j oint conference on Digit al libraries, Chapel Hill, N.C., USA. ( pp. 187- 188) . New York, NY: ACM Press. Cockburn, A. ( 2001) . Writ ing effect ive use cases. Bost on, MA: Addison- Wesley. Crow, R. ( 2002) . The case for inst it ut ional reposit ories: a SPARC posit ion paper. Washingt on, DC: The Scholarly Publishing & Academ ic Resources Coalit ion. Ret rieved 15 Novem ber, 2007 from ht t p: / / www.arl.org/ sparc/ bm ~ doc/ ir_final_release_102.pdf Day, M. ( 2003) . Prospect s for inst it ut ional e- print reposit ories in t he Unit ed Kingdom . EPrint s UK support ing st udy, no.1.. Ret rieved 16 Novem ber, 2007 from ht t p: / / eprint s- uk.rdn.ac.uk/ proj ect / docs/ st udies/ im pact / . Dhiensa, R. ( 2006) . Managers' percept ions of t heir inst it ut ional reposit ories. Unpublished m ast er's dissert at ion, Universit y of Loughborough, Loughborough, UK. Gadd, E., Oppenheim , C. & Probet s, S. ( 2003a) . RoMEO st udies 1: t he im pact of copyright ownership on academ ic aut hor self- archiving. Journal of Docum ent at ion, 5 3 ( 3) , 243- 277. Gadd, E., Oppenheim , C. & Probet s, S. ( 2003b) . RoMEO st udies 2: How academ ics want t o prot ect t heir openaccess research papers. Journal of I nform at ion Science, 2 9 ( 5) , 333- 356. http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories                        Page 25 of 28 Gadd, E., Oppenheim , C. & Probet s, S. ( 2003c) . RoMEO st udies 4: an analysis of j ournal publishers' copyright agreem ent s. Learned Publishing, 1 6 ( 4) , 293- 308. Genoni, P. ( 2004) . Cont ent in inst it ut ional reposit ories. Library Managem ent , 2 5 ( 6- 7) , 300- 306. Gibbons, S. ( 2004) . Defining an inst it ut ional reposit ory. Library Technology Report s, 4 0 ( 4) , 6- 10. Gladney, H. M. ( 1999) . Digit al dilem m a: int ellect ual propert y synopsis and views on t he st udy by t he Nat ional Academ ies' Com m it t ee on I nt ellect ual Propert y Right s and t he Em erging I nform at ion I nfrast ruct ure. D- Lib Magazine., 5 ( 22) Ret rieved 15 Novem ber, 2007 from ht t p: / / www.dlib.org/ dlib/ decem ber99/ 12gladney.ht m l. Guy, M., Powell, A. & Day, M. ( 2004) . I m proving t he qualit y of m et adat a in e- print archives. Ariadne, ( no. 38) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.ariadne.ac.uk/ issue38/ guy/ . Haj j em , C., Harnad, S. and Gingras, Y. ( 2005) . Ten- year cross- disciplinary com parison of t he growt h of open access and how it increases research cit at ion im pact . I EEE Dat a Engineering Bullet in, 2 8 ( 4) , 39- 47. Ret rieved 16 Novem ber, 2007 from ht t p: / / eprint s.ecs.sot on.ac.uk/ 12906/ Harnad, S. ( 2003) . Open access t o peer - reviewed research t hrough aut hor/ inst it ut ion self- archiving: m axim ising research im pact by m axim ising online access. Journal of Post graduat e Medicine, 4 9 ( 4) , 337- 342. Ret rieved 20 Novem ber, 2007 from ht t p: / / t inyurl.com / 3yrhky Harnad, S. ( 2006a, January 19) . Learning from t he successful OA I Rs. [ Msg 838] . Message post ed t o AMERI CAN- SCI ENTI ST- OPEN- ACCESS- FORUM archived at ht t p: / / t inyurl.com / 34xgnb. Harnad, S. ( 2006b) . Publish or perish? Self- archive t o flourish: t he green rout e t o open access. ERCI M News Online Edit ion ( no. 64) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.ercim .org/ publicat ion/ Ercim _News/ enw64/ harnad.ht m l. Hast ings, K., & Tennant , R. ( 1996) . How t o build a digit al librarian. D- Lib Magazine, 2 . Ret rieved May 7 2007 from ht t p: / / www.dlib.org/ dlib/ novem ber96/ ucb/ 11hast ings.ht m l Heery, R. & Anderson, S. ( 2005) . Digit al reposit ories review. Bat h, UK: UKOLN and Art s and Hum anit ies Dat a Service. Ret rieved 15 Novem ber, 2007 from ht t p: / / www.j isc.ac.uk/ uploaded_docum ent s/ digit al- reposit oriesreview - 2005.pdf. Hey, J. ( 2004, July) . Target ing academ ic research wit h Sout ham pt on's inst it ut ional reposit ory. Ariadne, 4 0 . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.ariadne.ac.uk/ issue40/ hey/ . Jacobs, N. ( Ed.) ( 2006) . Open access: key st rat egic, t echnical and econom ic aspect s. Oxford: Chandos Press. Jenkins, B. & Breakst one, E. ( 2005) . Cont ent in, cont ent out : t he dual roles of t he reference librarian in inst it ut ional reposit ories. Reference Services Review, 3 3 ( 3) , 312- 324. Johnson, R. K. ( 2002) . I nst it ut ional reposit ories: part nering wit h facult y t o enhance scholarly com m unicat ion. D- Lib Magazine, 8 ( 11) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.dlib.org/ dlib/ novem ber02/ j ohnson/ 11j ohnson.ht m l. Joint I nform at ion Syst em s Com m it t ee. ( 2005) . Digit al reposit ories program m e 2005- 7. London: Joint I nform at ion Syst em s Com m it t ee. Ret rieved May 7, 2007 from ht t p: / / www.j isc.ac.uk/ what wedo/ program m es/ program m e_digit al_reposit ories.aspx Jones, R. Andrew, T. & MacColl, J. ( 2006) . The inst it ut ional reposit ory. Oxford: Chandos Press. Kennan, M. A. & Wilson, C. S. ( 2006) . I nst it ut ional reposit ories: review and an inform at ion syst em s perspect ive. Library Managem ent , 2 7 ( 4/ 5) , 236- 248. Kling, R. ( 1999) . What is social inform at ics and why does it m at t er? D- Lib Magazine, 5 ( 1) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.dlib.org/ dlib/ j anuary99/ kling/ 01kling.ht m l. Koehler, A.E.C. ( 2006) . Som e t hought s on t he m eaning of open access for universit y library t echnical services. Serials Review, 3 2 ( 1) , 17- 21. Kurt z, M.J., Eichhorn, G., Accom azzi, A., Grant , C.S., Dem leit ner, M. & Murray, S.S. ( 2004) . The effect of use and access on cit at ions. I nform at ion Processing and Managem ent , 4 1 ( 6) , 1395- 1402. Lynch, C. ( 2003) . I nst it ut ional reposit ories: essent ial infrast ruct ure for scholarship in t he digit al age. port al: Libraries and t he Academ y, 3 ( 2) , 327- 336. McLean, N. & Lynch, C. ( 2004) . I nt eroperabilit y bet ween library and inform at ion services and learning http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories                 Page 26 of 28 environm ent s — bridging t he gaps. A j oint whit e paper on behalf of t he I MS Global Learning Consort ium and t he Coalit ion for Net worked I nform at ion. Ret rieved 15 Novem ber, 2007 from ht t p: / / www.im sglobal.org/ digit alreposit ories/ CNI andI MS_2004.pdf. Medeiros, N. ( 2003) . E- print s, inst it ut ional archives and m et adat a: dissem inat ing scholarly lit erat ure t o t he m asses. OCLC Syst em s & Services, 1 9 ( 2) , 51- 53. Mezick, E.M. & Koenig, M.E.D. ( 2008) . Educat ion for inform at ion science. Annual Review of I nform at ion Science and Technology, 4 2 , 593- 624 Moed, H. ( 2006) . The effect of “ Open Access” upon cit at ion im pact : an analysis of ArXiv's Condensed Mat t er sect ion [ ArXiv preprint v. 1. Novem ber 16 2006] . Ret rieved 15 Novem ber, 2007 from ht t p: / / arxiv.org/ abs/ cs.DL/ 0611060. Mullins, J. & Linehan, M. ( 2005) . The cent ral role of leaders in public libraries. Library Managem ent , 2 6 ( 6/ 7) , 386- 396. Payet t e, S. & Lagoze, C. ( 1998) . Flexible and Ext ensible Digit al Obj ect and Reposit ory Archit ect ure ( FEDORA) . I n C. Nikolaou & C. St ephanidis ( Eds.) , Research and Advanced Technology for Digit al Libraries: Proceedings of t he 2nd European Conference ( pp. 41- 60) . Heidelberg, Germ any: Springer - Verlag. ( Lect ure Not es in Com put er Science: Vol. 1513/ 1998.) Perry, C. A. ( 2005) . Educat ion for digit izat ion: how do we prepare? Journal of Academ ic Librarianship, 3 1 ( 6) , 523- 53. Phillips, H., Carr, R. & Teal, J. ( 2005) . Leading roles for reference librarians in inst it ut ional reposit ories. One library's experience. Reference Services Review , 3 3 ( 3) , 301- 311. Pinfield, S. ( 2003) . Open archives and UK inst it ut ions. An overview. D- LibMagazine, 9 ( 3) Available at ht t p: / / www.dlib.org/ dlib/ m arch03/ pinfield/ 03pinfield.ht m l. Pinfield, S., Gardener, M. & MacColl, J. ( 2002) . Set t ing up an inst it ut ional e- print archive. Ariadne, ( no. 31) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.ariadne.ac.uk/ issue31/ eprint - archives/ . Ray, M.S. ( 2001) . Shift ing sands: t he j urisdict ion of librarians in scholarly com m unicat ion. Paper delivered at ACRL's 10t h Nat ional Conference: Crossing t he Divide. March 15- 18, 2001, Denver, Colorado. Ret rieved 12 Oct ober 2006 fr om ht t p: / / www.ala.org/ ala/ acrl/ acrlevent s/ m ray.pdf. Rogers, S. A. ( 2003) . Developing an inst it ut ional knowledge bank at Ohio St at e Universit y: from concept t o act ion plan. port al: Libraries and t he Academ y, 3 ( 1) , 125- 136. Rosen, E. ( 2000) . The anat om y of buzz. London: Harper Collins Business. Sargeant , S.J.E. & Harrison, J. ( 2004) . Clinical librarianship in t he UK: t em porary t rend or perm anent profession? Part I : a review of t he role of t he clinical librarian. Healt h I nform at ion and Libraries Journal, 2 1 ( 3) , 173- 181. Sm it h, M., Bart on, M.R., Branschofsky, M., McClellan, G., Harford Walker, J, Bass, M. J. et al. ( 2003) . DSpace: an open source dynam ic digit al reposit ory. D- Lib Magazine, 9 ( 1) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.dlib.org/ dlib/ j anuary03/ sm it h/ 01sm it h.ht m l. Spink, A. & Cool, C. ( 1999) . Educat ion for digit al libraries. D- Lib Magazine, 5 ( 5) . Ret rieved 15 Novem ber, 2007 from ht t p: / / www.dlib.org/ dlib/ m ay99/ 05spink.ht m l. Thelwall, M. ( 2004) . Link analysis: an inform at ion science approach. San Diego, CA: Academ ic Press Appe n dix : M a n a ge m e n t I n t e r vie w Sch e dule Rationale for Creating the Repository   What was t he m ain reason for creat ing your reposit ory? When did you first realise t hat a reposit ory like t he one you have creat ed m ight be beneficial t o regist ered users? http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories  Page 27 of 28 Were t here any exam ples of reposit ories t hat you followed? Why? Development of the Repository      Who was involved in t he developm ent of your reposit ory? What was t he senior m anagem ent support like? What resources were required t o set up t he reposit ory? How did you decide what m at erials would be included in digit al reposit ory? Were any t raining program m es used for t hose involved wit h it s developm ent ? I f so, what were t hese program m es and how were t hey im plem ent ed? Did you experience any problem s/ difficult ies whilst you were set t ing up your reposit ory? What were t he difficult ies and did t hey affect your progress? Identification of Users and Publicizing the Repository        How did you ident ify pot ent ial users or user groups during t he early developm ent st age of your reposit ory? When did you int roduce your reposit ory t o ot her inst it ut ions and m em bers of t he general public? Was your digit al reposit ory publicised? How was t his done? Did you have any part icular st rat egy for recruit ing users? Do you have t raining program m es for users and how havet hey been im plem ent ed? Do you collect any usage st at ist ics of your reposit ory? Do you keep t rack of where your users are com ing from ? Who have you ident ified recent ly as t he m ain users of your reposit ory? Benefits of the Digital Repository  Have you com e across any benefit s t o users associat ed wit h t he reposit ory? Web Link Analysis       I n what ways does t he m anagem ent t eam for your reposit ory ext ract 'act ionable' inform at ion from t he Web in order t o bet t er serve users or pot ent ial users and t heir inform at ion needs? [ Present t he LexiURL Web link analysis report t o t he int erviewee and let him / her exam ine t he inform at ion. Answer any quest ions and or explain t he report det ails.] I s t here any inform at ion in t his report t hat is surprising t o you? Do you regard t he inform at ion provided in t his report t o be useful t o your reposit ory program m e? How? Do you t hink t hat a “ Web I nt elligence” report like t he one we present would be valuable t o you as part of a regular service? How oft en would you like t o receive a report like t his? ( e.g., m ont hly; four t im es per year?) I s t here any inform at ion in t his report t hat you would like t o see added? How to cite this paper Zu cca la , A., Oppe nhe im , C. & D hie nsa , R. ( 2 0 0 8 ) . " M a n a gin g a n d e va lu a t in g digit a l r e posit or ie s " I n for m a t ion Re se a r ch , 1 3 ( 1 ) pa pe r 3 3 3 . [ Ava ila ble 2 1 N ove m be r , 2 0 0 7 a t ht t p:/ / I n for m a t ion R.n e t / ir / 1 3 - 1 / pa pe r 3 3 3 .h t m l] http://informationr.net/ir/13-1/paper333.html 4/13/2009 Managing and evaluating digital repositories Page 28 of 28 Fin d ot h e r pa pe r s on t h is su bj e ct Schola r Se a r ch Google Se a r ch W in dow s Aca de m ic Book m a r k Th is Pa ge count W e b Cou n t e r © t he a ut hor s, 2 0 0 8 . La st u pda t e d: 2 0 N ove m be r , 2 0 0 7 Cont e n t s | Au t h or in de x | Subj e ct inde x | Se a r ch | H om e http://informationr.net/ir/13-1/paper333.html 4/13/2009