Applied digital library project management

The current issue and full text archive of this journal is available at www.emeraldinsight.com/1065-075X.htm OCLC 25,3 MANAGING DIGITAL LIBRARIES: THE VIEW FROM 30,000 FEET 162 Applied digital library project management Received April 2009 Reviewed April 2009 Accepted April 2009 Using paired comparison analysis to determine relative importance H. Frank Cervone Purdue University Calumet, Hammond, Illinois, USA Abstract Purpose – This paper sets out to define and describe paired comparison analysis as a method for prioritizing the factors that have the most impact in a digital library project. Design/methodology/approach – Using theory and example, the paper relates the use of paired comparison analysis to the successful prioritization of competing demands in digital library projects. Findings – Paired comparison analysis is useful to project managers and project teams alike as a means for prioritizing the issues, factors, and courses of action within a single project or among multiple projects, especially when these items are difficult to evaluate using an objective scheme. Originality/value – The paper fills a gap in the digital library project management literature by providing an overview of a useful tool to prioritize the issues, factors, and courses of action within a project or among several projects, particularly when these items cannot be compared based on similar measures. Keywords Digital libraries, Project management, Project planning, Mathematical analysis, Paired comparisons Paper type General review OCLC Systems & Services: International digital library perspectives Vol. 25 No. 3, 2009 pp. 162-166 q Emerald Group Publishing Limited 1065-075X DOI 10.1108/10650750910982548 As was noted in the last article in this series, project managers frequently face dilemmas when they are trying to prioritize action items and options in a digital library project. In many cases, the different tasks within a project usually do not have the same level of significance to the overall success of the project. The issue to be resolved is how to focus resources on the tasks that will have the largest impact. In these situations, Pareto analysis (Cervone, 2009) can be used to determine which tasks or options will have the biggest impact and benefit overall. However, not all the decisions a project manager must make are simply related to determining what tasks or options will have the greatest impact. A more difficult problem is balancing the conflicting courses of action in multiple projects. Furthermore, decisions about where to focus resources are made more on a political basis. An example of this is when the project manager must determine which option among the many that could possibly be worked on will be chosen in order to please the largest number of constituencies. For these types of decisions, Pareto analysis is not appropriate and we must use another methodology. In many cases, paired comparison (also known as paired choice) analysis may be useful. Paired comparison analysis provides a method for setting priorities but it differs from Pareto analysis in several ways. Pareto analysis assumes that there is an objective method for making decisions. Consequently, it will not work in the prior example because there is no objective data for making the decision. The decision to be made is much more subjective and there is no “hard and fast” rule for determining the relative importance of the various options. In fact, there may be several conflicting opinions on what the relative importance of the options are because we are trying to meet the needs of multiple constituencies. Paired comparison analysis, therefore, is useful when we do not have objective data related to the various tasks within a project or the courses of action among multiple projects. Perhaps one of the most important differences between Pareto analysis and paired comparison analysis is that paired comparison can be used in “apples to oranges” types of comparisons such as determining the relative priority of remodeling a reading room in the library versus redesigning the library’s website. Paired comparison analysis can also be used where the determination of priorities is ambiguous or deadlocked (Saaty, 2008). In both of these cases, the situation could be the result of differences in opinion among various staff members or because there really is no objective way of creating a prioritized list. Accordingly, the analysis primarily relies on the preferences of the people participating in the analysis rather than some type of objective criteria for decision-making. Nonetheless, the resulting outcome provides data for moving forward. Because of this, paired comparison analysis can be used in much larger number of situations that Pareto analysis can. Methodology Using paired comparison analysis is relatively straightforward. First, a list of the various options or tasks is defined. Each of the options is compared against each of the other options and ranked comparatively as to which is preferable. The results are summed and the options are listed with their sums in descending order, providing an ordered list of priorities. Although paired comparison analysis can be done individually, it can also be used with groups. In a group setting, this can be done in one of two ways. The group could come up with a ranking collectively through a process of negotiation related to each option in relationship to the others resulting in a group ranking of the relationship. A more effective, and frequently faster, method is to have each member of the group rank the relationships individually and then sum all of the individual rankings together to generate an overall ranking from the group. In most cases, a paired choice matrix is used to help perform the analysis. To effectively do this, each option is assigned a letter. Each option’s letter is then assigned as a row and column heading in the matrix. Cells where an option is compared with itself are blanked out, as are cells that duplicate a comparison. In effect, this creates a “half-matrix” with a dividing line running from the topmost left-hand corner to the bottom-most right-hand corner, with the bottom half of the matrix being unused. To perform the actual ranking within the matrix, the option in the row is compared with the option in the column. Within each cell, a decision is made as to which of the two options is the most important. This is supplemented with a numerical ranking from 0 to 3 using the following scale: Paired comparison analysis 163 OCLC 25,3 164 0. No difference between the two. 1. Small difference/low priority. 2. Medium difference/medium priority. 3. Significant difference/high priority. As previously discussed, the results are consolidated by summing the total of all the values of each option. In many cases, these values are then converted to percentages of the total. This is often done to provide a more intuitive indicator of relative importance. An example in practice Examples for the use of paired comparison analysis in digital library projects abound. Consider the case where a project manager has to make a decision about how to prioritize the following four projects: (1) upgrade the OpenURL software to the latest version; (2) implement a new search widget into the library’s main web page; (3) digitize a new collection of photographs the library has acquired; and (4) install a new digital repository package. One of the parameters the project manager is dealing with is that there are only enough resources available to work on one project at a time, so some type of prioritized list must be developed. Moreover, none of these projects is dependent on another and none of them have an inherently more pressing need for implementation according to the administration of the library. In this case, Pareto analysis would be of no use to the project manager. Paired comparison analysis, however, would be a useful tool for prioritizing these items. In this situation, the project manager would most likely gather together a group of interested parties from throughout the library as well as from various constituency groups. Each of these people would complete individual paired comparisons. Subsequently, these individual analyses would be rolled up into a single paired comparison that provides the final list of priorities. In working through this process, the first step for the project manager would be to develop a paired comparison matrix for the problem. Table I demonstrates the type of OpenURL upgrade (A) OpenURL upgrade (A) New Search widget (B) Table I. Example paired comparison analysis table Digitize photographs (C) Digital repository (D) Blocked out selfcomparison Blocked out duplicate comparison Blocked out duplicate comparison Blocked out duplicate comparison New search widget (B) Digitize photographs (C) Digital repository (D) Blocked out selfcomparison Blocked out duplicate comparison Blocked out duplicate comparison Blocked out selfcomparison Blocked out duplicate comparison Blocked out selfcomparison form that could be distributed to every member of the comparison team in this example. Each member of the team would then perform their ranking by comparing each option in the row with the option in the column. The team member makes an individual decision as to which of the two options is the most important and assigns a numerical ranking (as previously discussed), noting their decisions within the matrix. The resulting matrix would look similar to the one found in Table II. After the comparison team member is finished with their evaluation, they submit their matrix to the team leader. For each team member’s matrix, the team leader sums the results by adding each of the A, B, C, and D values. In the example in Table II, this would result in the following list: . A ¼ 3; . B ¼ 1; . C ¼ 0; and . D ¼ 4. Paired comparison analysis 165 When all the team members have completed their matrices, the results of all of the team members’ matrices are consolidated by summing the matrices. This can be facilitated by combining all the matrices into a single matrix. Assuming the comparison team in our example had three members, Table III demonstrates how the team’s ranking would be combined into a single matrix. Summing this matrix results in the following list: . A ¼ 10 (33.33 percent); . B ¼ 8 (26.67 percent); . C ¼ 3 (10.00 percent); and . D ¼ 9 (30.00 percent). In this example, the team leader has go a step further and converted the values to percentages of the total. As is demonstrated in Figure 1, converting the numbers to percentages can be rather informative. The percentages clearly demonstrate there is only a minor preference for option A over option D. The only thing that is clear is that option C is not really a priority for any of the team members. While this can be somewhat problematic as there is not a clear “winner” among A, B, and D, the process has provided the project manager with a relative mechanism for prioritizing the projects and moving forward with the projects. OpenURL upgrade (A) OpenURL upgrade (A) New search widget (B) Digitize photographs (C) Digital repository (D) New search widget (B) Digitize photographs (C) Digital repository (D) A,2 A,1 D,1 B,1 D,1 D,2 Table II. Example completed paired comparison analysis matrix OCLC 25,3 OpenURL upgrade (A) OpenURL upgrade (A) 166 Table III. Example consolidated paired comparison analysis matrix New search widget (B) Digitize photographs (C) New search widget (B) Digitize photographs (C) Digital repository (D) (A,2), (B,1), (A,3) (A,1), (C,1), (A,3) (D,1), (D,2), (A,1) (B,1), (B,3), (C,1) (D,1), (B,3), (D,3) (D,1), (C,1), (D,1) Digital repository (D) Figure 1. Consolidated percentage rollup of comparison analysis matrices Paired comparison analysis, therefore, is a useful mechanism for prioritizing issues, tasks, and courses of action when there is no objective mechanism for weighing the options. As seen in the example, it is a useful tool for balancing the demands of multiple projects as well. Relatively simple to use, it is a tool that every project manager should know about. References Cervone, H.F. (2009), “Applied digital library project management: using Pareto analysis to determine task importance rankings”, OCLC Systems and Services, Vol. 25 No. 2, pp. 76-81. Saaty, T.L. (2008), “Relative measurement and its generalization in decision making: why pairwise comparisons are central in mathematics for the measurement of intangible factors – the analytic hierarchy/network process”, Review of the Royal Spanish Academy of Sciences, Series A, Mathematics, Vol. 102 No. 2, pp. 251-318, available at: www.rac.es/ ficheros/doc/00576.PDF (accessed May 8, 2009). About the author H. Frank Cervone is the Vice Chancellor for Information Services, Purdue University, Hammond, Illinois, USA. H. Frank Cervone can be contacted at: [email protected] To purchase reprints of this article please e-mail: [email protected] Or visit our web site for further details: www.emeraldinsight.com/reprints