Available online at www.sciencedirect.

com

Physics Procedia 25 (2012) 1857 – 1862

2012 International Conference on Solid State Devices and Materials Science

An Evaluation Model of Software Testing Management

in Core Banking System Programme
Wei Zhang, Fuqiang Zhu
Shandong Provincial Key Laboratory of Computer Network,
Shandong Computer Science Center,
25000 Jinan, China

Abstract

Core banking system is the most important software system for commercial banks. The complexities of the core
banking system projects lead to that the financial software testing is very complex and difficult. This paper analyzes
the traditional software testing management evaluation criteria, and proposes a new evaluation model. By removing
the correlation between traditional evaluation criteria, the model can obtain a set of uncorrelated evaluation factors,
and these factors can be used for the evaluation of software testing management. Experiments have proved that this
model can be more scientific and objective for the evaluation of software testing management in core banking
systems programme.

© 2012
© 2011 Published by Elsevier Ltd.
B.V.Selection
Selectionand/or
and/orpeer-review
peer-reviewunder
underresponsibility ofof
responsibility [name
Garryorganizer]
Lee
Open access under CC BY-NC-ND license.
Keywords: software testing management; evaluation factors; evaluation model; core banking system programme

1 Introduction

With the development of financial industry and information technology, the increasing of commercial
banks’ development is directly dependent on the core banking systems. Software development, testing and
application of the core banking systems are becoming into decisive factors of the development of the
banking industry. As the increasingly demands of the customers, the core banking software has became
more and more complicated, and various aspects of the core banking software project are becoming
research hotspots gradually. Meanwhile, the software testing of the core banking system has been to the
joint between banking business and banking information system. In the projects of core banking system,
software testing is through every stage of project lifecycle and drives the progress of the project.
Compare with other software projects, the core banking system requires higher complexity, security
and accuracy. It will take the software testing and testing management a series of difficulty. Therefore,
how to evaluate the management quality of core banking system will become a very essential problem.
This paper analyzes the common evaluation criteria of software testing management in core baking

1875-3892 © 2012 Published by Elsevier B.V. Selection and/or peer-review under responsibility of Garry Lee
Open access under CC BY-NC-ND license. doi:10.1016/j.phpro.2012.03.322
1858 Wei Zhang and Fuqiang Zhu / Physics Procedia 25 (2012) 1857 – 1862

system, and focuses on the short point of traditional criteria. With orthogonalizing and standardizing the
programme matrix and the evaluation criteria matrix, this paper puts forward a new kind of software
testing management evaluation model, which has got good efforts in practical application.

2 Software Testing Management

Financial software testing project management is quite different with the traditional information system
testing project management. Because of the complexity of banking operations, core banking system project
is often posed by a number of sub-projects. For example, banking operations include deposits, loans, bank
cards, gross settlement, etc. Each banking operations system projects have become a separate sub-project.
These sub-projects constitute the core banking system programme. Programme is a collection of
interrelated projects, which are managed at the same time, but independent of each other [1]. The main
feature of the programme is that between sub-projects are interrelated, and requirement analysis, designing,
coding, testing of these sub-projects should be done in roughly the same time. The programme is divided
into two categories [2]. The first category is the discrete programme, which main characteristic is that IT
outsourcing companies are taken as the main cluster of the project management organization and structure.
The second category is the continuous programme, and this kind is based on the user side as the main body
to undertake the organization and structure of the project management [3].
The software testing management is also interrelated among these sub-projects in the core banking
system progranmme. It has become a difficult problem to evaluate the quality of software testing
management in these sub-projects. The method for evaluating the quality of the software testing which
is measured by measuring software reliability growth can be applied [4]. This method also can be
advanced by analysis of software adequacy criteria and fault-detecting ability measurement theory [5].
And the common method of simple to use is giving the sub-projects scores by the traditional evaluation
criteria. There are many commonly used evaluation criteria, such as test case coverage rate, case tested
rate, test hit rate, average case execution time. Table I lists some commonly used evaluation criteria.
Through the evaluation, we can judge the quality of software testing management according to the
experience. But, we cannot judge by the simple addition of the evaluation criteria, due to the different
dimensions and the correlation of each other of the evaluation criteria.

Table 1. Commonly Used Evaluation Criteria

No Evaluation criteria Detailed definition

The proportion of function points included by caces in the total
1 Test case coverage rate
points
The proportion of cases have been tested in the total number of
2 Case tested rate
cases
3 Testing hit rate The proportion of problems have in the total number of cases
4 Average case execution time The average time spent by each case execution
5 State ratio The proportion of caces been testing and tested in the total cases

3 The Evaluation Model

In general, the scientific and objective evaluation system requires that all evaluation criteria should be
independent of each other; there is no correlation between the evaluation factors. There are a lot of
information overlaps among the traditional evaluation criteria. And in this paper, information overlaps are
picked with programme matrix, the evaluation criteria matrix, and the factor analysis method.
 Wei Zhang and Fuqiang Zhu / Physics Procedia 25 (2012) 1857 – 1862 1859

3.1 Subproject-Evaluation Matrix

According to the sub-projects and the corresponding traditional evaluation criteria, we can get the
Subproject-Evaluation Matrix I. Each column in the matrix represents the value of each sub-project at the
same evaluation criteria. And each line represents all the values of evaluation criteria in one sub-project.
ª i11 i12 ! i1n º
«i
« 21 i22 " i 2 n »»
I (1)
«# # % # »
« »
¬im1 i m 2 " imn ¼

3.2 Nondimensionalization and Normalization

Some indicators are positive indicators, which mean that the larger evaluation criteria value represents the
higher the quality of software testing, while the others are negative indicators. So we should normalize
and nondimensionalize the matrix with the following substitution. S m is the variance of evaluation
criteria of the sub-project m. And the Subproject-Evaluation Matrix I can be changed into the normalized
matrix I’.

imn  im
c
imn (2)
Sm

3.3 Uncorrelating

Between the various indicators are interrelated. The correlation matrix J can be obtained according to
the values of correlation between each two indicators. This matrix is symmetric. The overlapping
information can be picked according to the method of orthogonalizing the matrix. So, we calculate the
eigenvalues O1 , O 2 , ! O n , and the eigenvector corresponds to each eigenvalues [6].

ª1 j12 ! j1n º
«j 1 " j 2 n »»
J « 21 (3)
« # # % # »
« »
¬ j n1 jn2 " 1 ¼

3.4 The Evaluation Factors

For the sub-project, there is a group of the eigenvalues O1 , O 2 , ! O n . These eigenvalues represent weight
of each evaluation criteria. And using these eigenvalues, an evaluation model can be got with the
evaluation factors. We can get a more scientific rating with the following substitution. As the overlapping
information has been picked, The evaluation model is more objective characterization of the quality of
each sub-project software testing.
1860 Wei Zhang and Fuqiang Zhu / Physics Procedia 25 (2012) 1857 – 1862

n Op
Em ¦O '
imp (4)
p 1 1  O2  "  On

Each sub-project is evaluated through this model. It can be more clearly shown which sub-project
software testing management is more effective. For sub-projects which got lower scores, testing
administrators can deploy more resources.

4 Application of the Model and Analysis

Shandong City Commercial Bank Alliance project (SCCBA) is a typical programme of core banking
system, which not only constitute the various sub-project of each outsourcing company, but also
distinguish the sub-project under the banking business. So the project management is very complicated.
In the system's software testing management, if only the traditional testing management evaluation was
used, it is difficult to evaluate the quality of software testing management correctly. We selected the data
of the SCCBA project in a day.

Table 2. Data of SCCBA Programme

No 1 2 3 4 5

Deposits 95.20% 97.37% 2.50% 1.2day 78.03%

Loans 89.53% 98.80% 0.81% 1.8day 57.44%
Bank cards 96.40% 96.05% 4.82% 0.8day 84.60%
Gross settlement 86.60% 94.30% 1.40% 1.1day 60.21%
Customer management 90.80% 89.58% 2.90% 0.6day 70.59%
Table 2 shows the software testing management situations of the five sub-projects. They are deposits,
loans, bank cards, gross settlement, and customer management. And the evaluation criteria are Test case
coverage rate, Case tested rate, Testing hit rate, Average case execution time, and State ratio. The number
of criteria is same as witch in Table 1.
The table above constitutes the Subproject-Evaluation Matrix I. the normalized matrix I’ can be got
from the equations (1) and (2). The result is the following matrix. Evaluation of each subproject cannot
be easily seen, because there are the information overlaps among the various values of the matrix I’.
ª0.83 0.64 0.83 0.54
0.76º
«0.76 0.69 0.73 0.58»»
0.48
«
I' «0.89 0.60 0.96 0.82»
0.68 (5)
« »
«0.67 0.62 0.78 0.50
0.61»
«¬ 0.81 0.57 0.82 0.70 0.70¼»

We need to eliminate these information overlaps. According to the data provided, the correlation
matrix J is the equation (6) as follows.
ª1.00 0.10 0.02 0.02 0.04º
«0.10 1.00 0.80 0.32 0.85»»
«
J «0.02 0.80 1.00 0.12 0.82» (6)
« »
«0.02 0.32 0.12 1.00 0.70»
¬«0.04 0.85 0.82 0.70 1.00 ¼»
 Wei Zhang and Fuqiang Zhu / Physics Procedia 25 (2012) 1857 – 1862 1861

By calculating the matrix J eigenvalues O1 , O2 ,! On , the following table shows the weight of each
evaluation criteria [6].

Table 3. Weight of Each Evaluation Criteria

No 1 2 3 4 5
On 0.03 1.93 1.24 0.68 2.67
Weight 0.46% 29.47% 18.93% 10.38% 40.76%

ach sub-project can be integrated, as shown in Table IV. We can easily evaluate the situations of
software testing management of these sub-projects. This evaluation is difficult to obtain through the
traditional evaluation criteria shown in Table II. It can be seen through the ranks, that software testing
situation of Sub-project Deposits is better, and the situation of Sub-project Customer management is
worse. These ranks have brought great convenience for the management of software testing in the
SCCBA project. The model in this programme has been a good application, reflects the very good results.

5 Conclusions

Software testing management of core banking system programme is very complex. It is not easy to
evaluate the quality of the software testing with the traditional evaluation criteria. Based on the traditional
evaluation criteria with nondimensionalization and normalization, a new evaluation model has been put
forward in this paper. The application has proved that the model can facilitate an objective evaluation of
software testing management in core banking system programme, and bright great convenience for the
software project management.

Table 4. Scores of Sub-projects

Sub-project Score Em Rank

Deposits 0.7386 1
Loans 0.7178 3
Bank cards 0.7245 2
Gross settlement 0.6441 4
Customer management 0.6236 5

Acknowledgment

This work was supported by the National Natural Science Foundation of China under Grant No.
61070039, the Fund for the Doctoral Research of Shandong Academy of Sciences under Grant No. 2010-
12, and the Outstanding Young and Middle-aged Scholars Foundation of Shandong Province of China
under Contract No. BS2011DX033.
1862 Wei Zhang and Fuqiang Zhu / Physics Procedia 25 (2012) 1857 – 1862

References

[1] Jeffry K. Pingto, “Project management: Achieving competitive advantage”.Person Education, Inc.,2007, pp.
84-85.
[2] YANG Fu-Qing, “Thinking on the development of software engineering technology,” Journal of Software,
vol.16No.1, 2005, pp.1-7.
[3] XIE Meilong, LI Congdong, XUE Qiong, ZHU Konglai, “Research on executive performance of financial
software programme,” Journal of Xidian University(Social Science Edition), vol.19No.2,pp.35-39, 2009
[4] Phyllis G. Frankl, Richard G. Hamlet, Bev Littlewood, “Evaluating testing methods by delivered
reliability,” IEEE Transactions on Software Eng, vol.24No.8, 1998.
[5] Xu Zhongwei,Wu Fangmei, “Software testing quality measurement,” Computer Engineering and
Applications, vol.21, 2002, pp.101-102.
[6] LIU Ye-ling and JI Zhan-huai, “The method of computing the eigenvalues and eigenvectors of real and
symmetric matrix,” Journal of Xi’an University of Science and Technoligy, vol.27No.2, 2007