Software security characteristics for function point analysis

Software Security Characteristics for Function Point Analysis N. A. S. Abdullah1, R. Abdullah2, M. H. Selamat2, A. Jaafar2 1 2 Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Shah Alam, Selangor, Malaysia Faculty of Computer Science and Information Technology, Universiti Putra Malaysia, Serdang, Selangor, Malaysia ([email protected]; rusli, hasan, [email protected]) Abstract – Software cost estimation (SCE) has been an important but difficult task since the beginning of the computer era. It considers list of parameters to estimate accurately the software cost. However, security cost is always excluded in most of the parametric cost estimation models. It is because of the security aspect is normally considered late in the software development. To overcome this problem, we proposed an enhancement to one of the parametric estimation models, which is Function Point Analysis (FPA), to address the security concerns. The enhancement suggests potential software security characteristics during system development life cycle (SDLC). These characteristics are then integrated into FPA calculation to encounter the security cost estimation. This paper also points to the validation of a survey findings and calibration of the FPA calculation. II. METHODOLOGY A. Security Review on Parametric Estimation Models Several parametric estimation models have been evaluated from the security viewpoint [9]. The models include Albrecht’s Function Point Analysis (FPA), [3][14][24], Putnam’s Software Life Cycle Model (SLIM) [6][17], COSMIC-FPP [18], Boehm’s COCOMO II [2], and Symon’s Mk II Function Point Analysis (Mk II FPA) [5][19]. It reveals how existing models addressed security and found them deficient [9]. References [13], [15] and [16] have shared the same idea. According to [16], many commercial firms are interested in determining how the costs of implementing different security strategies. They also extended COCOMO II to COSECMO [13] and considered industry practices with security respects [15]. From these reviews, we believe that the existing parametric estimation models unable to estimate the security cost in the software cost estimation. Keywords – Function Point Analysis, Security Cost Estimation, Software Cost Estimation, Software Security Characteristics. I. INTRODUCTION B. Security Characteristics Derivation Software cost estimation (SCE) is a complex activity that requires knowledge of a number of key attributes about the project for which the estimate is being constructed [1]. It has been an important but difficult task since the beginning of the computer era in the 1940s [1]. SCE can be vary depends on different models. But most of the cost estimation models do not consider the costs that invoke security while developing software [10]. It is because security is often an afterthought when developing software and is often bolted on late in development or even during deployment or maintenance [10]. Nevertheless, secure software development has gained momentum. Software houses are now keen to produce secure software as requested by customers’ desire with respect to security and quality of their products [11]. Engineering security will substantially raise software cost and there has been wide variation in the amount of added cost estimated by different models [13][15]. Due to the insufficiency of parametric software cost estimation models in security concerns and the desire of software houses in producing secure software, there is a need to include the security cost during the software cost estimation. To estimate the security cost, we have proposed a software security characteristics formulation [9]. These characteristics are then integrated to FPA to become one of the General System Characteristics (GSCs). 978-1-4244-4870-8/09/$26.00 ©2009 IEEE To develop meaningful security characteristics, the relative importance of these characteristics needs to be established, along with some understanding as to why certain characteristics are perceived to be more important than others [7][12]. Based on the meta-data analysis [9] and supporting studies [12][13][15][16], a software security characteristics formulation has been proposed. Four security standards are analyzed to produce this formulation [9]. These standards include Information Technology Security Cost Estimation Guide [20], Common Criteria for Information Technology Security Evaluation [21], Open Web Application Security Project [22], and Control Objectives for Information and related Technology [23]. These common standards are chosen based on the literature reviews. The security aspects are prearranged into five phases as in Table 1. There are Plan (P), Design (D), Code (C), Test (T) and Deploy (E). In each phase, there are interrelated security aspects. Each security aspect considers the related software security characteristics. From the reviews, there are 48 security characteristics [9] (see Table 2). These characteristics are arranged according to the respective security aspects. They will be the characteristics to be considered whenever the software developers want to estimate for the security cost. These characteristics are integrated in existing FPA as GSC. 394 Proceedings of the 2009 IEEE IEEM TABLE 1 PROPOSED SECURITY CHARACTERISTICS FORMULATION Step Plan (P) Design (D) Security Aspects (1) Security Requirements (SR) As mentioned, the total number of UAF is accumulated from five components as in (2). The simplified equation is as follows: UAF = EI + EO + EQ + ILF + EIF. (2) where EI = External Inputs EO = External Outputs EQ = External Inquires ILF = Internal Logical Files EIF = External Interface Files Security Features (SF); Functional Features (FF) Code (C) Attack Planning (AP); Formal Review and Sign-off (FR); Secure Coding, Review and Audit (SCR) Test (T) Software Security Assurance (SSA); Final Security Review (FSR); Infrastructure Application Security Measures (ASM) Deploy (E) FP = UAF * VAF where UAF = Unadjusted Function Point VAF = Value Adjusted Factor Software Hardening Monitoring (SHA) & Application Security C. Measurement Instrument Setting The weights are assigned to each component based on transactional and data function types. For VAF, it is calculated from the summation of 14 GSCs as in (3). The proposed software security characteristics might or might not involve during the real world practices of SDLC in SCE. In response, we hypothesized that H0 : µsecurity ≥ µm where µsecurity = Software security characteristics from meta-data analysis µm = Malaysian software developers’ awareness towards the listed security characteristics. A survey has been conducted to test the proposed hypothesis [8]. The result indicates the extent of software security characteristics implemented in Malaysia software developers. To validate the formulation with current developers’ practices, a questionnaire has been setup as measurement instrument. The security characteristics are transformed into questions (refer as items in RASCH) that scale from 1 to 4 to indicate the degree of importance. The sample unit is the respondents that representing Multimedia Super Corridor (MSC) software houses and directly involved in software development related works. The results [8] from the survey were tabulated and run in Winsteps, RASCH [4] analysis software, to obtain the logit values. It helps to obtain the relative importance for the security characteristics to enhance the FPA model. ∑ Ci )/100]. 14 VAF = 0.65 + [( (3) i =1 where Ci = degree of influence for each Genaral System Characteristic i = is from 1 to 14 representing each GSC Σ = is summation of all 14 GSCs. From the literature [8][9], we strongly recommended that security can be treated as one of the general system characteristics in the current real world application estimation. Therefore, we proposed considering security as individual characteristic in GSCs in FPA model. However, this proposed enhancement is still need to be proven. We have initiated an effort to validate experimentally that the model can predicts the impact of added security cost. This experiment can calibrate the model to collect and analyze the actual impact of security characteristics. III. RESULTS A. Survey Results D. Review on Function Point Analysis for Enhancement Possibility 1) Person Measure Summary Statistics: The major finding is the Person Mean, µperson = 1.59logit, hence person awareness of the security characteristics is positive, which above the item mean. The person reliability is excellent, which is 0.97. 2) Item Measure Summary Statistics: The Item Summary gives poor separation, G=1.08 and poor Reliability = 0.54, which is lower than threshold 0.67. However, it has good item spread of 3.74logit with SDi = 0.70. It requires review on the both end of very difficult and very easy items, which shows a hollow area need to be patched up. In this analysis, items are referred to security characteristics. Therefore, most of the security characteristics (N=19) are basically implemented by all the respondents. This number is used in the score in GSC. Function Point Analysis (FPA) [3][14] consists of two main parts in the measurement. First part is five components; include External Inputs (EI), External Outputs (EO), External Inquires (EQ), Internal Logical Files (ILF) and External Interface Files (ELF). These components are evaluated by complexity weights to produce Unadjusted Function Point (UAF). The second part is 14 General System Characteristics (GSCs) that measured from 0 to 5 nominal scales. These characteristics contribute to Value Adjusted Factor (VAF). Based on the Function Point Analysis Training Course by [24], the final function point count is obtained by multiplying the VAF times the Unadjusted Function Point (UAF). The standard equation for estimation is: 395 Proceedings of the 2009 IEEE IEEM 3) Person-Item Distribution Map: The analysis reveals a person spread of 5.52logit with good separation, G=3.64 and excellent Reliability of Cronbach-α = 0.97, which means the survey outcome is acceptable due to Cronbach-α is greater than 0.67. From the calculations and results [8], generally the respondents have high level of awareness in implementing security characteristics throughout SDLC; µperson of 83.06%, which is higher than 60% threshold limit. Hereby, the Person Mean, µperson = 1.59 ≥ 0.00, with significant of p=0.05. Therefore, the H0 is accepted. It means the proposed security characteristics are valid, relevant and implemented by the software houses as current practices in Malaysia. B. Security Enhancements for FPA After analyzing the results [8], we intended to increase another General System Characteristic for security concern. The enhancement for VAF is proposed to do some modification on (3) and produce the following equation. ∑ Ci + Security) /100]. (4) i =1 where Ci = degree of influence for each Genaral System Characteristic i = is from 1 to 14 representing each GSC Σ = is summation of all 14 GSCs Security = Degree of Influence for Security Characteristics. SF User Identify Management; Interpreter Injection; Authorization; User Attribute Definition; Protection of Security Technology; Data Validation; Authentication; Session Management; Cryptography; Exchange of Sensitive Data; Verification of Secrets; Data Integrity FF Management Security Functions Behavior; Production and Input/Output Controls; File System; Web Services; Incident Response Capability; Handling E-Commerce Payments; Configuration; Buffer Overflow; Administrator Interfaces; Personnel Security SCR Audit Trial; Error Handling, Auditing and Logging; Cheat Sheets SSA Security Testing; Security Surveillance; Security Monitoring FSR Final Review of Security Controls; Final Security Attribute based Access Control; Final Logical Access Controls; Final Audit Data Generation ASM Security Awareness; Security Education; Security Training SHA Hardware Maintenance; Documentation; System Software Maintenance 0 1 2 3 4 5 Descriptions To Determine Degree of Influence None of the above Below 20% (1 ≤ N ≤ 10) 20% to 39% (11 ≤ N < 20) 40% to 63% (20 ≤ N ≤ 30) 64% to 79% (31 ≤ N ≤ 38) 80% and above (N > 38) IV. DISCUSSION The proposed software security characteristics are treated as items in RASCH analysis method. This analysis method has validated the items (refer to security characteristics) that practiced by the software houses in Malaysia. Therefore, the item validity from the RASCH method has supported the acceptance of security characteristics. However, the proposed enhancement on FPA model is currently transformed as online estimation tool and still in progress of testing. This tool needs to collect data through controlled experiments and verifies the proposed enhancement of FPA model. Security Characteristics Specification of Management Functions; Abuse Cases; Threat Risk Modeling; Risk Analysis; Management of IT Security; PHP Guidelines; Security Rules; IT Security Plan Review of Security Controls; Logical Access Controls; Audit Data Generation; Security Attribute based Access Control Score As TABLE 2 SECURITY CHARACTERISTICS SR FR TABLE 3 DEGREE OF INFLUENCE FOR SECURITY CHARACTERISTICS For ease of giving degree of influence, we listed the security characteristics (see Table 2). The user can identify the number of characteristics that might take into consideration during development. The number will indicate the score for degree of influence. Table 2 and Table 3 will be the additional characteristic sheet for GSCs in FPA model. Security Aspects Phishing; Denial of Service Attacks; Non-bypass Ability of TSP; Malicious Software Prevention, Detection and Correction As in the survey result, 19 items or security characteristics are basically implemented by all the respondents. Therefore, 19/48 x 100% = 39.58% ≈ 40%. The basic and common implementation is normally given score 3 in GSCs. Therefore, 20 items and above will get 3 as degree of influence. Based on the Measure Order of Difficulty for the items [8], the suggested score is as below. 14 VAF = 0.65 + [( AP V. CONCLUSION The importance of estimating security cost is highlighted in few studies. In response, we proposed an enhancement on FPA model to encounter the security 396 Proceedings of the 2009 IEEE IEEM [10] S. Ardi, D. Byers, N. Shahmehri, “Towards a Structured Unified Process for Software Security”, in Proc. ICSESESS, 2006, pp. 3-9. [11] Jari, R, “Contracting over the Quality Aspect of Security in Software Product Markets”, in Proc. of 2nd ACM workshop on Quality of Protection, 2006. pp. 19-26. [12] Frederick, L. “The Importance of Identifying Risk during Project Planning”; in Proc. of Project Management in Practice-The 2006 Project Risk And Cost Management Conference, 2006, pp. 74-83. [13] Colbert E., Wu D., Chen Y., and Boehm B, “Cost Estimation for Secure Software and Systems 2006 Project Update”, University of Southern California: Center of Software Engineering, USA, 2006. [14] Albrecht, A. J., “Measuring application development productivity”, in Proc. of IBM Application Development Symp, 1979, pp. 83-92. [15] Wu, D., and Yang, Y., “Towards An Approach for Security Risk Analysis in COTS Based Development”, Center for Software Engineering, University of Southern California, Los Angeles, CA. Proceedings of Software Process Workshop on Software Process Simulation 2006, USCCSE-2006-605, May 2006 [16] Reifer, D.J., Boehm, B.W., and Gangadharan, M., “Estimating the Cost of Security for COTS Software”, Center for Software Engineering, University of Southern California, Los Angeles, CA. Technical Report USC-CSE2003-502, 2003. [17] Panlilio N.Y., “Software Estimation Using the SLIM Tool”, IBM Canada Ltd Laboratory Technical Report 74.102, 1994. [18] C. Symons. (2007, Sep.). The COSMIC Functional Size Measurement Method Version 3.0 – Measurement Manual. Common Software Measurement International Consortium, UK. [Online]. Available: http://www.gelog.etsmtl.ca/cosmic-ffp [19] UKSMA Metrics Practices Committee. (1998, Sep.). Mk II Function Point Analysis – Counting Practices Manual Version 1.3.1. United Kingdom Software Metrics Association (UKSMA), UK. [Online]. Available: http://www.eee.metu.edu.tr/~bilgen/MARK%20II%20FP% 20Guide.pdf [20] Department of Education, USA. (2002, Nov.). Information Technology Security Cost Estimation Guide. United States of America. [Online]. Available: http://csrc.nist.gov/groups/SMA/fasp/documents/pm/ED_IT _Security_Cost_Estimation_Guide_NIST.doc [21] CCRA Working Group, (2007, Sep.). Common Criteria for Information Technology Security Evaluation, version 3.1. [Online]. Available: http://www.commoncriteriaportal.org/ [22] The Open Web Application Security Project, (2005, Jul.). “A Guide to Building Secure Web and Web Services”, Black Hat 2nd Edition. [Online]. Available: http://www.owasp.org/index.php/Category:OWASP_Guide _Project [23] ISACA, IT Governance Institute, USA, (2007). “Control Objectives for Information and related Technology (COBIT) – Framework, Control Objectives, Management Guidelines, and Maturity Models”. [Online]. Available: http://www.isaca.org/template.cfm?Section=COBIT6 [24] Longstreet, D., (2004, Oct.). Function Points Analysis Training Course. Blue Springs. [Online]. Available: http://www.softwaremetrics.com/freemanual.htm cost. Meta-data analysis on software security characteristics has produced the software security formulation. This formulation is then transformed as items in a measurement instrument. The result of the survey was tabulated and analyzed by using RASCH analysis method. This method verifies the person and item validity. The result of analysis supported the hypothesis that the formulation is valid, relevant and currently implemented by the software houses. These proposed security characteristics are integrated as one of the GSCs in FPA model. It proposed an addition sheet of GSCs with responsive degree of influences. However, the results of the calculations still need to be verified through experiments. Online estimating tool has be setup to collect data for further verification purpose. ACKNOWLEDGMENT N. A. S. Abdullah thanks The Ministry of Higher Education Malaysia and Universiti Teknologi MARA for giving financial support and scholarship for this study. REFERENCES [1] T.C., Jones, Estimating Software Costs. United States of America: McGraw-Hill, 1998, pp. 3-35. [2] Boehm, B., Abts, C., Brown, A.W., Chulani, S., Clark, B.K., Horowitz, E., Modachy, R., Reifer, D., Steece, B., Software Cost Estimation with COCOMO II. New Jersey: Prentice Hall, Inc, 2000, pp. 145-151, pp. 284-291. [3] D. Garmus and D. Herron, Function Point Analysis: Measurement Practices for Successful Software Projects. United States of America: Addison-Wesley, 2001, pp. 83171. [4] T.G. Bond and C. M. Fox, Applying the Rasch Model: fundamental measurement in the Human Sciences, 2nd ed. New Jersey: Lawrence Erlbaum Associates, Inc., 2007, pp. 29-48. [5] Symons, C., Software Sizing and Estimating – Mark II FPA. United Kingdom: Wiley Series in Software Engineering, 1991. [6] Putnam, L.H., “A general empirical solution to the macro software sizing and estimation problem”, IEEE Transactions on Sofiware Engineering, ~01.4, no. 4, pp. 345-381, Jul. 1978. [7] Keil, M., Cule, P. E., Lyytinen, K. and Schmidt, R.C. “A Framework for Identifying Software Project Risks”, Communication of the ACM, vol 41, no II, pp. 76 – 83, Nov. 1998. [8] Abdullah, N.A.S.; Abdullah, R.; Selamat, M.H.; Jaafar, A., “Validation of Security Awareness in Software Development by using RASCH Measurement”, Pacific Rim Objective Measurement Symposium 2009 Hong Kong (PROMS 09 HK), pp. 100, Jul 2009. [9] Abdullah, N.A.S.; Abdullah, R.; Selamat, M.H.; Jaafar, A., “Potential security factors in software cost estimation”, Information Technology, 2008. ITSim 2008. International Symposium vol 3. pp. 1 – 9, 26-28 Aug. 2008. Digital Object Identifier: 10.1109/ITSIM.2008.4631983 397