Scribd Logo
Upload

Welcome to Scribd!

  • 349 views

    Software Test Metrics

    Uploaded by

    G.C.Reddy
    For more QTP Information & Scripts visit: www.gcreddy.com Software Test Metrics I) Introduction When we can measure what we are speaking about and express it in numbers, we know something about it; but when we cannot measure, when we cannot express it in numbers, our knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but we have scarcely, in your thoughts, advanced to the stage of science. Why we need Metrics? “We cannot improve what we cannot measure.” “We

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as DOC, PDF, TXT or read online from Scribd

    Software Test Metrics

    Uploaded by

    G.C.Reddy
    349 views4 pages
    For more QTP Information & Scripts visit: www.gcreddy.com Software Test Metrics I) Introduction When we can measure what we are speaking about and express it in numbers, we know something about it; but when we cannot measure, when we cannot express it in numbers, our knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but we have scarcely, in your thoughts, advanced to the stage of science. Why we need Metrics? “We cannot improve what we cannot measure.” “We

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    DOC, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    For more QTP Information & Scripts visit: www.gcreddy.com Software Test Metrics I) Introduction When we can measure what we are speaking about and express it in numbers, we know something about it; but when we cannot measure, when we cannot express it in numbers, our knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but we have scarcely, in your thoughts, advanced to the stage of science. Why we need Metrics? “We cannot improve what we cannot measure.” “We

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as DOC, PDF, TXT or read online from Scribd
    Download as doc, pdf, or txt
    349 views4 pages

    Software Test Metrics

    Uploaded by

    G.C.Reddy
    For more QTP Information & Scripts visit: www.gcreddy.com Software Test Metrics I) Introduction When we can measure what we are speaking about and express it in numbers, we know something about it; but when we cannot measure, when we cannot express it in numbers, our knowledge is of a meager and unsatisfactory kind: it may be the beginning of knowledge, but we have scarcely, in your thoughts, advanced to the stage of science. Why we need Metrics? “We cannot improve what we cannot measure.” “We

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as DOC, PDF, TXT or read online from Scribd
    Download as doc, pdf, or txt
    of 4

    For more QTP Information & Scripts visit: www.gcreddy.

    com

    Software Test Metrics


    I) Introduction

    When we can measure what we are speaking about and express it in numbers, we know
    something about it; but when we cannot measure, when we cannot express it in numbers,
    our knowledge is of a meager and unsatisfactory kind: it may be the beginning of
    knowledge, but we have scarcely, in your thoughts, advanced to the stage of science.

    Why we need Metrics?

    “We cannot improve what we cannot measure.”


    “We cannot control what we cannot measure”

    AND TEST METRICS HELPS IN

     Take decision for next phase of activities


     Evidence of the claim or prediction
     Understand the type of improvement required
     Take decision on process or technology change

    II) Type of metrics

    Base Metrics (Direct Measure)

    Base metrics constitute the raw data gathered by a Test Analyst throughout the testing
    effort. These metrics are used to provide project status reports to the Test Lead and
    Project Manager; they also feed into the formulas used to derive Calculated Metrics.

    Ex: # of Test Cases, # of Test Cases Executed

    Calculated Metrics (Indirect Measure)

    Calculated Metrics convert the Base Metrics data into more useful information. These
    types of metrics are generally the responsibility of the Test Lead and can be tracked at
    many different levels (by module, tester, or project).

    Ex: % Complete, % Test Coverage

    Base Metrics & Test Phases

    • # of Test Cases (Test Development Phase)


    • # of Test Cases Executed (Test Execution Phase)
    • # of Test Cases Passed (Test Execution Phase)
    • # of Test Cases Failed (Test Execution Phase)
    • # of Test Cases Under Investigation (Test Development Phase)
    • # of Test Cases Blocked (Test dev/execution Phase)

    For more Manual Testing Docs visit: www.gcreddy.net 1


    For more QTP Information & Scripts visit: www.gcreddy.com

    • # of Test Cases Re-executed (Regression Phase)


    • # of First Run Failures (Test Execution Phase)
    • Total Executions (Test Reporting Phase)
    • Total Passes (Test Reporting Phase)
    • Total Failures (Test Reporting Phase)
    • Test Case Execution Time ((Test Reporting Phase)
    • Test Execution Time (Test Reporting Phase

    Calculated Metrics & Phases

    The below metrics are created at Test Reporting Phase or Post test Analysis phase
    • % Complete
    • % Defects Corrected
    • % Test Coverage
    • % Rework
    • % Test Cases Passed
    • % Test Effectiveness
    • % Test Cases Blocked
    • % Test Efficiency
    • 1st Run Fail Rate
    • Defect Discovery Rate
    • Overall Fail Rate

    III) Crucial Web Based Testing Metrics

    Test Plan coverage on Functionality

    Total number of requirement v/s number of requirements covered through test scripts.
    • (No of requirements covered / total number of requirements) * 100
    Define requirements at the time of Effort estimation

    Example: Total number of requirements estimated are 46, total number of requirements
    tested 39; blocked 7…define what is the coverage?

    Note: Define requirement clearly at project level

    Test Case defect density

    Total number of errors found in test scripts v/s developed and executed.
    • (Defective Test Scripts /Total Test Scripts) * 100

    Example: Total test script developed 1360, total test script executed 1280, total test script
    passed 1065, total test script failed 215
    So, test case defect density is
    215 X 100
    ---------------------------- = 16.8%
    1280

    This 16.8% value can also be called as test case efficiency %, which is depends upon total
    number of test cases which uncovered defects

    For more Manual Testing Docs visit: www.gcreddy.net 2


    For more QTP Information & Scripts visit: www.gcreddy.com

    Defect Slippage Ratio

    Number of defects slipped (reported from production) v/s number of defects reported
    during execution.
    • Number of Defects Slipped / (Number of Defects Raised - Number of Defects
    Withdrawn)

    Example: Customer filed defects are 21, total defect found while testing are 267, total
    number of invalid defects are 17

    So, Slippage Ratio is


    [21/ (267-17)] X 100 = 8.4%

    Requirement Volatility

    Number of requirements agreed v/s number of requirements changed.


    • (Number of Requirements Added + Deleted + Modified) *100 / Number of Original
    Requirements
    • Ensure that the requirements are normalized or defined properly while estimating
    Example: VSS 1.3 release had total 67 requirements initially, later they added another 7
    new requirements and removed 3 from initial requirements and modified 11 requirements

    So, requirement Volatility is


    (7 + 3 + 11) * 100/67 = 31.34%
    Means almost 1/3 of the requirement changed after initial identification

    Review Efficiency

    The Review Efficiency is a metric that offers insight on the review quality and testing
    Some organization also use this term as “Static Testing” efficiency and they are aiming to
    get min of 30% defects in static testing

    Review efficiency=100*Total number of defects found by reviews/Total number of project


    defects

    Example: A project found total 269 defects in different reviews, which were fixed and test
    team got 476 defects which were reported and valid

    So, Review efficiency is [269/(269+476)] X 100 = 36.1%

    Efficiency and Effectiveness of Processes

    • Effectiveness: Doing the right thing. It deals with meeting the desirable attributes
    that are expected by the customer.

    • Efficiency: Doing the thing right. It concerns the resources used for the service to
    be rendered

    For more Manual Testing Docs visit: www.gcreddy.net 3


    For more QTP Information & Scripts visit: www.gcreddy.com

    Metrics for Software Testing

    • Defect Removal Effectiveness

    DRE= Defects removed during development phase x100%


    Defects latent in the product

    Defects latent in the product = Defects removed during development


    Phase+ defects found later by user

    • Efficiency of Testing Process (define size in KLoC or FP, Req.)

    Testing Efficiency= Size of Software Tested


    Resources used

    For more Manual Testing Docs visit: www.gcreddy.net 4

    You might also like