Scribd Logo
Upload

Welcome to Scribd!

  • 11 views

    Verification and Testing of Embedded Systems: Non-Functional Test Bug Characteristics, Lifecycle & Reporting

    Uploaded by

    Ahmad Gafar
    This document discusses non-functional testing and bug reporting. It defines non-functional testing as measuring system behavior and performance under various loads and conditions using specialized tools. It also describes common testing terms like performance testing and load testing. The document defines a bug, lists bug characteristics, and explains the components of a bug report like the reproducible steps, failure description, and severity. It outlines the typical lifecycle of a bug from initial reporting to resolution.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Verification and Testing of Embedded Systems: Non-Functional Test Bug Characteristics, Lifecycle & Reporting

    Uploaded by

    Ahmad Gafar
    11 views20 pages
    This document discusses non-functional testing and bug reporting. It defines non-functional testing as measuring system behavior and performance under various loads and conditions using specialized tools. It also describes common testing terms like performance testing and load testing. The document defines a bug, lists bug characteristics, and explains the components of a bug report like the reproducible steps, failure description, and severity. It outlines the typical lifecycle of a bug from initial reporting to resolution.

    Original Title

    ISTQB (Part 5)

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document discusses non-functional testing and bug reporting. It defines non-functional testing as measuring system behavior and performance under various loads and conditions using specialized tools. It also describes common testing terms like performance testing and load testing. The document defines a bug, lists bug characteristics, and explains the components of a bug report like the reproducible steps, failure description, and severity. It outlines the typical lifecycle of a bug from initial reporting to resolution.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    11 views20 pages

    Verification and Testing of Embedded Systems: Non-Functional Test Bug Characteristics, Lifecycle & Reporting

    Uploaded by

    Ahmad Gafar
    This document discusses non-functional testing and bug reporting. It defines non-functional testing as measuring system behavior and performance under various loads and conditions using specialized tools. It also describes common testing terms like performance testing and load testing. The document defines a bug, lists bug characteristics, and explains the components of a bug report like the reproducible steps, failure description, and severity. It outlines the typical lifecycle of a bug from initial reporting to resolution.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 20

    Verification and Testing of Embedded Systems

    Module 5
    Non-functional Test
    Bug Characteristics, Lifecycle & Reporting

    Prepared By: AMIT Learning


    Outline

    • Non-functional Testing

    • Common Testing Terms

    • Bug Definition, Characteristics, Life cycle and Reporting

    Prepared By: AMIT Learning


    Intro to Performance Test

    • Non-functional test aims to measuring the system behavior and performance


    under loads and variant conditions.

    • Requires accurate measures and massive triggers.

    • Performed using specialized tools, usually cannot be held by manual test.

    Prepared By: AMIT Learning


    Non-functional Testing Types

    • Performance Test: evaluate the SUT performance in terms of the following:

    • Speed: processing time and through put (incase of network applications).

    • Accuracy: precision and losses rate

    • Resources Consumption: CPU load and memory consumption.

    Prepared By: AMIT Learning


    Non-functional Testing Types

    • Load Test: loading the SUT gradually till the usage approaches the system’s
    maximum capacity

    • Stress Test: loading the SUT rapidly till the usage goes beyond the system’s
    maximum capacity in an attempt to break down the system

    Prepared By: AMIT Learning


    Non-functional Testing Types

    • Interoperability Test: test the interaction between the different parts of the
    same system, either from the same vendor or from different vendors

    • Compatibility Test: check that two different systems can work properly
    sharing the same platform

    • Conformance Test: test the conformance to the standardized specs against


    reference devices

    Prepared By: AMIT Learning


    Non-functional Testing Types

    • Reliability Test: test the stability of the SUT over long time of continuous
    operation and under different conditions, “long time” differs from a system to
    another

    • Robustness Test: test the capability of the SUT to tolerate and over come
    invalid inputs and conditions

    Prepared By: AMIT Learning


    Non-functional Testing Types

    • Configuration Test: test the behavior of the SUT under different combinations
    of the possible configurations

    • Usability Test: test how friendly the system is to new users, usually uses the
    Test-as-you-go technique

    Prepared By: AMIT Learning


    Common Testing Terms

    • Monkey Test: unplanned testing, testing randomly without Test Cases

    • Environmental Test: test tolerance for heat, motion, humidity, .etc

    • Alpha Test: actual testing by potential user/customer at the development site

    • Beta Test: actual testing by potential user/customer at the users’ site, the
    product is released to a selected group of users

    Prepared By: AMIT Learning


    Common Testing Terms

    • Regression Test: verify that the updates in the SUT to fix a recently reported
    bug or to integrate a new feature/component did not negatively impacted the
    already passing TCs

    • Regression Test is pretty important and exists along all of the test life
    cycle

    • Performed at the baselines, the builds that imply major changes, and
    periodically

    Prepared By: AMIT Learning


    Bug Definition

    • A Bug is an error or flaw in a computer program that may prevent it from


    working correctly or produce an incorrect or unintended result.

    • Any anomaly from the intended system design, the organization standards,
    the system functional requirements, or the non-functional requirements is
    considered a bug.

    Prepared By: AMIT Learning


    Characteristics of a Bug

    • A bug is said to be consistent if there is a fixed procedure that reproduces


    the bug every time it is executed under the same conditions

    • A bug is said to be inconsistent if the reproduction procedure does not show


    the bug every time it is executed under the same conditions

    • It is necessary to state it if the bug is inconsistent and recommended to


    associate it with the consistency ratio

    Prepared By: AMIT Learning


    Components of a Bug Report

    • Descriptive Summary

    • Avoid vague summaries

    • Avoid threatening and personal impressions!

    • Keep it short and to the point

    • Should help the development team to prioritize the bugs

    • Bug Priority

    • Prioritization indicates which bugs to be fixed first

    • Usually a Technical Manager’s decision

    Prepared By: AMIT Learning


    Components of a Bug Report

    • Environment Setup

    • Full detailed environment setup stating the effective parameters

    • Contains the platform, the tools, the surrounding environment, .etc

    • May refer to the setup stated in the test design document

    • Build number and Feature Under Test (FUT)

    Prepared By: AMIT Learning


    Components of a Bug Report

    • Reproducible Procedure

    • The procedure that gives the highest probability of reproducing the bug, incase it is
    inconsistent

    • The best bug is are producible bug

    • It may refer to a typical TC procedure, only TC identification is required in this case

    • Failure description

    • Hit the root cause and avoid misleading aspects

    • Mention the suspected component, based on the root cause

    Prepared By: AMIT Learning


    Components of a Bug Report

    • Bug Severity

    • Critical / Blocking: An item that prevents further testing of the product or function under test
    can be classified as Critical Bug. No workaround is possible for such bugs.

    • Major / High: A defect that does not function as expected/designed or cause other
    functionality to fail to meet requirements can be classified as Major Bug. A workaround can
    be provided for such bugs.

    • Average / Medium: The defects which do not conform to standards and conventions can be
    classified as Medium Bugs. Easy workarounds exists to achieve functionality objectives.

    • Minor / Low: Cosmetic defects which does not affect the functionality of the system can be
    classified as Minor Bugs.

    Prepared By: AMIT Learning


    Bug Lifecycle

    Prepared By: AMIT Learning


    Sample Test Report

    Prepared By: AMIT Learning


    Q&A

    Prepared By: AMIT Learning


    THANK YOU!

    You might also like