Verification and Testing of Embedded Systems

Module 3
Software Dynamic Verification

Prepared By: AMIT Learning

 Outline

• Dynamic Verification; Definition and Scope

• Objectives of Software Testing

• Common Testing Terms and Concepts

• Testing Levels; Definitions and Sequence

• Unit Test

• Integration Test

• System Test

• Acceptance Test
Prepared By: AMIT Learning
 Definition & Scope of Testing

• Definition: Testing is the process of executing a program with the intention of

finding errors.

• Scope: Testing can not demonstrate the absence of errors. It can only
demonstrate the presence of errors.

• It represents the ultimate verification of the requirements specification, the

design, and the code.

• Many organizations spend 40-50% of the development time in testing.

Prepared By: AMIT Learning

 What is Software Testing?

• Software Testing is the process of establishing the existence of errors; The

main aim of testing is to find errors.

• Software Testing is any activity aimed at evaluating an attribute or capability

of a program or system and determining that it meets its required results.

• Even if the best inspection methods are used, testing is necessary.

Prepared By: AMIT Learning

 Objectives of Software Testing

• Make sure the software works as expected.

• Make sure that software meets the requirements.

• Find as many problems as possible.

• Make sure the application is a quality application.

• Make sure that the system does not do anything that it is not supposed to do.

• Try to minimize the risk that your customers would have by using the system

Prepared By: AMIT Learning

 Common Testing Terms

• Error is a mistake made by a human (in a software development activity)

• Defect (or fault) is the result of introducing an error into a software artifact

• Failure is a departure from the required behavior for a system

• The failure mechanism:

• Error -> Defect ->Failure

Prepared By: AMIT Learning

 Common Testing Terms

• Debugging is concerned with finding where defects occur (in code, design or
requirements) and removing them; fault identification and removal.

• Build Verification Test (Build Acceptance Test) is a set of tests run on each
new build of a system to verify that the build is testable before the build is
released into the hands of the test team.

Prepared By: AMIT Learning

 Common Testing Terms

• Positive Testing is the process of executing a program or a system with the

intent of making sure that it does everything it is supposed to do.

• Performed through providing valid data and/ or valid conditions that will be
processed through the normal paths.

• Negative Testing is the process of executing a program or a system with the

intent of making sure the system does not do anything it is not supposed to do.

• Performed through providing invalid data and/ or invalid conditions that will
be processed through the error handling paths.

Prepared By: AMIT Learning

 Common Testing Terms Test Case

• Test Case is a collection of input values (test data) and a collection of expected output
values designed to test a specific situation of a running software system or part of a
system.

• Test case specification lists inputs, expected outputs, features to be tested by this
test case, and any other special requirements e.g. setting of environment variables
and test procedures. Dependencies with other test cases are also specified.

Prepared By: AMIT Learning

 Common Testing Terms Test
Scenario
• Test Scenario is a situation that could possibly happen when the system is in
operation.

• Relation between requirements and scenarios is many - to - many; a requirement may

has more than one scenario and a scenario may touch on more than one requirement.

• If a requirement was written to be very specific, it is possible for a requirement to

have only one scenario to be tested.

Prepared By: AMIT Learning

 Common Testing Terms Test
Scenario: Example 1
• Requirement:

• System shall allow customers to check the status of their orders.

• Scenarios:

• Check the status of an order that does not exist

• Check the status of an order that was shipped

• Check the status of an order that has not been shipped.

• Check the status of an order that has been cancelled.

Prepared By: AMIT Learning

 Common Testing Terms Test
Scenario: Example 2
• Requirements:

• Seniors will receive a 10% discount on all tickets

• Children under the age of 12 will be charged only 60% of the price unless
they are traveling by them selves in which case they will be charged 80%
of the price.

• Scenario:
• Test a reservation for a child under the age of 12 traveling with a senior

Prepared By: AMIT Learning

 Common Testing Terms Test
Scenario: Example 3
• Requirements:
• The Vehicle is able to park automatically in parallel slots or perpendicular

• Scenario:
• Test auto parallel parking on the right side of the road
• Test auto parallel parking on the left side of the road
• Test auto perpendicular backward parking in slot = vehicle dimensions +
20%

Prepared By: AMIT Learning

 Relationship between Testing
Elements
• Extract Test Scenarios from Requirements then create the
different cases of each scenario

Requirements -> Test Scenarios -> Test Cases

Prepared By: AMIT Learning

 Testing Levels

• Unit testing: testing of individual components

• Integration: testing collections of units integrated into sub – systems or

major component

• System testing: testing the complete system prior to delivery

• Acceptance testing: testing by users to check that the system satisfies

requirements, sometimes called alpha testing

Prepared By: AMIT Learning

 Testing Levels Sequence

Unit Test ->

Module Test ->

Integration Test ->

System Test ->

Acceptance Test

Prepared By: AMIT Learning

 Unit Testing

• Unit Testing checks that an individual program unit (subprogram, object

class, package, module) behaves correctly.

• There is some debate about what constitutes a “unit”. Here some common
definitions of a unit:

• The smallest chunk that can be compiled by itself

• A stand alone procedure or function

• Developed by a single person

Prepared By: AMIT Learning

 Unit Testing
Stubs and Drivers
• In order to test individual modules/units we have to use stubs and/or drivers.

• Driver is a module which simulates the environment or the entity that

demands a service.

• It provides the tested module with test cases, and displays results produced
by the tested module.

• Stub is a module which simulates in a simple way the behavior of a sub

ordinate module that provides a service to the demanding module.

Prepared By: AMIT Learning

 Unit Testing
Stubs and Drivers
• Hierarchy of Modules

Prepared By: AMIT Learning

 Unit Testing
Stubs and Drivers
• Setup to Test Module M2

Prepared By: AMIT Learning

 Integration Testing

• Integration Testing is the phase of software testing in which individual

software modules are combined and tested as a group

• Integration testing takes a sits input modules that have been unit tested,
groups them in larger aggregates, applies tests defined in an integration test
plan to those aggregates, and delivers a sits output the integrated system
ready for system testing

Prepared By: AMIT Learning

 Need for Integration Testing

• Reasons why integration test is mandatory:

• Differences in the understanding of business requirements between multiple

developers

• Different interface assumptions; structures and ranges

• The modules still have errors that were not uncovered in unit testing

• Time spent on integration test is impacted by the clarity of the interfaces design and
the strength of the unit test

Prepared By: AMIT Learning

Integration Testing Approaches: Big-
Bang
• Big Bang Approach ties all of the system modules together at one shot.

• Advantages:
• Suitable for small-sized and simple systems with limited number of components
• Saves the time of developing drivers and stubs
• Suitable when modules have been heavily unit tested
• Helps for showing the overall system behavior early

• Disadvantages:
• Risky for systems with large number of components
• Complicates the debugging process
• Integration cannot start before all system modules are ready

Prepared By: AMIT Learning

Integration Testing Approaches: Big-
Bang
• All of the system
• modules together
• at one shot.

Prepared By: AMIT Learning

 Integration Testing Approaches
Incremental -Top-Down

• In the Incremental Top-Down approach, the general modules at the top of the
overall logic path are tested first, using stubs for the called modules, then the
called modules are added using stubs for modules they might call.

• Advantages: easier to logically understand and helps for seeing the big
picture first.

• Disadvantages: in most cases, it defers the bulk of the complex processing

until later.

Prepared By: AMIT Learning

 Integration Testing Approaches
Incremental -Top-Down

• M1 with M2

• M1 with M3

• M3 with M6

• M2 with M4

• M2 with M5

Prepared By: AMIT Learning

 Integration Testing Approaches
Incremental -Bottom-Up

• In the Incremental Bottom-Up approach, start off with modules

at the lower level and use driver programs to call them, then
replace the drivers with the actual programs and move up the
program hierarchy until all the modules are in place.

• Advantages: in most cases, it checks for the bulk of the

complex processing at the beginning.

• Disadvantages: harder to logically understand and harder to see

the big picture until later.

Prepared By: AMIT Learning

 Integration Testing Approaches
Incremental -Top-Down

• M6 with M3

• M4 with M2

• M5 with M2

• M2 with M1

• M3 with M1

Prepared By: AMIT Learning

 Integration Testing Approaches
Incremental Pros and Cons
• Advantages:
• Safer for complicated systems with large number of components
• Easier and efficient in locating bugs
• Integration can start without waiting for all the system components
• Works even if the system modules didn’t go through a rigorous unit test

• Disadvantages:
• Requires developing drivers and stubs
• Doesn’t allow showing the overall system behavior early

Prepared By: AMIT Learning

 System Testing

• Executing test cases on the entire system in the intention of

finding defects and anomalies from both the intended design
and the system requirements.

• The whole system is a black box, implementation details are

totally hidden.

• Both verification and validation are applicable.

• Both functional and non-functional are applicable.

Prepared By: AMIT Learning

 Acceptance Testing

• A set of formal tests run for the client, and specified by the client. When the
system passes these tests, the software has been accepted by the client as
meeting the requirements.

• Based on end-user scenarios.

• Usually a sub set of the System Test Cases pool.

• Main difference from System Test is that it doesn’t contain corner scenarios.

• Goal is to make sure the system satisfies customer requirements.

Prepared By: AMIT Learning

 Verification… or Validation?

• Inspection Either

• Unit testing Verification

• Integration Testing Verification

• System testing Either

• Acceptance testing Validation

Prepared By: AMIT Learning

 Q&A

Prepared By: AMIT Learning

THANK YOU!