Software Testing

Testing Fundamentals

Phan Thi Hong

Lecturer
Faculty of Information Technology
Van Lang University

© 2016
Lecture Learning Objectives
 Objective
▪ Provides an overview about software testing and its
importance in software development.

 Outcomes
▪ Understand what software testing is and why it is
important

▪ Identify the primary objectives of Software Testing

▪ Understand the typical difference in the role of

software testers and software developers
2
Outline

 Definition, Primary Objectives, and Importance of

Software Testing

 Definition of Some Basic Terms

 Software Testers vs. Software Developers

 Principles of Software Testing

3
01 | Testing Fundamentals
1.1. Definition, primary
objectives, and importance
of software testing
Section Overview
 In this section, we will review the following:
▪ Definition, role, and objectives of software testing
▪ Testing everything is impossible
▪ Importance of software testing

5
Guiding Questions

 What is software testing?

 Why is it impossible to test everything?

 What are important objectives of software

testing?

 Why is software testing important?

6
What is Software Testing?
 At its core, testing is the process of comparing
"what is" with "what ought to be”

 “Testing is a concurrent lifecycle process of

engineering, using and maintaining testware in
order to measure and improve the quality of
the software being tested.” - Rick D. Craig and
Stefan P. Jaskiel, Systematic Software Testing, Artech
House, 2002

9
The Impossibility of Testing Everything
 Consider the following program:
int blech (int j) {
j = j - 1; // should be j = j + 1
j = j / 30000;
return j;
}

 If integers are implemented using 16 bits on

the computer executing this program, there
are 65,536 possible inputs into this tiny
program. Will you have the time (and the
stamina) to create 65,536 test cases?

10
The Impossibility of Testing Everything
 Consider the following input values and their ability to
detect this defect.
Input (j) Expected Actual
int blech (int j) { Result Result
j = j - 1; //should be j = j + 1 1 0 0
j = j / 30000; 42 0 0
return j;
40000 1 1
}
-64000 -2 -2

 Oops! None of the test cases chosen have detected

this defect.

 Which inputs to the blech routine will find the hidden

defect? How did you determine them?
11
Primary Objectives of Testing

 Ensure that the application under test (AUT) fulfills all

requirements of customer

 Find out the defects or issues occurring in the

application before it is encountered by the end-user

 Evaluate the overall performance of the application

 Provide a high quality product to the end user

12
Why is Software Testing Important?

Source: http://www.guru99.com/software-testing-introduction-importance.html

13
 Section Questions

What is software testing?

Why is it impossible to test everything?

What are important objectives of software

testing?

Why is software testing important?

01 | Testing Fundamentals

1.2. Definition of some basic

terms
Section Overview
 In this section, we will review the following:
▪ Definition of some basic terms. Topics include:
distinguish errors, faults (defects) and failures, test
case and test suite, verification and validation,
testing vs. debugging, QC vs. QA
▪ How can software quality be determined?

16
Guiding Questions
 What is the difference between
▪ errors, faults (defects) and failures?
▪ test case and test suite?
▪ verification and validation?
▪ QC and QA

 What is the difference between testing and

debugging? What specific tasks are involved in each?
Which groups should have responsibility for each of
these processes?

 How can software quality be determined?

17
Errors, Faults (Defects) and Failures

 An error is a mistake, misconception, or

misunderstanding on the part of a software developer.

 A fault (defect) is introduced into the software as the

result of an error. It is an anomaly in the software that
may cause it to behave incorrectly, and not according
to its specification.

 A failure is the inability of a software system or

component to perform its required functions within
specified performance requirements.

18
Test Case and Test Suite

 A test case is a specification of the inputs, execution

conditions, testing procedure, and expected results
that define a single test to be executed to achieve a
particular software testing objective, such as to
exercise a particular program path or to verify
compliance with a specific requirement.

 A group of related tests that are associated with a

database, and are usually run together, is sometimes
referred to as a test suite.

19
Verification and Validation
 Verification is the process of evaluating a software system
or component to determine whether the products of a given
development phase satisfy the conditions imposed at the
start of that phase.
 Verification answers for the following question: ”Are we
building the product right?”

 Validation is the process of evaluating a software system or

component during, or at the end of, the development cycle in
order to determine whether it satisfies specified
requirements.

 Validation answers for the following question: ”Are we

building the right product?”

20
 Quality Control vs. Quality Assurance
Criteria Software Quality Assurance Software Quality Control
(SQA/QA) (SQC/QC)
Definition SQA is a set of activities for ensuring SQC is a set of activities for
quality in software engineering ensuring quality in software
processes (that ultimately result in products. The activities focus
quality in software products). The on identifying defects in the
activities establish and evaluate the actual products produced.
processes that produce products.
Focus Process focused Product focused
Orientation Prevention oriented Detection oriented
Breadth Organization wide Product/project specific
Scope Relates to all products that will ever Relates to specific product
be created by a process
Activities Process Definition & Implementation, Reviews, Testing
Audits, Training
Table 1. The differences between QC and QA
21
Testing vs. Debugging – 1
 At its core, testing is a process of finding "unknown"
faults (defects) whereas debugging is a process of
removing "known" faults (defects).

 Testing as a process has following aspects: economic,

technical, and managerial

 Testing itself is related to two other processes called

verification and validation

22
Testing vs. Debugging – 2
 Debugging, or fault localization is the process of
1. locating the fault or defect
2. repairing the code
3. retesting the code

 The debugging process begins after testing has been

carried out and the tester has noted that the software is
not behaving as specified.

23
Quality Attributes
 Quality Attributes are characteristics that reflect quality
of a system, system component, or process.

 Some examples of quality attributes for software have

been described in IEEE Standards for Software Quality
Metrics Methodology:
▪ Correctness
▪ Reliability
▪ Usability
▪ Integrity
▪ Portability
▪ Maintainability
▪ Interoperability
25
Metrics and Quality Metrics

 A metric is a quantitative measure of the degree to

which a system, system component, or process
possesses a given attribute.

 Quality metrics are a special kind of metric. A quality

metric is a quantitative measurement of the degree to
which an item possesses a given quality attribute.

26
Quality Must Be Built In
 Testing is concerned with what is in the product or
system and what is missing.

 Testing can only verify the product or system and its

operation against predetermined criteria (requirements).

 Testing neither adds nor takes away anything.

 Quality is an issue that is determined during the

requirements and design phases by the development
project stakeholders or requesting customers. It is not
decided at testing time.

27
 Section Questions
What is the difference between
• errors, faults (defects) and failures?
• test suite, test oracle and test bed?
• verification and validation?
• QC and QA

What is the difference between testing and debugging?

What specific tasks are involved in each? Which groups
should have responsibility for each of these processes?

How can software quality be determined?

According to you, how can you add value to program

under test?
01 | Testing Fundamentals

1.3. Software testers vs.

Software developers
Section Overview
 In this section, we will review the following:
▪ Role and needed skills of software testing
professionals

▪ Who should be on the test team?

30
Guiding Questions

 What is the career path of a software tester?

 Which skills should a software testing

professional have?

 What are the works that a software tester do?

 Why should software testers have

programming skill?

 Who should be on the test team?

31
The Skills of Testing Professionals
 The best software testers must have advanced
skills drawn from many software professions
including:
▪ Software developers

▪ Database developers

▪ Network developers

▪ System administrator

32
Why testers should know programming

 To understand how programs are built

 To get insights of when programs can fail

 To automate some tasks that can help testing

 To understand creative side of developer

33
The Main Role of Software Testers
 The software testers’ main role is that of a verifier.
They should:
▪ Take an objective look at the software in progress that is
independent of the authors of development documents and of
program code.

▪ Determine through repeated testing whether the software

matches its requirements and specifications.

▪ Be expected to tell the development team which requirements

and specifications are met and which requirements and
specifications are not met.

34
What Else Does a Software Tester Do?
 The professional answer is plan, plan, and plan.

 The expert tester must identify the critical areas of

software to be tested and the most efficient ways to
complete that testing.

 The tester must manage the plan and schedule to

complete the testing.

35
Who Is on the Test Team?
 A good test team has a mix of skill levels (entry,
intermediate, advanced) represented by its
members.

 This enables:
▪ The more experienced testers to be responsible for the test
planning, scheduling, and analysis of test results.

▪ The intermediate-level testers can work within the test plan

to create the test scenarios, cases, and scripts that follow the
plan.

▪ Then, with the advice and mentoring of the senior testers, a

mix of intermediate-level and entry-level testers executes the
tests.
36
 Section Questions

Which skills should a software testing

professional have?

What are the works that a software tester do?

Why should software testers have programming

skill?

Who should be on the test team?

01 | Testing Fundamentals

1.4. Principles of Software

Testing
Section Overview
 In this section, we will review the following:
▪ Some important guidelines for designing and
executing test cases

39
7 Principles of Software Testing

No Principle Description
1 Testing shows the • Testing shows that defects are present but
presence of defects does not prove that there are no more
defects.
2 Exhaustive testing • Testing all the combinations of inputs and
is impossible preconditions is not feasible except for trivial
cases.
3 Early testing • Testing should begin as early as possible in
SDLC.
4 Defect clustering • Testing effort should be focused on the areas
where defects are expected and modules
with higher defect density.

Table 2. Principles of Software Testing

40
7 Principles of Software Testing

No Principle Description
5 Pesticide paradox • Repeatedly running the same tests (test
cases) will reduce the probability of finding
new defects
6 Testing is context • Testing should be done differently for
dependent different contexts.

7 Absence of error • The system built should be usable and fulfil

fallacy the users needs and expectations else finding
and fixing defects does not help.

Table 2. Principles of Software Testing

41
Summary
 Testing is a concurrent lifecycle process of engineering, using, and
maintaining testware in order to measure and improve the quality of the
software being tested. (Craig and Jaskiel)

 Testing should start at the beginning of a development project because

everything produced by the project is an excellent test candidate.

 Recalling the objectives of testing, we must design tests that have the
highest likelihood of finding the most errors with a minimum amount of
time and effort. (Pressman)

 Software Quality can be determined by measurement of the degree to

which a system, system component, or process possesses a given
quality attribute with quality metrics. (Burnstein)

 The real challenge for experienced testers is to identify what can go

untested with the least impact to the business.

43
References
1. Lee Copeland, A Practitioner's Guide to Software Test Design,
Artech House, 2004: Chapter 1

2. Glenford J Mayers, Tom Badgett and Corey Sandler, The Art of

Software Testing, 3rd edition, John Wiley & Sons, Inc., 2012:
Chapter 2

3. Ilene Burnstein, Practical Software Testing - A Process-Oriented

Approach, Springer-Verlag New York Inc., 2003: Chapter 1 & 2

4. Cem Kaner, Jack Falk and Hung Quoc Nguyen, Testing

Computer Software, 2nd edition: Chapter 4

5. http://www.guru99.com/software-testing-introduction-
importance.html

6. https://en.wikipedia.org/wiki/Test_case (access on Sep, 2nd 2018)

7. http://softwaretestingfundamentals.com/software-testing-basics/
44
Questions & Answers

45