TESTING & AGILE METHODS PMIT 6111 ST & QA

SOFTWARE TESTING
Testing is the process of exercising a program with the specific intent of
finding errors prior to delivery to the end user.
 When you test software, you execute a program using artificial data.
 You check the results of the test run for errors, anomalies or information about the
program’s non-functional attributes.
 Can reveal the presence of errors NOT their absence.
 Testing is part of a more general verification and validation process, which also
includes static validation techniques.

2
WHAT IS AGILE?
The word ‘agile’ means −
 Able to move your body quickly and easily.
 Able to think quickly and clearly.

Agile is an iterative development methodology, where both development and testing

activities are concurrent.
Testing is not a separate phase; Coding and Testing are done interactively and
incrementally, resulting in quality end product, which the meets customer requirements.
Further, continuous integration results in early defect removal and hence time, effort
and cost savings.
CHARACTERISTICS OF AGILE SOFTWARE DEVELOPMENT

 Light Weighted methodology

 Small to medium sized teams

 vague and/or changing requirements

 vague and/or changing techniques

 Simple design

 Minimal system into production

WATERFALL VS AGILE IN SMALL PROJECTS

Source: The CHAOS Manifesto,

The Standish Group, 2012
EXISTING AGILE METHODS
Extreme Programming

Scrum

Crystal Methods

Feature Driven Development

Lean Development

Dynamic Systems Development Methodology (DSDM)

EXTREME PROGRAMMING -- XP
Most prominent Agile Software development method

Prescribes a set of daily stakeholder practices

“Extreme” levels of practicing leads to more responsive software.

Changes are more realistic, natural, inescapable.

EXTREME…
Taking proven practices to the extreme
If testing is good, let everybody test all the time
If code reviews are good, review all the time
If design is good, refactor all the time
If integration testing is good, integrate all the time
If simplicity is good, do the simplest thing that could possibly work
If short iterations are good, make them really, really short
EXTREME PROGRAMMING PRACTICES (A)
Principle or practice Description
Incremental planning Requirements are recorded on story cards and the stories to be included in a release are
determined by the time available and their relative priority. The developers break these
stories into development ‘Tasks’. See Figures 3.5 and 3.6.
Small releases The minimal useful set of functionality that provides business value is developed first.
Releases of the system are frequent and incrementally add functionality to the first
release.
Simple design Enough design is carried out to meet the current requirements and no more.
Test-first development An automated unit test framework is used to write tests for a new piece of functionality
before that functionality itself is implemented.
Refactoring All developers are expected to refactor the code continuously as soon as possible code
improvements are found. This keeps the code simple and maintainable.
EXTREME PROGRAMMING PRACTICES (B)
Principle or practice Description
Pair programming Developers work in pairs, checking each other’s work and providing the support to
always do a good job.
Collective ownership The pairs of developers work on all areas of the system, so that no islands of expertise
develop and all the developers take responsibility for all of the code. Anyone can change
anything.
Continuous integration As soon as the work on a task is complete, it is integrated into the whole system. After
any such integration, all the unit tests in the system must pass.

Sustainable pace Large amounts of overtime are not considered acceptable as the net effect is often to
reduce code quality and medium term productivity

On-site customer A representative of the end-user of the system (the customer) should be available full
time for the use of the XP team. In an extreme programming process, the customer is a
member of the development team and is responsible for bringing system requirements to
the team for implementation.
XP RELEASE CYCLE

Source: Tutorials Points

 SCRUM
Definition from rugby
football:
a scrum is a way to
restart the game after an
interruption, where the
forwards of each side
come together in a tight
formation and struggle to
gain possession of the ball
when it is tossed in among
them

13
SCRUM - AN AGILE PROCESS
SCRUM is an agile, lightweight process for managing and controlling software and
product development in rapidly changing environments.

 Iterative, incremental process

 Team-based approach
 developing systems/ products with rapidly changing requirements
 Controls the chaos of conflicting interest and needs
 Improve communication and maximize cooperation
 Protecting the team form disruptions and impediments
 A way to maximize productivity

JASS 2006 14
FUNCTIONALITY OF SCRUM

JASS 2006 15
WHAT IS AGILE TESTING?
Agile Testing is a software testing practice that follows the principles of agile software
development.
Agile Testing involves all members of the project team, with special expertise contributed
by testers. Testing is not a separate phase and is interwoven with all the development
phases such as requirements, design and coding and test case generation. Testing takes
place simultaneously through the Development Life Cycle.
Furthermore, with testers participating in the entire Development Lifecycle in conjunction
with cross-functional team members, the contribution of testers towards building the
software as per the customer requirements, with better design and code would become
possible.
Agile Testing covers all the levels of testing and all types of testing.
 AGILE TESTING ACTIVITIES
The Agile Testing Activities during an iteration include −
The Agile Testing Activities at Project Level
are −
Release Planning (Test Plan) • Participating in iteration planning
 For every Iteration, • Estimating tasks from the view of testing
 Agile Testing Activities during an Iteration • Writing test cases using the feature descriptions
• Unit Testing
Regression Testing
• Integration Testing
Release Activities (Test Related) • Feature Testing
• Defect Fixing
• Integration Testing
• Acceptance Testing
• Status Reporting on Progress of Testing
• Defect Tracking
AGILE TESTING
CONTINUOUS INTEGRATION, CONTINUOUS QUALITY
Entire team is involved in all activities of an iteration.
After each iteration there is release.
Continuous Integration is the key of the success of Agile Development.
Integrate frequently, at least daily such that you are ready for a release as and
when required. Testing in Agile becomes an essential component of all the phases of
the development, ensuring continuous quality of the product. Constant feedback from
everyone involved in the project adds to the quality of the product.
In Agile, communication is given utmost importance and the customer requests are
received as and when necessary. This gives the satisfaction to the customer that all the
inputs are considered and working quality product is available throughout the
development.
AGILE TESTING METHODOLOGIES
Agile Testing Methodology mostly based on Scrum.
The other commonly used Agile Testing Methodologies are −
Test-Driven Development (TDD) − Test-Driven Development (TDD) is based on
coding guided by tests.
Acceptance Test-Driven Development (ATDD) − Acceptance Test-Driven
Development (ATDD) is based on communication between the customers, developers
and testers and driven by pre-defined Acceptance Criteria and Acceptance Test
Cases.
Behavior-Driven Development (BDD) − In Behavior-Driven Development (BDD)
testing is based on the expected behavior of the software being developed.
AGILE TESTING LIFECYCLE
In Scrum, the Testing activities include −
 Identify expected behavior of the system to derive test cases from User
Stories
 Release Planning based on Test Effort and Defects
 Sprint Planning based on User Stories and Defects
 Sprint Execution with Continuous Testing
 Regression Testing after the completion of Sprint
 Reporting Test Results
 Automation Testing
SPRINT BASED LIFECYCLE
A ‘PRESCRIBING MEDICATION’
STORY
EXAMPLES OF TASK CARDS
FOR PRESCRIBING
MEDICATION
TESTING IN EXTREME PROGRAMMING
Testing is central to XP and XP has developed an approach where
the program is tested after every change has been made.
 Unit Tests and Functional Tests
 Test a little, code a little…“Test-first programming”
 Tests become the specification
 Tests give confidence in the system as user is involved
 Tests give courage to change the system
 Automated test harnesses are used to run all component tests each time that
a new release is built.
TEST-FIRST DEVELOPMENT
•Writing tests before code clarifies the requirements to be implemented.

•Tests are written as programs rather than data so that they can be
executed automatically. The test includes a check that it has executed
correctly.
• Usually relies on a testing framework such as JUNIT.

•All previous and new tests are run automatically when new functionality is
added, thus checking that the new functionality has not introduced errors.
UNIT TESTS
CUSTOMER INVOLVEMENT
The role of the customer in the testing process is to help develop acceptance
tests for the stories that are to be implemented in the next release of the
system.
The customer who is part of the team writes tests as development proceeds. All
new code is therefore validated to ensure that it is what the customer needs.
However, people adopting the customer role have limited time available and
so cannot work full-time with the development team. They may feel that
providing the requirements was enough of a contribution and so may be
reluctant to get involved in the testing process.
TEST CASE DESCRIPTION FOR DOSE CHECKING
 TEST AUTOMATION
Test automation means that tests are written as executable components before the
task is implemented
 These testing components should be stand-alone, should simulate the submission of input to be
tested and should check that the result meets the output specification. An automated test
framework (e.g. JUnit) is a system that makes it easy to write executable tests and submit a set of
tests for execution.
As testing is automated, there is always a set of tests that can be quickly and easily
executed
 Whenever any functionality is added to the system, the tests can be run and problems that the new
code has introduced can be caught immediately.
XP TESTING DIFFICULTIES
Programmers prefer programming to testing and sometimes they take short cuts
when writing tests.
 For example, they may write incomplete tests that do not check for all possible exceptions that
may occur.

Some tests can be very difficult to write incrementally.

 For example, in a complex user interface, it is often difficult to write unit tests for the code that
implements the ‘display logic’ and workflow between screens.

It difficult to judge the completeness of a set of tests. Although you may have a lot
of system tests, your test set may not provide complete coverage.
TEST-DRIVEN DEVELOPMENT
Test-driven development (TDD) is an approach to program development in which you
inter-leave testing and code development.
Tests are written before code and ‘passing’ the tests is the critical driver of
development.
You develop code incrementally, along with a test for that increment. You don’t move
on to the next increment until the code that you have developed passes its test.
TDD was introduced as part of agile methods such as Extreme Programming.
However, it can also be used in plan-driven development processes.

32
TEST-DRIVEN DEVELOPMENT

33
TDD PROCESS ACTIVITIES
Start by identifying the increment of functionality that is required. This should
normally be small and implementable in a few lines of code.
Write a test for this functionality and implement this as an automated test.
Run the test, along with all other tests that have been implemented. Initially, you have
not implemented the functionality so the new test will fail.
Implement the functionality and re-run the test.
Once all tests run successfully, you move on to implementing the next chunk of
functionality.

34
TDD EXAMPLE

Source: Tutorials Points

CONTEXT OF TESTING
Valid inputs
Invalid inputs
Errors, exceptions, and events
Boundary conditions
Everything that might break
BENEFITS OF TEST-DRIVEN DEVELOPMENT
Code coverage
 Every code segment that you write has at least one associated test so all code written has at least one
test.

Regression testing
 A regression test suite is developed incrementally as a program is developed.

Simplified debugging
 When a test fails, it should be obvious where the problem lies. The newly written code needs to be
checked and modified.

System documentation
 The tests themselves are a form of documentation that describe what the code should be doing.

37
 TESTER IN A TEAM
An Agile tester should have traditional testing skills. In addition, Agile tester needs −
 Good interpersonal skills.
 Ability to act positive and solution-oriented with team members and stakeholders.
 Ability to display critical, quality-oriented, skeptical thinking about the product.
 Aptitude to be pro-active to actively acquire information from the stakeholders.
 Skills to work effectively with customers and stakeholders in defining testable User Stories, the Acceptance Criteria.
 Talent to be a good team member working with developers in producing quality code.
 Usability of testing skills to have the right test cases at the right time and at the right level and executing them well
within the duration of the sprint.
 Ability to evaluate and report test results, test progress and the product quality.
 Openness to respond to changes quickly, including changing, adding or improving test cases.
 Potential to self-organize work.
 Enthusiasm to continuous skill growth.
 Competency in Test Automation, Test-driven Development (TDD), Acceptance Test-driven Development (ATDD),
Behavior Driven Development (BDD) and experience based Testing.
ROLE OF TESTER IN AGILE TEAM
Ensuring proper use of testing tools.
Configuring, using and managing the test environments and the test data.
Mentoring other team members in relevant aspects of testing.
Ensuring that appropriate testing tasks are scheduled during the release and sprint planning.
Understanding, implementing and updating test strategy.
Collaborating with developers, customer and stakeholders in clarifying requirements, in terms of
testability, consistency and completeness.
Performing the right tests at the right time and at right test levels.
Reporting defects and working with the team in resolving them.
Measuring and reporting test coverage across all applicable coverage dimensions.
Participating in sprint retrospectives, proactively suggesting and implementing improvements.
ROLE OF TESTER IN AGILE TEAM -- SUMMERY
Teamwork
Test Planning
Sprint Zero
Integration
Agile Testing Practices
TEAMWORK - I
Collaborative Approach − Working with cross-functional team members on Test
Strategy, Test Planning, Test Specification, Test Execution, Test Evaluation, and Test
Results Reporting. Contributing the testing expertise in conjunction with other team
activities.
Self-organizing − Planning and organizing well within the sprints to achieve the
targets of testing by amalgamating expertise from other team members as well.
Empowerment − Making appropriate technical decisions in achieving the team’s
goals.
Commitment − Committing to understanding and evaluating the product’s behavior
and characteristics as required by the customers and stakeholders.
TEAMWORK - II
Transparent − Open, Communicating and Accountable.
Credibility − Ensuring the credibility of the test strategy, its implementation, and
execution. Keeping the customers and stakeholders informed on the test strategy.
Open to Feedback − Participating in sprint retrospectives to learn from both successes
and failures. Seeking customer feedback and acting quickly and appropriately to
ensure quality deliverables.
Resilient − Responding to changes.
TEST PLANNING
Defining test scope, extent of testing, test and sprint goals.
Deciding on the test environment, test tools, test data and configurations.
Assigning testing of features and characteristics.
Scheduling test tasks and defining frequency of tests.
Identifying test methods, techniques, tools and test data.
Ascertaining prerequisites such as predecessor tasks, expertise and training.
Identifying dependencies such as functions, code, system components, vendor, technology, tools, activities,
tasks, teams, test types, test levels and constraints.
Setting priorities considering the customer/user importance and dependencies.
Arriving at the time duration and effort required to test.
Identifying tasks at each sprint planning.
SPRINT ZERO
Sprint Zero involves preparation activities before the first sprint. A tester needs to
collaborate with the team on the following activities −
 Identifying scope
 Dividing user stories into sprints
 Creating system architecture
 Planning, acquiring and installing tools (including testing tools)
 Creating the initial test strategy for all the test levels
 Defining test metrics
 Specifying the acceptance criteria, also called the definition of “Done”
 Defining exit criteria
 Creating Scrum board
 Setting the direction for testing throughout the sprints
INTEGRATION
In Agile, a quality working product should be ready for release at any point of time
in the development lifecycle. This implies continuous integration as a part of
development. An Agile tester needs to support continuous integration with continuous
testing.
To accomplish this, a tester needs to −
Understand the integration strategy.
Identify all dependencies between functions and features.
AGILE TESTING PRACTICES
An Agile tester needs to adapt Agile practices for testing in an agile project.
 Pairing − Two team members work together at the same keyboard. As one of them tests, the other
reviews/analyzes testing. The two team members can be
 One tester and one developer
 One tester and one business analyst
 Two testers
 Incremental Test Design − Test cases are built from user stories, starting with simple tests and
moving to more complex tests.
 Mind Mapping − A mind map is a diagram to organize the information visually. Mind mapping
can be used as an effective tool in Agile testing, using which information regarding the necessary
test sessions, test strategies and test data can be organized.
REFERENCE
1. Ian Sommerville, “Software Engineering”, 9th Edition, Chapter 3 – Agile Software
Development
2. Tutorials Point

T hank You