Shu-Ha-Ri

Measuring Maturity of Agile Teams

Dan Fuller
May 21, 2014

| 2014, Cognizant
Contents
2
1 Introduction(Shu-Ha-Ri)
2 Visualizing the Metaphor
3 Shu (Stage 1) Team Behaviors
4 Ha (Stage 2) Team Behaviors
5 Ri (Stage 3) Team Behaviors
6 Shu-Ha-Ri Applied to Agile Teams
7 Provably Better Agile Teams




| 2014, Cognizant
Shu-Ha-Ri
3
Shu-Ha-Ri is the journey that Aikido martial arts
students go through from novice towards mastery.

It is becoming the metaphor the Agile community is
using to measure maturity of Agile teams.

This session will explain Cognizants implementation
of Shu-Ha-Ri to measure Agile Teams
Source: Alastair Cockburn @ http://alistair.cockburn.us/Shu+Ha+Ri
| 2014, Cognizant
Visualizing the Metaphor
4
Nascent
Teams

Follow the
Rules
Proficient
Teams

Break the
Rules
Hyper
Performing
Teams

BE the Rule
-Shu
(Stage 1)
-Ha
(Stage 2)
-Ri
(Stage 3)
| 2014, Cognizant
Shu (Stage 1) Team Behaviors
5
-Shu
(Stage 1)
-Ha
(Stage 2)
-Ri
(Stage 3)
Learning the process basics, & team
members
Growing their rapport both
professionally and socially
Mimicking the practices
Learning to work together
Following the rules
Learning the process
A collection of skilled individuals
learning their roles
| 2014, Cognizant
Ha (Stage 2) Team Behaviors
6
-Ha
(Stage 2)
Starting to question practices that
seemingly may not work for them.
-Shu
(Stage 1)
Understanding the practices, and the
importance of the principles and values.
Coming to a deeper understanding of the
art than pure repetitive practice will
allow.
Beginning to move beyond a collection of
skilled individuals.
Developing their own personalities.
Beginning to break the rules.
-Ri
(Stage 3)
| 2014, Cognizant
Ri (Stage 3) Team Behaviors
7
-Ri
(Stage 3)
-Ha
(Stage 2)
Creating output greater than the sum
of the work of the individuals.
Breaking the rules to gain an
advantage.
Progressing more through self-
discovery than instruction.
Consistently thinking and acting as a
unit.
Adjusting with little friction
-Shu
(Stage 1)
Shu-Ha-Ri Applied to Agile Teams



| 2014, Cognizant
What do we evaluate and why?
9
We chose 12 particular dimensions that cover the
entire breadth of Agile methods from Product
Backlogs to Working Software.

Factoring in the roles of Product Owners, Scrum
Masters, Developers and Testers

Some are Quantitative, others are Qualitative.

Lets take a look at the 12 dimensions
| 2014, Cognizant
Quantitative Dimensions
10
Velocity
Story Creation
Quality
Accuracy of Commitment
Accuracy of Estimates
Overtime
How much working software is being
created?
How good is the working software?
How comfortable are we with creating
user stories?
Is the team delivering their
commitments?
How well are we estimating relative
size and effort?
How sustainable is the pace?
| 2014, Cognizant
Qualitative Dimensions
11
Sprint Planning
Daily Scrum
Release Planning
Story Estimation
Release Frequency
Retrospectives
How effective are we conducting our
sprint planning sessions?
How far out into the future are we
planning our work?
How effective is the facilitation of the
standup?
How smooth is the estimation
process?
How often is working software
released?
Are we actually doing meaningful
process improvement?
| 2014, Cognizant
Visualizing the Results
12
All teams start at Stage 1 (Shu)
To achieve Stage 2 (Ha) you must be at Stage 2 for all 12 dimensions,
likewise for Stage 3 (Ri)
Think of it like the rings on
a tree (Cambia),
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation
Process
Effort
Estimation
1
2
3
Stage 3 (Ri) - Outer Ring
Bark
Stage 1 (Shu) - Inner Ring
- Pith
Stage 2 (Ha) - Middle
Ring - Xylem & Floem
| 2014, Cognizant
13
Sprint Planning
Stage 1 Shu : Represented in the center of the chart
Team struggles with
the process, has
trouble defining tasks
& duration.
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
?
| 2014, Cognizant
14
Sprint Planning
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Team is comfortable
with the process, is
able to do sprint
planning in 3-4 hours.
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
3 4 Hours
| 2014, Cognizant
15
Sprint Planning
Stage 3 Ri : Represented in the outer circle of the chart
Team is identifying
tasks and durations in
advance and meeting
is fast and efficient
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
Quick Meeting
| 2014, Cognizant
16
Release Planning
Stage 1 Shu : Represented in the center of the chart
Team is not sure what
it will be doing next
sprint
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
!
! !
| 2014, Cognizant
17
Release Planning
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Team knows what it
will be working on 2-3
sprints into the future
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation 2 - 3 Sprints
| 2014, Cognizant
18
Release Planning
Stage 3 Ri : Represented in the outer circle of the chart
Team knows what it
will be working on 3 or
more sprints out into
the future
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
>3 Sprints
_
| 2014, Cognizant
19
Daily Scrum
Stage 1 Shu : Represented in the center of the chart
Lots of off-topic
discussion, resembles
a status meeting
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
---
---
---
-
-
| 2014, Cognizant
20
Daily Scrum
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Everyone is
participating and the
3 basic questions are
being addressed
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
To Do
Accomplished
Impediments
| 2014, Cognizant
21
Daily Scrum
Stage 3 Ri : Represented in the outer circle of the chart
Executed with
precision, nothing
extraneous,
transparency & truth
are shining through
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation

| 2014, Cognizant
22
Quality
Stage 1 Shu : Represented in the center of the chart
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
Defect Density > 25%
| 2014, Cognizant
23
Quality
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
Defect Density between
25% and 10%
!
| 2014, Cognizant
24
Quality
Stage 3 Ri : Represented in the outer circle of the chart
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
Defect Density < 10%
| 2014, Cognizant
25
Estimation
Stage 1 Shu : Represented in the center of the chart
Team struggles
gaining consensus on
story point estimates,
meeting takes a long
time
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
26
Estimation
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Team is able to
estimate about 6
stories per hour
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
1Hour / 6 Stories
| 2014, Cognizant
27
Estimation
Stage 3 Ri : Represented in the outer circle of the chart
Team is able to
estimate more than
8 stories per hour.
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
1Hour / 8+ Stories
| 2014, Cognizant
28
Story Creation Process
Stage 1 Shu : Represented in the center of the chart
Unpredictable Story
Creation Rate, Story
Authors Uncomfortable
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
29
Story Creation Process
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Story Authors starting
to become comfortable,
story production rate is
rising
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
30
Story Creation Process
Stage 3 Ri : Represented in the outer circle of the chart
Story Production Rate
>= Velocity of Teams
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
31
Velocity
Stage 1 Shu : Represented in the center of the chart
Velocity is
unpredictable, its up,
its down from sprint
to sprint
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
32
Velocity
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Velocity growth trend
is increasing for three
sprints in a row
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
33
Velocity
Stage 3 Ri : Represented in the outer circle of the chart
Velocity growth trends
slows, levels off, is
consistent &
predictable
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
34
Retrospectives
Stage 1 Shu : Represented in the center of the chart
Team seems to be
going through the
motions on the Retro
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
Collect Data
| 2014, Cognizant
35
Retrospectives
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Team has positive
discussions aligned
with Agile Manifesto
themes and values
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
---
---
---
-
-
| 2014, Cognizant
36
Retrospectives
Stage 3 Ri : Represented in the outer circle of the chart
Team is instituting
meaningful process
improvement every
sprint
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
37
Overtime
Stage 1 Shu : Represented in the center of the chart
Team frequently has
to put in overtime to
deliver against
commitments
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
?
| 2014, Cognizant
38
Overtime
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Overtime occurs, but
is minimal and team
is working at a
sustainable pace
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
!
| 2014, Cognizant
39
Overtime
Stage 3 Ri : Represented in the outer circle of the chart
Overtime is rare, and
it is usually the teams
choice to overachieve
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
Sustainable
| 2014, Cognizant
40
Accuracy of Estimates
Stage 1 Shu : Represented in the center of the chart
Initial Story Point
Estimates are
frequently revised
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
41
Accuracy of Estimates
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Initial estimates are
within 25% of actuals
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
0 25 50 75 100
| 2014, Cognizant
42
Accuracy of Estimates
Stage 3 Ri : Represented in the outer circle of the chart
Initial estimates are
within 10% of actuals
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
0 25 50 75 100
| 2014, Cognizant
43
Accuracy of Commitments
Stage 1 Shu : Represented in the center of the chart
Team frequently doesnt
deliver against story
point commitments
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
44
Accuracy of Commitments
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Team usually delivers
against its sprint
commitment but often
has adopted work
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
!
| 2014, Cognizant
45
Accuracy of Commitments
Stage 3 Ri : Represented in the outer circle of the chart
Team always delivers
against its sprint
commitment and
adopted work is rare
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
46
Release Frequency
Stage 1 Shu : Represented in the center of the chart
Team struggles to get
working software out
the door every sprint
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
?
| 2014, Cognizant
47
Release Frequency
Stage 2 Ha : Represented in the 2
nd
circle of the chart
Most sprints result in a
good build with
occasional build issues
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
48
Release Frequency
Stage 3 Ri : Represented in the outer circle of the chart
Every sprint results in
a good build of
working software, no
exceptions
Sprint Planning
Release Planning
Daily Scrum
Quality
Release Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation Process
Effort Estimation
| 2014, Cognizant
49
Measure Teams over Time
Sprint
Planning
Release
Planning
Daily
Scrum
Quality
Release
Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation
Process
Effort
Estimation
Sprint
Planning
Release
Planning
Daily
Scrum
Quality
Release
Frequency
Accuracy of
Commitments
Accuracy of
Estimates
Overtime
Retrospectives
Velocity
Story Creation
Process
Effort
Estimation
Team advanced from Stage 1 to 2
Sprint 10 Sprint 15
Team is at Stage 1 due to Overtime
Overtime moved from Stage 1 to 2
Velocity moved from Stage 2 to 3
| 2014, Cognizant
50
Data Collection
Qualitative data should come from Agile Lifecycle Management
tools (ALM) and Integrated Testing Suites, e.g.:
Rally
VersionOne

2 Approaches to Data Collection for Qualitative Dimensions:
Coach Led Interview of Product Owner and Scrum Master (Benefit
Rich conversation results in rich data)
Team Self Reported via Survey (Benefit Scalable on very large
programs)

Suggested Interval:
Every Sprint Ideal if enough Agile Coaching available
Every Release Minimal recommended interval
| 2014, Cognizant
51
Analyzing the Results
Team level assessment
Warning: This is not intended to determine good or bad teams.
This is a tool teams can use to come up with thoughtful ways during
retrospective to improve the way the team works.

Compare overall results against goals aligned with overall Agile
vision,
Goal is Faster Time to Market BUT teams scoring low on Release
Frequency
Goal is more Productivity BUT teams scoring low on Velocity
Goal is higher Quality BUT teams scoring low on Defect Density
Provably Better Agile Teams
| 2014, Cognizant
53
Becoming Ri (Hyper-Performing, Provably Better)
Do Stage 3 Teams Exist?
Many factors must come together. These teams typically are:
Stable in terms of staffing over time
Product Owner role cannot change on a project
Team is small; less than 10 people
Empowered to change its process
Incorporates the core values in their work
Bonding via a difficult challenge
Committed to shared success
Hungry to learn
Willing to accept change, and
Eager to experiment with new ideas
Caveat: Fewer than 5% of teams achieve Ri (Stage 3), and there is
no average timeline for a team to achieve the highest stage.
| 2014, Cognizant
54
Stage 2 and 3 Benefits
Statistics quoted from QSMA, Rico, and DDJ in Succeeding with Agile, Mike Cohn, Jan 2010
Higher Productivity
Lower Costs
Faster Time to Market
Higher Quality
Improved Job Satisfaction
Improved Stakeholder Satisfaction
14% 88% 384%
10% 26% 70%
5% 37% 64%
10% 63% 84%
52% 74% 92%
53% 78% 93%
S
t
a
g
e

2

S
t
a
g
e

3

| 2014, Cognizant
55
Benefits of Stage 3 RI Teams
Higher Quality - Fewer defects will result from their work.
Predictable and Consistent - Their velocity is well known,
consistent, and range-bound, allowing for greater confidence
in future delivery estimates.
Improved Morale - Members want to work together more for
the betterment of the team.
Improved Stakeholder Satisfaction - Business sponsors
working with these teams will enjoy working with the team.
Provably Better - We dont just think we are outperforming
waterfall, we have metrics to prove it.
Hyper-productivity - The team will be able to produce quality
software at an amazing rate enabling a faster time to market
and lower IT costs.
Extremely Efficient - The team will efficiently estimate, plan,
execute and report progress and impediments..
Ri
Benefits
| 2014, Cognizant
56
Case Study of a Stage 3 (Ri) Team
Results
A large enterprise project (56+ developers) run with distributed integrated Scrums
was as productive as a small co-located Scrum team
8X more productive than a parallel project performed with Waterfall, resulting in
more functionality and higher quality.
Characteristics
2-week sprints, with a few use cases, but mostly user stories
Teams in Provo, Utah and St Petersburg, Russia, with a few key individuals working
from Seattle, Denver, St Louis and Waterloo, Canada
Used Jira / Greenhopper ALM tool
Best Practices
Daily scrum team meetings from multiple sites
Daily meetings of Product Owners
Automated builds from one central repository
No distinction between team members at different sites
Followed sound engineering practices constantly
Seamless integration of some XP practices, refactoring, continuous integration,
limited pair programming on most complex technical components
SirsiDynix 2005 Distributed integrated Scrums that achieved a hyper-productive state

Thank you!

Dan Fuller
[email protected]

May 21, 2014