Decision-Making in Oil & Gas -

The Good, Bad, and the Ugly

Reidar B Bratvold
University of Stavanger
Norwegian Inst. of Technology
University of Texas at Austin
Acknowledgement
! Steve Begg and Matthew Welsh, U of Adelaide
! Eric Bickel, U of Texas at Austin
! Bart Willigers, BG Group
! Current and former graduate students at UoA,
UiS, NTNU, and UT Austin

R.B. Bratvold EDPN - October 2012 2

What is the Basis for my Discussion?
! Have spent 30 years in oil & gas working as a
roughneck, reservoir engineer, geostatistician,
executive, and academic
! A set of formal and informal polls and
questionnaires addressing oil & gas
professionals (engineers, geoscientists,
economists managers and non-managers)
! Experience drawn from having provided short
courses and consulting services to a large
number of oil & gas companies

R.B. Bratvold EDPN - October 2012 3

Oil & Gas is often referred to as an industry where
Decision Analysis is broadly and successfully adopted

Chevron DAS Award Raiffas Oil Wildcatter

R.B. Bratvold EDPN - October 2012 4

Decision Analysis in Oil & Gas

!"#$%&' !"#$%&'
,)$'-)' ())*+'
())*+'

R.B. Bratvold EDPN - October 2012 5

Decision Analysis (DA) is a systematic procedure for
transforming opaque decision problems into transparent
decision problems by a sequence of transparent steps
?)/'@)A#3*'

Decision
Problem

Framing, Evaluating, Assessing &

./&01"$' ./&01"$'
Structuring Modeling Deciding

23)45*6357'
4)890/#:)/');' Action

$))<&'='>3)45&&5&'
R.B. Bratvold EDPN - October 2012 6
Decision Analysis leverages processes and tools such as
the Dialogue Decision Process, Elements of Decision
Quality, Strategy Tables, and Sensitivity Analysis !
Decision-Maker(s) +
1'2")#*."'+

BB' 3*-4&0++
Framing, Evaluating, Assessing & ,-%&)'*./&0+ #/*++
Insight Insight
Structuring Modeling Deciding 5)*6&7"80+
;-&#&'%0+
"2+<&=$0$"'+
>4*-$%?+
()*#&+ 9&*0"'$':+

Analysis team

!"##$%#&'%'

R.B. Bratvold EDPN - October 2012 7

! however, the main value of decision analysis
is not in the specific numbers that are generated
in the process but in the !
Structured thinking and insight that the process
engenders.
The resulting transparency and record.

!"#$%$&'()')*+%$%($%(),&-.(/&0(.&(
)#/$"1"(#*)2$.+(&3()#4&'($'(
5)6$'7(8"#$%$&'%()'89("1"'(5&2"(
3-'8)5"'.)**+9(/&0(.&()#/$"1"(
#*)2$.+(&3(./&-7/.:((
( ( ((;&0)28(

R.B. Bratvold EDPN - October 2012 8

 The use of probabilistic modeling in the oil and
gas industry has increased significantly over the
last 20 years
Has the decision making ability in the oil & gas
industry improved as a result of this?

R.B. Bratvold EDPN - October 2012 9

 Decision making ability does not appear to be
increasing in tandem with uncertainty quantification

To what degree has your organization's ability to quantify

uncertainty increased in the last five years?

To what degree has your organizations decision-making

ability improved in the last five years?
70%
Uncertainty Quantification
60% Decision making
Percentage of Respondents

50%

40%

30%

20%

10%

0%
Regressed/No Change Some Increase Significant Increase No Opinion

R.B. Bratvold EDPN - October 2012 10

Whats Not So Good?
! Lack of decision oriented project
management processes

R.B. Bratvold EDPN - October 2012 11

The industry is often using management
processes that are not decision oriented
! Stage-gate project management processes (PMP) are
quite common in oil & gas companies
! The purpose of a PMP is to maximize value creation
through better decision-making and improved
execution

Planning Execution
Front End Loading (FEL)

R.B. Bratvold EDPN - October 2012 12

In practice, a significant gap exists
between the ideals of the PMP and reality
! Improving decision quality (DQ) is one of the reasons to
use a PMP, but too often no one explicitly owns DQ
! During FEL, the emphasis is on improved decision-
making
- However, the links between PMP, decision quality, and improved
value creation have been severed at most companies
- The PMP often gets linked to compliance and assurance

- It has morphed from a value-maximizing endeavor to a risk-

mitigation process. While risk management is important, it is a
poor proxy for value maximization

! Most PMP implementations do not include a dialogue

decision process
- Advocacy is the most commonly used process despite ample
evidence demonstrating its ineffectiveness
R.B. Bratvold EDPN - October 2012 13
Whats Not So Good?
! Lack of decision oriented project
management processes
! Poor understanding of the purpose/role of
technical work

R.B. Bratvold EDPN - October 2012 14

The main role of a Geoscientist, Engineer, or
Economist is to support decision-making
Technical work in the oil & gas industry is
fundamentally about uncertainty assessment for
the purpose of making decisions
First priority: Accurate (=unbiased) uncertainty
assessment
Second priority: Uncertainty reduction but only if it
is value adding in the context of the
decision(s) at hand

Many oil & gas professionals consider uncertainty

assessments to be of secondary importance
something that can be added after the main
(deterministic) work is done
R.B. Bratvold EDPN - October 2012 15
Whats Not So Good?
! Lack of decision oriented project
management processes
! Poor understanding of the purpose/role of
technical work
! Confusing uncertainty reduction with improved
decision quality

R.B. Bratvold EDPN - October 2012 16

We often forget that the goal is to make good decisions
which will lead to good outcomes not to reduce uncertainty
The main enemy of good decision making is
uncertainty bias, not the uncertainty being too large

Bias in central value (mean):

e.g. usually optimism (rose-tinted glasses),
sometimes pessimism - both destroy value!

Bias in width of distribution:

e.g. assessing the range of uncertainty to be much
less than it really is with respect to your true state of
knowledge (overconfidence)

Have we moved from a state of Blissful Ignorance

to Uncertainty Induced Confusion?
R.B. Bratvold EDPN - October 2012 17
Whats Not So Good?
! Lack of decision oriented project
management processes
! Poor understanding of the purpose/role of
technical work
! Confusing uncertainty reduction with improved
decision quality
! Poor understanding of the nature of probabilities

R.B. Bratvold EDPN - October 2012 18

In the words of someone who knows !

"If you don't get this elementary, but mildly

unnatural, mathematics of elementary
probability into your repertoire, then you go
through a long life like a one-legged man in an
ass-kicking contest.

Charlie Munger
(Vice-Chairman of Berkshire Hathaway
Corporation and lifelong business
partner with Warren Buffett)

R.B. Bratvold EDPN - October 2012 19

 Having a good understanding of probability
concepts is essential for good decision making
Large independent:
The actual performance of our key assets wasnt even
within the P1 to P99 range.
Common response to the question: What is the
definition of Expected Value?
The 50/50 of an outcome.

CEO to manager:
I want your guarantee that we will not spend more than
the P50 on this project!
Team manager:
I prefer to evaluate your project deterministically
because either youre right or youre wrong and if
youre good, youll be right!
R.B. Bratvold EDPN - October 2012 20
The nature of Probability:
Probability is a state of mind, not a state of things
Throw a die and hide top face. What is the
probability of the top face being a 3?

1/6

Now get information. Cut away center piece.

Has the top face changed? No
Has the probability that its a 3 changed? Yes
What is the probability of a 3 now? 1/3

Uncertainty is a function of what you know.

There is no right uncertainty (or PDF)!
R.B. Bratvold EDPN - October 2012 21
Uncertainty is in our heads its a function of our
state of knowledge
Probability is not an inherent
parameter of the system
The systems that we deal with are essentially deterministic,
we use probability to express our state of knowledge about
them
The Die:
- when the die is tossed, there will be one face that comes up
- in theory, if we knew (precisely) its initial conditions and we could model
(precisely) all of the processes involved with tossing it, we could predict how it
would land but that is, practically, impossible

And so with geology, oil price etc:

- we use the language of probability to quantify our lack of knowledge
- for events that have occurred, we could, theoretically, collect information to
reduce uncertainty - complete information would resolve it
R.B. Bratvold EDPN - October 2012 22
Probability is subjective (personal) and
depends upon your information

Shared
Person A
Info Person B
Info
Info

Person A PDF
Person B PDF
R.B. Bratvold EDPN - October 2012 23
Poor probabilities understanding is also
preventing us from learning from our mistakes

! Overconfidence survey
! Oil & gas professionals (mostly engineers,
geoscientists, and economists) were asked 10
questions to check how well calibrated they
are wrt their lack of knowledge
- Survey
o Total respondents: ~400
o Managers: ~10%
o Male: ~75%
o Female: ~25%

R.B. Bratvold EDPN - October 2012 24

Questions used to check for
overconfidence

R.B. Bratvold EDPN - October 2012 25

Oil & Gas professionals are as overconfident
as anyone else ->)$')/'
CD534)/E*5/$'' !5<<'4#<093#$5*'
F0/435#&0/1'$)'$"5'<5GH'

I5#/'7JKL'

R.B. Bratvold EDPN - October 2012 26

People often confuse probabilities with
other quantified valued
! Many participants choose not to answer the questions:
- I have no idea.
- How can I provide you with a range when I dont
even know what the correct answer is?

! Probabilities are not

point estimates of
some quantity
! Probabilities are used
to convey our
uncertainty about
the quantity Variable

! If I had precise knowledge about the

quantity, I wouldnt need probabilities at all
R.B. Bratvold EDPN - October 2012 27
 We use models that depend heavily on subjective estimates
and yet ignore the research about how well experts estimate
and how to adjust for their errors

Probability judgments are not

simply waiting in our heads to
be pulled out when needed
Discovering and developing these
requires hard and systematic thinking
about the important aspects of a decision
Human beings are imperfect information processors
Personal insights about uncertainty and preference can be
both limited and misleading
An awareness of human cognitive limitations is helpful in
developing the necessary judgmental inputs

A disciplined elicitation and calibration process, not

individual genius, is the key to improved assessments
R.B. Bratvold EDPN - October 2012 28
 We dont show much interest in learning from
our mistakes
Prof Daniel Kahneman
Winner of 2002 Nobel Prize in
Economics
Psychologist and Decision Analyst

" The thing that astonishes me when I talk to business

people in the context of decision analysis is that you
have an organization thats making lots of decision
and theyre not keeping track. Theyre not trying to
learn from their own mistakes; theyre not investing
the smallest amount in trying to actually figure out
what theyve done wrong. And thats not an accident:
They dont want to know.
R.B. Bratvold EDPN - October 2012 29
Even when we are explicitly shown that our
thinking is erroneous, we tend to explain it away

MNO'#/&A53'NKL'
PLO'#/&A53'NKJL'
LO'#/&A53'PQR' S)3354$'

MNO'#/&A53'NKL'
PLO'#/&A53'NKJL'
LO'#/&A53'PQR'

R.B. Bratvold EDPN - October 2012 30

Even when we are explicitly shown that our
thinking is erroneous, we tend to explain it away
! People tend to explain away their own faulty thinking
after they are shown to be wrong:
T <3(./"(=-"%4&'()'8(,""'(>2&>"2*+(>/2)%"8(<(0&-*8(/)1"(>2&1$8"8(./"(
#&22"#.()'%0"2(
T ?*./&-7/(./"(>2"8$#."8("1"'.(8$8('&.(&##-29((
o it eventually will (off on timing)
o it nearly occurred (the close call)
o it would have but for ! (the exogenous shock)

! Bad luck is a far more popular explanation for forecast

failure than good luck is for forecasting success
To improve the quality of your own predictions, keep brutally
honest score. Enjoy being wrong, admitting to it and learning
from it, as much as you enjoy being right - Tetlock
R.B. Bratvold EDPN - October 2012 31
Whats Good?
! 2010 DA Practice Award to Chevron

R.B. Bratvold EDPN - October 2012 32

Decision Analysis Practice Award for 2010 was given
to Chevron

U"5'V540&0)/'W/#<X&0&'23#4:45'
WA#3*'0&'10D5/'#//6#<<X'$)'$"5'95&$'
*540&0)/'#/#<X&0&'#>><04#:)/Y'#&'
Z6*15*'9X'#'>#/5<');'-)405$X'
858953&K'U"5'0/$5/$');'$"0&'#A#3*'
0&'$)'354)1/0[5Y'>3)8)$5Y'#/*'
>69<040[5'1))*'*540&0)/'#/#<X&0&'
>3#4:45K''

R.B. Bratvold EDPN - October 2012 33

Without the commitment to have quality assurance
and lookback/feedback as part of the DA process, just
doing DA calculations probably wont create much
value

R.B. Bratvold EDPN - October 2012 34

Whats Good?
! 2010 DA Practice Award to Chevron
! Increasingly accepting the existence,
if not quantification, of uncertainty

R.B. Bratvold EDPN - October 2012 35

Whats Good?
! 2010 DA Practice Award to Chevron
! Increasingly accepting the existence,
if not quantification, of uncertainty
! Islands of advanced uncertainty modeling skills within
many oil & gas companies

R.B. Bratvold EDPN - October 2012 36

Whats Good?
! 2010 DA Practice Award to Chevron
! Increasingly accepting the existence,
if not quantification, of uncertainty
! Islands of advanced uncertainty modeling skills within
many oil & gas companies
! Conferences, Forums, and Workshops focused on
DA (sort of)

R.B. Bratvold EDPN - October 2012 37

Whats Good?
! 2010 DA Practice Award to Chevron
! Increasingly accepting the existence,
if not quantification, of uncertainty
! Islands of advanced uncertainty modeling skills within
many oil & gas companies
! Conferences, Forums, and Workshops focused on
DA (sort of)
! Increasing number of oil & gas focused research
papers discussing DA (sort of)

R.B. Bratvold EDPN - October 2012 38

A new journal SPE Economics & Management was
established 3 years ago

R.B. Bratvold EDPN - October 2012 39

Whats Good?
! 2010 DA Practice Award to Chevron
! Increasingly accepting the existence,
if not quantification, of uncertainty
! Islands of advanced uncertainty modeling skills within
many oil & gas companies
! Conferences, Forums, and Workshops focused on
DA (sort of)
! Increasing number of oil & gas focused research
papers discussing DA (sort of)
! Increasing number of consulting firms and software
developers specializing in DA for oil & gas

R.B. Bratvold EDPN - October 2012 40

Concluding
! The oil & gas industry is not bad in
terms of its willingness and ability to
implement decision analysis but !
also not great
! Other industries are probably as good or better; e.g.
- Automotive
- Armed forces

- Pharmaceuticals

- Others ??

!"#$'#35'X)63'5B>5305/45&+'

R.B. Bratvold EDPN - October 2012 41