Coherence and Correspondence: Public Administration and Policy PAD634 Judgment and Decision Making Behavior

Public Administration and Policy
PAD634 Judgment and Decision Making Behavior
Coherence and correspondence
Thomas R. Stewart, Ph.D.

Center for Policy Research
Rockefeller College of Public Affairs and Policy
University at Albany
State University of New York
[email protected]
Copyright © Thomas R. Stewart

Coherence research
Coherence research measures the quality
of judgment against the standards of
logic, mathematics, and probability theory.
Coherence theory argues that decisions
under uncertainty should be coherent,
with respect to the principles of
probability theory.
coherence-correspondence.ppt 2
Coherence
Representativeness problem
There are two programs in a high school. Boys

are a majority (65%) in program A, and a
minority (45%) in Program B. There is an equal
number of classes in each of the two programs.
You enter a class at random, and observe that
55% of the students are boys. What is your best
guess--does the class belong to program A or
program B? (Kahneman and Tversky, 1972)
Coherence
Most subjects (67 out of 89) picked program A

because the class had more boys, and so did
program A. This is an example of an important
heuristic: “representativeness.” We judge an
event to be a member of a set of events if its
characteristics are similar to our belief about
members of that set. This is a good rule of
thumb in a lot of situations, but not this one.
Coherence
Some people might feel that the answer

could be “either program” because 55%
is midway between 45% and 65%.
They would have noticed a quantitative
relationship, and made a reasonable
guess.
But they would be wrong.
Coherence
The correct answer is Program B.

The variance of a binomial will be larger for
p = .45 than for p = .65. Therefore, there
will be more variability in the number of
boys in Program B’s classes, and it is
more likely that a class in Program B will
have as many as 55% boys.
Coherence
Why does it matter that students (in this case

Israeli high school students) fail such a difficult
question? Unfortunately, they also fail easier
questions. Furthermore, its not only students
who make these mistakes. These kinds of
errors have been found in studies of scientists,
statisticians, physicians, and other
professionals.
Coherence
Mammography illustration
David Eddy (1982) asked us to imagine that a physician

has a patient with a breast mass that he thinks is
probably benign. Lets interpret “probably” as a
probability of .99. The physician is 99% sure, but not
completely sure, that the mass is benign. This is his
“prior probability.” The physician orders a
mammogram and receives a report that the
radiologist thinks the mass is malignant. What
should the physician do?
Coherence
If he read the literature available in 1982,

he would find statements like:
• “The accuracy of mammography is
approximately 90 percent”
• “A positive report of carcinoma is highly
accurate.”
• “The accuracy of mammography in correctly
diagnosing malignant lesions averages 80 to
85 percent.”
Coherence
Note that this is NOT a decision table.
If he wanted more specific data, he could find these.
Mammograpy outcomes (from Eddy, 1982)
If malignant lesion If benign lesion (no

Results of X ray (cancer) cancer)
P(Positive test) = .792 .096
P(Negative test) = .208 .904

Sensitivity = .792 Specificity = .904
Coherence
The physician’s subsequent action for this

patient might be influenced by his
judgment about the likelihood that the
woman has a malignant lesion. What
would be your judgment about that
likelihood based on these data?
Coherence
• Eddy sampled 100 physicians and

found that most of them thought the
probability would be around 75 percent.
• We can calculate that likelihood using

the data given and Bayes theorem:
Coherence
Bayes Theorem
P(pos|ca)P(ca)
P(ca|pos) 
P(pos|ca)P(ca)  P(pos|benign)P(benign)
ca = malignant lesion (cancer) benign = benign lesion (no cancer) pos = positive test
(0.792)(0.01)
P(ca| pos)   0.077
(0.792)(0.01)  (0.096)(0.99)
Coherence
• So, the actual probability is about .08. The

physicians’ judgments were off by almost a
factor of 10!
• Does this matter?
– It only matters if the previous statement is true,
that is, “the physician’s subsequent action might
be influenced by his judgment about the likelihood
that the woman has a malignant lesion.” In fact,
this is a basic tenet of coherence research.
Coherence
Fundamental tenet of coherence

research
"Probabilistic thinking is important if people are to
understand and cope successfully with real-world
uncertainty."
Example:
"Accurate estimations of probability in diagnosis and prognosis are
important for physicians, because these probabilities influence their
diagnostic and therapeutic decisions. Objective probabilities, based on
epidemiological research, are often not available. In these instances,
physicians must rely on their own judgment and apply their probability
estimates to individual patients. Research has shown that the accuracy of
probability estimates of both experts, such as physicians and of lay people
in general is often inadequate...." Timmermans et al. (1996, pp. 107-108)
Coherence
Fundamental tenet of coherence

research
"Probabilistic thinking is important if people are to
understand and cope successfully with real-world
uncertainty."
Example:
"Accurate estimations of probability in diagnosis and prognosis are
important• for
Two elements
physicians, of thisthese
because quote:
probabilities influence their
• Probabilitydecisions.
diagnostic and therapeutic Objective
judgments are probabilities,
important because they based on
epidemiological research,
influenceare often not
decisions available.
made In these instances,
under uncertainty.
physicians must •rely on their
People own
are not judgment
very and apply
good at making their probability
probability
estimates to individual patients. Research has shown that the accuracy of
judgments.
probability estimates of both experts, such as physicians and of lay people
in general is often inadequate...." Timmermans et al. (1996, pp. 107-108)
Correspondence
Correspondence research
Correspondence research measures the
quality of judgment against the standards
of empirical accuracy. Correspondence
theory argues that decisions under
uncertainty should result in the least
number of errors possible, within the
limits imposed by irreducible uncertainty.
Correspondence
Fundamental tenet of correspondence

research
"Human competence in making judgments
and decisions under uncertainty is
impressive. Sometimes performance is
not. Why? Because sometimes task
conditions degrade the accuracy of
judgment."
Hammond, K. R. (1996). Human Judgment and Social Policy:

Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice.
New York, Oxford University Press (p. 282).
Coherence and correspondence
theories of competence
Coherence theory of competence

Uncertainty irrationality error
Correspondence theory of competence

Uncertainty inaccuracy error
What is the relation between coherence and

correspondence?
Comparison
Comparison of Coherence and
Correspondence Theories
•Sources of error
•Sources of disagreement
•Methods for improving judgment
•Value judgments
•Non-repetitive decisions
•Sources of competence
Comparison
Sources of error
•Coherence
"Irrationality" with respect to a normative system
•Correspondence
•Multiple fallible indicators:
Unreliability in information acquisition
Unreliability in information processing
Match between environment and judge
Fidelity of information system
Environmental uncertainty
Bias
Comparison
Sources of disagreement
•Coherence
Use of different heuristics
•Correspondence
Same as sources of error:
Fidelity of information system
Environmental uncertainty
Bias
Comparison
Methods for improving judgment
•Coherence
Decomposition and mechanical recomposition
Training in probability and statistics
Formal elicitation of subjective probabilities
•Correspondence
Environmental changes, training, and cognitive aids to
address:
Bias
Comparison
Value judgments
•Coherence
Decision theory assumes values can be
elicited.
•Correspondence
Has little to say. Correspondence not
relevant.
Comparison
Non-repetitive decisions
•Coherence
Decision theory provides prescriptive method.
•Correspondence
Not clear how to apply correspondence theory
when decisions are not repetitive.
Comparison
Sources of competence
•Coherence
Must be taught, usually in a formal setting, like
a school.
•Correspondence
Use of multiple fallible indicators is innate.
Integration of coherence and
correspondence theories
•Learn/teach identification of tasks

demanding coherence.
•Learn/teach identification of tasks

demanding correspondence.

Coherence and Correspondence: Public Administration and Policy PAD634 Judgment and Decision Making Behavior

Uploaded by

Copyright:

Available Formats

Coherence and Correspondence: Public Administration and Policy PAD634 Judgment and Decision Making Behavior

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Coherence and Correspondence: Public Administration and Policy PAD634 Judgment and Decision Making Behavior

Uploaded by

Copyright:

Available Formats

Public Administration and Policy

PAD634 Judgment and Decision Making Behavior

Coherence and correspondence

Thomas R. Stewart, Ph.D.

Copyright © Thomas R. Stewart

There are two programs in a high school. Boys

Most subjects (67 out of 89) picked program A

Some people might feel that the answer

The correct answer is Program B.

Why does it matter that students (in this case

David Eddy (1982) asked us to imagine that a physician

If he read the literature available in 1982,

If malignant lesion If benign lesion (no

P(Positive test) = .792 .096

P(Negative test) = .208 .904

The physician’s subsequent action for this

• Eddy sampled 100 physicians and

• We can calculate that likelihood using

• So, the actual probability is about .08. The

Fundamental tenet of coherence

Fundamental tenet of coherence

Fundamental tenet of correspondence

Hammond, K. R. (1996). Human Judgment and Social Policy:

Coherence theory of competence

Correspondence theory of competence

What is the relation between coherence and

Methods for improving judgment

•Learn/teach identification of tasks

•Learn/teach identification of tasks

You might also like