Psychiatry Research 105 Ž2001.

255᎐264

Inpatient diagnostic assessments: 1. Accuracy of

structured vs. unstructured interviews 夽

Paul R. Miller U , Robert Dasher, Rodney Collins, Pamela Griffiths,

Fred Brown
Department of Psychiatry, School of Medical Sciences, Uni¨ ersity of California, Los Angeles, 2406 Astral Dri¨ e, Los Angeles,
CA 90046, USA

Received 26 February 2001; received in revised form 5 September 2001; accepted 20 September 2001

Abstract

This study compared structured vs. unstructured interviews for making psychiatric diagnoses. Three clinicians
independently diagnosed 56 inpatient-subjects, each using a different method: Ž1. the unstructured Traditional
Diagnostic Assessment ŽTDA., the standard method of clinical practice; Ž2. the Structured Clinical Interview for
DSM-Clinical Version ŽSCID-CV., a widely used structured method; and Ž3. the Computer Assisted Diagnostic
Interview ŽCADI., a structured computer-based method. Once finished, the three clinicians developed a Consensus
diagnosis, using Spitzer’s LEAD Standard ŽL s Longitudinal evaluation of symptomatology, E s Evaluation by expert
consensus, ADs All Data from multiple sources.. Diagnoses were assigned to one of 10 groups Žcognitive
impairment, general medical condition-induced, alcohol-induced, drug-induced, mania, depression, schizophrenia,
schizoaffective, psychosis NOS, and anxiety.. Diagnostic accuracy for each method, measured against Consensus, was
as follows: TDA ᎏ agreement s 53.8%, kappas 0.4325 Ž‘fair’.; SCID-CV ᎏ agreement s 85.7%, kappas 0.8189
Ž‘excellent’.; CADI ᎏ agreement s 85.7%, kappas 0.8147 Ž‘excellent’.. All three methods reached acceptable levels
of diagnostic accuracy. Structured methods ŽSCID-CV, CADI. were significantly better than the unstructured TDA.
䊚 2001 Elsevier Science Ireland Ltd. All rights reserved.

Keywords: DSM-IV; Diagnosis; Validity; Reliability

夽
Similar findings with an overlapping sample were presented at the Annual Meeting, American Psychiatric Association, 6 May
2000, in Chicago, Illinois.
U
Corresponding author. Tel.: q1-323-876-2831; fax: q1-323-876-2831.
E-mail address: [email protected] ŽP.R. Miller..

0165-1781r01r$ - see front matter 䊚 2001 Elsevier Science Ireland Ltd. All rights reserved.
PII: S 0 1 6 5 - 1 7 8 1 Ž 0 1 . 0 0 3 1 7 - 1
256 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264

1. Introduction diagnostic questions are asked’ ŽFrances et al.,

1995, p. 66..
Accurate diagnosis is pragmatically as well as
1.1. E¨ olution of DSMs and psychiatric diagnosis
theoretically important, because ‘treatment plans
are often based on . . . diagnostic type. The advent
DSM-I Ž1952. and DSM-II Ž1968. defined syn- of disease-specific treatment protocols has height-
dromes more as psychological reactions than as ened the necessity for accurate diagnostic proce-
biopsychosocial conditions and solicited clini- dures’ ŽBasco et al., 2000..
cians’ subjective evaluations Ž‘clinical judgment’.
as much as their objective assessments. Conse- 1.2. TDA: diagnostic reliability and ¨ alidity
quently, many clinicians focused more on patient’s
‘problems’ arising from their psychodynamic sta-
Textbooks seem not to express concerns or list
tus than on mental disorders generated by biopsy-
references regarding the validity and reliability of
chosocial conditions; treatment of choice was
the TDA ŽOthmer and Othmer, 1994; Hales et
usually psychotherapy ŽWhitehorn, 1944; Sullivan,
al., 1995; Kaplan and Sadock, 1998.. Core Read-
1970.. Prior to 1980, ‘the classification system and
ings in Psychiatry ŽSacks et al., 1995., the APA
specialty of psychiatry wwerex often held as less
bibliography, has 11 references about diagnostic
‘medical’ or scientific than other branches of
validity, but none about TDA.
medicine. Extensive efforts to correct this percep-
Researchers have reported the diagnostic vari-
tion resulted in a paradigmatic shift from
ance of the TDA ŽWilliams et al., 1992; McGorry
hermeneutic winterpretive . . . theory basedx to em-
et al., 1995; Mojtabai and Nicholson, 1995; Hill et
pirically based approaches, and the development
al., 1996; van Praag, 1997., but these findings
of a nosology intended to increase diagnostic
have not led to significant changes in clinical
reliability and facilitate research efforts. These
practice.
changes are embodied in . . . DSM-III w1980x and
Although clinicians use the TDA with unques-
DSM-IV w1994x’ ŽBogenschutz and Nurnberg,
tioning faith, researchers mostly avoid using it as
2000, p. 824..
an exclusive diagnostic method in clinical trials.
The Traditional Diagnostic Assessment ŽTDA.,
Researchers also tend to avoid using TDAs exclu-
the unstructured interview that is the standard of
sively in their work to validate syndromes for
practice for that task throughout psychiatry
ŽOthmer and Othmer, 1994; Hales et al., 1995; DSM-IV ŽWidiger et al., 1994, 1996..
Kaplan and Sadock, 1998., has evolved alongside
DSM. To increase diagnostic validity and reliabil- 1.3. Reasons for de¨ eloping the Computer Assisted
ity, clinicians since 1980 have expanded the Men- Diagnostic Inter¨ iew (CADI)
tal Status Examination ŽMSE. and the history of
present and past illnesses, and are using more Following DSM-III’s introduction in 1980, re-
structured formats: ‘Most clinicians are now im- searchers began to evaluate how clinicians used
bued not only with the content of the DSM-IV it. Lipton and Simon Ž1985, p. 370. found that
criteria sets but also with a different method of ‘documentation of DSM-III criteria for assigned
interviewing patients and eliciting psy - chart diagnoses was not present in 80% of the
chopathology. Compared with pre-DSM-III days, 131 charts reviewed’. Skodol et al. Ž1984. found
clinical evaluations are now much more likely to that 75% of incorrect diagnoses made with DSM-
be more semi-structured and less open-ended. III resulted from incorrect application of criteria.
Clinicians are much more likely to ask specific Robinson et al. Ž1985. found that a university-af-
qu estion s to elicit th e sym ptom s an d filiated faculty misidentified 13᎐48% of DSM-III
course information wi.e. MSE and history of ill- criteria for major depression. Greist Ž1998. ex-
ness; emphasis addedx necessary to make a DSM amined how clinicians followed diagnostic rules
diagnosis...no evaluation is complete unless the and found error rates between 10 and 37%. Garb
 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264 257

Table 1
Mental status examination using Key Criteria

DSM-IV diagnostic groups Linked Key Criteria s data items

I. Cognitive impairment disorders 1. Attention᎐consciousness

2. Memory᎐orientation
3. Cognition
II. Disorders due to general 4. Evidence from medical history,
medical condition physical, laboratory, imaging
III. Alcohol disorders 5. Alcohol use
IV. Drug disorders 6. Drug use
V. Psychotic disorders 7. Hallucinations
8. Delusions
9. Disorganized speech
10. Disorganized behavior
11. Flat affect
12. Alogia
13. Avolition
VI. Mood disorders 14. Depression
15. Loss of pleasure Žanhedonia.
16. Elevatedrexpansive mood
17. Irritability
VII. Anxietyrdissociation 18. Anxiety
19. Dissociation
VIII. Somatoformrconversionrfactitiousr 20. Unexplained physical symptomsr
malingeringrhypochondriasis fearsrcomplaints
IX. Sexual disorders 21. Sexual patterns
X. Eating disorders 22. Eating patterns
XI. Sleep disorders 23. Sleep patterns
XII. Adjustment disordersrPTSD 24. Stress
XIII. Impulse-control disorders 25. Impulsive behaviors

Ž1998. reviewed research on clinical judgment and Criteria as MSE items. Key Criteria are those
found that clinicians often failed to adhere to criteria Žlisted first in DSM algorithms. that must
diagnostic criteria. Such reports justified the de- be evaluated before the linked diagnosis can be
velopment of methods to improve diagnostic ac- ruled in or ruled out. Table 1 shows 13 diagnostic
curacy of the TDA. groups and their linked 25 Key Criteria. If any
In 1980, the corresponding author undertook to Key Criterion is positive, the linked diagnosis
‘computerize’ DSM. This took its present form in must be evaluated completely. If all Key Criteria
1994 with the advent of CADI, which directs the for a linked diagnosis are negative, that diagnosis
clinician to evaluate all relevant criteria in DSM- can be ruled out. Thus, CADI operates in the
IV algorithms, by sequentially displaying ques- same way as ‘Decision Trees for Differential Di-
tions and answers on the computer screen. It is a agnosis’ ŽDSM-IV, Appendix A..
structured interview. Clinicians enter their assess- The CADI and the paper-and-pencil SCID both
ments into the computer, and the program assess DSM algorithms. The CADI differs in that
matches them with DSM-IV algorithms to make it has an MSE Žbased on Key Criteria., while
diagnoses. CADI has been in beta testing since SCID does not. The CADI makes diagnoses pro-
1996. CADI has been previously reported at na- grammatically, while the paper-and-pencil SCID
tional meetings ŽMiller, 1996, 1998, 1999, 2000a,b.. requires the clinician to choose the diagnoses.
The MSE underlies DSM diagnostics, so the Lieff Ž1987., Taintor et al. Ž1997., and Blacker
corresponding author restructured it to use Key Ž2000. have described different computerized di-
258 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264

agnostic interviews, a topic outside the compass 5% Ž3r56. married, 2% Ž1r56. widowed; domicile
of this article. ᎏ 36% Ž20r56. relative, 20% Ž11r56. friend,
20% Ž11r56. homeless, 14% Ž8r56. alone, 9%
1.4. Purposes of this study Ž5r56. board-and-care home, 2% Ž1r56. spouse;
work ᎏ 95% Ž53r56. unemployed, 4% Ž2r56.
The purposes of the study were twofold: Ž1. to part-time job; 2% Ž1r56. homemaker; support ᎏ
compare structured vs. unstructured methods for 45% Ž25r56. public funding, 27% Ž15r56. none,
making psychiatric diagnosis; Ž2. to assess the use 21% Ž12r56. relative, 4% Ž2r56. self, 4% Ž2r56.
of computer assistance for structured diagnosis friend; education ᎏ 12.5 mean years Žrange 7᎐17;
and the validity and reliability of the CADI. S.D.s 3.2..
We used the paper-and-pencil format of the Mean test scores were PANSSs 74.7 Žpositive
SCID, because it is the format most widely used, symptomss 20.1, negative symptomss 16.9, gen-
and because it enabled the comparison of three eral symptomss 37.7.; BPRSs 44.2. Mean time
distinct methods ᎏ unstructured paper-and- since first hospitalization was 5.6 years Žrange s
pencil ŽTDA ., structured paper-and-pencil 0᎐18; S.D.s 5.4.. Mean number of prior hospital-
ŽSCID-CV., and structured computer-assisted izations was 3.4 Žrange s 0᎐35; S.D.s 6.1..
ŽCADI.. In summary, these subjects were mostly chroni-
cally ill with multiple diagnoses Žsee below., cur-
rently hospitalized for an acute episode, and had
2. Methods never achieved a stable relationship, living situa-
tion, job, or financial self-support.
2.1. Subjects
2.3. In¨ estigators for CADI and SCID-CV
The 56 subjects were recruited from psychiatric
inpatients at a university-affiliated publicly funded The corresponding author has used the SCID
hospital. Qualifications included minimum age 18, since the 1980s and the CADI since 1994, and has
enough English fluency and cognitive function to taught structured and unstructured interviewing
comprehend written and oral descriptions of the methods for three decades. The other four inves-
study and to sign informed consent, and sufficient tigators were 4th᎐5th year residentsrfellows who
verbal fluency and memory to answer questions in were fulfilling residencyrfellowship requirements
the CADI and SCID-CV interviews. The first for doing research.
patient admitted each week was screened. If The corresponding author trained each co-in-
patient 噛1 did not meet criteria, we screened vestigator to do the SCID-CV and the CADI, by
patient 噛2, etc. Nineteen of 75 Ž25.3%. possible demonstrating how to use them with both simu-
subjects did not participate: eight were not suffi- lated and real patients, and then supervising each
ciently fluent in English, seven were too acutely with several patients until they achieved pro-
disordered, two withdrew after starting, one was ficiency. One-to-one instruction averaged 15 h.
mentally retarded, and one refused to participate. Each investigator did a minimum of 10 SCID-CVs
and 10 CADIs. Ventura et al. Ž1998, p. 163. found
2.2. Demographics that with sufficient training there are ‘no signifi-
cant differences between experienced and neo-
The 56 subjects had the following characteris- phyte interviewers in interrater reliability or diag-
tics: age ᎏ 36.9 years Žmean; range s 19.0᎐59.0, nostic validity’.
S.D.s 12.4.; gender ᎏ 55% Ž31r56. male, 45%
Ž25r56. female; race ᎏ 52% Ž29r56. White, 21% 2.4. Examiners for the TDA
Ž12r56. Latino, 16% Ž9r56. African-American,
4% Ž2r56., Asian, 7% Ž4r56. Other; marital sta- Six faculty clinicians Žfive psychiatrists, one PhD
tus ᎏ 63% Ž35r56. single, 30% Ž17r56. divorced, psychologist., with an average of 20 years of clini-
 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264 259

cal experience, developed the TDA discharge di- CV, 1᎐2 CADIs. plus ‘All Data’ Žsee above,
agnoses for this study. They used ‘All Data’ Žbut Section 2.4..
not the SCID-CV or the CADI. ᎏ Ž1. their own
TDA; Ž2. two previous hospital TDAs Žemergency Mihalopoulos et al. Ž2000. affirmed that using
room and inpatient admission, usually done by all clinical data improves procedural validity and
other clinicians .; Ž3. social work histories; Ž4. reliability. In choosing their Consensus primary
nursing records; Ž5. informants’ histories; Ž6. pre- diagnosis, the clinicians followed explicit guide-
sent and past laboratoryrimaging studies; and Ž7. lines: Ž1. use ‘All Data’; Ž2. match data accurately
‘Longitudinal’ medical records from other hospi- with DSM-IV algorithms; and Ž3. consider all
talsrclinicsrphysicians. possible diagnoses. Because the four co-investiga-
tors were near the end of their training, and
2.5. Primary data because all had been chief residents in the pro-
gram, faculty clinicians readily accepted them as
These were the primary Axis I diagnoses, de- peers in the Consensus process.
fined as the mental disorder responsible for this
hospitalization. Primary diagnoses were devel- 2.6. Other data
oped in this sequence:
Once all primary diagnoses were made, we
䢇 1 and 2: SCID-CV and CADI ᎏ done blind examined each write-up word-by-word to record
Žno data available to the investigator, other Ž1. all references to Key Criteria; and Ž2. all the
than what the subject said in that interview. other Axis I diagnoses as well as the primary
by two different investigators, during days 1᎐5 diagnosis.
of hospitalization, within 24 h of each other.
The SCID-CV and CADI alternated in se- 2.7. Diagnostic groups
quence for different patients. Investigators did
not share their information with each other The 10 diagnostic groups followed the model of
until the Consensus conference. Basco et al. Ž2000., with various modifications
䢇 Optional CADI ᎏ done blind for 27 of the 56 ŽTable 2..
subjects by a third investigator, within 1 day of
the first CADI, to test interrater reliability. 2.8. Data analysis
䢇 3: TDA ᎏ done by the six faculty clinicians as
the inpatient discharge assessment. The dis- The primary diagnoses made by TDA, SCID-
charge᎐not admitting᎐diagnosis was used, be- CV, and CADI were compared with the Consen-
cause other clinicians often did the admitting sus primary diagnosis. Diagnoses had to be in the
diagnosis, and because the faculty clinician same diagnostic group to count as agreement.
made this diagnosis after obtaining ‘All Data’ These were analyzed for overall agreement and
Žsee above, Section 2.4 Examiners for TDA.. kappa value.
䢇 4: Consensus Diagnosis ᎏ done by the above To assess interrater reliability for the CADI, 27
three to four clinicians, within 1᎐10 days after of the 56 subjects had a second CADI. Diagnoses
subject was discharged, using Spitzer’s ŽSpitzer, had to be in the same diagnostic group to count
1983. ‘LEAD Standard’: L s Longitudinal as agreement.
evaluation of symptomatology, E s Evaluation
by expert consensus, ADs All Data from
multiple sources. These consultations aver- 3. Results
aged 1.5 h Žrange 1᎐3.5 h. of meetings among
all the clinicians who developed the primary 3.1. Primary diagnosis
diagnoses. They used the three to four re-
search assessments Ždischarge TDA, SCID- Frequencies found by the different methods
260 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264

Table 2
Groups of syndromes

Diagnostic group Specific syndromes

1. Cognitive impairment All 290.xx ᎏ dementias

293.0-Delirium
All 294.xx ᎏ amnesias and dementias
2. General medical All 293.xx ᎏ disorders due to a general
condition-induced medical condition except 293.0 Ždelirium.
3. Alcohol-induced All 291.xx ᎏ alcohol-induced disorders
All 303.xx ᎏ alcohol intoxication and dependence
305.00 ᎏ alcohol abuse
4. Drug-induced All 292.xx ᎏ drug-induced disorders
All 304.xx ᎏ drug dependence
All 305.xx ᎏ drug abuse except
305.00 Žalcohol abuse.
5. Mania 296.4x ᎏ Bipolar I, manic
296.6x ᎏ Bipolar I, mixed
296.80 ᎏ Bipolar I, NOS
296.89 ᎏ Bipolar II
6. Depression 296.2x ᎏ Major depression, single episode
296.3x ᎏ Major depression,
multiple episodes
296.5x ᎏ Bipolar I depressed
309.0 ᎏ Adjustment disorder with
depressed mood
311 ᎏ Depression NOS
7. Schizophrenia All 295.xx ᎏ schizophrenia except
295.70 Žschizoaffective.
297.10 ᎏ Delusional disorder
8. Schizoaffective 295.70 ᎏ Schizoaffective disorder
9. Psychosis NOS 298.90 ᎏ Psychosis not otherwise specified
10. Anxiety All 300.xx ᎏ anxiety, somatoform, etc.
309.81 ᎏ Posttraumatic stress disorder

are in columns A᎐D of Table 3 Žsee bottom two 3.2. Other data results
lines for rates of agreement compared with Con-
sensus .. TDA had agreement s 53.8% and kappa TDA users evaluated on average only 53%
s 0.4325; SCID-CV had agreement s 85.7% and Ž9.5r18. of Key Criteria; structured interviews
kappas 0.8189; CADI had agreement s 85.7% evaluated 100% Ž18r18.. TDA users found on
and kappas 0.8147. Although the SCID and the average 1.54 Ž86r56. diagnoses per patient, while
CADI had the same overall agreement Ž48r56., a CADI users on average found 3.04 Ž170r56. per
patient. So the TDA found only 50.7% Ž86r170.
comparison of columns B and C shows that they
as many total diagnoses.
had minor differences in eight of 10 individual
groups. Using Fleiss’s Ž1973. standards for kappa, 3.3. Validity and reliability of CADI
the TDA was ‘fair;’ and both SCID-CV and CADI
were ‘excellent’. The structured SCID-CV and Both SCID-CV and Consensus diagnoses Žusing
CADI were significantly better than the unstruc- Spitzer’s Lead Standard. are accepted as diagnos-
tured TDA for agreement with Consensus ᎏ tic standards ŽBlacker, 2000.. The comparison of
␹ 2 s 12.2, d.f.s 1, P- 0.001 ŽTable 3.. the CADI with the SCID-CV was a test of crite-
 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264 261

Table 3
Frequency of diagnoses Ž N s 56.

Diagnostic A B C D
group TDA SCID-CV CADI Consensus
wagreex wagreex wagreex

1. Cognitive impairment 1 w1x 1 w1x 0 w0x 1

2. General medical
condition-induced 0 w0x 1 w1x 1 w1x 1
3. Alcohol-induced 1 w1x 2 w1x 1 w1x 1
4. Drug-induced 2 w2x 2 w2x 3 w3x 3
5. Mania 6 w3x 8 w6x 6 w6x 6
6. Depression 16 w10x 12 w11x 15 w12x 12
7. Schizophrenia 11 w5x 11 w8x 10 w7x 9
8. Schizoaffective 11 w7x 17 w16x 19 w17x 21
9. Psychosis NOS 7 w0x 0 w0x 0 w0x 0
10. Anxiety Žand PTSD. 1 w1x 2 w2x 1 w1x 2

Totals 56 w30x 56 w48x 56 w48x 56

% Agree with consensus 53.8% 85.7% 85.7%
Kappa 0.4325 0.8189 0.8147

For columns A᎐C, numbers in brackets indicate the agreement between that diagnostic category ŽTDA, SCID-CV, or CADI.
and Consensus. For example, row 6, column A, shows that TDA diagnosed depression 16 times, and w10x of those agreed with
Consensus.

rion validity Ž‘results of the test instrument Overall, structured interviews were significantly
wCADIx are compared with results of a similar, better than unstructured interviews for diagnostic
but presumably more accurate, instrument agreement with Consensus. Being computer-based
wSCID-CVx’. ŽRegier and Burke, 2000, p. 506.: did not detract and may have helped: ‘use of the
results were in agreement s 80.4% and kappas computer can often enhance standardization of a
0.7510 Ž‘excellent’.. The comparison of the CADI test because the conditions under which the test
with Consensus Diagnosis was a test of content is administered can be more precisely regulated’
validity Ž‘a systematic examination of the new ŽMiller, 2000a, p. 784..
instrument wCADIx by an expert wthree to four
clinicians doing the Consensusx in the area to
ensure that it covers the type of information 4. Discussion
needed for later interpretation and scoring’. ŽRe-
gier and Burke, 2000, p. 506.: results were in
4.1. Limitations of this study
agreement s 85.7% and kappas 0.8147 Ž‘excel-
lent’.. For interrater reliability in a test᎐retest
format, CADI had 92.6% Ž25r27. agreement and 4.1.1. Regionalism
kappas 0.91 Ž‘excellent’.. Gutkind et al. Ž2001. Los Angeles County, with its urban population
found that ‘clarity of . . . criteria positively corre- of 13 million and its high racial-ethnic mix, makes
lated with . . . interrater reliability’, and that it non-representative of the USA.
applies to CADI, which defines each criterion
exactly and provides a model question for the 4.1.2. Subjects
investigator to use in the interview. Provisionally, Subjects may represent psychiatric inpatients
CADI appears to have criterion and content va- from the public sector ŽPollack et al., 1996., but
lidity and to have interrater reliability. not all patients from all sectors.
262 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264

4.1.3. Clinicians this topic ŽVentura et al., 1998; Shear et al.,

Clinicians who did the TDAs may represent 2000.. Both those studies found low levels of
clinicians in publicly funded academic hospitals, agreement between TDA and SCID.
but not all psychiatrists. Given these limitations,
results are provisional. 4.5. Diagnostic accuracy in medical practice

4.2. Diagnostic complications Diagnostic accuracy is a problem throughout

medicine. Pre-morbid clinical diagnosis for cause
Comorbidity confounds the diagnostic process, of death disagrees 40% of the time with the
and our subjects had multiple diagnoses Žmean of autopsy findings, a figure that has not changed
3.04rpatient., similar to what others have found between 1938 and 1998 ŽLundberg, 1998.. Given
ŽBasco et al., 2000; Shear et al., 2000; Zimmer- that medicine, with many objective tests, has such
man and Mattia, 1999.. In addition, most subjects problems, psychiatry’s task appears daunting,
Ž86%, 48r56. had disorders associated with since interviews ᎏ not objective tests ᎏ supply
diminished insight Žschizophrenia, mania, severe most of our data.
depression., and most subjects Ž71%, 40r56. were
hospitalized involuntarily, both of which may have 4.6. Computerized diagnostics
altered the subjects’ motives and abilities to par-
ticipate. Greist Ž1998. reviewed the computer’s uses in
psychiatry and concluded that it could help diag-
4.3. Hypothesis to explain the differences in nostics. Lieff Ž1987. described many conflicting
diagnostic agreement concerns and attitudes: ‘the possibility of com-
puter diagnosis has been an emotional and con-
TDA’s agreement with Consensus diagnosis was troversial area. Computers will never aid much in
significantly lower than SCID-CV’s or CADI’s. the diagnosis of the truly difficult ambiguous
Our hypothesis is based on our finding that TDA cases, and therefore the role of the physician is
users evaluated only 53% of Key Criteria, thus quite safe’ Žpp. 160᎐161.. Nonetheless, this study
failing to search for all possible diagnoses and found that ‘difficult ambiguous cases’ could be
increasing the likelihood of missing the Consen- evaluated with computer assistance at the same
sus Diagnosis. Table 3 is consistent with the hy- levels of agreement achieved by clinician-driven
pothesis. TDA was the only method that diag- SCID-CV and Consensus. Lieff continued, ‘ . . . the
nosed Psychosis NOS, a likely consequence of not computer is much more compulsive than the
gathering enough data to make a more specific physician in remembering to ask all relevant
diagnosis. TDAs also diagnosed Schizoaffective questions’, and that seems a reasonable explana-
Disorder less often, a diagnosis that requires the tion for the computer-driven results in this study.
clinician to evaluate 11 Key Criteria Ž7 for Lieff added, ‘ . . . the physician is more intuitive
schizophrenia, 2 for mania, 2 for depression.. The than the computer in using experience to focus
differences between TDA and Consensus in those on specific important areas’. Therefore, we should
two diagnostic groups accounted for 65% Ž17 of consider combining the best of both worlds ᎏ
26. of their differences. the clinician’s intuitiveness and the computer’s
inclusiveness ᎏ to enhance data collection and
4.4. Comparison of structured ¨ s. unstructured integrate diagnosis.
inter¨ iews
4.7. Conclusions
A search on MEDLINE Žkeywords: DSM-IV,
diagnosis, validity, accuracy, reliability, structured The initial diagnostic interview is fundamental
interview; time 1994᎐August 2001. found 319 ci- to clinical practice. This study found that diagnos-
tations, only two of which specifically addressed tic accuracy could be improved with use of struc-
 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264 263

tured methods, including those that are com- Hales, R.E., Yudofsky, S.C., Talbott, J.A. ŽEds.., 1995.
puter-assisted. Textbook of Psychiatry, 2nd ed. American Psychiatric Press,
Washington, DC.
Two evolving goals in clinical psychiatry today Hill, C., Keks, N., Roberts, S., Opeskin, K., Dean, B., MacKin-
are evidence-based practices ŽDrake et al., 2001; non, A., Copolov, D., 1996. Problem of diagnosis in post-
Torrey et al., 2001. and best practices ŽGlazer, mortem brain studies of schizophrenia. American Journal
1998.. The goals of each are stated in their titles. of Psychiatry 153, 533᎐537.
Computer-assisted structured interviews strongly Kaplan, H.I., Sadock, B.J. ŽEds.., 1998. Kaplan and Sadock’s
Synopsis of Psychiatry. Behavioral SciencesrClinical Psy-
support both: data are formatted for immediate
chiatry, 8th ed. Lippincott Williams & Wilkins, Baltimore.
use as evidence, and data-analysis Žfacilitated by Lieff, J.D., 1987. Computer Applications in Psychiatry. Ameri-
how data are stored and retrieved. can be done can Psychiatric Press, Washington, DC.
readily to evaluate which practices might be best. Lipton, A.A., Simon, F.S., 1985. Psychiatric diagnosis in a state
hospital: Manhattan State revisited. Hospital and Commu-
nity Psychiatry 36, 368᎐373.
Lundberg, G.D., 1998. Low-tech autopsies in the era of high-
Acknowledgements tech medicine: continued value for quality assurance and
patient safety. Journal of the American Medical Associa-
tion 2801, 1273᎐1274.
This study was supported by Eli Lilly and Com- McGorry, P.D., Mihalopoulos, C., Henry, L., Dakis, J., Jack-
pany. son, H.J., Flaum, M., Harrigan, S., McKenzie, D., Kulkarni,
H., Karoly, R., 1995. Spurious precision: procedural validity
of diagnostic assessment in psychotic disorders. American
References Journal of Psychiatry 152, 220᎐223.
Mihalopoulos, C., McGorry, P., Roberts, S., McFarlane, C,
Basco, M.R., Bostic, J.Q., Davies, D., Rush, A.J., Witte, B., 2000. The procedural validity of retrospective case note
Hendrickse, W., Barnett, V., 2000. Methods to improve diagnosis. Australian and New Zealand Journal of Psychia-
diagnostic accuracy in a community mental health setting. try 34, 154᎐159.
American Journal of Psychiatry 157, 1599᎐1605. Miller, M.J., 2000a. Computer-based testing of the psychiatric
Blacker, D., 2000. Psychiatric rating scales. In: Sadock, B.J., patient. In: Sadock, B.J., Sadock, V.A. ŽEds.., Kaplan and
Sadock, V.A. ŽEds.., Kaplan and Sadock’s Comprehensive Sadock’s Comprehensive Textbook of Psychiatry, 7th ed.
Textbook of Psychiatry, 8th ed. Lippincott Williams & Lippincott Williams & Wilkins, Philadelphia, pp. 783᎐788.
Wilkins, Philadelphia, pp. 755᎐783. Miller, P.R., 2000b. Traditional diagnostic assessments: reli-
Bogenschutz, M.P, Nurnberg, H.G., 2000. Classification of ability, completeness, and accuracy. Presentation at Sympo-
mental disorders. In: Sadock, B.J., Sadock, V.A. ŽEds.., sium, ‘Psychiatry and the Future,’ American Psychiatric
Kaplan and Sadock’s Comprehensive Textbook of Psychia- Association Annual Meeting, May 6, 2000.
try, 8th ed. Lippincott Williams & Wilkins, Philadelphia, Miller, P.R., 1996. The Computer Assisted Diagnostic Inter-
pp. 824᎐839. view. Multimedia Presentation, Annual Meeting May 1996.
Drake, R.E., Goldman, H.H., Leff, H.S., Lehman, A.F., Dixon, American Psychiatric Association, New York.
L., Mueser, K.T., Torrey, W.C., 2001. Implementing evi- Miller, P.R., 1998. The Computer Assisted Diagnostic Inter-
dence-based practices in routine mental health service set- view. Course Presentation, Annual Meeting October 1998.
tings. Psychiatric Services 52, 179᎐182. Institute of Psychiatric Services, Los Angeles.
Fleiss, J.L., 1973. Statistical Methods for Rates and Propor- Miller, P.R., 1999. The Computer Assisted Diagnostic Inter-
tions. John Wiley and Sons, New York. view. Course Presentation, Annual Meeting May 1999.
Frances, A., First, M.B., Pincus, H.A., 1995. DSM-IV Guide- American Psychiatric Association, Washington, DC.
book. American Psychiatric Press, Washington, DC. Mojtabai, R., Nicholson, R.A., 1995. Interrater reliability of
Garb, H.N., 1998. Studying the Clinician: Judgment Research ratings of delusion and bizarre delusions. American Journal
and Psychological Assessment. American Psychological As- of Psychiatry 152, 1804᎐1806.
sociation, Washington, DC. Othmer, E., Othmer, S.C., 1994. American Psychiatric Press,
Glazer, W.M., 1998. Defining best practices: a prescription for Washington, DC.
greater autonomy. Psychiatric Services 49, 1013᎐1016. Pollack, M.H., Otto, M.W., Rosenbaum, J.F. ŽEds.., 1996.
Greist, J.H., 1998. The computer as clinician assistant: assess- Challenges in Clinical Practice: Pharmacologic and Psy-
ment made simple. Psychiatric Services 49, 467᎐468, 472. chosocial Settings. Guilford Press, New York.
Gutkind, D., Ventura, J., Barr, C., Shaner, A., Green, M., Regier, D.A., Burke, J.D., 2000. Epidemiology. In: Sadock,
Mintz, J., 2001. Factors affecting reliability and confidence B.J., Sadock, V.A. ŽEds.., Kaplan and Sadock’s Comprehen-
of DSM-III-R psychosis-related diagnosis. Psychiatry Re- sive Textbook of Psychiatry, 8th ed. Lippincott Williams &
search 101, 269᎐275. Wilkins, Philadelphia, pp. 500᎐522.
264 P.R. Miller et al. r Psychiatry Research 105 (2001) 255᎐264

Robinson, E.P., Asnis, G.M., Karkavy Friedman, J.M., 1985. van Praag, H.M., 1997. The future of biological psychiatry.
Knowledge of the criteria for major depression: a survey of CNS Spectrums. International Journal of Neuropsychiatric
mental health professionals. Journal of Nervous and Men- Medicine 2, 18᎐25.
tal Disease 176, 480᎐484. Ventura, J., Liberman, R.P., Green, M.F., Shaner, A., Mintz,
Sacks, M.H., Sledge, W.H., Warren, C. ŽEds.., 1995. Core J., 1998. Training and quality assurance with the Structured
Readings in Psychiatry. An Annotated Guide to the Litera- Clinical Interview for DSM-IV ŽSCID-IrPO.. Psychiatry
ture, 2nd ed. American Psychiatric Press, Washington, DC. Research 70, 163᎐173.
Shear, M.K., Breeno, C., Kang, J., Ludewig, D., Frank, E.,
Whitehorn, H.C., 1944. Guide to interviewing and clinical
Swartz, H., Hanekamp, M., 2000. Diagnosis of nonpsychotic
personality study. Archives of Neurology and Psychiatry 52,
patients in community clinics. American Journal of Psychi-
197᎐216.
atry 157, 581᎐587.
Skodol, A.E., Williams, J.B.W., Spitzer, R.L., Gibbon, M., Widiger, T.A., Frances, A.J., Pincus, H.A., First, M.B., Ross,
Kass, F., 1984. Identifying common errors in the use of R., Davis, W. ŽEds.., 1994. DSM-IV Sourcebook, 1. Ameri-
DSM-III through supervision. Hospital and Community can Psychiatric Association, Washington, DC.
Psychiatry 35, 251᎐255. Widiger, T.A., Frances, A.J., Pincus, H.A., Ross, R., First,
Spitzer, R.L., 1983. Psychiatric diagnosis: are clinicians still M.B., Davis, W.W. ŽEds.., 1996. DSM-IV Sourcebook, 2.
necessary. Comprehensive Psychiatry 24, 399᎐411. American Psychiatric Association, Washington, DC.
Sullivan, H.S., 1970. The Psychiatric Interview. W.W. Norton, Williams, J.B., Gibbon, M., First, M.B., Spitzer, R.L., Davies,
New York. M., Borus, J., Howes, M.J., Kane, J., Pope, H.G., Rounsa-
Taintor, Z., Schwartz, M., Miller, M., 1997. Computers, the ville, B., Wittchen, H.U., 1992. The Structured Clinical
patient, and the psychiatrist. In: Dickstein, L.J., Riba, M.B., Interview for DSM-III-R ŽSCID.. Archives of General Psy-
Oldham, J.M. ŽEds.., Review of Psychiatry, 16. American chiatry 49, 630᎐636.
Psychiatric Press, Washington, DC, pp. 15᎐47. Zimmerman, M., Mattia, J.I., 1999. Psychiatric diagnosis in
Torrey, W.C., Drake, R.E., Dixon, L., Burns, B.J., Flynn, L., clinical practice: is comorbidity being missed? Comprehen-
Rush, A.J., Clark, R.E., Klatzker, D., 2001. Implementing sive Psychiatry 40, 182᎐191.
evidence-based practices for persons with severe mental
illnesses. Psychiatric Services 52, 45᎐50.