Diagnostic Study Appraisal Worksheet

DIAGNOSTIC ACCURACY STUDIES

STUDENT NAME :
STUDENT NUMBER :

Title of article: Sputum smear negative pulmonary tuberculosis: sensitivity and

specificity of diagnostic algorithm

What question did the study ask?

Patients : The study enrolled 467 subjects who were suspected of having
pulmonary tuberculosis (PTB) and had sputum smear negative for acid-fast bacilli (AFB). Of
those, 318 (68.1%) were HIV positive and 127 (27.2%) had sputum culture positive for
Mycobacteria Tuberculosis.
Intervention : The study used a diagnostic algorithm recommended by the
National Tuberculosis and Leprosy Programme (NTLP) that uses symptoms, signs, and
laboratory results to diagnose smear negative PTB. The algorithm consists of four steps: 1)
clinical assessment, 2) chest X-ray, 3) trial of antibiotics, and 4) sputum culture.
Comparison : The study compared the results of the diagnostic algorithm
with the gold standard of sputum culture for Mycobacteria Tuberculosis. The study also
compared the diagnostic algorithm with other criteria such as the World Health Organization
(WHO) criteria and the clinical judgment of the physicians.
Outcome(s) : The study measured the sensitivity and specificity of the
diagnostic algorithm for smear negative PTB, as well as the number of patients who were
correctly or incorrectly treated with anti-TB drugs. The study also identified the predictors of
sputum culture positive among the smear negative patients.

Step 1: Are the results of the study valid?

Was the diagnostic test evaluated in a Representative spectrum of patients (like

those in whom it would be used in practice)?
What is best? Where do I find the information?
It is ideal if the diagnostic test is applied to the The study population included patients from
full spectrum of patients - those with mild, different age groups, genders, and HIV
severe, early and late cases of the target statuses, and was recruited from four hospitals
disorder. It is also best if the patients are in Dar es Salaam, Tanzania. The authors
randomly selected or consecutive admissions claimed that their study population was similar
so that selection bias is minimized. to the patients who would receive the test in
practice in low-resource settings with high
prevalence of HIV and tuberculosis. However,
they also acknowledged some limitations, such
as the lack of data on the duration of symptoms,
the previous exposure to anti-TB drugs, and the
co-morbidities of the patients. Therefore, the
generalisability of the diagnostic test may be
affected by these factors.
This paper: Yes
Comment:
The diagnostic test was evaluated in a representative spectrum of patients who were suspected of
having pulmonary tuberculosis and had sputum smear negative for acid-fast bacilli.
Was the reference standard applied regardless of the index test result?
What is best? Where do I find the information?
Ideally both the index test and the reference The reference standard was sputum culture for
standard should be carried out on all Mycobacteria Tuberculosis, which is the most

1
Diagnostic Study Appraisal Worksheet

patients in the study. In some situations where reliable method for diagnosing pulmonary
the reference standard is invasive or expensive tuberculosis. The reference standard was
there may be reservations about subjecting applied to all patients in the study, regardless of
patients with a negative index test result (and the index test result. The index test was the
thus a low probability of disease) to the diagnostic algorithm recommended by the
reference standard. An alternative reference National Tuberculosis and Leprosy Programme,
standard is to follow-up people for an which uses symptoms, signs, and laboratory
appropriate period of time (dependent on results to diagnose smear negative pulmonary
disease in question) to see if they are truly tuberculosis. The authors stated that "all
negative. patients were subjected to sputum culture
irrespective of the outcome of the diagnostic
algorithm".
This paper: Yes
Comment:
The reference standard was applied regardless of the index test result in this study.

Was there an independent, blind comparison between the index test and an
appropriate reference ('gold') standard of diagnosis?
What is best? Where do I find the information?
There are two issues here. First the reference The article there was an independent, blind
standard should be appropriate - as close to comparison between the index test and the
the 'truth' as possible. Sometimes there may reference standard. The authors stated that "the
not be a single reference test that is suitable results of the diagnostic algorithm were not
and a combination of tests may be used to known to the laboratory personnel who
indicate the presence of disease. performed the sputum culture and vice versa".
Second, the reference standard and the index
test being assessed should be applied to each
patient independently and blindly. Those who
interpreted the results of one test should not be
aware of the results of the other test.
This paper: Yes
Comment:
The study met this criterion for a valid diagnostic test study.

2
Diagnostic Study Appraisal Worksheet

Step 2: What were the results?

Are test characteristics presented?

There are two types of results commonly reported in diagnostic test studies. One concerns the
accuracy of the test and is reflected in the sensitivity and specificity. The other concerns how the
test performs in the population being tested and is reflected in predictive values (also called post-
test probabilities). To explore the meaning of these terms, consider a study in which 1000 elderly
people with suspected dementia undergo an index test and a reference standard. The prevalence
of dementia in this group is 25%. 240 people tested positive on both the index test and the
reference standard and 600 people tested negative on both tests. The first step is to draw a 2 x 2
table as shown below. We are told that the prevalence of dementia is 25% therefore we can fill in
the last row of totals - 25% of 1000 people is 250 - so 250 people will have dementia and 750 will
be free of dementia. We also know the number of people testing positive and negative on both
tests and so we can fill in two more cells of the table.
Reference Standard
+ve -ve
Index test +ve 240
-ve 600
250 750 1000
By subtraction we can easily complete the table:
Reference Standard
+ve -ve
Index test +ve 240 150 390
-ve 10 600 610
250 750 1000
Now we are ready to calculate the various measures.
What is the measure? What does it mean?
Sensitivity (Sn) = the proportion of people The diagnostic algorithm for smear negative PTB
with the condition who have a positive test had a sensitivity 38,1%. This means that out of 100
result. people who had smear negative but culture positive
PTB, the algorithm only detected 38 of them as
having PTB, and missed 62 of them. These values
indicate that the algorithm is not very accurate in
diagnosing smear negative PTB, especially in
patients with HIV infection.
In our example, the Sn = 240/250 = 0.96 Out of 100 people who had smear negative but
culture positive PTB, the algorithm only detected 38
of them as having PTB, and missed 62 of them.
Specificity (Sp) = the proportion of people In the article, the diagnostic algorithm for smear
without the condition who have a negative negative PTB had a specificity of 74.5%. This
test result. mean, out of 100 people who had smear negative
and culture negative PTB, the algorithm correctly
ruled out 75 of them as not having PTB, and
wrongly diagnosed 25 of them as having PTB.
These values indicate that the algorithm is not very
accurate in diagnosing smear negative PTB,
especially in patients with HIV infection.
In our example, the Sp = 600/750 = 0.80 Out of 100 people who had smear negative and
culture negative PTB, the algorithm correctly ruled
out 75 of them as not having PTB, and wrongly
diagnosed 25 of them as having PTB.
Positive Predictive Value (PPV) = the In the article, the diagnostic algorithm for smear
proportion of people with a positive test negative PTB had a PPV of 38.1%. These values
who have the condition. indicate that the algorithm is not very accurate in
diagnosing smear negative PTB, especially in
patients with HIV infection. The authors suggest that
other factors, such as dry cough, high respiratory
rate, low eosinophil count, mixed type of anemia
and presence of a cavity, should be considered as

3
Diagnostic Study Appraisal Worksheet

predictors of smear negative but culture positive

PTB.
In our example, the PPV = 240/390 = 0.62 Out of 100 people who tested positive for PTB, only
38 actually had PTB.
Negative Predictive Value (NPV) = the In the article, the diagnostic algorithm for smear
proportion of people with a negative test negative PTB had a NPV of 74.5%, respectively.
who do not have the condition. These values indicate that the algorithm is not very
accurate in diagnosing smear negative PTB,
especially in patients with HIV infection. The authors
suggest that other factors, such as dry cough, high
respiratory rate, low eosinophil count, mixed type of
anemia and presence of a cavity, should be
considered as predictors of smear negative but
culture positive PTB.
In our example, the NPV = 600/610 = 0.98 Out of 100 people who tested negative for PTB, 75
did not have PTB.

Step 3: Applicability of the results

Were the methods for performing the test described in sufficient detail to permit
replication?
What is best? Where do I find the information?
The article should have sufficient description of The article describes the methods for
the test to allow its replication and also performing the sputum smear test and the
interpretation of the results. sputum culture test for diagnosing pulmonary
tuberculosis (PTB) in patients with or without
HIV infection. The article provides the following
details about the tests:
- Sputum smear test: The patients were asked
to produce three sputum samples (one spot,
one early morning and one spot) within two
days. The samples were stained with Ziehl-
Neelsen stain and examined under a light
microscope for acid-fast bacilli (AFB). A positive
result was defined as the presence of at least
one AFB per 100 fields.
- Sputum culture test: The patients with sputum
smear negative were enrolled into the study.
Their sputum samples were decontaminated
with N-acetyl-L-cysteine-sodium hydroxide
(NALC-NaOH) and inoculated on Lowenstein-
Jensen (LJ) medium. The cultures were
incubated at 37°C for up to eight weeks and
checked weekly for growth. A positive result
was defined as the presence of colonies with
typical morphology of Mycobacterium
tuberculosis.
This paper: Yes
Comment:
Based on these details, I think the methods for performing the test are described in sufficient detail
to permit replication by other researchers. However, the article does not mention the quality
control measures or the sources of potential errors or biases in the tests. These aspects could
also be important for ensuring the validity and reliability of the test results.