Cochrans Q Test

NCSS Statistical Software NCSS.
com
Chapter 521
Cochran’s Q Test
Introduction
This procedure computes the non-parametric Cochran’s Q test for related categories where the response is binary.
Cochran’s Q is used for testing k = 2 or more matched sets, where a binary response (e.g. 0 or 1) is recorded from
each category within each subject. Cochran’s Q tests the null hypothesis that the proportion of “successes” is the
same in all groups versus the alternative that the proportion is different in at least one of the groups.
Cochran’s Q test is an extension of the McNemar test to a situation where there are more than two matched
samples. When Cochran’s Q test is computed with only k = 2 groups, the results are equivalent to those obtained
from the McNemar test (without continuity correction). Cochran’s Q is also considered to be a special case of the
non-parametric Friedman test, which is similar to repeated measures ANOVA and is used to detect differences in
multiple matched sets with numeric responses. When the responses are binary, the Friedman test becomes
Cochran’s Q test.
This procedure also computes two-sided, pairwise multiple comparison tests that allow you to determine which of
the individual groups are different if the null hypothesis in Cochran’s Q test is rejected. The individual alpha level
is adjusted using the Bonferroni method to control the overall experiment-wise error rate.
This procedure is based on the results and formulas given in chapter 26 of Sheskin (2011). We refer you there for
additional information about Cochran’s Q test.
Experimental Design
A typical design for this scenario involves N individuals where a binary measurement (e.g. 0 or 1) is made on
each individual for each of k categories, where k ≥ 2. Typical data might appear as
Subject Condition 1 Condition 2 Condition 3

1 0 1 0
2 1 1 0
3 1 1 1
4 0 0 0
5 1 0 0
6 0 1 1
7 0 0 0
8 1 1 0
9 0 1 0
10 1 1 1
11 0 1 0
12 1 1 0
where, in this case, each subject responds to 3 different conditions with either a Yes (1) or No (0). In NCSS, the
responses may be coded as either text values (e.g. Yes, No) or numeric values (e.g. 0, 1).
521-1
© NCSS, LLC. All Rights Reserved.
NCSS Statistical Software NCSS.com
Cochran’s Q Test
Technical Details
Suppose we have k binary measurements on each of 𝑁𝑁 subjects (where the “subject” may be a set of matched
individuals). Let 𝑌𝑌𝑖𝑖,𝑗𝑗 be the binary response from subject i in category j (i = 1 to N, j = 1 to k), with success = 1
and failure = 0. Let the proportions, 𝜋𝜋1 , 𝜋𝜋2 , … , 𝜋𝜋𝑘𝑘 , represent the proportion of “successes” in each of the k
groups.
Cochran’s Q is used to test the null hypothesis
𝐻𝐻0 : 𝜋𝜋1 = 𝜋𝜋2 = … = 𝜋𝜋𝑘𝑘
versus the alternative
𝐻𝐻𝐴𝐴 : 𝜋𝜋a ≠ 𝜋𝜋b for at least one pair 𝜋𝜋a , 𝜋𝜋b, with a ≠ b and 1 ≤ a, b ≤ k.
In NCSS, these proportions, 𝜋𝜋a and 𝜋𝜋b, are displayed as percentages.
Assumptions
The Cochran’s Q test and associated multiple comparisons require the following assumptions:
1. Responses are binary and from k matched samples.
2. The subjects are independent of one another and were selected at random from a larger population.
3. The sample size is sufficiently “large”. (As a rule of thumb, the number of subjects for which the
responses are not all 0’s or 1’s, n, should be ≥ 4 and nk should be ≥ 24. This assumption is not required
for the exact binomial McNemar test.)
Cochran’s Q Test Statistic

For binary responses, 𝑌𝑌𝑖𝑖,𝑗𝑗 , in k matched groups from N subjects, the Cochran’s Q test statistic is computed as
(𝑘𝑘 − 1)[𝑘𝑘𝑘𝑘 − 𝑇𝑇 2 ]
𝑄𝑄 =
𝑘𝑘𝑘𝑘 − 𝑅𝑅
where
𝑘𝑘 𝑁𝑁 2
𝐶𝐶 = � �� 𝑌𝑌𝑖𝑖,𝑗𝑗 �
𝑗𝑗=1 𝑖𝑖=1
𝑁𝑁 𝑘𝑘
𝑇𝑇 = � �� 𝑌𝑌𝑖𝑖,𝑗𝑗 �
𝑖𝑖=1 𝑗𝑗=1
2
𝑁𝑁 𝑘𝑘
𝑅𝑅 = � �� 𝑌𝑌𝑖𝑖,𝑗𝑗 �
For “large” samples, the test statistic, Q, is distributed as chi-square with 𝑘𝑘 − 1 degrees of freedom. As in the
McNemar test, only subjects who do not have the same response in all categories contribute to the overall Q
statistic.
521-2
Cochran’s Q Test
The p-value for the test is computed as

2
P-Value = Pr�𝑄𝑄 > 𝜒𝜒1−𝛼𝛼,𝑘𝑘−1 �
2
where 𝜒𝜒1−𝛼𝛼,𝑘𝑘−1 is the value of the (1 − 𝛼𝛼) quantile of the chi-square distribution with 𝑘𝑘 − 1 degrees of freedom.
Multiple Comparisons
When the null hypothesis of success proportion equality is rejected by Cochran’s Q test, you can proceed to
determine which of the groups are different by computing multiple pairwise comparisons.
Pairwise tests between groups “a” and “b” test the null hypothesis
𝐻𝐻0 : 𝜋𝜋a = 𝜋𝜋b
versus the alternative
𝐻𝐻𝐴𝐴 : 𝜋𝜋a ≠ 𝜋𝜋b
In NCSS, these proportions, 𝜋𝜋a and 𝜋𝜋b, are displayed as percentages.
NCSS provides two methods as described in chapter 26 of Sheskin (2011). The first method identifies the
minimum required difference, MRD, needed to declare a pair of experimental conditions as significantly different.
The second method simply employs pairwise McNemar tests among groups to find significant differences.
Both multiple comparison methods use the Bonferroni alpha adjustment to control the overall experiment-wise
error of the tests. The adjustment simply divides the overall required alpha, 𝛼𝛼, by the number of pairwise tests, c,
where
𝑘𝑘(𝑘𝑘 − 1)
𝑐𝑐 = .
2
The alpha-level for each individual test, 𝛼𝛼𝑎𝑎𝑎𝑎𝑎𝑎 , is

𝑎𝑎
𝛼𝛼𝑎𝑎𝑎𝑎𝑎𝑎 = .
𝑐𝑐
For example, if the desired overall alpha were 0.05 and three groups were being compared, then the individual
alpha level for each test would be 0.05/[3(2)/2] = 0.05/3 = 0.0167.
Minimum Required Difference

For sufficiently large sample sizes (i.e. n ≥ 4 and nk ≥ 24, where n is the number of subjects for which the
responses are not all 0’s or 1’s), the minimum required difference in proportions for any pair of k experimental
groups to be declared different is
𝑘𝑘𝑘𝑘 − 𝑅𝑅
𝑀𝑀𝑀𝑀𝑀𝑀 = 𝑧𝑧𝑎𝑎𝑎𝑎𝑎𝑎 �2 � �
𝑁𝑁 2 𝑘𝑘(𝑘𝑘 − 1)
where N, T, and R are defined as in Cochran’s Q statistic, with

𝑁𝑁 𝑘𝑘
𝑇𝑇 = � �� 𝑌𝑌𝑖𝑖,𝑗𝑗 �
2
𝑁𝑁 𝑘𝑘
𝑅𝑅 = � �� 𝑌𝑌𝑖𝑖,𝑗𝑗 �
521-3
Cochran’s Q Test
and
𝑧𝑧𝑎𝑎𝑎𝑎𝑎𝑎 is the value of the �1 − 𝛼𝛼𝑎𝑎𝑎𝑎𝑎𝑎 /2� quantile from the standard normal distribution.
Two groups are declared to be significantly different with protected overall alpha, 𝛼𝛼, if their absolute difference in
proportions is greater than MRD, that is if
|𝜋𝜋a − 𝜋𝜋b | > 𝑀𝑀𝑀𝑀𝑀𝑀
McNemar Tests
The McNemar test statistic for each pair of groups is computed as
(𝑛𝑛1 − 𝑛𝑛2 )2
𝑀𝑀 =
𝑛𝑛1 + 𝑛𝑛2
where
𝑛𝑛1 = number of subjects where group “a” response = 0 and group “b” response = 1.
𝑛𝑛2 = number of subjects where group “a” response = 1 and group “b” response = 0.
Large-Sample (Asymptotic)
For sufficiently large sample sizes (i.e. n ≥ 4 and nk ≥ 24, where n is the number of subjects for which the
responses are not all 0’s or 1’s), the test statistic, M, is asymptotically distributed as chi-square with 1 degree of
freedom. The p-value for the individual test with protected overall alpha, 𝛼𝛼, is computed as
2
P-Value = Pr �𝑀𝑀 > 𝜒𝜒1−𝛼𝛼 𝑎𝑎𝑎𝑎𝑎𝑎 ,1
�
2
where 𝜒𝜒1−𝛼𝛼 𝑎𝑎𝑎𝑎𝑎𝑎 ,1
is the value of the �1 − 𝛼𝛼𝑎𝑎𝑎𝑎𝑎𝑎 � quantile of the chi-square distribution with 1 degree of freedom.
Exact Test
Exact p-values for the McNemar test can also be computed by enumerating and summing individual binomial
probabilities of results more extreme than the observed. The exact test results are more accurate than the
asymptotic test results because there is no approximation.
A Note on the Power of the Two Multiple Comparison Tests

The Minimum Required Difference multiple comparison method uses all of the available information in the data
in its calculations, but the multiple McNemar tests comparison method uses just the values from subjects who
have different responses for the two categories, not all of the data. For this reason some argue that the Minimum
Required Difference method is more powerful for finding differences than using multiple McNemar tests (see
Note #9 on page 1135 of Sheskin (2011)).
Data Structure
The data may be entered in two formats, as shown in the examples below. The examples give the binary responses
of 12 subjects to each of 3 experimental conditions.
The first format, shown in the first table below, puts the responses for each group in separate columns; that is,
each column contains all responses for each condition. Each row corresponds to a single subject. This format
allows for the use of an additional optional frequency variable for summarized data.
The second format, shown in the second table below, arranges the data so that all responses are entered in a single
column. A grouping variable contains an index that gives the group (Condition 1, 2, or 3) to which each row of
data belongs. The subject variable specifies the individual to which each response belongs. This second format
allows you to specify multiple response variables; a separate analysis is carried out for each response variable.
521-4
Cochran’s Q Test
Multiple Response Variables

Subject Condition 1 Condition 2 Condition 3
1 0 1 0
2 1 1 0
3 1 1 1
4 0 0 0
5 1 0 0
6 0 1 1
7 0 0 0
8 1 1 0
9 0 1 0
10 0 1 0
11 0 1 0
12 0 1 0
Response Variable, Grouping Variable, and a Subject Variable

Subject Condition Response
1 1 0
1 2 1
1 3 0
2 1 1
2 2 1
2 3 0
3 1 1
3 2 1
3 3 1
4 1 0
4 2 0
4 3 0
5 1 1
5 2 0
5 3 0
6 1 0
6 2 1
6 3 1
7 1 0
7 2 0
7 3 0
8 1 1
8 2 1
8 3 0
9 1 0
9 2 1
9 3 0
10 1 0
10 2 1
10 3 0
11 1 0
11 2 1
11 3 0
12 1 0
12 2 1
12 3 0
521-5
Cochran’s Q Test
Procedure Options
This section describes the options available in this procedure.
Variables Tab
This panel specifies the variables used in the analysis.
Input Type
In this procedure, there are two ways to format the data for analysis. Specify which method you want to use.
• Multiple Response Variables, One Variable per Group Category

The dataset includes two or more binary response variables. A single analysis is performed for all variables at
once. Each category is represented by a separate column. All information about a specific subject is entered
on a single row.
Example Dataset
R1 R2 R3
0 1 1
1 0 1
1 0 0
• Response Variable(s), a Grouping Variable, and a Subject Variable

The dataset includes a subject variable, a grouping variable, and one or more binary response variables. A
separate analysis is performed for each response variable.
Example Dataset
Subj Grp Resp
1 A 1
1 B 1
1 C 0
2 A 0
2 B 1
2 C 1
Variables
Response Variables (Displayed when Input Type = Multiple Response Variables)
Enter two or more binary response variables. Values may be text or numeric.
You can enter the column names or numbers directly, or double-click in the box to display a Column Selection
window that will let you select the variables from a list.
The dataset must include two or more binary response variables. Each category is represented by a separate
column. All information about a specific subject is entered on a single row. A single analysis is performed for all
of the variables at once.
Example Dataset
R1 R2 R3
0 1 1
1 0 1
1 0 0
521-6
Cochran’s Q Test
Frequency Variable (Displayed when Input Type = Multiple Response Variables)

Specify an optional frequency (count) variable. This data column contains integers that represent the number of
observations (frequency) associated with each row of the dataset.
If this option is left blank, each dataset row has a frequency of one. This variable lets you modify that frequency.
This may be useful when your data are tabulated and you want to enter counts.
Response Variable(s) (Displayed when Input Type = Response Variable(s), a Grouping Variable,
and a Subject Variable)
Enter one or more binary response variables. Values may be text or numeric. A separate analysis is performed for
each response variable.
The dataset includes a subject variable, a grouping variable, and one or more binary response variables. In the
example data below “Resp” is the response variable.
Example Dataset
Subj Grp Resp
1 A 1
1 B 1
1 C 0
2 A 0
2 B 1
2 C 1
Grouping Variable (Displayed when Input Type = Response Variable(s), a Grouping Variable,
and a Subject Variable)
Enter a single categorical grouping variable. The values of this variable indicate which category each response
belongs in. Values may be text or numeric.
example data below “Grp” is the grouping variable.
Example Dataset
Subj Grp Resp
1 A 1
1 B 1
1 C 0
2 A 0
2 B 1
2 C 1
Subject Variable (Displayed when Input Type = Response Variable(s), a Grouping Variable, and a
Subject Variable)
Enter a single categorical subject variable. The values of this variable indicate which subject each response came
from. Values may be text or numeric.
example data below “Subj” is the subject variable.
521-7
Cochran’s Q Test
Example Dataset
Subj Grp Resp
1 A 1
1 B 1
1 C 0
2 A 0
2 B 1
2 C 1
Reports Tab
This tab controls which statistical reports and tables are displayed in the output.
Data Summary
Data Summary Report
Check this option to display a summary of the data used in the analysis with key statistics and helpful
information.
Count and Percentage Tables

Show Combined Table
Check this option to display a single table containing the selected statistics. After activating this option, you must
specify which items you would like to display in the table.
The items to choose from are:
• Counts
• Table Percentages
• Row Percentages
• Column Percentages
Show Individual Tables
Check this option to display a separate table for each statistic. After activating this option, you must specify which
tables you would like to display.
The tables to choose from are:
• Counts
• Table Percentages
• Row Percentages
• Column Percentages
Tests
Cochran’s Q Test
Check this option to output Cochran’s Q Test. This is the primary test for this procedure.
Multiple Comparisons using Minimum Required Absolute Difference
Check this option to output a report with pairwise comparisons between all groups, made using the Minimum
Required Absolute Difference. This multiple comparison procedure uses the Bonferroni alpha-level adjustment so
that the overall experiment-wise alpha level is achieved.
521-8
Cochran’s Q Test
A Note on Power
This multiple comparison method uses all of the data, not just the values from subjects who have different
responses for the two categories, so some argue that this method is more powerful than using multiple McNemar
tests (see Note #9 on page 1135 of Sheskin (2011)).
Multiple Comparisons using the McNemar Test
Check this option to output a report with pairwise comparisons between all groups, made using the McNemar
Test. The Bonferroni adjustment is applied and the alpha level is adjusted based the number of pairwise
comparisons so that the overall experiment-wise alpha level is achieved.
A Note on Power
This multiple comparison method uses just the values from subjects who have different responses for the two
categories, not all of the data. For this reason some argue that this method is less powerful than using the
Minimum Required Absolute Difference (see Note #9 on page 1135 of Sheskin (2011)).
Alpha for Test and Multiple Comparisons

Alpha
Enter the value of alpha to be used for all hypothesis tests in this procedure. The probability level (p-value) is
compared to alpha to determine whether to reject the null hypothesis.
Report Options Tab

The following options control the format of the reports.
Report Options
Variable Names
Specify whether to use variable names, variable labels, or both to label output reports. In this discussion, the
variables are the columns of the data table.
• Names
Variable names are the column headings that appear on the data table. They may be modified by clicking the
Column Info button on the Data window or by clicking the right mouse button while the mouse is pointing to
the column heading.
• Labels
This refers to the optional labels that may be specified for each column. Clicking the Column Info button on
the Data window allows you to enter them.
• Both
Both the variable names and labels are displayed.
Comments
1. Most reports are formatted to receive about 12 characters for variable names.
2. Variable Names cannot contain blanks or math symbols (like + - * / . ,), but variable labels can.
521-9
Cochran’s Q Test
Value Labels
Value Labels are used to make reports more legible by assigning meaningful labels to numbers and codes.
The options are
• Data Values
All data are displayed in their original format, regardless of whether a value label has been set or not.
• Value Labels
All values of variables that have a value label variable designated are converted to their corresponding value
label when they are output. This does not modify their value during computation.
• Both
Both data value and value label are displayed.
Example
A variable named GENDER (used as a grouping variable) contains 1’s and 2’s. By specifying a value label for
GENDER, the report can display “Male” instead of 1 and “Female” instead of 2. This option specifies whether
(and how) to use the value labels.
Table Formatting
Column Justification
Specify whether data columns in the contingency tables will be left or right justified.
Column Widths
Specify how the widths of columns in the contingency tables will be determined.
The options are
• Autosize to Minimum Widths

Each data column is individually resized to the smallest width required to display the data in the column. This
usually results in columns with different widths. This option produces the most compact table possible,
displaying the most data per page.
• Autosize to Equal Minimum Width

The smallest width of each data column is calculated and then all columns are resized to the width of the
widest column. This results in the most compact table possible where all data columns have the same with.
This is the default setting.
• Custom (User-Specified)
Specify the widths (in inches) of the columns directly instead of having the software calculate them for you.
521-10
Cochran’s Q Test
Custom Widths (Single Value or List)

Enter one or more values for the widths (in inches) of columns in the contingency tables. This option is only
displayed if Column Widths is set to “Custom (User-Specified)”.
• Single Value
If you enter a single value, that value will be used as the width for all data columns in the table.
• List of Values
Enter a list of values separated by spaces corresponding to the widths of each column. The first value is used
for the width of the first data column, the second for the width of the second data column, and so forth. Extra
values will be ignored. If you enter fewer values than the number of columns, the last value in your list will
be used for the remaining columns.
Type the word “Autosize” for any column to cause the program to calculate it's width for you. For example,
enter “1 Autosize 0.7” to make column 1 be 1 inch wide, column 2 be sized by the program, and column 3 be
0.7 inches wide.
Wrap Column Headings onto Two Lines
Check this option to make column headings wrap onto two lines. Use this option to condense your table when
your data are spaced too far apart because of long column headings.
Decimal Places
Item Decimal Places
These decimal options allow the user to specify the number of decimal places for items in the output. Your choice
here will not affect calculations; it will only affect the format of the output.
• Auto
If one of the “Auto” options is selected, the ending zero digits are not shown. For example, if “Auto (0 to 7)”
is chosen,
0.0500 is displayed as 0.05
1.314583689 is displayed as 1.314584
The output formatting system is not designed to accommodate “Auto (0 to 13)”, and if chosen, this will likely
lead to lines that run on to a second line. This option is included, however, for the rare case when a very large
number of decimals is needed.
Omit Percent Sign after Percentages
The program normally adds a percent sign, %, after each percentage. Checking this option will cause this percent
sign to be omitted.
521-11
Cochran’s Q Test
Example 1 – Multiple Response Variables

This section presents an example of how to run an analysis on hypothetical data where the responses are stored in
separate columns, one for each category. In this example, a series of physical exams were given to 12 subjects. If
the subject passed the exam, a “1” was recorded, otherwise a “0” was recorded for failure. They wish to determine
if there is a difference among the three exams.
You may follow along here by making the appropriate entries or load the completed template Example 1 by
clicking on Open Example Template from the File menu of the procedure window.
1 Open the PhysExam1 example dataset.

• From the File menu of the NCSS Data window, select Open Example Data.
• Click on the file PhysExam1.NCSS.
• Click Open.
2 Open the Cochran’s Q Test procedure window.

• Using the Analysis menu or the Procedure Navigator, find and select the Cochran’s Q Test procedure.
• On the menus, select File, then New Template. This will fill the procedure with the default template.
3 Specify the variables.

• Select the Variables tab.
• Leave Input Type as Multiple Response Variables, One Variable per Group Category.
• Double-click in the Response Variables text box. This will bring up the variable selection window.
• Select Exam1, Exam2, and Exam3 from the list of variables and then click OK. “Exam1-Exam3” will
appear in the Response Variables box.
• Leave the Frequency Variable box empty.
4 Specify the reports and plots.

• Leave all report and plot options at their default settings.
5 Run the procedure.

• From the Run menu, select Run Procedure. Alternatively, just click the green Run button.
Data Summary Section

Data Summary
Rows Processed: 12
Rows with Missing Values: 0
Rows Used in the Analysis: 12
Responses: 2 (0, 1)
Groups (k): 3 (Exam1, Exam2, Exam3)
Subjects or Blocks (N): 12
• Number with responses
that are not all equal (n): 11 (nk = 33)*
* Large-sample (asymptotic) test results should be used only if n ≥ 4 and nk ≥ 24.

Status: Conditions are met.
The Data Summary report gives a summary description of the data used in the analysis. The summary indicates
that the large-sample conditions (n ≥ 4 and nk ≥ 24) are met. One subject passed every exam, so his/her responses
will not contribute to the Cochran’s Q test statistic.
521-12
Cochran’s Q Test
Combined Table Section

Combined Table
Response
Variable
0 1 Total
Exam1 Count 6 6 12
% within Group 50.00 50.00 100.00
Exam2 Count 2 10 12
% within Group 16.67 83.33 100.00
Exam3 Count 9 3 12
% within Group 75.00 25.00 100.00
Total Count 17 19 36
% within Group 47.22 52.78 100.00
This report give the counts and percentages of each response within each category. Exam 2 has the highest pass
rate with 83.33%, while Exam 3 has the lowest pass rate with only 25%.
Cochran’s Q Test Section

Cochran's Q Test
(Exam1 by Exam2 by Exam3)
H0: The proportions of [Response = "1"] in all groups are equal.
H1: The proportion of [Response = "1"] in at least one group is different.
Chi-Square Asymptotic
Statistic Prob Reject H0
Test Type Q DF Level at α = 0.05?
Cochran's Q 2-Sided 6.7273 2 0.03461 Yes
Cochran’s Q test has a p-value of 0.03461, indicating that the success rate for at least one group is different from
the others. Because this result is significant, we can proceed to consider the multiple comparison tests.

Section
H0: πᵢ = πⱼ (The proportions of [Response = "1"] in the two groups are equal.)
H1: πᵢ ≠ πⱼ (The proportions of [Response = "1"] in the two groups are not equal.)
Number of Comparisons (c): 3
Minimum
Absolute Required Reject H0
Difference Absolute with Overall
Comparison* πᵢ (%) πⱼ (%) |πᵢ - πⱼ| Difference α = 0.05?†
Exam1 vs. Exam2 50.00 83.33 33.33 54.02 No
Exam1 vs. Exam3 50.00 25.00 25.00 54.02 No
Exam2 vs. Exam3 83.33 25.00 58.33 54.02 Yes
* These tests should only be considered if the null hypothesis of equality was rejected by Cochran's Q Test.
† Individual Comparison Alpha = (Overall Alpha)/c = 0.05/3 = 0.01667.
These multiple comparison results indicate that exams 2 and 3 are significantly different from each other with an
absolute different of 58.33%.
521-13
Cochran’s Q Test
Multiple Comparisons using the McNemar Test Section

Reject H0
Chi-Square Prob with Overall
Comparison* πᵢ (%) πⱼ (%) Test Value DF Level α = 0.05?†
Exam1 vs. Exam2 50.00 83.33 Asymptotic 2.0000 1 0.15730 No
Binomial Exact 0.28906 No


These multiple comparison results indicate that there are no pairs significantly different from one another. It’s
interesting to note here that no pairs were found to be different even though the overall Cochran’s Q test found a
significant difference and the multiple comparisons test using minimum required absolute difference found a
difference between exams 2 and 3. This is likely due to the fact that McNemar test has lower overall power
because only discordant pairs are used in the computation of the test statistic. The other multiple comparison
procedure uses all of the data and, thus, has more power.
Plots Section
Plots Section
This section provides a graphical representation of the counts and percentages for each response within each
category.
521-14
Cochran’s Q Test
Example 2 – Multiple Response Variables with a Frequency

Variable
Continuing from Example 1, we’ll now show you how to analyze data that has been tabulated (summarized) using
a frequency variable. The data for this example is exactly the same as that from Example 1 except that it has been
summarized. The variable “Count” indicates how many subjects correspond to each series of responses.
1 Open the PhysExam2 example dataset.

• Click on the file PhysExam2.NCSS.
• Click Open.


• Select Exam1, Exam2, and Exam3 from the list of variables and then click OK. “Exam1-Exam3” will
appear in the Response Variables box.
• For Frequency Variable enter Count.


Output
Data Summary
Rows Processed: 8
Rows with Missing Values: 0
Rows Used in the Analysis: 8
Responses: 2 (0, 1)
Groups (k): 3 (Exam1, Exam2, Exam3)
Subjects or Blocks (N): 12
• Number with responses
that are not all equal (n): 11 (nk = 33)*
* Large-sample (asymptotic) test results should be used only if n ≥ 4 and nk ≥ 24.

Status: Conditions are met.
521-15
Cochran’s Q Test
Cochran's Q Test
(Exam1 by Exam2 by Exam3)
The Data Summary report indicates that only 8 rows were processed this time (12 rows were processed in
Example 1), but the number of subjects, 12, is the same as in Example 1 because some of the rows represent more
than one individual. The Cochran’s Q test results are exactly the same as in Example 1.
Example 3 – Responses All in a Single Column

This section presents an example of how to run an analysis on hypothetical data where the responses are stored in
a single column with a separate subject and grouping variable. In this example, responses from 20 individuals are
recorded for each of 4 pain relief drugs (A, B, C, and D). One hour after the administration of each drug, each
subject was asked whether the medication was effective for them in controlling pain. Responses were recorded as
either “Yes” or “No”. Drugs were administered in random order, each after a washout period.
1 Open the PainDrug example dataset.

• Click on the file PainDrug.NCSS.
• Click Open.


• Change Input Type to Response Variable(s), a Grouping Variable, and a Subject Variable.
• Double-click in the Response Variable(s) text box. This will bring up the variable selection window.
• Select Response from the list of variables and then click OK.
• Double-click in the Grouping Variable text box. This will bring up the variable selection window.
• Select Drug from the list of variables and then click OK.
• Double-click in the Subject Variable text box. This will bring up the variable selection window.
• Select Subject from the list of variables and then click OK.
4 Specify the reports.

• Click on the Reports tab.
• Deselect all reports except the Combined Table (with Counts and Percentages within Groups) and
Cochran’s Q Test.
• Click on the Report Options tab.
• For Variable Names select Labels.
• Click on the Plots tab.
• Uncheck Show Plots.
521-16
Cochran’s Q Test

Output
Combined Table
Was the Drug Effective?

Drug
No Yes Total
A Count 7 13 20
% within Group 35.00 65.00 100.00
B Count 11 9 20
% within Group 55.00 45.00 100.00
C Count 7 13 20
% within Group 35.00 65.00 100.00
D Count 14 6 20
% within Group 70.00 30.00 100.00
% within Group 48.75 51.25 100.00
Cochran's Q Test
(A by B by C by D)
H0: The proportions of [Response = "Yes"] in all groups are equal.
H1: The proportion of [Response = "Yes"] in at least one group is different.
Cochran's Q 2-Sided 6.2239 3 0.10121 No
Cochran’s Q test indicates that there is not a significant difference among the 4 medications. There is no reason to
look at the multiple comparison tests at this point.
521-17
Cochran’s Q Test
Example 4 – Validation of Cochran’s Q and Multiple Comparison

Tests using Sheskin (2011)
Sheskin (2011) presents an example of computing Cochran’s Q test and the associated multiple comparisons in
chapter 26, starting on page 1120. The data for the example consists of responses from 12 female subjects about
whether or not they would purchase an automobile manufactured by three different companies: Chenesco,
Howasaki, and Gemini. The data for this validation example are contained in the dataset called “Sheskin”.
Sheskin (2011) computes the group proportions for Response = “1” in Chenesco, Howasaki, and Gemini as 0.25
(25%), 0.75 (75%), and 0.25 (25%), respectively. They compute a Cochran’s Q value of 8.0. They do not
compute the p-value directly, but state that it is between 0.01 and 0.05. For the multiple comparison test using the
minimum required absolute difference with an overall alpha level of 0.05, they find the minimum required
difference to be 0.49 (or 49%). They conclude that Howasaki is different from both Chenesco and Gemini, each
with differences of 50%, which are both greater than 49%. Sheskin further computes the binomial exact test p-
values for the multiple McNemar test comparisons as 0.0312 for Chenesco vs. Howasaki and 0.0704 for
Howasaki vs. Gemini. Both are greater than the Bonferroni-adjusted alpha level of 0.0167 so both tests fail to
reject the null hypothesis.
The results from NCSS match all of these results, with slight difference due to rounding.
1 Open the Sheskin example dataset.

• Click on the file Sheskin.NCSS.
• Click Open.


• Select Chenesco, Howasaki, and Gemini from the list of variables and then click OK. “Chenesco-
Gemini” will appear in the Response Variables box.
• Leave the Frequency Variable box empty.


521-18
Cochran’s Q Test
Output
Combined Table
Response
Variable
0 1 Total
Chenesco Count 9 3 12
% within Group 75.00 25.00 100.00
Gemini Count 9 3 12
% within Group 75.00 25.00 100.00
Howasaki Count 3 9 12
% within Group 25.00 75.00 100.00
% within Group 58.33 41.67 100.00
Cochran's Q Test
(Chenesco by Gemini by Howasaki)

Minimum
Absolute Required Reject H0
Difference Absolute with Overall
Comparison* πᵢ (%) πⱼ (%) |πᵢ - πⱼ| Difference α = 0.05?†
Chenesco vs. Gemini 25.00 25.00 0.00 48.87 No
Chenesco vs. Howasaki
25.00 75.00 50.00 48.87 Yes
Gemini vs. Howasaki 25.00 75.00 50.00 48.87 Yes

Reject H0
Chi-Square Prob with Overall
Comparison* πᵢ (%) πⱼ (%) Test Value DF Level α = 0.05?†
Chenesco vs. Gemini 25.00 25.00 Asymptotic 0.0000 1 1.00000 No
Chenesco vs. Howasaki

25.00 75.00 Asymptotic 6.0000 1 0.01431 Yes
Gemini vs. Howasaki 25.00 75.00 Asymptotic 4.5000 1 0.03389 No

The results from NCSS match Sheskin (2011), with slight differences due to rounding. Key matched items are
highlighted in purple.
521-19

Cochrans Q Test

Uploaded by

Copyright:

Available Formats

Cochrans Q Test

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cochrans Q Test

Uploaded by

Copyright:

Available Formats

NCSS Statistical Software NCSS.

Subject Condition 1 Condition 2 Condition 3

Cochran’s Q Test Statistic

The p-value for the test is computed as

The alpha-level for each individual test, 𝛼𝛼𝑎𝑎𝑎𝑎𝑎𝑎 , is

Minimum Required Difference

where N, T, and R are defined as in Cochran’s Q statistic, with

A Note on the Power of the Two Multiple Comparison Tests

Multiple Response Variables

Response Variable, Grouping Variable, and a Subject Variable

• Multiple Response Variables, One Variable per Group Category

• Response Variable(s), a Grouping Variable, and a Subject Variable

Frequency Variable (Displayed when Input Type = Multiple Response Variables)

Count and Percentage Tables

Alpha for Test and Multiple Comparisons

Report Options Tab

• Autosize to Minimum Widths

• Autosize to Equal Minimum Width

Custom Widths (Single Value or List)

Example 1 – Multiple Response Variables

1 Open the PhysExam1 example dataset.

2 Open the Cochran’s Q Test procedure window.

3 Specify the variables.

4 Specify the reports and plots.

5 Run the procedure.

Data Summary Section

* Large-sample (asymptotic) test results should be used only if n ≥ 4 and nk ≥ 24.

Combined Table Section

Cochran’s Q Test Section

Multiple Comparisons using Minimum Required Absolute Difference

Multiple Comparisons using the McNemar Test Section

Exam1 vs. Exam3 50.00 25.00 Asymptotic 1.8000 1 0.17971 No

Exam2 vs. Exam3 83.33 25.00 Asymptotic 5.4444 1 0.01963 No

Example 2 – Multiple Response Variables with a Frequency

1 Open the PhysExam2 example dataset.

2 Open the Cochran’s Q Test procedure window.

3 Specify the variables.

4 Specify the reports and plots.

5 Run the procedure.

* Large-sample (asymptotic) test results should be used only if n ≥ 4 and nk ≥ 24.

Example 3 – Responses All in a Single Column

1 Open the PainDrug example dataset.

2 Open the Cochran’s Q Test procedure window.

3 Specify the variables.

4 Specify the reports.

5 Run the procedure.

Was the Drug Effective?

Example 4 – Validation of Cochran’s Q and Multiple Comparison

1 Open the Sheskin example dataset.

2 Open the Cochran’s Q Test procedure window.

3 Specify the variables.

4 Specify the reports and plots.

5 Run the procedure.

Multiple Comparisons using Minimum Required Absolute Difference

Multiple Comparisons using the McNemar Test

Chenesco vs. Howasaki

Gemini vs. Howasaki 25.00 75.00 Asymptotic 4.5000 1 0.03389 No

You might also like