See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/326275068

A review of the application of logistic regression in educational research:

common issues, implications, and suggestions

Article  in  Educational Review · July 2018

DOI: 10.1080/00131911.2018.1483892

CITATIONS READS

11 3,777

1 author:

Lian Niu
Iowa State University
10 PUBLICATIONS   383 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Lian Niu on 21 February 2019.

The user has requested enhancement of the downloaded file.

 1

A Review of the Adoption of Logistic Regression in Educational Research: Common Issues,

Implications, and Suggestions

Lian Niu

Correspondence concerning this article should be addressed to Lian Niu at

[email protected]

This is the author’s copy before final acceptance by the journal. For the published version of
this article, please refer to the publisher’s website at
https://doi.org/10.1080/00131911.2018.1483892

To cite this article:

Niu, L. (2018). A review of the adoption of logistic regression in educational research:

Common issues, implications, and suggestions. Educational Review. Advance online
publication. doi: 10.1080/00131911.2018.1483892

 
 2

A Review of the Application of Logistic Regression in Educational Research: Common

Issues, Implications, and Suggestions

Abstract

This study reviews the international literature of empirical educational research to examine

the application of logistic regression. The aim is to examine common practices of the report

and interpretation of logistic regression results, and to discuss the implications for

educational research. A review of 130 studies suggests that: (a) the majority of studies report

statistical significance and sign of predictors but do not interpret relationship magnitude in

terms of probabilities; (b) odds ratio is the most commonly reported effect size, and it tends

to be incorrectly interpreted as relative risk, which leads to significant exaggeration of the

association magnitude and misleading conclusions; and (c) marginal effects and predicted

probabilities are reported by only 10.7% of reviewed studies, and the specification of

independent variables’ values is frequently missing. It is suggested that marginal effects and

predicted probabilities be reported more frequently to fully utilize the information provided

by logistic regression results.

Keywords: logistic regression, odds ratio, relative risk, marginal effects, effect size

exaggeration

 
 3

Introduction

Logistic regression is the commonly used statistical method in empirical studies

involving categorical dependent variables. As Allison (1999) demonstrates, a dichotomous

dependent variable violates the assumptions of homoscedasticity and normality of the error

term for linear regression model. Consequently, the estimates of the standard error will not be

consistent estimates of the true standard errors, and the coefficient estimates will no longer be

efficient. In addition, estimating a linear probability model with the ordinary least squares

technique will lead to predicted values that are outside the plausible range of the probability

(0,1). For these reasons, the logistic regression model is used when the dependent variable is

dichotomous. This model transforms probability to odds and then takes the logarithm of the

odds. By doing so, both the lower and upper bound of the probability is removed. The logistic

regression model takes the following form (Allison 1999, 13):

log [ ] = α + β1 xi1 + β2xi2 + … + βkxik 1-1

where i denotes individual, pi represents the probability of the event occurring, 1- pi

represents the probability of the event not occurring, the ratio of the two represents the odds

of the event, and the left-hand side expression represents the log-odds, or the logit. On the

right-hand side of the equation, α represents the intercept, β represents the regression

coefficient, and x represents the independent variable (for more detailed background of

logistic regression, see also Kleinbaum et al. 1998; McCulloch and Searle 2001; Menard

2010; Pedhazur 1997; Rencher 2000; Tabachnick and Fidell 2001).

As can be seen from the right-hand side of equation 1-1, the specification of the

logistic regression model is very similar to that of the linear regression model in terms of

independent variables. Like linear regression, logistic regression can handle both continuous

and categorical independent variables. Major statistical softwares include easy-to-use

procedures for logistic regression models. Although it is relatively straightforward to specify

 
 4

and estimate logistic regression models, the interpretation of the results is more complicated

and less intuitive compared to linear regression. This is due to the fact that in the logistic

regression model, the relationship between the probabilities and the set of independent

variables is not linear; instead, it is the relationship between the logit and the set of

independent variables that is assumed to be linear. As a result, the estimates of the

coefficients represent the change in the log of odds corresponding to a certain amount of

change in the independent variable. Acknowledging the difficulty of interpreting the log of

odds, the common practice is to exponentiate the estimates of the coefficients to obtain the

odds ratio. This step eliminates the complication of interpreting the log, and the odds ratio

becomes the standard effect size output by statistical softwares. Nevertheless, the concept and

proper interpretation of the odds ratio still leave many researchers confused. A browse of the

current educational research literature shows that different approaches to interpreting logistic

regression results have been adopted. For example, the odds ratio is interpreted as a relative

risk in some studies (Chiang et al. 2012; Sullivan and Cosden 2015) but not in others (Jaeger

and Eagan 2011). Some studies do not interpret the odds ratio at all but report and interpret

marginal effects and predicted probabilities instead (King 2015; Ko and Jun 2015). Which

approach is most commonly adopted in educational research? Is the most common approach

the best? What are the implications of taking various approaches?

The aim of this study is to review current literature in educational research where

logistic regression is used to examine the application of logistic regression, focusing on the

presentation and interpretation of logistic regression results. The goal is to provide

educational researchers and research literature readers with a better understanding of how

logistic regression results can be reported and accurately interpreted to better answer

educational research questions. Specifically, this study examines the following aspects of the

application of logistic regression: (a) the statistics being reported as logistic regression

 
 5

results, and (b) the interpretation of statistical significance level, direction, and magnitude of

the association between predictor and event probability. The goals of this study are to reveal

any common issues in using logistic regression to explain educational phenomena, to

compare different approaches of interpreting logistic regression results, and to provide

suggestions of preferred approaches to enhance the quality of the application of this statistical

method.

Review Method

The first step of the study was to search for relevant literature, namely empirical

educational research studies in which logistic regression was used. To obtain a sample size

manageable for a review study, the search was limited to studies that were written in English,

published in peer-reviewed journals, and published from 2010 to 2016. The online databases

searched included Education Source by EBSCO, ERIC, PsycINFO, and Web of Science. The

following search terms were used to search anywhere in the document: logistic regression,

education. The search was limited to studies published in peer-reviewed journals to include

research that had been evaluated through rigorous peer review processes. The search yielded

143 articles. These documents then were reviewed, and studies that did not include empirical

analysis using logistic regression model were excluded. This process resulted in a list of 130

empirical research articles. The articles were then coded in terms of the statistics reported and

the interpretation (or the lack thereof) of such statistics. The coding was based on contents

anywhere in the articles, including tables, figures, texts, and appendices. If the detected

statistically significant relationship between the predictor and the event occurrence was

interpreted in terms of probabilities, then an article was coded as having interpreted the

magnitude of association. If odds ratio was explicitly interpreted as relative risk, namely if

the definition of relative risk was used to explain the numeric value of reported odds ratio,

then an article was coded as having treated odds ratio as relative risk. Given that this is a

 
 6

single-authored study, no additional coder was involved to check the coding of articles.

Instead, the author used Table B1 in Appendix B to double-check the coding and to present a

summary of the reviewed studies. Given the straightforward binary (yes/no) nature of all

categories of the coding, the author believes that the lack of a second coder should not impact

the accuracy of the coding.

Because the inclusion criterion was the application of logistic regression, the included

studies demonstrated a wide range of research topics. The most common outcome variables

concerned students’ choice among two or more options, such as student enrollment pattern

(Campbell and Mislevy 2013), students’ intention of urban/rural placement (Jones, Bushnell,

and Humphreys 2014), college major choice (Ferguson et al. 2012; Pinxten et al. 2015),

participation in student organizations (Case 2011), pursuit of financial aid (Radey and

Cheatham 2013), and decision to pursue graduate education (d’Aguiar and Harrison 2016).

Another common type of dependent variable is student outcome such as degree attainment

(Flynn 2014), academic success (Fong, Melguizo, and Prather 2015; Stegers-Jager et al.

2012), and retention (Evans 2013; O’Neill et al. 2011; Pruett and Absher, 2015; Santelices et

al. 2016). Students’ health and social behaviors also were among the examined outcomes

(Taliaferro and Muehlenkamp 2015; Titzmann, Brenick, and Silbereisen 2015).

The manuscripts then were analyzed to determine the type of statistics being reported,

as well as the accuracy and extent of the interpretation of the significance, direction, and

magnitude of the association between predictors and event probability. The following section

reports the results of the analysis, summarizes the frequency and type of statistics being

reported as logistic regression results, determines the suitability of reported effect sizes and

the accuracy of their interpretation, and analyzes the scope of information revealed by the

reported effect sizes and the interpretation. A discussion section then follows to compare

 
 7

approaches of interpreting logistic regression results, to identify common issues and their

implications, and to make suggestions of how to overcome such issues.

Results

This section examines the application of logistic regression in two aspects. The first

examines the statistics being reported as logistic regression results. The second examines how

significance level, direction, and magnitude of the association between predictor and event

probability are interpreted.

Statistics Being Reported as Logistic Regression Results

Table 1 presents the frequency of relevant statistics reported as logistic regression

results in the reviewed studies. The majority of studies (81.5%) reported the odds ratio (OR)

as effect size. The confidence interval of either the coefficient estimate or the odds ratio was

reported by less than half (43.1%) of the studies, while the p value of the significance test

was reported by most studies (86.9%). This suggests that p values are more commonly used

than confidence intervals as indication of the significance level of independent variables.

Given the fact that odds ratio, confidence interval, and p value are among the standard output

of major statistical packages such as STATA and SAS, it is unsurprising to see that these

statistics frequently are reported as logistic regression results.

[Table 1 near here]

Compared to the odds ratio, alternative measures of relationship magnitude were

much less frequently reported. As can be seen from Table 1, marginal effects and predicted

probabilities each were reported by 5.4% of the reviewed studies. This might be because

researchers do not find it necessary to report and interpret effect sizes other than odds ratio. It

also is plausible that the additional steps such as code writing and specification of

independent variable values required to obtain these statistics affect their popularity.

The Interpretation of Significance and Direction of Association

 
 8

Before reviewing the included studies’ interpretation of model results, a brief

overview of the basics of logistic regression is necessary. The goal of using logistic

regression model is to understand the correlational relationship between independent

variables (predictors) and the likelihood of the event. As indicated in equation 1-1, the left-

hand side of logistic regression model is the log of odds, and coefficient βk is the change in

the log of odds for one-unit change in xk, holding all other independent variables constant. To

eliminate the complication of interpreting βk in terms of changes in log odds, the common

practice is to exponentiate both sides of the equation to obtain the odds ratio, namely the ratio

of the odds of an event for two individuals differing by one-unit on x. The odds ratio can be

expressed as follows:

/
Odds ratio = Exp(β) = 2-1
/

where p represents the probability of the event occurring, x represents independent variable,

and x+1 represents one-unit change in independent variable. It can be seen that although the

odds ratio is related to event probability and it measures a ratio between two individuals

differing by one-unit on the independent variable, it does not measure the ratio of

probabilities. Instead, it measures the ratio of odds. The ratio of probabilities, also called

relative risk or risk ratio, measures the ratio of the probability of an event occurring in one

group (exposed to certain treatment/condition) to the probability of the same event occurring

in another group (unexposed to such treatment/condition). The difference between the two

ratios is easier to understand by comparing the formulae. Relative risk is expressed as

follows:

Relative risk = 2-2

where p represents the probability of the event occurring, x represents independent variable,

and x+1 represents one-unit change in independent variable.

 
 9

Comparing equations 2-1 and 2-2, the difference between the odds ratio and the

relative risk is obvious. This can be demonstrated by a simple example with hypothetical

event probabilities. Let the hypothetical probabilities of obtaining college degree be 0.6 and

0.8 for men (reference group) and women (comparison group), respectively. The relative risk

of women compared to men therefore is 0.8/0.6 = 1.33. The odds for women are 0.8/(1-0.8) =

4, and the odds for men are 0.6/(1-0.6) = 1.5. The odds ratio of women compared to men

equals (0.8/(1-0.8))/(0.6/(1-0.6)) = 4/1.5 = 2.66. It can be seen that for the hypothetical

probabilities, the odds ratio between the two genders is much larger than the gender relative

risk.

The interpretations of the odds ratio and the relative risk also are different. Relative

risk is easier and more intuitive to understand. Using the previous example, the probability of

women receiving a college degree is 1.33 times as large as the probability of men, or 33%

higher. We can say that women are 33% more likely to receive a college degree compared to

men. It is less easy to interpret the odds ratio. An odds ratio of 2.66 does not mean that

women are 2.66 times as likely as men, or 1.66 times more likely than men, to receive a

college degree. In other words, the ratio of odds does not translate into the ratio of probability

automatically. Therefore, it is important that the odds ratio is not mistaken for the ratio of

probabilities (namely relative risk). It also is important that the odds ratio not be interpreted

as relative risk in wordings such as “n percent more likely to” or “n times as likely to.” Doing

so would inflate the difference of probabilities between the comparison and reference groups

and therefore, be misleading.

If the odds ratio should not be interpreted as relative risk, how should it be

interpreted? Using the same example, we can say that the odds of women receiving a college

degree are 2.66 times as large as the odds for men. But what exactly does this statement

reveal? In fact, not much can be inferred directly from this statement except that women are

 
 10

more likely than men to receive a college degree (2.66> 1). It is, however, not clear how

much more likely women are to receive a college degree than men because the odds ratio

does not correspond to one probability ratio but rather to a series of probability ratios. Let the

probability of obtaining a college degree for men and women be 0.43 and 0.67, respectively.

It can be calculated that the odds ratio of women to men is still 2.66, but the relative risk is

1.56 instead of 1.33 as calculated before with the other pair of probabilities (0.6 and 0.8). The

odds ratio reveals the direction of the relationship but not the magnitude in terms of

probabilities. Knowing the value of the odds ratio does not indicate how much more or less

likely the event is to occur for two individuals differing by one-unit on the independent

variable.

As discussed above, p value of significance test was reported by most studies, and less

than half of the studies reported the confidence interval of the coefficient estimate or the odds

ratio. Both statistics were used to interpret whether independent variable was statistically

significant in predicting the event likelihood. Also, the direction of the association could be

easily determined by examining the value of the effect size statistic. Table 2 presents the

frequency and percentage of studies that explicitly interpreted the significance of independent

variables and direction of the association.

[Table 2 near here]

It can be seen that the majority of the studies explicitly interpreted the significance of

predictors and the direction of the associations. Typically, these two interpretations are

combined. For example, Mendez and Mendez (2016, 11) made the following statement in

their study on college students’ perception of faculty and physical appearance: “As we

hypothesized, perceived age statistically significantly predicted attractiveness, knowledge and

approachability. First, younger faculty members were perceived to be more attractive than

older faculty, confirming Hypothesis 1. This result is statistically significant (p = 0.00)”.

 
 11

Similarly, Pinxten et al. (2015, 1930) reported the association between students’ course-

taking background and college major choice as follows: “students with a

mathematical/science course-taking background were significantly more inclined to opt for

an engineering, economics, (para)medical or science major over a major in literature, history,

or arts”. These interpretations provide complete information on whether a correlational

relationship exists, and whether significant independent variables positively or negatively

predict event probability.

There are a few studies where either significance or direction was not sufficiently

interpreted. For example, in a study that examined students’ satisfaction of a training

program, Aine et al. (2014, 197) reported the association between students’ evaluation of

learning and overall satisfaction as follows: “In the GP group, good evaluation of diagnostic

skills learning during GP training was predictive of overall training satisfaction”. Although

this interpretation is correct, it is not complete and does not explain whether good evaluation

of diagnostic skills positively or negatively predicts student satisfaction. Similarly, in a study

on college students’ sexual activities, Burke, Gabhainn, and Young (2014, 41) reported the

results regarding the predictor “student status,” stating: “Univariate regression models

identified student status to be a significant predictor of having four or more sexual partners in

their life”. This statement combined with the omission of predictor level specification in the

text or in the tables makes it unclear whether being a student positively or negatively

predicted certain sexual activities. These incomplete and rather vague interpretations decrease

the amount of information made available to the readers; impact the readability of the studies;

and make it difficult to understand, assess, and adopt the specific findings. Table 2 shows that

about 15% of the reviewed studies failed to provide interpretation of the association direction,

and about 9% did not discuss the significance of predictors in text. Considering that predictor

significance and direction of the association are the primary results of logistic regression and

 
 12

should be discussed sufficiently in text, these percentages suggest that there is room for

improvement in educational research in this regard.

The Interpretation of Magnitude of Association

Once individual predictor significance and the direction of association are determined,

the crucial question to be answered is: how strong is the detected relationship? Using the

habit formed by interpreting linear regression results, one would want to know how much the

event probability changes for a certain amount of change in the predictor. However, unlike

linear regression, where coefficient estimate measures the change in the dependent variable

for certain change in the independent variable, logistic regression does not yield such a

measure. Logistic regression does not model the linear relationship between the predictors

and the event probability. As previously discussed, the odds ratio, which is a transformation

of the logistic regression coefficient estimate, is not a measure in terms of probability. This

does not mean, however, that the magnitude of the predictor cannot be properly interpreted to

help us better understand educational phenomena. The following sections provide a detailed

review of the included studies to examine if the magnitude of the significant predictors was

interpreted, what methods were used, and if the methods were appropriate.

Quantification of Association Magnitude

Fifty-two studies, or 40% of all included studies, attempted to interpret the magnitude

of statistically significant relationships in terms of probabilities (Table 2). This percentage is

much lower than those that included interpretation of predictor significance and direction of

association. The fact that 60% of the studies didn’t interpret the strength of the detected

relationships in terms of probabilities may be because of various reasons. Researchers may

find it sufficient to reveal the direction of the relationship and unnecessary to quantify the

strength of the relationship. It also is plausible that some researchers may be uncertain about

how to interpret the magnitude in probabilities, and choose to only interpret the direction to

 
 13

avoid errors. A typical example can be found in a study on the relationship between new

degree structure and medical students’ career considerations (van den Broek et al. 2010), in

which it was concluded that:

In the third year of medical school, pre-BaMa students were significantly more likely

to consider a temporary stop than BaMa students. Apparently, those who cannot

receive a bachelor degree think more about the opportunity to interrupt the course

than those who can receive one. (1000)

Although this conclusion reveals that the new degree structure is related to lower preference

for a temporary stop during the program, it is unknown how large the difference in preference

is before and after the introduction of the new degree structure. Not only was the difference in

probabilities missing, even the meaning of odds ratio values was not interpreted.

Odds Ratio

As discussed previously, the odds ratio is easily available because it is part of the

standard output for logistic regression by major statistical software packages. Not

surprisingly, it is commonly used as a measure of magnitude of detected relationship (Pruett

and Absher 2015; Radey and Cheatham 2013; Swecker, Fifolt, and Searby 2013; Wood

2013). This statistic frequently is interpreted in text similar to the following: “The odds that

students with visual impairments received accommodations and other disability-specific

services at 2-year colleges were more than 7 times those for students with learning disabilities

(OR = 7.1, p < .001)” (Newman and Madaus 2015, 213). This interpretation is not incorrect;

it explains that the odds of receiving accommodations for students with visual impairments

are seven times as large as the odds for students with learning disabilities. But what exactly

does that mean? This interpretation still does not eliminate the confusion caused by the

concept of odds, and therefore is not easy to comprehend. Explaining the odds ratio using this

approach only is potentially problematic, because the interpretation could be easily

 
 14

misunderstood as a comparison of event probabilities by readers confused by the concepts of

odds and probability.

Some studies show such confusion on the researcher’s part. Besides its ready

availability, odds ratio also appears to be appealing as a measure of relationship magnitude

because it measures a ratio between the comparison and the reference group corresponding to

a certain amount of change in the predictor. It is rather intuitive to interpret this statistic as a

ratio of event probabilities. This is shown explicitly by a study on the relationship between

higher education and students’ entrepreneurial intentions, where it is concluded that “The

odds ratio of 1.573 indicates that senior students have a probability of stating an

entrepreneurial intention 1.5 times higher than the freshmen students” (Ertuna and Gurel

2011, 395). Clearly, odds ratio is incorrectly interpreted as ratio of probabilities. A review of

the included studies shows that 36 studies interpreted the value of odds ratio as relative risk

(ratio of probabilities). That is 69.2% of the 52 studies that attempted to interpret relationship

strength in terms of probabilities (see Table 2, row 3). For example, in a study on the

relationship between lecture quiz score and final exam performance, Wambuguh and Yonn-

Brown (2013) discussed the findings as follows:

Students attaining combined quizzes scores of at least 70% are approximately 9 times

(OR value of 8.78 in Table 2) as likely to pass the final examination with the same

score or better compared to those who got scores below 70% (χ2= 160.53, p<.001) .

(3)

In a study on the relationship between usage of student services and college students’

experiences of difficulty and distress, Julal (2013, 419, 421) explained the relationship

between experiencing personal difficulty and using student services as follows: “The odds

ratio of 34.34 showed that students who were faced with personal difficulty were more likely

to use one or more of the available support services. ... [They] were 34 times more likely to

 
 15

use one or more support services”. Similarly, Baker-Eveleth, O’Neill, and Sisodiya (2014, 66)

described the association between students’ performance in predictor and subsequent courses

as follows: “as students perform worse in the [predictor] course [Business Law] they are

nearly 10 times more likely to score a D or F in the module rather than an A –

Exp(B)=9.807”.

Similar interpretation of odds ratio is abundant (Beran et al. 2012; Lee 2014;

Lombardi et al. 2013; Luna and Fowler 2011; Lyons and Akroyd 2014; Munisamy, Jaafar,

and Nagaraj 2014). As discussed previously, the odds ratio tends to be larger than relative

risk. Interpreting odds ratio as relative risk could lead to an exaggeration of the predictor’s

relationship with event probability. Table 3 presents overestimation of relative risks

calculated based on the odds ratio reported by Julal’s (2013) study and hypothetical event

probabilities. Using equations 2-1 and 2-2, it can be demonstrated that relative risk can be

expressed in odds ratio and event probability using the following formulae:

Relative risk = 2-3
∗

Relative risk = Odds ratio – (Odds ratio – 1) * p(x+1) 2-4

where p(x) represents the event probability for the reference group, and p(x+1) represents the

event probability for the comparison group. Hypothetical probabilities of using student

services for students without personal difficulty (reference group) are shown in the second

column of Table 3. Using equation 2-1 and the odds ratio reported by Julal (2013) comparing

students with and without personal difficulty, hypothetical probabilities for students with

difficulty are calculated and shown in the first column of Table 3. Using equation 2-3 (or

equation 2-2), relative risks are calculated and shown in the third column of Table 3.

[Table 3 near here]

Comparison of the calculated relative risks and the reported odds ratio shows that the

latter overestimates the former extensively. If students without personal difficulty rarely use

 
 16

student services as suggested by the hypothetical probability of 0.05, the odds ratio would

overestimate relative risk by a factor of 2.67. With the increase of the event occurrence for

the comparison group, the overestimation becomes stronger. For example, if 40% of students

without difficulty use student services, interpreting the odds ratio as relative risk would

exaggerate the relationship magnitude by a factor of 14.34. The exact extent of the

exaggeration is unknown because the exact event probabilities are unavailable, but it is

obvious that the study’s (Julal 2013) conclusion is not valid. The calculated relative risks are

still large and indicate strong predictive relationship between personal difficulty and usage of

student services, but they are not nearly as dramatic as a factor of 34 as suggested by Julal’s

(2013) conclusion.

Table 4 lists examples of studies that made similar exaggerating conclusions by

interpreting odds ratio as relative risk. Hypothetical probabilities are approximated based on

descriptive statistics reported in the studies. For example, the overall completion rate of the

Free Application for Federal Student Aid (FAFSA) among single mother students is 87%

(Radey and Cheatham 2013). Based on this percentage, when the probability of completing

FAFSA among single mother students at public universities (reference group) is set at 0.8, the

odds ratio would overestimate relative risk by a factor of 8. Single mother students at for-

profit institutions (comparison group) would be estimated to be 22% more likely (or 122% as

likely) than their counterparts at public universities to complete FAFSA instead of nearly 10

times more likely to do so as claimed by Radey and Cheatham (2013). Looking at the

claimed difference of nearly 10 times, readers would conclude that students at for-profit

institutions are dramatically more inclined to apply for financial aid, while the actual

difference (here calculated based on approximated probabilities) is much less extreme.

Similarly, college students with a history of high risk factors are estimated to be about 41%

more likely to engage in heavy drinking and about 93% more likely to experience blackouts

 
 17

compared to students with a history of low risk factors, not 6.04 times and 2.51 times more

likely to do so, respectively; and high risk factor college students are estimated to be about

15% more likely to drink heavily compared to moderate risk factor students, not 2.85 times

more likely to do so as suggested by Sullivan and Cosden’s (2015) conclusion. It can be seen

that interpreting odds ratio as relative risk has led to inaccurate understanding of educational

phenomena, and the extent of the inaccuracy is too large to be ignored.

[Table 4 near here]

Marginal Effects and Predicted Probabilities

Because the odds ratio does not reveal relationship strength in terms of probability,

alternative measures are needed to quantitatively interpret logistic regression results. Agresti

(2013) acknowledged the fact that it may be difficult to understand odds or odds ratio effects,

and suggested the use of instantaneous rate of change in the probability or estimated

probabilities at selected values (such as sample means or quartiles) of the independent

variables to address such difficulty. The current review shows that marginal effects and

predicted probabilities have been used to provide probability-based interpretations of logistic

regression results. Marginal effects are partial derivative or instantaneous rates of change for

continuous independent variables and discrete changes for categorical independent variables.

Marginal effects measure the change in predicted probabilities corresponding to a certain

change in the independent variable of interest, holding all other independent variables

constant. The interpretation of marginal effects for categorical and continuous variables is

different. For categorical variables, marginal effects measure the change in predicted

probability as the independent variable of interest changes from 0 to 1 while holding all other

independent variables constant. For continuous variables, marginal effects provide good

approximation to the amount of change in predicted probability for a one-unit change in the

independent variable while holding all other independent variables constant. The difference

 
 18

lies in that marginal effects for continuous independent variables are approximation because

the relationship between the independent variable and predicted probability is not linear

(Williams 2016). Predicted probabilities are probabilities calculated using logistic regression

coefficient estimates and selected values of independent variables.

Both marginal effects and predicted probabilities are dependent on the values of

independent variables, which are selected by researchers based on substantive considerations

or following common practices. For example, to avoid the impact of outliers, the upper and

lower quartiles could be used instead of the smallest and largest values of the independent

variable when calculating estimated probabilities (Agresti 2013). It is important that the

selection of independent variable values is described and if possible, justified. It also is

important that the implications of the selected values of independent variables be reflected in

the interpretation of these two measures, as demonstrated in the examples provided by

Agresti (2013). Specifically, marginal effects should not be interpreted as constant effects and

should always be interpreted in relation to the selected values of independent variables that

are held constant. The same applies to the calculated values of predicted probabilities.

Table 1 shows that seven studies claimed to report marginal effects to interpret the

magnitude of relationship strength. Four of these studies used this statistic correctly. Marginal

effects at the means were reported by three studies (Fernández-Macías et al. 2013; Jaeger and

Eagan 2011; Pursel et al. 2016), where the independent variables other than the one of

interest were held constant at their means. Average marginal effect was reported by one study

(Bielby et al. 2014), where the marginal effect of the independent variable of interest is

calculated for each case holding other independent variables at their respective values and the

average of all individual marginal effects is calculated.

Two of the seven studies did not report complete information in relation to marginal

effects. Ko and Jun (2015) used a figure to report marginal effects of the independent variable

 
 19

“chance to benefit society” on college students’ preference for public sector jobs. The values

of the calculated marginal effects were not reported in the figure or in text, and the

specification of independent variables’ values was not reported. As a result, the readers still

were not provided with quantitative measures of the relationship strength. It also is difficult

to adopt the finding because the values of the other independent variables are unknown to the

readers. Melguizo, Torres, and Jaime’s (2011) study also suffered from this issue by omitting

the specification of independent variables’ values. One of the seven studies used the term

marginal effect incorrectly. Santelices et al. (2016) interpreted coefficient estimates as

predicted probabilities and incorrectly calculated the marginal effect.

Table 1 also shows that seven studies reported predicted probabilities to quantify the

relationship strength. Similar to Ko and Jun’s study (2015), four of the seven studies did not

report the exact values of predicted probabilities, but instead used figures to visually show

differences in predicted probabilities (Flynn 2014; Jowett et al. 2014; King 2015; Pleitz et al.

2015). One study omitted the calculation method of predicted probabilities and specification

of independent variables’ value (Pleitz et al. 2015).

It can be seen that the reporting of marginal effects and predicted probabilities in the

quantitative interpretation of relationship magnitude suffers not only from low quantity, but

also from quality issues. The omission of specification of independent variables’ values is a

common issue, leading to interpretation of predictive relationship magnitude without clearly

stated conditions. Another common issue is the omission of exact values of marginal effects

or predicted probabilities, providing the readers with only vague information about the

relationship strength. Even for the very few studies that reported the selected values of

independent variables, the interpretation of the marginal effects tends to lack emphasis on the

conditional nature of this statistic. For example, Jaeger and Eagan (2011) interpreted

marginal effects at the means (referred to as ME) as follows:

 
 20

Black students (ME = 0.04, p < .001) were significantly more likely to be retained

than their White counterparts. In-state students had a significantly higher probability

of retention compared to students who resided outside the state of the institution (ME

= 0.08, p < .001). (522)

The interpretation does not make explicit that the marginal effects, or the changes in

predicted probabilities, hold only for otherwise average individuals. Marginal effects and

predicted probabilities are dependent on the selected values of independent variables. For

individuals with values much higher or lower than the average on independent variables, the

marginal effect of 0.04 for being Black and the marginal effect of 0.08 for being an in-state

student might not hold, and the conclusion might be affected. Emphasizing the conditional

nature of marginal effects and predicted probabilities could make the interpretation and

conclusion more accurate and enable readers to properly adopt the findings.

Discussion

The review of 130 empirical educational studies applying logistic regression reveals

three major issues that warrant attention: (a) a common lack of quantitative interpretation of

the magnitude of detected predictive relationship (60% of the studies did not attempt to

provide such in terms of probabilities); (b) common practice of interpreting odds ratio as

relative risk (69.2% of the studies that attempted to interpret strength of predictive

relationship in terms of probabilities explained odds ratio value as difference in probability);

and (c) omission of specification of independent variables’ values in the calculation and

interpretation of marginal effects and predicted probabilities.

Lack of Quantitative Interpretation of Relationship Magnitude in Terms of Probabilities

The current review indicates that most educational studies using logistic regression

report significance and sign of predictors, but do not quantitatively interpret the predictive

relationship strength in terms of probabilities. This issue can be attributed in part to the

 
 21

confusion caused by the concept of the odds ratio, which is the standard output by major

statistical software packages used to estimate logistic regression models. In the reviewed

studies, the odds ratio is the most commonly reported effect size, and the interpretation of its

meaning falls into three categories: no interpretation, interpreted as change in odds, and

interpreted as relative risk. It is not incorrect not to interpret odds ratio or to interpret it as the

change in odds corresponding to a certain change in the predictor, but the amount of

information revealed by these options is low. For some studies, the goal is to detect

significant predictor(s) and the direction of the relationship. The magnitude of the association

may not be of interest to the researcher. Although in such case it may not be necessary to

report quantitative measures such as marginal effects, doing so would better utilize the

readily available results to reveal deeper understanding of educational phenomena. Reporting

these statistics also could prevent the overestimation of the relationship strength. It is

suggested that researchers go beyond the value of the odds ratio and report marginal effects

or predicted probabilities. Technically, this could be done easily with the simple commands

used by major statistical software packages to calculate such statistics. For example, marginal

effects and predicted probabilities can be easily computed using the margins command in

STATA.

Interpreting Odds Ratio as Relative Risk

Interpreting odds ratio as relative risk causes more serious concern than the two

previous options because doing so leads to an overestimation of the predictive relationship as

demonstrated in Tables 3 and 4. Almost 70% of the studies that attempted to interpret the

results in terms of probabilities did so by interpreting the odds ratio as a ratio of probabilities;

this indicates a noteworthy weakness on educational researchers’ conceptual understanding of

logistic regression. Exaggerating relationship magnitude not only undermines the validity and

rigor of research studies, but it also could mislead educators, education administrators, and

 
 22

policy-makers. For example, by reading the finding of Radey and Cheatham (2013, 270) that

aid-eligible women at for-profit institutions are almost “10 times more likely” to complete an

application for financial aid than those at public institutions, a policy maker might draw the

conclusion that new policy and/or funding is needed to address this large gap and to urge

public institutions to make more efforts to assist students to seek financial aid. In fact, this

gap is overestimated extensively, and the actual gap could be in the range of about 22%

(Table 4). Empirical studies inform practice, research, and policy making of education. The

threat to well-informed decision making posed by overestimating predictive relationship

magnitude should not be overlooked. The relatively common presence of this issue in

educational research reviewed here reflects a conceptual weakness of both empirical

researchers and peer reviewers and warrants attention. Given the similarity of model

specification between linear and logistic regression, it is not surprising that researchers tend

to interpret logistic regression results in the same way they interpret linear regression results.

Educational researchers should carefully distinguish between the two types of regression

models, update conceptual understanding of odds ratio and its limitations, and use more

caution when interpreting predictive relationship strength detected by logistic regression

models. Also, peer reviewers should pay more attention to the statistics used as effect size

and how they are interpreted.

Omission of Specification of Independent Variables’ Values

Marginal effects and predicted probabilities both are dependent on the values of

independent variables other than the one of interest. This means that selecting different sets

of values for the other independent variables in the model would yield different marginal

effects and predicted probabilities. As a result, the calculated values of these statistics are

conditional. This review suggests that it is relatively common that the selection of

independent variables’ values is not reported and/or not incorporated in the interpretation of

 
 23

the calculated statistics. Ignoring the conditional nature of the statistics could make it difficult

for readers to understand the educational phenomena of interest, or they even could mistake

the conditional values as constant effects. It is suggested that educational researchers be more

cautious when reporting marginal effects and predicted probabilities as logistic regression

results, and to always discuss their conditions. Table 5 summarizes the approaches of

interpreting magnitude of predictive relationship revealed by this review.

[Table 5 near here]

Recommendations

In summary, this review indicates that there is much room for improvement in the

interpretation of logistic regression results in educational research. Researchers and peer

reviewers could consider the following recommendations when using logistic regression

models.

 Fully utilize the information available from logistic regression results

First, researchers should consider interpreting the magnitude of detected association

between predictors and event probabilities to fully utilize the information available from

logistic regression results. It is suggested that researchers calculate marginal effects and/or

predicted probabilities using methods discussed above in the section Marginal Effects and

Predicted Probabilities. Marginal effects and predicted probabilities could be easily computed

by major statistical software packages. These statistics provide a good approximation of the

change in event probability corresponding to a certain change in individual predictors, and

they provide much flexibility for researchers to compare probabilities corresponding to

predictor values of substantive interest. Currently, many articles only report and interpret

odds ratio and discuss the direction of the association. Doing so does not reveal changes in

event probability corresponding to certain change in predictors, and does not shed light on the

relative importance of the predictors. Even when relationship magnitude is not of primary

 
 24

interest to the researchers, the readers, especially educational policy makers and practitioners,

oftentimes need such information in order to prioritize and allocate scarce resources.

Computing and reporting statistics in terms of probability could provide readers with a deeper

understanding of the educational phenomena, and be more helpful in informing research,

practice, and policy making.

 Enhance conceptual understanding of logistic regression, especially of the

definition of the odds ratio

Second, conceptual understanding of logistic regression should be enhanced

especially to clarify the definition of the odds ratio. One common error to avoid is that the

odds ratio should not be interpreted as relative risk or a ratio of probabilities. Although it is

very tempting to do so, treating the odds ratio as relative risk leads to exaggeration of

prediction magnitude and misleading conclusions about educational phenomena. Instead,

researchers could compute statistics in terms of probability to quantitatively measure the

magnitude of the predictive relationship. Researchers should also explicitly remind readers

not to confuse odds ratio with relative risk.

 Avoid confusing the interpretation of logistic regression with that of linear

regression

Third, more attention should be paid to ensure that the interpretation of logistic

regression results is not confused with that of linear regression models. Specifically,

calculating marginal effects and/or predicted probabilities would require careful selection of

values of independent variables, such specification should always be reported, and the

conditional nature of these statistics should be emphasized in their interpretation and in the

overall conclusions.

Technical complexity may have been a factor that has kept researchers from moving

beyond odds ratio and towards calculated probabilities. However, advanced statistical

 
 25

software packages have made such calculating relatively simple. The main challenge faced

by researchers seems to lie in the conceptual understanding of the method of logistic

regression. Given its popularity in educational research, it is imperative that researchers

revisit the key concepts of this method. Some might argue that the extra burden of calculating

and reporting probability-based statistics is not worthwhile because oftentimes knowing the

significance and direction of the association is sufficient, and even if not, that odds ratio

approximates relative risk. This review, however, suggests that odds ratio could be very

misleading as a substitute for relative risk. This review serves as a starting point for

educational researchers interested in using logistic regression to revisit previous studies,

detect any misleading conclusions drawn, and more importantly, improve accuracy in their

own research. It would be helpful for future research to explore whether the issues identified

in this review, such as effect size exaggeration, have had any noticeable impact on

educational policy making or practices. It would also be interesting to compare the

application of logistic regression in educational research and other research fields.

Conclusion

The results of this review reveal several issues in the application of logistic regression

models in current educational research publications that warrant attention among researchers.

The primary concern is the weakness in conceptual understanding of the method, its

difference from linear regression model, and the meaning and limitation of the odds ratio.

Although logistic regression is a powerful tool in understanding correlational relationships,

this review indicates that some of the results being reported by educational research based on

logistic regression results might not be accurate and should be interpreted with caution by

readers. It is strongly suggested that educational researchers, including empirical researchers

and peer reviewers, properly address the issues discussed in this review to enhance the

quality of logistic regression studies. While this review has focused on the reporting and

 
 26

interpretation of logistic regression results, it should be noticed that successful use of logistic

regression also relies on aspects such as model selection, suitability of model for data

structure, model specification, model testing, and measurement of model fit. Due to the

limitation of the scope of this review, these aspects have not been examined or discussed.

Future review studies on these topics could further shed light on the adoption of logistic

regression in educational research.

References

* References marked with an asterisk indicate studies included in the review.

Agresti, A. 2013. Categorical Data Analysis. Hoboken: John Wiley & Sons.

* Aine, T., M. Sumanen, T. Heikkila, H. Hyppola, H. Halila, J. Vanska, S. Kujala, I. O. Virjo,

and K. Mattila. 2014. “Factors Associated with General Practice Specialty Training

Satisfaction - Results from the Finnish Physician Study.” Education for Primary Care

25: 194-201.

Allison, P. D. 1999. Logistic Regression Using SAS: Theory and Application. Cary, NC: SAS

Institute Inc.

* Baker-Eveleth, L. J., M. O’Neill, and S. R. Sisodiya. 2014. “The Role of Predictor Courses

and Teams on Individual Student Success.” Journal of Education for Business 89 (2):

59-70. doi:10.1080/08832323.2012.757541

* Beran, T. N., C. Rinaldi, D. S. Bickham, and M. Rich. 2012. “Evidence for the Need to

Support Adolescents Dealing with Harassment and Cyber-Harassment: Prevalence,

Progression, and Impact.” School Psychology International 33 (5): 562-576.

doi:10.1177/0143034312446976

 
 27

* Bielby, R., J. R. Posselt, O. Jaquette, and M. N. Bastedo. 2014. “Why are Women

Underrepresented in Elite Colleges and Universities? A Non-Linear Decomposition

Analysis.” Research in Higher Education 55: 735-760.

* Burke, L., S. N. Gabhainn, and H. Young. 2015. “Student Sex: More or Less Risky than

Other Young Adults?” Sex Education 15 (1): 31-47.

doi:10.1080/14681811.2014.947362.

* Campbell, C. M., and J. Mislevy. 2013. “Student Perceptions Matter: Early Signs of

Undergraduate Student Retention/Attrition.” Journal of College Student Retention:

Research, Theory & Practice 14 (4): 467-493. doi: 10.2190/CS.14.4.c.

* Case, K. F. 2011. “A Gendered Perspective on Student Involvement in Collegiate Clubs

and Organizations in Christian Higher Education.” Christian Higher Education 10:

166-195. doi: 10.1080/15363759.2011.576208.

* Chiang, H. M., Y. K. Cheung, L. Hickson, R. Xiang, and L. Y. Tsai. 2012. “Predictive

Factors of Participation in Postsecondary Education for High School Leavers with

Autism.” Journal of Autism and Developmental Disorders 42 (5): 685-696. doi:

10.1007/s10803-011-1297-7.

* d’Aguiar, S., and N. Harrison. 2016. “Returning from Earning: UK Graduates Returning to

Postgraduate Study, with Particular Respect to STEM Subjects, Gender and

Ethnicity.” Journal of Education and Work 29 (5): 584-613. doi:

10.1080/13639080.2014.1001332.

* Ertuna, Z. I., and E. Gurel. 2011. “The Moderating Role of Higher Education on

Entrepreneurship.” Education + Training 53 (5): 387-402.

doi:10.1108/00400911111147703

 
 28

* Evans, D. B. 2013. “Examining the Influence of Noncognitive Variables on the Intention of

Minority Baccalaureate Nursing Students to Complete Their Program of Study.”

Journal of Professional Nursing 29 (3): 148-154. doi:10.1016/j.profnurs.2012.04.016.

* Ferguson, E., D. James, J. Yates, and C. Lawrence. 2012. “Predicting who Applies to Study

Medicine: Implication for Diversity in UK Medical Schools.” Medical Teacher 34

(5): 382-391. doi: 10.3109/0142159X.2012.652237.

* Fernández-Macías, E., J. Antón, F. Braña, and R. M. De Bustillo. 2013. “Early School-

Leaving in Spain: Evolution, Intensity and Determinants.” European Journal of

Education 48 (1): 150–164.

* Flynn, D. 2014. “Baccalaureate Attainment of College Students at 4-year Institutions as a

Function of Student Engagement Behaviors: Social and Academic Student

Engagement Behaviors Matter.” Research in Higher Education 55 (5): 467-493. doi:

10.1007/s11162-013-9321-8.

* Fong, K. E., T. Melguizo, and G. Prather. 2015. “Increasing Success Rates in

Developmental Math: The Complementary Role of Individual and Institutional

Characteristics.” Research in Higher Education 56: 719-749. doi: 10.1007/s11162-

015-9368-9.

* Jaeger, A. J., and M. K. Eagan. 2011. “Examining Retention and Contingent Faculty Use in

a State System of Public Higher Education.” Educational Policy 25 (3): 507-537.

doi:10.1177/0895904810361723

* Jones, M. P., J. A. Bushnell, and J. S. Humphreys. 2014. “Are Rural Placements Positively

Associated with Rural Intentions in Medical Graduates?” Medical Education 48 (4):

405-416. doi: 10.1111/medu.12399.

* Jowett, T., J. Harraway, B. Lovelock, S. Skeaff, L. Slooten, M. Strack, and K. Shephard.

2014. “Multinomial-Regression Modeling of the Environmental Attitudes of Higher

 
 29

Education Students Based on the Revised New Ecological Paradigm Scale.” The

Journal of Environmental Education 45 (1): 1-15.

* Julal, F. S. 2013. “Use of Student Support Services Among University Students:

Associations with Problem-Focused Coping, Experience of Personal Difficulty and

Psychological Distress.” British Journal of Guidance & Counselling 41 (4): 414-425.

doi:10.1080/03069885.2012.741680

* King, B. 2015. “Changing College Majors: Does it Happen More in STEM and Do Grades

Matter?” Journal of College Science Teaching 44 (3): 44-51.

doi:10.2505/4/jcst15_044_03_44

Kleinbaum, D. G., L. L. Kupper, K. E. Muller, and A. Nizam. 1998. Applied Regression

Analysis and Other Multivariable Methods. 3rd ed. Pacific Grove: Duxbury Press.

* Ko, K., and K.-N. Jun. 2015. “A Comparative Analysis of Job Motivation and Career

Preference of Asian Undergraduate Students.” Public Personnel Management 44 (2):

192-213. doi:10.1177/0091026014559430

* Lee, A. 2014. “Students with Disabilities Choosing Science Technology Engineering and

Math (STEM) Majors in Postsecondary Institutions.” Journal of Postsecondary

Education and Disability 27 (3): 261-272.

* Lombardi, A., B. Doren, J. M. Gau, and L. E. Lindstrom. 2013. “The Influence of

Instructional Settings in Reading and Math on Postsecondary Participation.” Journal

of Disability Policy Studies 24 (3): 170-180. doi:10.1177/1044207312468766

* Luna, G., and M. Fowler. 2011. “Evaluation of Achieving a College Education Plus: A

Credit-Based Transition Program.” Community College Journal of Research and

Practice 35 (9): 673-688.

* Lyons, F. W., and D. Akroyd. 2014. “The Impact of Human Capital and Selected Job

Rewards on Community College Faculty Job Satisfaction.” Community College

 
 30

Journal of Research and Practice 38 (2-3): 194-207.

doi:10.1080/10668926.2014.851965

McCulloch, C. E., and S. R. Searle. 2001. Generalized, Linear, and Mixed models. New York:

John Wiley & Sons.

* Melguizo, T., F. S. Torres, and H. Jaime. 2011. “The Association Between Financial Aid

Availability and the College Dropout Rates in Colombia.” Higher Education 62 (2):

231-247. doi:10.1007/s10734-010-9385-8

Menard, S. 2010. Logistic Regression: From Introduction to Advanced Concepts and

Applications. Thousand Oaks, CA: SAGE.

* Mendez, J. M., and J. P. Mendez. 2016. “Student Inferences Based on Facial Appearance.”

Higher Education 71 (1): 1-19. doi: 10.1007/s10734-015-9885-7.

* Munisamy, S., N. I. M. Jaafar, and S. Nagaraj. 2014. “Does Reputation Matter? Case study

of Undergraduate Choice at a Premier University.” Asia-Pacific Education

Researcher 23 (3): 451-462.

* Newman, L. A., and J. W. Madaus. 2015. “An Analysis of Factors Related to Receipt of

Accommodations and Services by Postsecondary Students with Disabilities.”

Remedial and Special Education 36 (4): 208-219. doi:10.1177/0741932515572912

* O’Neill, L., J. Hartvigsen, B. Wallstedt, L. Korsholm, and B. Eika. 2011. “Medical School

Dropout - Testing at Admission Versus Selection by Highest Grades as Predictors.”

Medical Education 45 (11): 1111-1120. doi: 10.1111/j.1365-2923.2011.04057.x.

Pedhazur, E. J. 1997. Multiple Regression in Behavioral Research: Explanation and

Prediction. 3rd ed. Fort Worth, TX: Harcourt Brace College Publishers.

* Pinxten, M., B. De Fraine, W. Van Den Noortgate, J. Van Damme, T. Boonen, and G.

Vanlaar. 2015. “‘I Choose so I am’: A Logistic Analysis of Major Selection in

 
 31

University and Successful Completion of the First Year.” Studies in Higher Education

40 (10): 1919-1946. doi: 10.1080/03075079.2014.914904.

* Pleitz, J. D., A. E. MacDougall, R. A. Terry, M. R. Buckley, and N. J. Campbell. 2015.

“Great Expectations: Examining the Discrepancy between Expectations and

Experiences on College Student Retention.” Journal of College Student Retention:

Research, Theory & Practice 17 (1): 88-104.

* Pruett, P. S., and B. Absher. 2015. “Factors Influencing Retention of Developmental

Education Students in Community Colleges.” Delta Kappa Gamma Bulletin 81 (4):

32-40.

* Pursel, B. K., L. Zhang, K. W. Jablokow, G. W. Choi, and D. Velegol. 2016.

“Understanding MOOC Students: Motivations and Behaviours Indicative of MOOC

Completion.” Journal of Computer Assisted Learning 32: 202-217.

* Radey, M., and L. P. Cheatham. 2013. “Do Single Mothers Take Their Share?: FAFSA

Completion Among Aid-Eligible Female Students.” Journal of Diversity in Higher

Education 6 (4): 261-275. doi: 10.1037/a0035089.

Rencher, A. C. 2000. Linear Models in Statistics. New York: John Wiley.

* Santelices, M. V., X. Catalán, D. Kruger, and C. Horn. 2016. “Determinants of Persistence

and the Role of Financial Aid: Lessons from Chile.” Higher Education 71: 323-342.

doi: 10.1007/s10734-015-9906-6.

* Stegers-Jager, K. M., E. W. Steyerberg, J. Cohen-Schotanus, and A. P. N. Themmen. 2012.

“Ethnic Disparities in Undergraduate Pre-Clinical and Clinical Performance.”

Medical Education 46 (6): 575-585. doi: 10.1111/j.1365-2923.2012.04265.x.

* Sullivan, K., and M. Cosden. 2015. “High School Risk Factors Associated with Alcohol

Trajectories and College Alcohol Use.” Journal of Child & Adolescent Substance

Abuse 24: 19-27.

 
 32

* Swecker, H. K., M. Fifolt, and L. Searby. 2013. “Academic Advising and First-Generation

College Students: A Quantitative Study on Student Retention.” NACADA Journal 33

(1): 46-53.

Tabachnick, B. G., and L. S. Fidell. 2001. Using Multivariate Statistics. 4th ed. Boston: Allyn

and Bacon.

* Taliaferro, L. A, and J. J. Muehlenkamp. 2015. “Risk Factors Associated with Self-

Injurious Behavior Among a National Sample of Undergraduate College Students.”

Journal of American College Health 63 (1): 40-48. doi:

10.1080/07448481.2014.953166.

* Titzmann, P. F., A. Brenick, and R. K. Silbereisen. 2015. “Friendships Fighting Prejudice:

A Longitudinal Perspective on Adolescents' Cross-Group Friendships with

Immigrants.” Journal of Youth and Adolescence 44 (6): 1318-1331. doi:

10.1007/s10964-015-0256-6.

* van den Broek, S., B. Muller, N. Dekker, A. Bootsma, and O. T. Cate. 2010. “Effect of the

New Bologna Bachelor Degree on Career Considerations of Medical Students in One

Medical School.” Medical Teacher 32 (12): 997-1001.

* Wambuguh, O., and T. Yonn-Brown. 2013. “Regular Lecture Quizzes Scores as Predictors

of Final Examination Performance: A Test of Hypothesis Using Logistic Regression

Analysis.” International Journal for the Scholarship of Teaching and Learning 7 (1):

1-10. doi: 10.20429/ijsotl.2013.070107.

Williams, R. 2016. “Marginal Effects for Continuous Variables.” Retrieved from

https://www3.nd.edu/~rwilliam/stats3/Margins02.pdf

* Wood, J. L. 2013. “The Same ... but Different: Examining Background Characteristics

Among Black Males in Public Two-Year Colleges.” The Journal of Negro Education

82 (1): 47-61.

 
 33

* Yusif, H., I. Yussof, and Z. Osman. 2013. “Public University Entry in Ghana: Is it

Equitable?” International Review of Education 59 (1): 7-27.

 
 1

A Review of the Application of Logistic Regression in Educational Research: Common

Issues, Implications, and Suggestions

Table 1. Frequency and percentage of reported statistics as logistic regression results

Statistic Frequency Percentage
Odds ratio 106 81.5%
Confidence interval of coefficient or odds ratio 56 43.1%
p value of significance test 113 86.9%
Marginal effects 7 5.4%
Predicted probability 7 5.4%
Note: N = 130

 
 2

Table 2. Frequency and percentage of interpretation of independent variable

Interpretation Frequency Percentage
Significance of independent variable 118 90.8%
Direction of association 110 84.6%
Magnitude of association in terms of probabilities 52 40.0%
Note. N = 130

 
 3

Table 3. Relative risk based on hypothetical probabilities and reported odds ratio
Hypothetical Hypothetical Hypothetical Reported odds Overestimation
probability of probability of relative risk, ratio, students of relative risk
using student using student students with with difficulty (odds ratio /
service for service for difficulty vs. vs. students relative risk)
students with students without students without without
difficulty difficulty difficulty difficulty
0.64 0.05 12.88 34.34 2.67
0.79 0.10 7.92 34.34 4.33
0.86 0.15 5.72 34.34 6.00
0.90 0.20 4.48 34.34 7.67
0.92 0.25 3.68 34.34 9.34
0.94 0.30 3.12 34.34 11.00
0.95 0.35 2.71 34.34 12.67
0.96 0.40 2.40 34.34 14.34
Note. Odds ratio reported by Julal (2013).

 
 4

Table 4. Overestimation of relationship magnitude by interpreting odds ratio as relative risk

Hypothetical Hypothetical
probability, probability,
Reported comparison reference Hypothetical Over-
Conclusion odds ratio group group relative risk estimation
Students attaining combined quizzes scores of at least 70% are
approximately 9 times (OR value of 8.78 in Table 2) as likely to
pass the final examination with the same score or better compared 8.78 0.93 0.6 1.55 5.67
to those who got scores below 70% (χ2 = 160.53, p < .001).
(Wambuguh & Yonn-Brown, 2013, p.3)

The likelihood of entering public university increases by a factor of

6.781 if a student has one sibling and it is significant at the 1 per 6.781 0.91 0.6 1.52 4.47
cent level. (Yusif, Yussof, & Osman, 2013, p. 22)

Students in the high HSR [high school risk] group were 6.04 times
6.04 0.92 0.65 1.41 4.28
more likely to engage in heavy episodic drinking (p < .001) and
2.51 times more likely to experience a blackout in college (p <.001)
than were students in the low HSR group (see Table 7). Students in
the high HSR group were also 2.85 times more likely to engage in 2.51 0.39 0.2 1.93 1.30
heavy episodic drinking in college as compared to the moderate
HSR group, p<.05. (Sullivan & Cosden, 2015, p. 24) 2.85 0.92 0.8 1.15 2.48
Those attending for-profit institutions were nearly 10 times more
likely to complete FAFSAs than aid-eligible women at public 9.75 0.98 0.8 1.22 8.00
universities. (Radey & Cheatham, 2013, p. 270)
Note. Hypothetical probabilities are approximated based on descriptive statistics reported in respective studies.

 
 5

Table 5. Approaches of interpreting magnitude of relationship revealed by the current review

Approach Revealed issue of approach Suggestion

Report and interpret
No quantitative Does not fully utilize logistic quantitative
interpretation of regression results to understand measurements of
relationship magnitude educational phenomena predictive relationship

Difficult to comprehend; causes

Interpretation of odds
confusion; might be understood as Use statistics in terms of
ratio as change in odds
ratio of probabilities; does not probabilities as
corresponding to
measure prediction in terms of supplement
change in predictor
probabilities
Compute statistics in
terms of probabilities,
Interpretation of odds Exaggerates predictive relationship;
such as marginal effects
ratio as ratio of leads to inaccurate conclusion; might
and predicted
probabilities mislead readers
probabilities

Report selected values

Omission of the selection of of other independent
Calculation of
independent variables’ values in the variables in the model
marginal effects or
calculation and interpretation of these and emphasize the
predicted probabilities
statistics conditional nature of
calculated statistics

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 1
 

Appendix A. List of Reviewed Studies not Cited in the Text

Abdulghani, H. M., G. Al-Shaikh, A. K. Alhujayri, N. S. Alohaideb, H. A. Alsaeed, I. S.

Alshohayeb, M. M. Alyahya, A. I. Alhaqwi, and S. A. Shaik. 2013. “What Determines

the Selection of Undergraduate Medical Students to the Specialty of Their Future

Careers?” Medical Teacher 35: S25-S30. doi:10.3109/0142159x.2013.765548

Aikins, R. D., A. Golub, and A. S. Bennett. 2015. “Readjustment of Urban Veterans: A Mental

Health and Substance Use Profile of Iraq and Afghanistan Veterans in Higher

Education.” Journal of American College Health 63 (7): 482-494.

doi:10.1080/07448481.2015.1068173

Andersen, L., and T. J. Ward. 2014. “Expectancy-Value Models for the STEM Persistence Plans

of Ninth-Grade, High-Ability Students: A Comparison Between Black, Hispanic, and

White Students.” Science Education 98 (2): 216-242. doi:10.1002/sce.21092

Angelin, M., B. Evengård, and H. Palmgren. 2015. “Illness and Risk Behaviour in Health Care

Students Studying Abroad.” Medical Education 49 (7): 684-691.

doi:10.1111/medu.12753

Barrett, K. L., W. G. Jennings, and M. J. Lynch. 2012. “The Relation Between Youth Fear and

Avoidance of Crime in School and Academic Experiences.” Journal of School Violence,

11 (1): 1-20. doi:10.1080/15388220.2011.630309

Barron, J. S., E. Bragg, D. Cayea, S. C. Durso, and N. S. Fedarko. 2015. “The Short-Term and

Long-Term Impact of a Brief Aging Research Training Program for Medical Students.”

Gerontology & Geriatrics Education 36 (1): 96-106. doi:10.1080/02701960.2014.942036

Becerra, D. (2012). “Perceptions of Educational Barriers Affecting the Academic Achievement

of Latino K-12 Students.” Children and Schools 34 (3), 167-177. doi:10.1093/cs/cds001

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 2
 

Bingham, C. R., A. I. Barretto, M. A. Walton, C. M. Bryatn, J. T. Shope, and T. E. Raghunathan.

2011. “Efficacy of a Web-Based, Tailored, Alcohol Prevention/Intervention Program for

College Students: 3-month Follow-Up.” Journal of Drug Education 41 (4): 405-430.

doi:10.2190/DE.41.4.e

Blackford, K., and J. Khojasteh. 2013. “Closing the Achievement Gap: Identifying Strand Score

Differences.” American Journal of Educational Studies 6 (2): 5-15.

Boek, S., S. Bianco-Simeral, K. Chan, and K. Goto. 2012. “Gender and Race are Significant

Determinants of Students’ Food Choices on a College Campus.” Journal of Nutrition

Education and Behavior 44 (4): 372-378. doi:10.1016/j.jneb.2011.12.007

Bouck, E. C., and G. S. Joshi. 2015. “Does Curriculum Matter for Secondary Students with

Autism Spectrum Disorders: Analyzing the NLTS2.” Journal of Autism and

Developmental Disorders 45 (5): 1204-1212. doi: 10.1007/s10803-014-2281-9

Champion, J., F. Parker, B. Mendoza-Spencer, and A. Wheeler. 2011. “College Algebra

Students’ Attitudes Toward Mathematics in Their Careers.” International Journal of

Science and Mathematics Education 9 (5): 1093-1110. doi:10.1007/s10763-010-9246-z

Chandauka, R. E., J. M. Russell, J. Sandars, and P. Vivekananda-Schmidt. 2015. “Differing

Perceptions Among Ethnic Minority and Caucasian Medical Students Which May Affect

Their Relative Academic Performance.” Education for Primary Care 26 (1): 11-15.

Cogan, M. F. 2011. “Predicting Success of Academically Dismissed Undergraduate Students

Using Quality Point Status.” Journal of College Student Retention: Research, Theory and

Practice 12 (4): 387-406. doi: 10.2190/CS.12.4.a

De Backer, L., H. Van Keer, and M. Valcke. 2015a. “Socially Shared Metacognitive Regulation

During Reciprocal Peer Tutoring: Identifying its Relationship with Students’ Content

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 3
 

Processing and Transactive Discussions.” Instructional Science 43 (3): 323-344.

doi:10.1007/s11251-014-9335-4

De Backer, L., H. Van Keer, and M. Valcke. 2015b. “Exploring Evolutions in Reciprocal Peer

Tutoring Groups’ Socially Shared Metacognitive Regulation and Identifying its

Metacognitive Correlates.” Learning and Instruction 38: 63-78.

doi:0.1016/j.learninstruc.2015.04.001

Eckles, J. E., and E. G. Stradley. 2012. “A Social Network Analysis of Student Retention Using

Archival Data.” Social Psychology of Education 15 (2): 165-180. doi:10.1007/s11218-

011-9173-z

England-Siegerdt, C. 2010. “Do Loans Really Expand Opportunities for Community College

Students?” Community College Journal of Research and Practice 35 (1-2): 88-98.

doi:10.1080/10668926.2011.525180

Epping, A. S., M. P. Myrvik, R. F. Newby, J. A. Panepinto, A. M. Brandow, and J. P. Scott.

2013. “Academic Attainment Findings in Children with Sickle Cell Disease.” Journal of

School Health 83 (8): 548-553.

Flannery, K. B., J. L. Frank, and M. M. Kato. 2012. “School Disciplinary Responses to Truancy:

Current Practice and Future Directions.” Journal of School Violence 11 (2): 118-137.

doi:10.1080/15388220.2011.653433

Fox, R. S., E. L. Merz, M. T. Solorzano, and S. C. Roesch. 2013. “Further Examining Berry’s

Model: The Applicability of Latent Profile Analysis to Acculturation.” Measurement and

Evaluation in Counseling and Development 46 (4): 270-288.

doi:10.1177/0748175613497036

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 4
 

Furlong, M., and M. Quirk. 2011. “The Relative Effects of Chronological Age on Hispanic

Students’ School Readiness and Grade 2 Academic Achievement.” Contemporary School

Psychology 15 (1): 81-92.

Gil-Flores, J., T. M. Padilla-Carmona, and M. Suárez-Ortega. 2011. “Influence of Gender,

Educational Attainment and Family Environment on the Educational Aspirations of

Secondary School Students.” Educational Review 63 (3): 345-363.

doi:10.1080/00131911.2011.571763

Gonzalez, L. M. 2012. “College-Level Choice of Latino High School Students: A Social-

Cognitive Approach.” Journal of Multicultural Counseling and Development 40 (3): 144-

155. doi:10.1002/j.2161-1912.2012.00014.x

Greenberg, J. P. 2013. “Determinants of After-School Programming for School-Age Immigrant

Children.” Children and Schools 35 (2): 101-111. doi: 10.1093/cs/cdt002

Greulich, L., S. Al Otaiba, C. Schatschneider, J. Wanzek, M. Ortiz, and R. Wagner. 2014.

“Understanding Inadequate Response to First Grade Multi-Tier Intervention: Nomothetic

and Ideographic Perspectives.” Learning Disability Quarterly: Journal of the Division for

Children with Learning Disabilities 37 (4): 204-217. doi: 10.1177/0731948714526999

Haas, A. L., S. K. Smith, and K. Kagan. 2013. “Getting “Game”: Pregaming Changes During the

First Weeks of College.” Journal of American College Health 61 (2): 95-105.

doi:10.1080/07448481.2012.753892

Harrell, I. L., and B. L. Bower. 2011. “Student Characteristics That Predict Persistence in

Community College Online Courses.” American Journal of Distance Education 25 (3):

178-191. doi: 10.1080/08923647.2011.590107

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 5
 

Hawkins, A., C. R. Graham, R. R. Sudweeks, and M. K. Barbour. 2013. “Academic

Performance, Course Completion Rates, and Student Perception of the Quality and

Frequency of Interaction in a Virtual High School.” Distance Education 34 (1): 64-83.

doi:10.1080/01587919.2013.770430

Hayes, J. B., R. A. Price, and R. P. York. 2013. “A Simple Model for Estimating Enrollment

Yield From a List of Freshman Prospects.” Academy of Educational Leadership Journal

17 (2): 61-68.

Hickman, G. P., and D. Wright. 2011. “Academic and School Behavioral Variables as Predictors

of High School Graduation Among at-Risk Adolescents Enrolled in a Youth-Based

Mentoring Program.” The Journal of At-Risk Issues 16 (1): 25-33.

Hillman, N. W. 2013. “Cohort Default Rates: Predicting the Probability of Federal Sanctions.”

Educational Policy 29 (4): 559-582. doi: 10.1177/0895904813510772

Honney, K., M. Buszewicz, W. Coppola, and M. Griffin. 2010. “Comparison of Levels of

Depression in Medical and Non-Medical Students.” Clinical Teacher 7 (3): 180-184.

doi:10.1111/j.1743-498X.2010.00384.x

Houser, L. C.-S., and S. An. 2015. “Factors Affecting Minority Students’ College Readiness in

Mathematics.” Urban Education 50 (8): 938-960. doi: 10.1177/0042085914536998

Hu, S., A. C. McCormick, and R. M. Gonyea. 2012. “Examining the Relationship Between

Student Learning and Persistence.” Innovative Higher Education 37 (5): 387-395.

doi:10.1007/s10755-011-9209-5

Jamali, A., S. Tofangchiha, R. Jamali, S. Nedjat, D. Jan, A. Narimani, and A. Montazeri. 2013.

“Medical Students’ Health-Related Quality of Life: Roles of Social and Behavioural

Factors.” Medical Education 47 (10): 1001-1012. doi: 10.1111/medu.12247

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 6
 

Jelfs, A., and J. T. E. Richardson. 2013. “The Use of Digital Technologies Across the Adult Life

Span in Distance Education.” British Journal of Educational Technology 44 (2): 338-351.

doi:10.1111/j.1467-8535.2012.01308.x

Jonesa, L. Ø., T. Mangerb, O.-J. Eikeland, and A. Asbjørnsen. 2013. “Participation in Prison

Education: Is it a Question of Reading and Writing Self-Efficacy Rather Than Actual

Skills?” Journal of Correctional Education 64 (2): 41-62.

Kolenovic, Z., D. Linderman, and M. M. Karp. 2013. “Improving Student Outcomes via

Comprehensive Supports: Three-Year Outcomes From CUNY’s Accelerated Study in

Associate Programs (ASAP).” Community College Review 41 (4): 271-291.

doi:10.1177/0091552113503709

Kovner, C. T., C. Brewer, C. Katigbak, M. Djukic, and F. Fatehi. 2012. “Charting the Course for

Nurses’ Achievement of Higher Education Levels.” Journal of Professional Nursing 28

(6): 333-343. doi:10.1016/j.profnurs.2012.04.021

Kushimoto, T. 2010. “Outcomes Assessment and its Role in Self-Reviews of Undergraduate

Education: In the Context of Japanese Higher Education Reforms Since the 1990s.”

Higher Education 59 (5): 589-598. doi:10.1007/s10734-009-9266-1

Kushner, J., and G. M. Szirony. 2014. “School Counselor Involvement and College Degree

Attainment: A Quantitative Conundrum.” Online Journal of New Horizons in Education

4 (1): 75-79.

Kwenda, M. 2011. “Factors Affecting Performance in an Introductory Sociology Course.”

College Teaching 59: 60-65. doi: 10.1080/87567550903418578

Larsen, J. K., M. Kleinjan, R. C. M. E. Engels, J. O. Fisher, and R. Hermans. 2014. “Higher

Weight, Lower Education: A Longitudinal Association Between Adolescents’ Body Mass

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 7
 

Index and Their Subsequent Educational Achievement Level?” Journal of School Health

84 (12): 769-776. doi: 10.1111/josh.12212

Lauderdale-Littin, S., E. Howell, and J. Blacher. 2013. “Educational Placement for Children with

Autism Spectrum Disorders in Public and Non-Public School Settings: The Impact of

Social Skills and Behavior Problems.” Education and Training in Autism and

Developmental Disabilities 48 (4): 469-478.

Lee, M., H. Kim, and M. Kim. 2014. “The Effects of Socratic Questioning on Critical Thinking

in Web-Based Collaborative Learning.” Education as Change 18 (2): 285-302.

doi:10.1080/16823206.2013.849576

Leppo, R. H. T., S. W. Cawthon, and M. P. Bond. 2014. “Including Deaf and Hard-of-Hearing

Students with Co-Occurring Disabilities in the Accommodations Discussion.” Journal of

Deaf Studies and Deaf Education 19 (2): 189-202. doi:10.1093/deafed/ent029

Lewis, T. F., and E. Wahesh. 2015. “Perceived Norms and Marijuana Use at Historically Black

Colleges and Universities.” Journal of College Counseling 18 (2): 130-143.

doi:10.1002/jocc.12010

Liu, L. 2012. “Factors Influencing Students' Preference to Online Learning: Development of an

Initial Propensity Model.” International Journal of Technology in Teaching and Learning

7 (2): 93-108.

Liu, Z., and Y. Gao. 2015. “Family Capital Social Stratification, and Higher Education

Attainment—An Empirical Study Based on Jiangsu Province.” Chinese Education &

Society 48 (3): 218-230. doi:10.1080/10611932.2015.1085773

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 8
 

Long, G. L., C. Marchetti, and R. Fasse. 2011. “The Importance of Interaction for Academic

Success in Online Courses with Hearing, Deaf, and Hard-of-Hearing Students.”

International Review of Research in Open and Distance Learning 12 (6): 1-19.

Magalhães, E., P. Costa, and M. J. Costa. 2012. “Empathy of Medical Students and Personality:

Evidence from the Five-Factor Model.” Medical Teacher 34 (10): 807-812.

doi:10.3109/0142159X.2012.702248

McGinley, M., K. M. Rospenda, L. Liu, and J. A. Richman. 2015. “Chronic Generalized

Harassment During College: Influences on Alcohol and Drug use.” Journal of Youth and

Adolescence 44 (10): 1898-1913. doi:10.1007/s10964-015-0313-1

McLaughlin, M. J., K. E. Speirs, and E. D. Shenassa. 2012. “Reading Disability and Adult

Attained Education and Income: Evidence from a 30-year Longitudinal Study of a

Population-Based Sample.” Journal of Learning Disabilities 47 (4): 374-386.

doi:10.1177/0022219412458323

McVie, S. 2014. “The Impact of Bullying Perpetration and Victimization on Later Violence and

Psychological Distress: A Study of Resilience Among a Scottish Youth Cohort.” Journal

of School Violence 13 (1): 39-58. doi:10.1080/15388220.2013.841586

Mendoza, P., and J. P. Mendez. 2012. “The Oklahoma’s Promise Program: A national model to

promote college persistence.” Journal of College Student Retention: Research, Theory

and Practice 14 (3): 397-421. doi:10.2190/CS.14.3.g

Newman, M. D., and J. M. Petrosko. 2011. “Predictors of alumni association membership.”

Research in Higher Education 52 (7): 738-759. doi: 10.1007/s11162-011-9213-8

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 9
 

Novak, H., and L. McKinney. 2011. “The Consequences of Leaving Money on the Table :

Examining Persistence Among Students Who Do not File a FAFSA.” Journal of Student

Financial Aid 41 (3): 5-23.

Nunez, A.-M., and G. Crisp. 2012. “Ethnic Diversity and Latino/a College Access: A

Comparison of Mexican American and Puerto Rican Beginning College Students.”

Journal of Diversity in Higher Education 5 (2): 78-95. doi: 10.1037/a0026810

Owens, J. 2010. “Foreign Students, Immigrants, Domestic Minorities and Admission to Texas’

Selective Flagship Universities Before and After the Ban on Affirmative Action.”

Peabody Journal of Education 85 (4): 486-510. doi:10.1080/0161956X.2010.518046

Pérez, A., T. Rodríguez, M. J. López, X. Continente, M. Nebot. 2016. “Adoption of Preventive

Measures and Attitudes Toward the H1N1 Influenza Pandemic in Schools.” Journal of

School Health 86 (7): 534-542. doi: 10.1111/josh.12406

Price, J. H., G. M. Kirchofer, J. Khubchandani, J. Kleinfelder, and M. Bryant. 2013.

“Development of a College Student’s Mistrust of Health Care Organizations Scale.”

American Journal of Health Education 44 (1): 19-25.

doi:10.1080/19325037.2012.749705

Prins, E., S. Monnat, C. Clymer, and B. W. Toso. 2015. “How is Health Related to Literacy,

Numeracy, and Technological Problem-Solving Skills Among U.S. Adults? Evidence

From the Program for the International Assessment of Adult Competencies (PIAAC).”

Journal of Research and Practice for Adult Literacy, Secondary, and Basic Education 4

(3): 22-42.

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 10
 

Rohr, S. L. 2012. “How Well Does the SAT and GPA Predict the Retention of Science,

Technology, Engineering, Mathematics, and Business Students.” Journal of College

Student Retention: Research, Theory & Practice 14 (2): 195-208. doi: 10.2190/CS.14.2.c

Rojas, Y. 2013. “School Performance and Gender Differences in Suicidal Behaviour - A 30-year

Follow-Up of a Stockholm Cohort Born in 1953.” Gender and Education 25 (5): 578-

594. doi:10.1080/09540253.2013.797955

Rye, B. J., C. Mashinter, G. J. Meaney, E. Wood, and S. Gentile. 2015. “Satisfaction With

Previous Sexual Health Education as a Predictor of Intentions to Pursue Further Sexual

Health Education.” Sex Education 15 (1): 93-107. doi:10.1080/14681811.2014.967389

Schripsema, N. R., A. M. van Trigt, J. C. C. Borleffs, and J. Cohen-Schotanus. 2014. “Selection

and Study Performance: Comparing Three Admission Processes Within One Medical

School.” Medical Education 48 (12): 1201-1210. doi:10.1111/medu.12537

Schumacher, P., A. Olinsky, J. Quinn, and R. Smith. 2010. “A Comparison of Logistic

Regression, Neural Networks, and Classification Trees Predicting Success of Actuarial

Students.” Journal of Education for Business 85 (5): 258-263.

doi:10.1080/08832320903449477

Shen, F. C. 2015. “The Role of Internalized Stereotyping, Parental Pressure, and Parental

Support on Asian Americans’ Choice of College Major.” Journal of Multicultural

Counseling and Development 43 (1): 58-73. doi:10.1002/j.2161-1912.2015.00064.x

Song, Y., P. Loyalka, and J. Wei. 2013. “Determinants of Tracking Intentions, and Actual

Education Choices among Junior High School Students in Rural China.” Chinese

Education & Society 46 (4): 30-42. doi:10.2753/CED1061-1932460403

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 11
 

Stein, C. H., L. A. Osborn, and S. C. Greenberg. 2016. “Understanding Young Adults' Reports of

Contact With Their Parents in a Digital World: Psychological and Familial Relationship

Factors.” Journal of Child and Family Studies 25 (6): 1802-1814. doi:10.1007/s10826-

016-0366-0

Strayhorn, T. L. 2010. “Money Matters: The Influence of Financial Factors on Graduate Student

Persistence.” Journal of Student Financial Aid 40 (3): 4-25.

Sullivan, A. L., D. A. Klingbeil, and E. R. Van Norman. 2013. “Beyond Behavior: Multilevel

Analysis of the Influence of Sociodemographics and School Characteristics on Students’

Risk of Suspension.” School Psychology Review 42 (1): 99-114.

Trant, E. C., K. E. Crabtree, D. J. Ciancio, L. A. Hart, T. B. Watson, and R. L. Williams. 2015.

“Why Some HOPE Scholarship Recipients Retain the Scholarship and Others Lose It.”

Innovative Higher Education 40 (3): 201-214. doi:10.1007/s10755-014-9306-3

Turcios-Cotto, V. Y., and S. Milan. 2013. “Racial/Ethnic Differences in the Educational

Expectations of Adolescents: Does Pursuing Higher Education Mean Something

Different to Latino Students Compared to White and Black Students?” Journal of Youth

and Adolescence 42 (9): 1399-1412. doi:10.1007/s10964-012-9845-9

Tuttle, L. V., and G. D. Musoba. 2013. “Transfer Student Persistence at a Hispanic-Serving

University.” Journal of Latinos and Education 12 (1): 38-58.

doi:10.1080/15348431.2013.734248

van Rooij, S. W. 2012. “Open-Source Learning Management Systems: A Predictive Model for

Higher Education.” Journal of Computer Assisted Learning 28 (2): 114-125.

doi:10.1111/j.1365-2729.2011.00422.x

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 12
 

Wei, X., E. R. A. Christiano, J. W. Yu, J. Blackorby, P. Shattuck, and L. A. Newman. 2014.

“Postsecondary Pathways and Persistence for STEM Versus Non-STEM Majors: Among

College Students With an Autism Spectrum Disorder.” Journal of Autism and

Developmental Disorders 44 (5): 1159-1167. doi:10.1007/s10803-013-1978-5

Wells, K., C. Makela, and C. Kennedy. 2014. “Co-Occurring Health-Related Behavior Pairs in

College Students: Insights for Prioritized and Targeted Interventions.” American Journal

of Health Education 45 (4): 210-218. doi:10.1080/19325037.2014.916637

Whannell, R. 2013. “Predictors of Attrition and Achievement in a Tertiary Bridging Program.”

Australian Journal of Adult Learning 53 (2): 280-301.

Wilkins, S., and M. S. Balakrishnan. 2013. “Assessing Student Satisfaction in Transnational

Higher Education.” International Journal of Educational Management 27 (2): 143-156.

doi:10.1108/09513541311297568

Willits, K. A., M. L. Troutman-Jordan, M. A. Nies, E. F. Racine, E. Platonova, and Harris, H. L.

2013. “Presence of Medical Home and School Attendance: An Analysis of the 2005-2006

National Survey of Children with Special Healthcare Needs.” Journal of School Health

83 (2): 93-98. doi:10.1111/josh.12003

Wilson, I., B. Griffin, L. Lampe, D. Eley, G. Corrigan, B. Kelly, and P. Stagg. 2013. “Variation

in Personality Traits of Medical Students Between Schools of Medicine.” Medical

Teacher 35 (11): 944-948. doi:10.3109/0142159X.2013.827331

Wolf, S., J. L. Aber, and P. A. Morris. 2015. “Patterns of Time Use Among Low-Income Urban

Minority Adolescents and Associations with Academic Outcomes and Problem

Behaviors.” Journal of Youth and Adolescence 44 (6): 1208-1225. doi:10.1007/s10964-

015-0294-0

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 13
 

Zhao, D., and B. Parolin. 2014. “Merged or Unmerged School? School Preferences in the

Context of School Mapping Restructure in Rural China.” Asia-Pacific Education

Researcher 23 (3): 547-563. doi:10.1007/s40299-013-0129-2

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 14
 

Appendix B. Summary of the Reviewed Studies

Table B1. Summary of the reviewed studies

Report of Statistics Interpretation of Statistics

Est. Direct. Treated

Probab. Pred. Stat. of Magnitude OR as
Author(s) and year OR CI p ME RR Change Probab. Sig. Assoc. of Assoc. RR
Abdulghani et al. (2013) Y Y N N N N N Y N N N
Aikins, Golub, and Bennett (2015) Y Y N N N N N Y Y N N
Aine et al. (2014) Y Y N N N N N Y N N N
Andersen and Ward (2014) Y Y Y N N N N Y Y Y Y
Angelin, Evengård, and Palmgren
Y Y N N N N N Y Y N N
(2015)
Baker-Eveleth, O’Neill, and
Y N Y N N N N Y Y Y Y
Sisodiya (2014)
Barrett, Jennings, and Lynch (2012) Y N N N N N N Y Y N N
Barron et al. (2015) Y Y Y N N N N Y N N N
Becerra (2012) Y N Y N N N N Y Y N N
Beran et al. (2012) Y Y Y N N N N Y Y Y Y
Bielby et al. (2014) N N Y Y N N N Y Y Y N
Bingham et al. (2011) Y Y N N N N N Y Y N N
Blackford and Khojasteh (2013) Y Y Y N N N N Y Y N N
Boek et al. (2012) Y Y Y N N N N Y Y N N
Bouck and Joshi (2015) Y N Y N N N N Y Y N N
Burke, Gabhainn, and Young (2015) Y Y Y N N N N Y N N N
Campbell and Mislevy (2013) Y N Y N N N N Y Y Y Y
Case (2011) Y N Y N N N N Y N N N

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 15
 

Report of Statistics Interpretation of Statistics

Est. Direct. Treated

Probab. Pred. Stat. of Magnitude OR as
Author(s) and year OR CI p ME RR Change Probab. Sig. Assoc. of Assoc. RR
Champion et al. (2011) N N N N Y N N Y Y Y N

Chandauka et al. (2015) N N Y N N N N Y Y N N

Chiang et al. (2012) Y Y Y N N N N Y Y Y Y
Cogan (2011) Y N Y N N N N Y Y N N
d’Aguiar and Harrison (2016) Y N Y N N N N Y Y Y Y
De Backer, Van Keer, and Valcke
Y Y Y N N N N Y Y Y Y
(2015a)
De Backer, Van Keer, and Valcke
Y Y Y N N N N Y Y Y Y
(2015b)
Eckles and Stradley (2012) Y N Y N N N N N N Y Y
England-Siegerdt (2010) N N N N N N N N Y N N
Epping et al. (2013) Y Y Y N N N N Y N N N
Ertuna and Gurel (2011) Y N Y N N N N Y Y Y Y
Evans (2013) N N N N N N N N Y N N
Ferguson et al. (2012) Y Y Y N N N N Y Y N N
Fernández-Macías et al. (2013) N N N Y N N N Y Y Y N
Flannery, Frank, and Kato (2012) Y Y Y N N N N Y Y N N
Flynn (2014) Y N Y N N N Y Y Y N N
Fong, Melguizo, and Prather (2015) Y N Y N N N N Y Y N N
Fox et al. (2013) Y Y Y N N N N Y Y N N
Furlong and Quirk (2011) Y Y Y N N N N Y Y N N
Gil-Flores, Padilla-Carmona, and
Y N Y N N N N Y Y Y Y
Suárez-Ortega (2011)
Gonzalez (2012) N N Y N N N N N N N N
Greenberg (2013) Y N Y N N N N Y Y N N
Greulich et al. (2014) Y Y Y N N N N Y N N N

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 16
 

Report of Statistics Interpretation of Statistics

Est. Direct. Treated

Probab. Pred. Stat. of Magnitude OR as
Author(s) and year OR CI p ME RR Change Probab. Sig. Assoc. of Assoc. RR
Haas, Smith, and Kagan (2013) Y Y Y N N N N Y N N N
Harrell and Bower (2011) Y Y Y N N N N Y Y N N
Hawkins et al. (2013) N N N N N N N N N N N
Hayes, Price, and York (2013) Y N Y N N N N N Y N N
Hickman and Wright (2011) Y N Y N N N N Y Y Y N
Hillman (2013) Y N Y N N N Y Y Y Y N
Honney et al. (2010) N N Y N N N N N Y N N
Houser and An (2015) Y Y Y N N N N N Y N N
Hu, McCormick, and Gonyea (2012) Y N Y N N N N Y Y N N
Jaeger and Eagan (2011) N N Y Y N N N Y Y Y N
Jamali et al. (2013) Y Y Y N N N N N N N N
Jelfs and Richardson (2013) N N N N N N N Y Y N N
Jones, Bushnell, and Humphreys
Y Y Y N N N N Y Y N N
(2014)
Jonesa et al. (2013) Y N Y N N N N Y Y Y Y
Jowett et al. (2014) Y Y N N N N Y Y Y N N
Julal (2013) Y Y Y N N N N Y Y Y Y
King (2015) Y N Y N N N Y Y Y N N
Ko and Jun (2015) Y N Y Y N N N Y Y N N
Kolenovic, Linderman, and Karp
Y N Y N N N N Y Y Y N
(2013)
Kovner et al. (2012) Y Y Y N N N N Y Y N N
Kushimoto (2010) Y N Y N N N N Y Y N N
Kushner and Szirony (2014) N N N N N N N Y Y N N

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 17
 

Report of Statistics Interpretation of Statistics

Est. Direct. Treated

Probab. Pred. Stat. of Magnitude OR as
Author(s) and year OR CI p ME RR Change Probab. Sig. Assoc. of Assoc. RR
Kwenda (2011) Y N Y N N N N Y Y N N
Larsen et al. (2014) Y Y Y N N N N Y Y N N
Lauderdale-Littin, Howell, and
Y N Y N N N N Y Y N N
Blacher (2013)
Lee (2014) Y N Y N N N N Y Y Y Y
Lee, Kim, and Kim (2014) Y Y Y N N N N Y Y Y Y
Leppo, Cawthon, and Bond (2014) Y Y Y N N N N Y Y Y Y
Lewis and Wahesh (2015) Y Y Y N N N N Y Y N N
Liu (2012) Y N Y N N N N Y Y Y Y
Liu and Gao (2015) Y Y Y N N N N Y Y Y Y
Lombardi et al. (2013) Y Y Y N N N N N N Y Y
Long, Marchetti, and Fasse (2011) N N Y N N N N Y N N N
Luna and Fowler (2011) Y N Y N N N N Y Y Y Y
Lyons and Akroyd (2014) Y N Y N N N N Y Y Y Y
Magalhães, Costa, and Costa (2012) Y Y Y N N N N Y Y Y N
McGinley (2015) Y N Y N N N N Y Y N N
McLaughlin, Speirs, and Shenassa
Y Y N N N N N Y Y Y Y
(2012)
McVie (2014) Y Y Y N N N N Y Y N N
Melguizo, Torres, and Jaime (2011) N N Y Y N N N Y Y Y N
Mendez and Mendez (2016) N N Y N N N N Y Y N N
Mendoza and Mendez (2012) Y N Y N N N N Y Y Y Y
Munisamy, Jaafar, Nagaraj (2014) Y N Y N N N N Y Y Y Y
Newman and Petrosko (2011) Y N Y N N N N N N N N
Newman and Madaus (2015) Y Y Y N N N N Y Y Y N
Novak and McKinney (2011) Y Y Y N N N N Y Y N N

 
LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 18
 

Report of Statistics Interpretation of Statistics

Est. Direct. Treated

Probab. Pred. Stat. of Magnitude OR as
Author(s) and year OR CI p ME RR Change Probab. Sig. Assoc. of Assoc. RR
Nunez and Crisp (2012) Y N Y N N N N Y Y N N
O’Neill et al. (2011) Y Y Y N N N N Y Y Y Y
Owens (2010) N N N N N Y Y Y Y Y N
Pérez et al. (2016) Y Y Y N N N N Y Y N N
Pinxten et al. (2015) N N Y N N N N Y Y N N
Pleitz et al. (2015) N N Y N N Y Y Y Y Y N
Price et al. (2013) Y Y Y N N N N Y Y N N

Prins et al. (2015) Y Y Y N N N N Y Y N N

Pruett and Absher (2015) Y N Y N N N N Y Y Y Y
Pursel et al. (2016) N N Y Y N N N Y Y Y N
Radey and Cheatham (2013) Y Y Y N N N N Y Y Y Y
Rohr (2012) Y N Y N N N N Y Y N N
Rojas (2013) Y Y Y N N N N Y Y Y Y
Rye et al. (2015) Y N Y N N N N Y N N N
Santelices et al. (2016) N N Y Y N N N Y Y Y N
Schripsema et al. (2014) Y N Y N N N N Y N N N

Schumacher et al. (2010) N N N N N N N N N N N

Shen (2015) Y Y Y N N N N Y Y N N
Song, Loyalka, and Wei (2013) N N Y N N N N Y Y N N
Stegers-Jager et al. (2012) Y Y Y N N N N Y Y N N
Stein, Osborn, and Greenberg (2016) N Y Y N N N N Y Y Y N
Strayhorn (2010) N N Y N N N N Y Y Y N

 
 LOGISTIC REGRESSION IN EDUCATIONAL RESEARCH 19
View publication stats

Report of Statistics Interpretation of Statistics

Est. Direct. Treated

Probab. Pred. Stat.
of Magnitude OR as
Author(s) and year OR CI p ME RR Change Probab. Sig.
Assoc. of Assoc. RR
Sullivan, Klingbeil, and Van
Y N Y N N N N Y Y Y Y
Norman (2013)
Sullivan and Cosden (2015) Y N Y N N N N Y Y Y Y
Swecker, Fifolt, and Searby (2013) Y Y Y N N N N Y Y N N
Taliaferro and Muehlenkamp (2015) Y Y Y N N N N Y N N N
Titzmann, Brenick, and Silbereisen
Y Y Y N N N N Y Y N N
(2015)
Trant et al. (2015) Y N Y N N N N Y Y Y N
Turcios-Cotto and Milan (2013) Y Y Y N N N N Y Y N N
Tuttle and Musoba (2013) Y N Y N N N N Y Y N N
van den Broek et al. (2010) Y Y Y N N N N Y Y N N
van Rooij (2012) Y N Y N N N N Y Y Y Y
Wambuguh and Yonn-Brown (2013) Y N Y N N N Y Y Y Y Y
Wei et al. (2014) Y Y Y N N N N Y Y N N
Wells, Makela, and Kennedy (2014) Y N Y N Y N N Y Y Y Y
Whannell (2013) Y Y Y N N N N Y Y N N
Wilkins and Balakrishnan (2013) Y N Y N N N N Y N N N
Willits et al. (2013) Y Y Y N N N N Y Y N N
Wilson et al. (2013) Y N Y N N N N Y Y N N
Wolf, Aber, and Morris (2015) Y N Y N N N N Y Y N N
Wood (2013) Y N Y N N N N Y Y Y Y
Yusif, Yussof, Osman (2013) Y N Y N N N N Y Y Y Y
Zhao and Parolin (2014) Y N Y N N N N Y Y N Y
Note. OR: odds ratio; CI: confidence interval; p: p value or significance level; ME: marginal effect; RR: relative risk. This table
summarizes the statistics reported and the statistics interpreted by each study included in the review. The last column reports whether
a study interpreted odds ratio as relative risk. If a statistic is reported or interpreted, a Y is noted; otherwise, an N is noted.