Logistic Regression

Introduction to R
History of R
Logistic regression is a type of regression analysis used for predicting the outcome of a Categorical dependent
variable (a dependent variable that can take on a limited number of categories) based on one or more predictor
variables (Continuous, Ordinal or Categorical).
Binary Logistic Regression has Outcome or Dependent Variable: Binary or Dichotomous i.e. 0 or 1
Example:
a customer will churn (1) or not (0)
a customer will respond to a campaign (1) or not (0)
should we grant a loan to a particular person (1) or not (0)
Logistic regression measures the relationship between a categorical dependent variable and independent variable (or
several), by converting the dependent variable to probability scores (P)
Probability score signifies the probability of event happening, for example probability of a customer to churn or
respond to a campaign
How is Logistic Regression different from Linear Regression
In Linear regression, the outcome variable is continuous and the predictor variables can be a mix of numeric and
categorical. But often there are situations where we wish to evaluate the effects of multiple explanatory variables on
a binary outcome variable
For example, the effects of a number of factors on the development or otherwise of a disease. A patient may be
cured or not; a prospect may respond or not, should we grant a loan to particular person or not, etc.
When the outcome or dependent variable is binary, and we wish to measure the effects of several independent
variables on it, we uses Logistic Regression
Probability of each observation will not be linearly distributed but more like sigmoid function i.e. values would be
closer to 0 and 1.
 The binary outcome variable can be coded as 0 or 1.
 The logistic curve is shown in the figure below:
Sigmoid Function
Concept of Sigmoid Function in Logistic Regression
 The sigmoid function is a bounded function.
If
 If b is –ve, thens shape will get reversed

 If we use linear regression, the predicted
value can become greater than one and less
than zero.
 Basically Y is a random variable having 0 or 1
outcome, which is a Bernoulli random
variable.
log of odds:
ln( p / 1  p)  a  bx
This is also called as a logit function
The estimation of parameters is done using Maximum Likelihood Estimate(for Non Linear
distribution) unlike Linear regression where method of Ordinary Least square is used.
Odds Ratio
Odds is calculated as P(Y = 1)/P(Y = 0)
Odds > 1 if Y = 1 is more likely
Odds < 1 if Y = 0 is more likely
This is called logit and it looks like “linear regression equation”

The bigger the logit is, bigger is P(Y = 1)
Quick Question 1
Suppose the coefficients of a logistic regression model with two
independent variables are as follows:
β0 = -1.5, β1 = 3, β2 = -0.5
And we have an observation with the following values of independent

variables:
x1 = 1, x2 = 5
What is the value of the Logit for this observation? Recall that the Logit is
log(Odds)
What is the value of the Odds for this observation? Note that you can
compute e^x, for some number x, in your R console by typing exp(x). The
function exp() computes the exponential of its argument
What is the value of P(y = 1) for this observation?

Applications of logistic regression in business
Response to an Subscriber
Churning up of
E-mail conversion after
subscribers
Campaign a Campaign
Response Conversion Attrition

Model Model Model
Cross Sell
Application Behavioral
Up Sell
Risk Model Risk Model
Model
Finding Credit Finding

Finding
card defaulters parameters that
probability of
by Demography boost Cross sell
loan defaults
and Behavior & Up sell
Just a few of them

Logistic Process
THE FRAMINGHAM HEART STUDY
Evaluating Risk Factors to Save Lives
Misconceptions in the first half of 20th Century about blood pressure

High blood pressure, dubbed hypertension, was considered important to
force blood through arteries and it was considered harmful to lower
blood pressure
In late 1940s, the US government set out to better understand
cardiovascular disease
The plan was to track a large cohort of initially healthy patients over
their lifetimes
A city was chosen, the city of Framingham, Massachusetts, to be the site
for the study
 Appropriate Size
 Stable population
5209 patients aged 30 – 59 enrolled
Patients were given questionnaire and exam every 2 years:
 Physical characteristics
 Behavioral characteristics
THE FRAMINGHAM HEART STUDY Contd..
Use the anonymized version of the original data that was collected
 Includes several demographic risk factors:
 the sex of the patient, male or female;
 the age of the patient in years;
 the education level coded as either 1 for some high school, 2 for a
high school diploma or GED, 3 for some college or vocational
school, and 4 for a college degree.
 Includes behavioral risk factors:
 Does the patient smoke (yes/no)
 Medical history - blood pressure medication, previously had a
stroke, hypertensive or not, diabetic or not
 Includes risk factors from physical examination:
 Cholesterol level
 Systolic/diastolic blood pressure
 Body mass index
 Heart rate
 Blood glucose level
THE FRAMINGHAM HEART STUDY Contd..
Building the model
Split our data randomly into training and testing set
Logistic regression to predict whether or not a patient experienced
Coronary Heart Disease within 10 years of first examination
After building the model, we will evaluate the predictive power of the
model on the test set
Threshold Value
The outcome of a logistic regression model is a probability
Often, we want to make a binary prediction – whether this person will
suffer from CHD or not
We can do this using a threshold value t
If P(CHD = 1) ≥ t, predict CHD

If P(PoorCare = 1) < t, predict healthy
What value should we pick?

Often selected based on which errors are better
Confusion Matrix
Compare actual outcomes to predicted outcomes using a confusion
matrix (classification matrix)
 Sensitivity = TP/(TP + FN)

 Specificity = TN/(TN + FP)
Receiver Operator Characteristic
 Receiver Operating Characteristic (ROC) curve is a graph between True Positive Rate (Sensitivity)
and False Positive Rate (1-Specificity)
 Accuracy is measured by the area under the ROC curve. The greater the area under curve better is
the model. An area of 1 represents a perfect test.
 Each point on the ROC curve represents a cutoff probability. These cutoff point represent the
tradeoff between sensitivity and specificity probabilities
 Ideally the goal should be to have high probabilities for both Sensitivity and Specificity
Selecting a Threshold using ROC
 Captures all thresholds simultaneously
 High threshold means High specificity and Low sensitivity
 Low Threshold means Low specificity and High sensitivity
 Choose best threshold for best trade off:
 cost of failing to detect positives
 costs of raising false alarms
Compute Outcome Measures
 Overall accuracy = (TN + TP)/N
 Overall error rate = (FP + FN)/N
 Sensitivity = TP/(TP + FN)
 Specificity = TN/(TN + FP)
 False negative error rate = FN/(TP + FN)
 False positive error rate = FP/(TN + FP)
Quick Question 2
Using the below confusion matrix, answer the following questions.
FALSE TRUE
0 1069 6
1 187 11
What is the sensitivity of our logistic regression model on the test set,
using a threshold of 0.5?
What is the specificity of our logistic regression model on the test set,
using a threshold of 0.5?
Thank You

Logistic Regression

Uploaded by

Copyright:

Available Formats

Logistic Regression

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Logistic Regression

Uploaded by

Copyright:

Available Formats

Introduction to R

a customer will churn (1) or not (0)

a customer will respond to a campaign (1) or not (0)

should we grant a loan to a particular person (1) or not (0)

 If b is –ve, thens shape will get reversed

This is called logit and it looks like “linear regression equation”

And we have an observation with the following values of independent

What is the value of P(y = 1) for this observation?

Response Conversion Attrition

Finding Credit Finding

Just a few of them

Misconceptions in the first half of 20th Century about blood pressure

If P(CHD = 1) ≥ t, predict CHD

What value should we pick?

 Sensitivity = TP/(TP + FN)

You might also like