Machine Learning For Predictive Data Analytics PDF

What is Predictive Data Analytics? What is ML? How Does ML Work?
Underfitting/Overfitting Lifecycle Summary
Fundamentals of Machine Learning for

Predictive Data Analytics
Machine Learning for Predictive Data Analytics
John Kelleher and Brian Mac Namee and Aoife D’Arcy
[email protected] [email protected] [email protected]

What is Predictive Data Analytics? What is ML? How Does ML Work? Underfitting/Overfitting Lifecycle Summary
1 What is Predictive Data Analytics?
2 What is Machine Learning?
3 How Does Machine Learning Work?
4 What Can Go Wrong With ML?
5 The Predictive Data Analytics Project Lifecycle:

Crisp-DM
6 Summary
What is Predictive Data

Analytics?
Predictive Data Analytics encompasses the business and

data processes and computational models that enable a
business to make data-driven decisions.
Figure: Predictive data analytics moving from data to insights to

decisions.
Example Applications:
Price Prediction
Fraud Detection
Dosage Prediction
Risk Assessment
Propensity modelling
Diagnosis
Document Classification
...
What is Machine Learning?

(Supervised) Machine Learning techniques automatically

learn a model of the relationship between a set of
descriptive features and a target feature from a set of
historical examples.
Figure: Using machine learning to induce a prediction model from a
training dataset.
Figure: Using the model to make predictions for new query instances.
L OAN -S ALARY
ID O CCUPATION AGE R ATIO O UTCOME
1 industrial 34 2.96 repaid
2 professional 41 4.64 default
5 industrial 48 3.80 default
6 industrial 61 2.52 repaid
7 professional 37 1.50 repaid
8 professional 40 1.93 repaid
What is the relationship between the descriptive features

(O CCUPATION, AGE, L OAN -S ALARY R ATIO) and the target
feature (O UTCOME)?
if L OAN -S ALARY R ATIO > 3 then

O UTCOME=’default’
else
O UTCOME=’repay’
end if

else
end if
This is an example of a prediction model


else
end if

This is also an example of a consistent prediction model

else
end if

This is also an example of a consistent prediction model
Notice that this model does not use all the features and the
feature that it uses is a derived feature (in this case a
ratio): feature design and feature selection are two
important topics that we will return to again and again.
What is the relationship between the descriptive features

and the target feature (O UTCOME) in the following
dataset?
Loan-
Salary
ID Amount Salary Ratio Age Occupation House Type Outcome
1 245,100 66,400 3.69 44 industrial farm stb repaid
2 90,600 75,300 1.2 41 industrial farm stb repaid
3 195,600 52,100 3.75 37 industrial farm ftb default
4 157,800 67,600 2.33 44 industrial apartment ftb repaid
5 150,800 35,800 4.21 39 professional apartment stb default
7 193,100 73,200 2.64 38 professional house ftb repaid
8 215,000 77,600 2.77 17 professional farm ftb repaid
10 186,100 49,200 3.78 30 industrial house ftb default
11 161,500 53,300 3.03 28 professional apartment stb repaid
12 157,400 63,900 2.46 30 professional farm stb repaid
13 210,000 54,200 3.87 43 professional apartment ftb repaid
15 143,200 65,300 2.19 32 industrial apartment ftb default
16 203,000 64,400 3.15 44 industrial farm ftb repaid
17 247,800 63,800 3.88 46 industrial house stb repaid
19 213,300 61,100 3.49 21 industrial apartment ftb default
21 154,000 48,900 3.15 49 professional house stb repaid
23 252,000 59,700 4.22 27 professional house stb default
24 175,200 39,900 4.39 37 professional apartment stb default
25 149,700 58,600 2.55 35 industrial farm stb default
if L OAN -S ALARY R ATIO < 1.5 then

else if L OAN -S ALARY R ATIO > 4 then
else if AGE < 40 and O CCUPATION =’industrial’ then
else
end if
if L OAN -S ALARY R ATIO < 1.5 then

else if L OAN -S ALARY R ATIO > 4 then
else if AGE < 40 and O CCUPATION =’industrial’ then
else
end if
The real value of machine learning becomes apparent in

situations like this when we want to build prediction models
from large datasets with multiple features.
How Does Machine Learning

Work?
Machine learning algorithms work by searching through a

set of possible prediction models for the model that best
captures the relationship between the descriptive features
and the target feature.

An obvious search criteria to drive this search is to look for
models that are consistent with the data.

An obvious search criteria to drive this search is to look for
models that are consistent with the data.
However, because a training dataset is only a sample ML
is an ill-posed problem.
Table: A simple retail dataset

ID B BY A LC O RG G RP
1 no no no couple
2 yes no yes family
3 yes yes no family
4 no no yes couple
5 no yes yes single
Table: A full set of potential prediction models before any training

data becomes available.
B BY A LC O RG G RP M1 M2 M3 M4 M5 ... M6 561
no no no ? couple couple single couple couple couple
no no yes ? single couple single couple couple single
no yes no ? family family single single single family
no yes yes ? single single single single single couple
...
yes no no ? couple couple family family family family
yes no yes ? couple family family family family couple
yes yes no ? single family family family family single
yes yes yes ? single single family family couple family
Table: A sample of the models that are consistent with the training
data
no no no couple couple couple single couple couple couple
no no yes couple single couple single couple couple single
no yes yes single single single single single single couple
...
yes no yes family couple family family family family couple
yes yes no family single family family family family single
Table: A sample of the models that are consistent with the training
data
no no no couple couple couple single couple couple couple
no no yes couple single couple single couple couple single
no yes yes single single single single single single couple
...
yes no yes family couple family family family family couple
yes yes no family single family family family family single
Notice that there is more than one candidate model left! It

is because a single consistent model cannot be found
based on a sample training dataset that ML is ill-posed.
Consistency ≈ memorizing the dataset.

Consistency with noise in the data isn’t desirable.
Goal: a model that generalises beyond the dataset and
that isn’t influenced by the noise in the dataset.
So what criteria should we use for choosing between
models?
Inductive bias the set of assumptions that define the model

selection criteria of an ML algorithm.
There are two types of bias that we can use:
1 restriction bias
2 preference bias
Inductive bias is necessary for learning (beyond the
dataset).
How ML works (Summary)

ML algorithms work by searching through sets of potential
models.
There are two sources of information that guide this
search:
1 the training data,
2 the inductive bias of the algorithm.
What Can Go Wrong With ML?

No free lunch!
What happens if we choose the wrong inductive bias:
1 underfitting
2 overfitting
Table: The age-income dataset.

ID AGE I NCOME
1 21 24,000
2 32 48,000
3 62 83,000
4 72 61,000
5 84 52,000
80000
●
60000
●
Income
●
●
40000
20000
0 20 40 60 80 100
Age
80000
●
60000
●
Income
●
●
40000
20000
0 20 40 60 80 100
Age
80000
●
60000
●
Income
●
●
40000
20000
0 20 40 60 80 100
Age
80000
●
60000
●
Income
●
●
40000
20000
0 20 40 60 80 100
Age
80000
80000
80000
80000
● ● ● ●
60000
60000
60000
60000
● ● ● ●
Income
Income
Income
Income
● ● ● ●
● ● ● ●
40000
40000
40000
40000
20000
20000
20000
20000
● ● ● ●
0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

Age Age Age Age
(a) Dataset (b) Underfitting (c) Overfitting (d) Just right
Figure: Striking a balance between overfitting and underfitting when

trying to predict age from income.
There are many different types of machine learning

algorithms.
In this course we will cover four families of machine
learning algorithms:
1 Information based learning
2 Similarity based learning
3 Probability based learning
4 Error based learning
The Predictive Data Analytics

Project Lifecycle: Crisp-DM
Business Data
Understanding Understanding
Data
Prepara1on
Deployment
Data Modeling
Evalua1on
Figure: A diagram of the CRISP-DM process which shows the six

key phases and indicates the important relationships between them.
This figure is based on Figure 2 of [1].
Summary
Machine Learning techniques automatically learn the

relationship between a set of descriptive features and a
target feature from a set of historical examples.
Machine Learning is an ill-posed problem:
1 generalize,
2 inductive bias,
3 underfitting,
4 overfitting.
Striking the right balance between model complexity and
simplicity (between underfitting and overfitting) is the
hardest part of machine learning.
[1] R. Wirth and J. Hipp.

Crisp-dm: Towards a standard process model for data mining.
In Proceedings of the 4th International Conference on the Practical Applications of Knowledge Discovery and
Data Mining, pages 29–39. Citeseer, 2000.
1 What is Predictive Data Analytics?
2 What is Machine Learning?
3 How Does Machine Learning Work?
4 What Can Go Wrong With ML?
5 The Predictive Data Analytics Project Lifecycle:

Crisp-DM
6 Summary

Machine Learning For Predictive Data Analytics PDF

Uploaded by

Copyright:

Available Formats

Machine Learning For Predictive Data Analytics PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Machine Learning For Predictive Data Analytics PDF

Uploaded by

Copyright:

Available Formats

What is Predictive Data Analytics? What is ML? How Does ML Work?

Underfitting/Overfitting Lifecycle Summary

Fundamentals of Machine Learning for

John Kelleher and Brian Mac Namee and Aoife D’Arcy

[email protected] [email protected] [email protected]

1 What is Predictive Data Analytics?

2 What is Machine Learning?

3 How Does Machine Learning Work?

4 What Can Go Wrong With ML?

5 The Predictive Data Analytics Project Lifecycle:

What is Predictive Data

Predictive Data Analytics encompasses the business and

Figure: Predictive data analytics moving from data to insights to

What is Machine Learning?

(Supervised) Machine Learning techniques automatically

What is the relationship between the descriptive features

if L OAN -S ALARY R ATIO > 3 then

if L OAN -S ALARY R ATIO > 3 then

This is an example of a prediction model

if L OAN -S ALARY R ATIO > 3 then

This is an example of a prediction model

if L OAN -S ALARY R ATIO > 3 then

This is an example of a prediction model

What is the relationship between the descriptive features

if L OAN -S ALARY R ATIO < 1.5 then

if L OAN -S ALARY R ATIO < 1.5 then

The real value of machine learning becomes apparent in

How Does Machine Learning

Machine learning algorithms work by searching through a

Machine learning algorithms work by searching through a

Machine learning algorithms work by searching through a

Table: A simple retail dataset

Table: A full set of potential prediction models before any training

Notice that there is more than one candidate model left! It

Consistency ≈ memorizing the dataset.

Inductive bias the set of assumptions that define the model

How ML works (Summary)

What Can Go Wrong With ML?

Table: The age-income dataset.

0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100

(a) Dataset (b) Underfitting (c) Overfitting (d) Just right

Figure: Striking a balance between overfitting and underfitting when

There are many different types of machine learning

The Predictive Data Analytics

Figure: A diagram of the CRISP-DM process which shows the six

Machine Learning techniques automatically learn the

[1] R. Wirth and J. Hipp.

1 What is Predictive Data Analytics?

2 What is Machine Learning?

3 How Does Machine Learning Work?

4 What Can Go Wrong With ML?

5 The Predictive Data Analytics Project Lifecycle:

You might also like